On the surface there doesn’t seem to be a lack of physical memory, the memory usage on the ESX host is usually 60 – 70%.

Again thank you.

It could be memory related. See if memory resources are scarce on your host (check for ballooning/swapping on running vms on that host). When there’s memory contention on your host, and that particular vm is idle, than the share-system in combition with the idle memory tax can reclaim memory from that vm. When you restart java, the java process reclaims the memory it needs, but when the host doesn’t have the memory available, it first has to free memory from other vms before it can be assigned to that vm and the java process. This will impact startup performance for java, since java claims a preconfigured amount of memory on startup. After the first startup, the memory is seen as active and not swapped out by the hypervisor directly, so any consecutive startups will have the memory available right away.
Try setting a reservation for that vm, to prevent ESX from swapping that memory out to disk.
As I said before, to be sure if the problem is memory related, you need to investigate what happens on your ESX host when you first restart that java app. Monitor ESX behaviour with esxtop.

-Arnim