We are running 10.2.1 Server on Windows 2012 Server Standard with a default cluster of 2 machines. The machine is heavily spec'd with 8 cores and 48GB of RAM. What we are seeing is this - The machine sits at about 14GB RAM with just the map services spun up. Once we start a caching job, the memory usage climbs and climbs until the cache process crashes and server starts kakking itself with out of resource errors. If we stop server, we see all the AGS processes disappear i.e. ArcGISServer.exe all the ArcSocs and the javaw processes but the memory usage however is not released - and using RAMMAP we see that all the memory is in the non-pooled page area. The memory isn't released until we do a reboot of the server.
describes a similar issue and this thread
talks about something similar, and possible relation to VMWare management software settings.
To quote from the thread:
"Turns out it wasn't an ArcGIS Server issue, but a setting that needed to be changed in our VM software after increasing the amount of RAM."
"We use VMWare, and from what my server guy can remember, it was the configuration manager software that was causing the problem."
So if anyone has done similar and knows what settings to tweak that would be grand.
We are running on VMWare 5.1.