Interesting stuff Frank.
One thing that gets confusing is talking about virtual memory.
Microsoft uses it to indicate Swap space, disk space acting as a place to hold RAM images (I know you know that, just putting it out there for clarification.)
So when you say you're up to 5GB of virtual memory, do you mean in your Windows Max Swap space size?
Or do you mean in your VMware allocation for the guest OS?
When we get into the virtual world where we're virtualizing servers, now virtual memory is something different.
It's now memory on the host being set aside for the virtual box's usage.
But the virtual box's guest OS, if Windows, is still doing it's own Virtual (swap) ram....
And I think most Hypervisors have their own method of doing Swap space so that if available physical ram is over allocated, swapping starts happening. In my experience, this causes virtual boxes to start to crawl.
So that's sort of a third class of virtual ram.
As you said, resource exhaustion is the bane of ArcServer. (or really, almost any situation)
I think in this day and age of cheap ram we should be trying to never have to use Swap space.
This can be much easier said than done unless you're lucky enough to have pretty much unlimited resources.
I've noticed substantial improvements in my local VM boxes by requiring they not swap out.
They allocate and receive their chunk of RAM and run or they don't get fired off.
Now this does not address the issue of Windows Virtual memory (swap space)
Again, though, if you allocate enough ram to begin with for the guest, then set the swap space to a low min, say 256MB, then you can run and then hope that the 256MB never grows. This means you're never going to swap space (or if you are it's small chunks < 256mb) and if it does grow, it can give you a good starting point for adding more actual ram (or actual virtual ram for the guest 😉
Besides watching the commit metric, one also wants to watch the Physical Memory %
When that number starts hitting >=80% (I think the magic number is 83%? ) Windows starts swapping big time.
There's quite a few articles out there on this.
Swapping is just bad for performance and all sorts of things. Especially on a server. Especially on a heavily loaded ArcServer with a lot of map services, the amount of disk thrashing that can go on can just paralyze a server.
I was just Goggling about the 80% number and came to realize that with the new flavors of Windows after XP, things have gotten a LOT more complicated at the OS level. As the first article states, "Windows memory management is rocket science"
Now add virtualization onto to this and ....
Here's an older article (2010 ) but worth the read from the perspective of trying to understand Virtual (swap) memory. And the link inside it to Russovich's (of SysInternals fame) 2008 article gets even deeper.
Windows 7 memory usage: What's the best way to measure? | ZDNet
They make some interesting comments about the Commit metric.