We've got similar issues, and we're at 10.2 so extremely keen to find out from ESRI what we can do about this. Our logging javaw.exe process has reached 1.5+gig on a number of occasions and won't let up. We have about 20 map services hooked in, so not a massive load as far as SOCs are concerned, but that log process concerns me when I read that some users are getting 3+ gig tied up there!
We're about to upgrade to 16gig RAM, across 4 servers/nodes (quad core) in the 'site' and will be hitting that up with some decent load tests in the next day or two, so will report back some findings if that helps anyone!
First and foremost - is there a way of restricting the amount of memory used for the logging process?? it seems the other javaw.exe has a maximum memory set in the string, but the logging one doesn't - or is this not a good idea? Just looking for some feedback on this if possible.