ArcGIS for Server 10.2.2 javaw

[DaveTenney]

does anyone know of any major issues with ArcGIS for Server javaw consuming massive amounts of memory and almost bringing down an entire server instance?

thanks

dave

[JonathanQuinn]

Great find!  The first thing you should do is consider adding a robots.txt file, like you mentioned, within your web server.

On the software side, I think it would be a good idea to get in touch with Support and have them investigate why you can still download a KMZ when the capability is not checked and also, why the CPU usage seems to jump so much when making those requests.  Those seem like areas for improvement within the software.

[RebeccaStrauch__GISP]

I know that if you have something on OpenData, there is no way for you to prevent KML as an option, even if you have it turned off on the ArcGIS Server service side.  (say that 10 times fast!) It would be nice if that was an option to turn off. 

Justin, I'm not sure if you have your services in OpenData or not, but something to keep in mind.  I haven't run into this issue yet since we don't have much out there yet, but my guess we will at some point).

[DaveTenney]

our vm  cpu spiked to 98-99% and was sustained at that usage. (we were forced to kills arcgis services and reboot the machine)

    at the time of spike we were running GP services and our script failed:

ExecuteError: ERROR 001470: Failed to retrieve the job status from server. The Job is running on the server, please use the above URL to check the job status.

Failed to execute (ManageMapServerCacheTiles).

like i said above, this is concerning since we installed the 2016 Security Patch update 2 last week.

thanks

dave

[JonathanQuinn]

What is the memory usage of the javaw.exe process when the CPU usage is high?  The symptoms do sound like BUG-000082423, (Under consistent load, the javaw.exe process at ArcGIS 10.2.2 for Server consumes 25% of the server's RAM, and any further request forces the process to use 100% of the machine's CPU), but you're right in that it should be resolved in that patch.

[JustinWhite2]

Quick follow up for anyone interested.

I would say with 99.9% certainty that the KMZ download combined with Google's relentless crawling of our services resulted in our incredibly high CPU usage and interestingly enough our high RAM usage. The javaw.exe process now spikes sometimes to 10-12% but immediately drops back down and is consistently consuming under 1GB of memory at the moment. 

The fix for me was a bit drastic - I tried the robots.txt approach but was unsure where that should go on the hard drive. I put a copy all over the place but it seemed to have no effect. I then blocked, in a software firewall, the IP range that Google uses to crawl sites. Fortunately, we really could care less if Google crawls this server as the few number of web applications are all direct links and the server's primary function is as an ArcGIS Server.

I have to believe that this is a bug in the ArcGIS Server platform, but perhaps it is a configuration or data issue on our end... I will try and submit a report as Jonathan suggests above, but unfortunately this already consumed several days of my time that I didn't have.

I hope this helps someone experiencing (whether known or not) a similar issue.

Cheers!

Justin

[RandallWilliams]

This can also be addressed by disabling the html representation of the services directory.

http://server.arcgis.com/en/server/10.3/administer/linux/disabling-the-services-directory.htm

http://server.arcgis.com/en/server/latest/administer/linux/best-practices-for-configuring-a-secure-e...