Orphaned ArcSOC.exe

Question

Just a few moments ago I came across a curious issue - the over-writing of one of my services continued to fail. This isn't the first time this has happened to me - and last time I fixed it when I re-ran the ArcGIS Server configuration - the one that you specify which user ArcGIS Server is running under. I kept getting the error...SEVERE

Sep 14, 2013 2:23:26 PM

Failed to create the service.: Unable to delete service input folder.

System/PublishingTools.GPServer

Let me continue with a back story. Last night I was fiddling around doing queries - for some reason through the REST interface my service will -never- return if I do an objectid lookup. I have to put OBJECTID=n into the WHERE instead. Me doing this last night caused one of my services to be become unstable - showing one constantly being in use. I shrugged - I know ArcGIS Server will kill off a service that's in use for too long.

However this morning that same one was still in use. I stopped my service and looked at task manager - the ArcSOC.exe for that service did not stop.

Then I decided to try to overwrite the service. Failure - however the service was now no longer part of my list of services on the manager portal. I was confused - I checked folder permissions and everything was in check. It turns out that orphaned ArcSOC.exe was holding onto that folder. I understand that.

What I do NOT understand is how I even GOT an orphaned ArcSOC. What about the automatic clean-up? Why did it miss this one? Perhaps I am misunderstanding "The maximum time a client can use a service:" - which I assumed was ticking down on the "Instances in Use: 1" - with the recycling options, which are set to 24 hours.

However, even STOPPING the service didn't kill off the ArcSOC, so I assume the recycling wouldn't have done it either.

Any advice would be fantastic. Sorry for the length. Try to fit as much as I can.

JustinHunter · Answer

This sounds like a custom SOE I could build that has no purpose other than to ensure the process does eventually end. What I'm worried about is that these processes are kept track of somewhere and my SOE that kills it may cause issues - such as ArcGIS Server thinking there are 5 processes already running when in fact only 4 are.

Thanks for your suggestion. I'm going to duplicate this and bring it to ESRI's attention.

Update.png

RichardWatson · Answer

The way that I solved this problem was when the SOE is Shutdown I start a background thread.  What the thread does is to wait some amount of time (say 5 minutes) and then it kills the process.  What this does is to give the process time to cleanly exit.  If it hangs then death is forced.  Doing this stops the zombie ArcSOC processes.

JustinHunter · Answer

My service does have an SOE that I built attached to it, however running it was not the cause of the original hanging - if that makes a difference.