topic Re: Orphaned ArcSOC.exe processes from printing tool in High Availability and Disaster Recovery Questions

Orphaned ArcSOC.exe processes from printing tool

StephanieRinguet2 — Fri, 27 Sep 2019 15:21:33 GMT

Hello,

We keep having orphaned ArcSOC.exe processes piling up on our server. They are all services from the Printing Utility service. We use the ArcGIS Server printing tool (not a custom one) and is set to recycle every 24 hours, but it does not appear to work because we find weeks old ArcSOC.exe processes still running. Pooling is set to 0 (minimum instances) and 2 (max instances). Only way to get rid of them is to kill them manually or reboot the server (which we do monthly). I just ended 49 orphaned ArcSOC processes and the memory on the server went from 90% to 65%. It is obviously not sustainable to keep monitoring this and addressing this manually. This has been going on for weeks but only finding the time to inquire about this now. Can someone please let me know how to fix this?

Thank you!

Re: Orphaned ArcSOC.exe processes from printing tool

JonathanQuinn — Fri, 27 Sep 2019 22:12:04 GMT

How are you identifying them as the Publishing Tools service? Did you scrub out the command line column for security reasons? The command line column will have a -Dservice parameter that associates the SOC process with the service name:

20190927000011.528837-420 ArcSOC.exe 16376 "C:\Program Files\ArcGIS\Server\bin\ArcSOC.exe" -XX:-CreateMinidumpOnCrash -Xmx64M -XX:+UseParallelGC -Dservice=System.PublishingTools.GPServerSync....

In your screenshot, I don't see the command line information.

Re: Orphaned ArcSOC.exe processes from printing tool

StephanieRinguet2 — Mon, 30 Sep 2019 12:42:01 GMT

I am not sure why but the commandline in Command Prompt does not get populated. However, when I look at processes in Task Manager using PID to cross-reference, I can see that all hung up processes are tied to the Printing Tools service.

Re: Orphaned ArcSOC.exe processes from printing tool

MichaelVolz — Mon, 30 Sep 2019 13:27:49 GMT

You should also be able to display the Command line column in Task Manager as well as the PID. I believe PID is a column that is visible by default, but Command line is not. Here is a screenshot of Command line in the Details pane of Task Manager. Right click anywhere on the column names and click on Select columns to change the display of columns available for you to evaluate:

Re: Orphaned ArcSOC.exe processes from printing tool

StephanieRinguet2 — Mon, 30 Sep 2019 13:31:36 GMT

Yes thank you that is what I was saying. That I could see the Commandline in Task Manager and I can see that all orphaned services are tied with Printing Tools.

Re: Orphaned ArcSOC.exe processes from printing tool

MichaelVolz — Mon, 30 Sep 2019 13:49:27 GMT

What version of ArcGIS Server are you using?

Where exactly do you use this printing service (e.g. AGOL web apps, custom apps, both, other)?

Have you tried creating your own vanilla printing service and see if that helps with the orphan processes?

Re: Orphaned ArcSOC.exe processes from printing tool

StephanieRinguet2 — Mon, 30 Sep 2019 13:59:39 GMT

We are currently on ArcGIS Server 10.6.1. We are planning an upgrade to 10.7.1 in late October.

This printing service is used in we apps hosted in our Production Portal. We observed the same behavior in our Staging/Test Portal. We have had issues in the past with our custom printing service so we reverted back to the default Server printing utility tools. What I am puzzled with is the fact that the printing services don't get "recycled", that they keep piling up, which quickly take a toll on the server. Only way to get rid of them is to kill them manually or reboot the server.

Re: Orphaned ArcSOC.exe processes from printing tool

StephanieRinguet2 — Mon, 30 Sep 2019 19:50:54 GMT

Do you suggest I contact ESRI Technical Support for this issue?

Re: Orphaned ArcSOC.exe processes from printing tool

MichaelVolz — Mon, 30 Sep 2019 19:54:51 GMT

Do you have the ability to test this print service outside of your portal to see if it's the portal that is causing the issue? I ask because I don't have these orphan print services, but my org has not yet set up portal.

Re: Orphaned ArcSOC.exe processes from printing tool

StephanieRinguet2 — Tue, 01 Oct 2019 12:04:34 GMT

What do you mean exactly by testing this outside of portal?

Re: Orphaned ArcSOC.exe processes from printing tool

MichaelVolz — Tue, 01 Oct 2019 14:17:00 GMT

The print service I have is only used in standalone web apps that are not connected to a portal (either AGOL or organization portal) and I have no orphan printing process issues. Maybe you could look for some simple sample HTML5/javascript web apps that ESRI provides to test to see if the print service generates the orphan processes when used in those web apps that are not associated with a portal. This is only a hypothesis and something to test, but it might be best just to contact tech support at this point.

Re: Orphaned ArcSOC.exe processes from printing tool

StephanieRinguet2 — Tue, 01 Oct 2019 15:48:18 GMT

I just opened a case with Technical Support

Re: Orphaned ArcSOC.exe processes from printing tool

tSteve_SchunzelAtTacoma — Tue, 01 Dec 2020 19:41:14 GMT

Did you ever find an acceptable solution with Tech Support? We are having the exact same issue. We are running a stand alone Server 10.8.1 and have installed all patches. Our current workaround is just to schedule a restart every evening, but our server admins aren't a big fan of this.

Re: Orphaned ArcSOC.exe processes from printing tool

AndresCastillo — Tue, 05 Oct 2021 15:34:50 GMT

taskkill /f /im arcsoc.exe /T

https://jeanboivin.wordpress.com/2010/07/18/201007kill-all-arcsoc-processes-html/

Re: Orphaned ArcSOC.exe processes from printing tool

nhetrick — Wed, 15 Dec 2021 19:38:43 GMT

There is a patch for this for 10.8.1 and 10.9 it is officially fixed in 10.9.1. Thanks to log4j bug we discovered this. https://support.esri.com/en/bugs/nimbus/QlVHLTAwMDEzNDY1Ng== https://support.esri.com/en/download/7916