Problems with Shared Instance Services

[Martin1]

We are working with ArcGIS Enterprise 11.3 in an environment with four servers (Portal, 2x ArcGIS Server, DataStore) running on Windows Server 2022. The data for our webservices is referenced from Enterprise-Geodatabases. 

We have 415 webservices (324 shared intances, 85 dedicated instances, 6 hosted)

We are having issues with the shared instance services that are not behaving properly. Sometimes they take a long time to show up or disappear during usage.

In the logs I see a lot of the following error for many different webservices:

• CreateDynamicMapService failed for service "..." 

 

Unfortunately, even when setting the log level to "Fine" there is not more information.

It seems that the DynamicMappingHost that is used to run the shared instance services has an issue, but I am not able to find out what the problem is. I have played around with the Shared Instance Settings, changed the values for javaHeapSize, SDEConnectionPoolSize for the DynamicMappingHost as well as the Web server maximum heap size for the ArcGIS Servers. I even adapted the non-interactive desktop heap size in the  Windows registry, but nothing seems to affect this problem?

Has anybody come across something similar or has any additional ideas, where this could be coming from?   

[rcGIS]

Hello @Martin1, there appear to be some issues with shared instances in ArcGIS Enterprise 11.3 and 11.4.

This has been an active post/thread on the topic: https://community.esri.com/t5/arcgis-enterprise-questions/upgrade-from-11-3-to-11-4-broke-shared-ins...

[Martin1]

@rcGIS Thanks for the tip. 
I tried an update to 11.5. Unfortunately, this did not solve the problems. It seems to come from somewhere else.

[rcGIS]

Thanks for the feedback, @Martin1. A pity. These two posts (1 and 2) might be of interest too. 

To summarize, some suggest limiting the number of shared instances to a certain %. For any services using relatively static data, using hosted services instead of referenced services could be a good workaround in terms of system performance.

[AdamMesser1]

@Martin1 Do you know if you have DynamicMappingHost ArcSOCs that are not being cleaned up? We have this same issue, which I am currently working with support on. I realized that over time our system became less stable. Even though the DMHost has a 24 hour recycle interval for midnight and 8 maximum instances, there more than 8 ArcSOCs in the task manager associated with the DMHost service. There are 8 from today as I would expect, but 3 extra from prior days. Our DEV environment has even more orphans consuming even more memory. We have been having to reboot the server to 'fix' our problems and I believe this may be the fundamental cause. It seems rebooting may be something other are doing as well per Solved: Server service needs restarted every 7 days or so - Esri Community.

[Martin1 · Accepted solution]

@AdamMesser1 @rcGIS Thanks a lot for your suggestions.

We played around with a lot of different parameters, added four more CPUs to each of our two ArcGIS Servers and made sure we are using the most current versions of Enterprise Geodatabases, ArcGIS Enterprise, etc... unfortunately, without success.

After some more analysis, I think we are facing the following known bug:

When using a web adaptor URL with http/2 enabled on Internet Information Services (IIS) and in the w...

I hope there will be a solution soon.