topic How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer? in ArcGIS Enterprise Questions

How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

Yogesh_Chavan — Mon, 13 May 2024 07:33:06 GMT

We are facing the below severe error logs on ArcGIS Server.

Trying to understand what could be triggering this or is there some ways to minimize/fix these errors?

Any help on this would be highly appreciated.

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

JakeSkinner — Mon, 13 May 2024 11:17:40 GMT

Hi @Yogesh_Chavan, are you services configured with Shared or Dedicated instances?

I've seen this error before when all services are configured with Shared. Try changing the services to Dedicated.

https://enterprise.arcgis.com/en/server/latest/administer/windows/configure-service-instance-settings.htm#ESRI_SECTION1_C6A867B5D3C543668798F75139028A99

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

Yogesh_Chavan — Mon, 13 May 2024 12:08:57 GMT

Thank you @JakeSkinner for your insights.

We have 94 services on the server and very few are using the dedicated pool and all the others are on shared pooling.

Do we have to change pooling for all the services?

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

JakeSkinner — Mon, 13 May 2024 12:23:17 GMT

I would highly recommend the more frequently used services be switched to dedicated. I typically only recommend a few services be set to shared instance type.

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

berniejconnors — Mon, 13 May 2024 18:11:54 GMT

I agree with @JakeSkinner. We configure our map services in a similar way. We reserve the use of shared instances for services we are quite certain will be used infrequently. About 20 map services (or 25%) use the shared pool.

Bernie.

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

DavidColey — Fri, 17 May 2024 15:02:33 GMT

I would agree with both @JakeSkinner and @berniejconnors - our Federated site is a 2 machine cluster, 8 cores each. So for the site, I have the dynamic mapping service pooling set to 8 number of instances, giving me 16. But you shouldn't set the number of instances it can use to more than the number of cores you have.

So as a rule of thumb for the dynamic service, I would publish no more than 30 simple map services services to the shared instance. In practice, I have about the same as Bernie, 15 -20 or so simple map image services.

As for the warning about the dynamic service crashing, I do not see that very often and since there is not much I can do about it, I just ignore it. The service recovers and I move on.

I use Monitor, and for my any of my shared or dedicated instances, I can see the % saturation. If I were to see one of my shared instances using up to like 75%, I would consider moving that instance to dedicated. Otherwise, that might be or maybe could be the cause of the dynamic map service instance crashing. . .

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

MattiasEkström — Mon, 13 Jan 2025 10:57:49 GMT

@DavidColey @berniejconnors @JakeSkinner

You all agree that the number of services that use a shared pool should be just "a few" / 15-20 / 25 % / no more than 30. In the documentation I can't see anything that points to that, and the shared pool is used as default. For us, the shared pool was great to save resources on our server when it was introduced. A couple of months ago, we had 10.9.1 with about 100 services in the shared pool and 25 with dedicated, and it worked fine. But then we upgraded to 11.3 which seemed to go well at first, but then we noticed performance issues which came from "Instance of the service 'System/DynamicMappingHost.MapServer'" crashing all the time.

I've tried switching about 15-20 of the most used services in the shared pool to dedicated, so maybe we have about 80 in the shared pool and 45 with dedicated, but it didn't make any difference. Switching all except 20 services to dedicated would make the memory on our server run out.

We have a single server deployment of ArcGIS Enterprise on a virtual server with 4 cores (limited to that by our license) and 32 GB of memory.

Any suggestions?
We have a lot of services with just a few layers. Maybe I have to merge as many as I can to end up with a lower number of services but more layers in each service?

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

JakeSkinner — Mon, 13 Jan 2025 13:44:50 GMT

@MattiasEkström for any services that have relatively static data, I would recommend publishing them as hosted feature services. Hosted feature services are very lightweight, and do not spin up any ArcSOC.exe processes.

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

MattiasEkström — Wed, 15 Jan 2025 10:19:13 GMT

Thanks @JakeSkinner
I didn't know there is a performance benefint to hosted feature services, we have some relatively static data that I could publish as hosted feature services instead.

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

DougBrowning — Mon, 15 Sep 2025 22:19:13 GMT

We are getting these error dumps as posted above also and "Failed to obtain item from data store for given id: ()" also. The admins set us up to do everything as shared to start and not use dedicated. As we migrate to the new server more and more we started getting these errors and really poor performance on some bigger layers like polygons and image services. Some will not draw at all and just site there. Often we get totally white maps.

We doubled the number of shared instances but still having issues. Seems like moving anything that has issues to dedicated has helped. But some still did crash out saying out of resources when set to 1 and 2 (I am thinking 0 min 2 max is best?). We have over 300 services now (but 50 some are just views) and have it set to 16 instances (was set on a 8 core with 50 for each (which I think is the default). Server CPU under 30% and RAM under 40% used. Its a Azure VM.

They come every second or so. Often we can hit one and its fine but hit a second and it starts in the log. We do not have too many users on yet so not sure what will happen then. Polygons in a Ops Dashboard make it the most mad.

I am thinking we wack-a-mole any service that is slow and change to dedicated? Or you seem to say just put the little ones that are not used much to shared then ones we know are big or used a lot to dedicated? And set to?

Not finding too much on this so any help is greatly appreciated! Been on it a week already.

thanks for any info @JakeSkinner I think we may have met in the past.

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

JakeSkinner — Mon, 15 Sep 2025 22:46:04 GMT

@DougBrowning I highly recommend heavily used services to be set to Dedicated. For the number of instances, there is no default answer. I would recommend starting at a min of 1 and a max of 4. Let the service receive requests for a some time (i.e. half day/full day) and take a look in ArcGIS Server Manager > Logs > Statistics. You can view the number of requests that particular service is receiving.

Check the Average Response Time statistic for the service.

I typically recommend keeping the response to 2 seconds or less. If you see excessive times, check the Service Maximum Running Instances for the service:

If the Max Running Instances are maxed out (i.e. 4 instances) this is indicating clients are waiting for a instance to handle their requests, which means you should increase the max instances for this service.

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

DougBrowning — Tue, 16 Sep 2025 16:56:38 GMT

Thanks a lot @JakeSkinner but some issues still I have questions on.

I cannot seem to run any of these reports. It lets me change the drop down but nothing updates. Is this a rights issue? I am not a full admin.

I started changing our services to dedicated, 1 and 4 as you said and its much better. But we still are getting this error in the log a lot. Even with only 1 user and 1 out of 4 instances we get it.

Failed to obtain item from data store for given id: ()

Happening on pretty much every service we try. We double checked, republished, etc but it just keeps coming up. Is it a managed data store config issue maybe? We have a lose admin team right now and hard to tell who setup what and if its correct. Portal is 11.3.

Posts like https://community.esri.com/t5/publishing-and-managing-services-questions/arcgis-server-log-severe-questions/m-p/1631990#M1409 point to a resources thing but we are only using 1 of 4. Is it resources or some other data store issue? The SDE this all connects to is super fast and no issues there.

Thanks a lot we are going a bit nuts here!

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

Scott-Shavnore — Wed, 08 Oct 2025 15:44:10 GMT

@DougBrowning which version of ArcGIS Pro are you publishing the layers from? You mentioned Portal/Federated Server is 11.3, so thinking you may be at ArcGIS Pro 3.3 but do you have any minor version patches installed? If publishing from Pro 3.3.5, you could be encountering this bug https://support.esri.com/en-us/bug/publishing-from-arcgis-pro-335-to-arcgis-enterprise-suc-bug-000175811

It was recently reported as fixed in Pro 3.3.7, I plan to begin testing this to see if it resolves the same issue in our environment.

Re: How to treat Severe ArcGIS Server Logs for System/DynamicMappingHost.MapServer?

DougBrowning — Wed, 08 Oct 2025 15:55:55 GMT

I am 3.3.5 yes. We have 3.5.2 now on our test builds but I have not tested yet (something big going on here). Will report back when we republish.