Editable Feature Service - Poor Performance and Reliability

[KenBragg]

Hi Folks,

We have an ArcGIS Enterprise Server (version 10.8.1) hosting many Map services most of them based on managed data sources (SDE).  One of the editable services (lets call it world_edit) is used heavily by about 12 ArcGIS Pro users for both view and edit.

This service has very poor performance including issues like:

• very slow loading of features
• layers not loading at all
• definition queries failing
• service not responding at all (without a restart)
• failure to sync when taken offline in ArcGIS Pro

Additional Information:

• we see from the server statistics that this world_edit services sometimes creates  1000s of requests to the REST service even when just one or two users are access it
• this service also seems to impact/slow or other services resulting wider problem to our infrastructure.
• 4 instances with 8 max are assigned to this service
• the available CPU is significant both for the server and the database
• Version 10.8.1
• Dedicated instances

Thanks for any feedback,

Ken

[AndrewKesterton]

Hi Ken, 

My reply isn't super focussed, but I thought I'd chip in as a neighbour 😉  

Did you recently upgrade to 10.9.1? We had a similar issue where following the upgrade the server behaved as if it was short of resources, some feature services (with the data in an enterprise geodatabase/Postgres) performing very poorly, or not starting at all.  Publishing new services was difficult.  

The logfiles in the ArcGIS Server Manager had lots of unhandled exceptions like "Failed to construct instance of service 'service_name.MapServer'. Unable to instantiate class for xml schema type: CIMMap" etc..  

Although the server did have plenty RAM and CPU capacity available it seemed to be due to some size limits being hit as this machine was also serving a near global audience with lots of services.  Behaviour similar to in this forum post - https://community.esri.com/t5/arcgis-enterprise-questions/large-arcgis-server-site-stability-issues/...

For us the solution was to switch to shared instances instead of dedicated, and this really helped. 

It may be an option, though doesn't explain the incredibly chatty requests to the REST service that you are seeing I'm afraid..  Other suggestions from me would be to check that the data is indexed, possibly drop and rebuild the spatial index if appropriate?  

Andrew

[KenBragg]

Thanks a lot for this Andrew,

In fact we recently upgraded to 10.8.1 (I should add this to initial post).  But your suggestion about shared vs dedicated instance is worth looking into again. We have been juggling that a bit and in this particular case the service uses a dedicated instance.

Cheers

Ken

[jcarlson]

I think the shared instances is really the best way to go, if you've got to keep your services on an SDE.

In our org, we took a hard look at all our services, and for anything that didn't truly require the capabilities of an SDE, we moved to hosted feature services, and have seem big performance improvements there, too.

I'd like to point out that a request to the service can come from all sorts of things, and it can be hard to correlate that with how it's being used. For instance, I have a service in a Dashboard Data Expression. Because of how the data expression works, it makes dozens of individual requests to the service every time it loads or refreshes. But the impact to the service itself and its performance is quite minimal.

Also, most services have a limit on how many records are returned by a query, so if you're retrieving more than that, it needs to make repeated queries and combine the results together. And finally, if you're using ArcGIS Pro to access your services, note that your caching settings will have a big impact on how much your machine needs to query the service.

[KenBragg]

Thanks a lot for this Josh,

Indeed Esri has suggested hosted services are faster also.  I think we kind of knew this but it has been a priority for us to stick with "managed" databases. I guess we may need to rethink that a bit. 

We'll need to take a look at adjusting those cache settings for our ArcGIS Pro users.

All the best

Ken

[SteveWaldronTG]

Hi Ken Bragg:

Did you find that cache settings improved performance? I'd like to learn more, especially if it pertains to editing from feature service.

[QuantitativeFuturist]

We are also having poor performance on SDE backed services hosted on Enterprise 10.9.1. Implementation done by esri so there should be no configuration issues. This was an expensive implementation and has high Azure running costs - the poor performance is quite shockingly bad and something that I hesitate to use in production. The problem being that a lot of functionality is simply not available on hosted feature services so we're stuck with an expensive poorly performing system. How is this acceptable in any way?

[SteveWaldronTG]

QuantitativeFuturist, 

Did you get a resolution to your Azure instance with poor performing ArcGIS Enterprise?