New Map Viewer is super slow to load when WMS and WMTS layers are in the map

[DougBrowning]

I have a map with 300 WMS and WMTS layers from outside my org.  I made the map in Pro and then published so that it was easy to add so many and so I could make nested groups easily.  (they are for reference by year and can vary in use a lot)

All layers are turned off.  Map Viewer classic loads them all fast no problem but of course there are no groups.

New Map Viewer totally hangs for several minutes.  Sometimes I get a memory error and other times it finally loads but its super slow to use.  I have a feeling the new map is trying to touch all the layers in some way.

Anyone know what is going on here?  Why it is touching all the layers when they are all off.

Open to any ideas.

thanks

[David_McRitchie]

Hey Martin,

I took a look at this again and I am still getting the same performance between both the Map Viewer and Classic. 

I would check to see if the issues are just happening on this one WMS on your device, or if it occurs for other WMS layers.

If it is happening on numerous layers then I would recommend raising a support case with your distributor. Key things I would recommend testing are if the performance is the same on different networks as well other types of data like Hosted Feature Layers.

I would also confirm that your network/firewall follows the domain requirements from the ArcGIS Trust site.

Many thanks,

David

[DougBrowning]

I can share my map to you if you send me your AGOL username and you can take a look and see the old vs new map viewer difference.  It is unusable now.  thanks

[RTPL_AU]

@David_McRitchie Anything new to share? 
I have noticed that when you have more than just a handful of external Esri REST services coming into a new map viewer it will start to grind when doing anything in the interface when interacting with our hosted feature services. (symbology, filters, grouping layers, etc). 
The latest project has to overlay project data over various agency & department open data sources - and no we're not going to download it all and republish 🙂

You earlier mention a limit of 10 layers - Is that an official Esri position that is documented?  Link or it doesn't count. Not sure if Esri UK is the same as Esri Australia (i.e. Esri in nothing but name) and caught in the middle between users and Esri Inc. but hopeful you have something to give this thread a decent conclusion.

It has happened enough that I feel the only common denominator is AGOL - different computers, different data sources (mix of Esri REST & WxS open data services by various agencies), internet speed increasing over time but currently 100mbit only. Cannot say I can see a difference compared to when we only had 24mbit.

EDIT:
Experience Builder apps' performance are fine - users use apps but I have to make the map to publish and it is my time that is compromised.

Classic Viewer didn't fall over so it must be a new thing - Maybe I should get ADSL again, run Classic Viewer on Internet Explorer on my old XP machine? Pity that won't work with the new feature service functions and Field Maps requirements.... 

Also refer to https://community.esri.com/t5/arcgis-online-questions/map-viewer-is-slow-including-setting-default/t... 
and the search that brought me here:
https://community.esri.com/t5/forums/searchpage/tab/message?filter=location&q=map%20viewer%20slow&lo...

Sorry I can't share a map - commercial in confidence etc.  I'll see if I can find the time to build a map to share with dummy data over the government data but seeing as this is not an uncommon issue, maybe Esri could better document the exact constraints to Viewer performance rather than us trying to hunt issues blindly?

Something like - If you have a top spec PC and fast internet (100mbit+) then you should be able to load 10 external REST endpoints (examples below)  + 10 organisation hosted layers. 

These are typical web service sources we use A LOT : 
https://spatial-gis.information.qld.gov.au/arcgis/rest/services/PlanningCadastre/LandParcelPropertyF...

https://spatial-gis.information.qld.gov.au/arcgis/rest/services/Economy/MinesPermitsCurrent/MapServe...

https://spatial-gis.information.qld.gov.au/arcgis/rest/services/Environment/MattersOfStateEnvironmen...

Regards,

Chris

[David_McRitchie]

Hey Chris,

10 layers is what I generally recommend, but it isn't a hard limit and not something that is possible to confidently document. The limit is dynamic and will be on a use case by case basis depending on the layer and map configuration. For example if we were loading a Hosted Feature layer with a few points, it will be a lot more performant than a large polygon layer with complex geometry. I have seen instances with 15 large services performing slowly, while in other cases Web Maps with around 60 layers have performed fine.

When starting out a new Web Map its good to go with 10 layers and see if performance is acceptable before adding additional layers.

Obviously, this can add a lot of complexity and potential red herrings in troubleshooting. A first step I usually use is to check if a new Web Map works with just one layer, then add services back in blocks to see if a particular service, or group of services is causing the slow down.

Web traffic is really useful, we can put exact time stamps for how long particular requests are taking, and observe where slow performance is coming from.

Republishing is a useful step, particularly if a service is coming from ArcGIS Enterprise/Server as it provides a new service, that isn't being used by other users. Often we can see that the tuning for a service is out of sync with what should be used as the service is seeing an increase in users.

Comparing Classic Map Viewer and Map Viewer can also yield useful results as we could be seeing an issue between the ArcGIS JavaScript SDK v3.x and v4.x. Generally from the performance perspective v4.x will subdivide query requests more, causing a large total volume of requests which typically is more performant, but can cause issues if there is an overwhelming number of requests coming in. This in my experience tends to be the case if we see identical services functioning fine in Classic Map Viewer.

In your use case Chris given that the services are external, I would suspect that perhaps these are Enterprise services that need additional instances assigned, or data that is in a different projection. Reprojecting on the fly can add significant delays in performance.

Hopefully that helps. If not then it is worth raising a Support Case. We occasionally do get performance issues and defects with the system and troubleshooting such issues can be easy, or very complex depending on the cause.

Thank you,

David

[DougBrowning]

10 is too few for just about every use case we have.  Esp with reference layers.  For sure needs to be updated and tested way larger than that.  Not handling enterprise class size data has been Esri's biggest downfall for years.  We broke WAB going over 250 fields once.  Every single thing should be tested at 300 fields, 100 layers, and 1 million records before it ever gets released.

[David_McRitchie]

Hey Doug,

Understandably 10 could be more than enough, or too few layers depending on the use case. When requirements begin to get large it is vital that performance testing is done to ensure that applications and maps are performant.

300 fields and 100 layers is a large example. I would expect a web map to perform poorly in this context. Even with most layers' visibility disabled, that is 100 requests that are being sent out each time a user pans or zooms on the map.

David

[DougBrowning]

If the map is still sending requests when a layer is off that is probably a big part of the problem.  No reason to do that and seems strange.  Again old map viewer it was not an issue.  We keep seeing this again and again that the JS 4 versions are having issues that the JS 3 versions had already figured out.  It seems like so much code is being rewritten when it could have been ported.  Been a big step back for us in many ways.

I know all of that is out of your realm.  It just has been hard to be a proponent of Esri when it just plain cannot keep up at the enterprise scale. 

[David_McRitchie]

Hey Doug,

Of course, and performance issues such as these can be a real frustration to encounter. I would encourage you to log a support case with your distributor if you believe there is an issue with the software, or something that could be improved. We are here to help, particularly when things are not working as well as they could.

On review you mentioned referenced data and I have seen a couple cases where performance has dipped significantly when moving to JavaScript v4.x with large referenced datasets. In these they got resolved by configuring the Map refresh rates 

Otherwise, I think we could benefit in checking the web traffic on this. Specifically looking at the query requests. If you copy the URL to these then copy and paste these into a new browser tab with the developer console open how long are they taking to process for the largest services?

The timing of these will let us know if the query itself, is slow to perform, or if it is a wider issue with the visualisation of the map, perhaps its an issue when all the queries are coming together in rapid succession. Generally with these, queries will be queued so sometimes all it takes is one area of slow performance to drag the rest of the performance down.

Another big difference that can improve or degrade performance is that the ArcGIS SDK for JavaScript v4.x introduces more usage of Feature Tiling so one request is sent per tile which results in many more requests being sent out compared with JavaScript v3.x. When dealing with referenced datasets this usually helps performance, but if the number of additional requests is overwhelming to the service instance pool then it can have the opposing effect.

Hopefully that gives a few more ideas to work with, but if not then a Support Case will allow someone to look at the issue in greater detail and help advise from there.

David

[RTPL_AU]

Hi @David_McRitchie 
I have issue with "begin to get large it is vital that performance testing is done"
Esri people keep use these phrases without qualifying key terms like "large".
You are the only ones that have all the data needed to quantify and define exactly what is possible and how it should perform - there by to set realistic and measurable expectations. 

Secondly - the problem here, and what others here also say, is not user side (user defined as the end user of the published web app, not the GIS person creating the map) but with New Map Viewer. Within reason, all other methods of displaying the map where compatible i.e. Map Viewer Classic or various web apps, has decent performance in my case.
New Map Viewer is apparently not performant in a way it should be to cater for publishing 'complex' or 'large' maps - Esri's words, not mine.

Going through a regional enforced monopoly reseller for support for this type of thing is near useless. They will just try to find a workaround (which already exists - download, optimise, republish) and close the case as fast as possible to not affect some performance KPI. Also not easy to just share a map with them due to them being a commercial competitor,  and data being sensitive, containing PII, or just of economic in confidence (NDAs in place etc). Recreating shareable equivalent maps without compensation is not economically viable for every issue I have. 

Please take a step back and rather than giving us (paying customers) more work to do, have a look at the data source I listed yesterday.
Loading various combinations of the themes I listed should give you all the data you need to escalate this with the product team. 
As a lot of work on the Queensland Globe services are supported by Esri Australia, you are in a unique position to work with a skilled regional reseller that is also consultant/service provider to the data sources causing issues with New Map Viewer. 
Please use this opportunity to do well for your customers rather than just pushing "do testing when data large".

Thank you.

[David_McRitchie]

Of course, large can be subjective, what is large for one use case is small for another. 

Anything over 100,000 is what I would consider large, of course this is small when we are looking at databases where millions can be the expectation. 

These services have been checked by a few of us on the post so very much agreed, we can rule out user-side. (Other than map level settings. Map refreshing seems to offer some improvement on the services, least in testing on my side)

Did you get a chance to look at the Web Traffic requests? When I have used the services the export bounding box requests have been slow such as this one below taking 2.6s. For example if we take the request below.

https://spatial-gis.information.qld.gov.au/arcgis/rest/services/PlanningCadastre/LandParcelPropertyF...

and try access it in a new browser window it is taking about 2.6s to complete. This to me would highlight that this is a service level issue and not a problem with Map Viewer. I saw similar behaviour in the other linked services, at least at the time of testing.

The service could be poorly tuned or seeing an increase in requests. If I was looking at the service backend my first thoughts would be testing an increase in the Instance configuration, and checking if the service is cached or being drawn dynamically. 

A republish, or temporarily making the service private could rule out issues from requesting volumes.

When we are encountering performance issues, particularly with services of this calibre we need to  measure performance and my recommendation is that this should be a continual process to ensure their is resiliency against any sudden changes, such as a service getting publicised in the news and suddenly seeing a large increase in usage.

I am part of a distributor so I am not able to directly discuss this with the product team over in the US. However they do keep a close eye on logged defects, enhancements and ideas here on the Esri Community. We are here to listen, learn, advise and help but we need these issues to be highlighted in cases to help bring them to the attention of the appropriate team.

I would flag with Queensland Globe services that some of their GIS services seem to be currently performing slowly. They should be able to check the service configuration to check for potential improvements. 

Full Stack GIS performance tuning is a tricky area, so I would still urge you to log a support case. If we can prove there is a limitation with the new JavaScript SDK then it helps build a case up for a defect to be logged. Fixing these things is easier when we can say that an issue can be easily replicated on different datasets/maps/apps etc.

Thank you,

David