ArcGIS Web Map Viewer Max Record

[GIS123EXPERT]

I am working on ArcGIS Enterprise system and I want to publish a layer consisting of 1.7 million points.

I publish this layer with ArcGIS Pro as a wfs service. I set the maximum records by server as 2000000 from the sharing settings.

After publishing the service, you can see the maximum number of records below.

When I open it with the viewer, all the records are not displayed and when I set the WFS OGC options on the Web Map Viewer, I can enter the maximum value of 30000.When I enter the value 30001, I get an error.

  How can I see all my records as WFS?

Thanks..

[ChristopherCounsell]

Thanks @GIS123EXPERT for clarifying.

You don't need to adjust the maxRecordCount here. That record limits the number of features that are returned as part of requests. If you try and load a map extent that exceeds the maxRecordCount of the service/client, it will only return that many features. If you zoom in on the map, the request will be made again, and if you zoom in far enough, the total number of features in the map will be below the max record count.

It is not a good idea to return all features at once. Your client (browser) and your server will have performance issues as it tries to return all the features and render them.

What you can do is use scale visibility ranges. This means the layer won't display on the map until the users zoom in. All 1.7 million points will be available for viewing. Just turn it off at a city/country/world level so that users don't try and request all of them at once. You want to choose a scale range that ideally only returns the max record count (1000) within the map extent. 

https://www.esri.com/arcgis-blog/products/arcgis-online/mapping/using-scale-visibility-ranges-for-sy...

Some web mapping applications also restrict the zoom levels, to help the user experience.

If you do want to visualize the buildings at the smaller scales such as a state or country, I would suggest swapping out individual feature rendering with one of the symbology methods mentioned earlier, like heat mapping or clustering. This allows users to visualize the building features in aggregate, then zoom in to see individual features at an appropriate scale.

https://doc.arcgis.com/en/arcgis-online/reference/best-practices-high-density-data.htm

[berniejconnors]

You said:

        "A public institution wants to see all the building numbers in the same service and at one time"

IMHO, it would be better to solve this issue by using clustering or a heat map at smaller scales and then show all the points at larger scales.  Nobody can visually discern 1.7 million points with labels on a single map.  Your users don't always understand the limits of ArcGIS Server.  It is part of our job to educate our users so they get what they need in a manner that is performant for them and for our servers.

I think Montgomery Scott said it best...

Cheers,

Bernie.

[nadja]

hey @GIS123EXPERT , @ChristopherCounsell , @berniejconnors , @A_Wyn_Jones 

we've just encountered aa similar issue. honestly, we dont care too much about how the map would look with that many points, for us it is far more important that an analysis is carried out with the entire dataset. it might be that the service will be slow, but there's certainly other ways to boost the performance. and even if not - ana anlysis without the entire dataset is simply pointless. 

here's an idea which might help: https://community.esri.com/t5/arcgis-pro-ideas/when-stting-up-a-wfs-connection-make-it-possible/idi-... 

[berniejconnors]

Nadja,

        I am not an expert but here are some opinions:

        If you need access to millions of data points why not just use desktop GIS to connect directly to your database?  Using a map service or feature service to acess the data is just going to slow down the analysis. 

        If you must use a map service or feature service there are methods to send multiple requests to retrieve all of the required records.  If the max record count is 2000 you can get 2000 records with the first request and then 2000 more with the second request, etc., etc., until your user has the entire dataset.  I expect this will require the creation of a custom widget (that you will have to create) to allow your users to retreive all of the records via an ArcGIS map or feature service.  Essentially, your users will be using a custom widget to access the service to retrieve an entire copy of the feature class and store it locally in a data structure on their own computer.  This does not sound efficient.  This is also the definition of "data scraping".

        Perhaps it would be smarter to create a geoprocessing service that you host on your server that can directly access the entire feature class from your database.

Good luck.

Bernie.

[nadja]

@berniejconnors 

well, I can't connect directly to the database, because it is not ours 😉 they provide just a WFS with which we have to do the analysis. so that's simply not an option. 

also we need an option which does not require much programming or data scraping or anything. we need a straightforward method  to access ALL the data which is offered by the data owner. currently we need to either ask what's the feature count or open the data in QGIS to see the number of features and then set the maximum number of features returned by the WFS accordingly. That's unnecessary. 

and since it's not our data, we have no possible to change the way the data is served. 

[berniejconnors]

Nadja,

        Thanks for the clarification.  I thought you were publishing the WFS.  Without doing some research and testing I think your only option is to scrape the data from the WFS service and then do your analysis locally.  You could also ask the data publisher if they also provide a download of the data.  Here is a Google search with some promising results.

Good luck,

Bernie.