Guidance Needed on Rendering Large Datasets with ArcGIS JS API

FactSetResearchSystems

I'm working with @ArcGIS/core@4.21.2 in a Vue application and facing some performance challenges when trying to display a large dataset on a map.

We’ve uploaded a CSV file with around 15 million rows into a hosted feature layer. When applying a filter, the resulting dataset includes approximately 350,000 points. However, the data is being fetched in batches, and seems to be limited by a maxRecordCount setting. Even though I attempted to update this setting using the ArcGIS updateDefinition API (as described here: https://developers.arcgis.com/rest/services-reference/online/update-definition-feature-service-.htm), the value still shows as 1000, and the actual requests seem capped at 4000 records.

In some cases, even fetching 4000 records takes up to 2 minutes, possibly due to how offset-based pagination is handled.

Given this situation:

• What is the best approach for efficiently displaying such large datasets on the map?

• Would it be better to use techniques like tiling, generalization, or server-side clustering?

Any recommendations or best practices would be greatly appreciated.

mgeorge

@EricLake1 yes this is using AGOL hosted here, though the cold cache times shouldn't be that bad for this either (you can test this yourself by adding a definition expression like 4=4 to the layer). Also note that if you change the initial map zoom for this to be something like 12, the map will instead use tile queries.

Is your backend actually bandwidth limited or are you seeing slow db timings? If it's the latter, deploying a cache in front of your service is probably the way to go.