BIG DATASTORE : Cannot fetch data beyond the 10 000th record using Result Offset

Yann-EricBoyeau — Tue, 18 Jan 2022 00:52:05 GMT

Hi,

We are currently testing the queries performance of an hosted feature layer (Big Datastore) . We implemented the FAA flight geoevent and BDS tutorial available here : https://www.arcgis.com/home/item.html?id=870b1bf0ad17472497b84b528cb9af00 to create the featureservice

Our Big Data Store is now hosting more than 1 million points, here is the exact count by querying the feature layer :

We were not able to query the data beyond the 10 000th record by using the result offset. For an example if we want to retrieve the 2 000 records between 9 000-11 000, the feature layer will only fetch 1 000 records.

the service is configured by default with Maxrecordcount=10 000 but I assume using a Result Offset, I do not have to change this setting to fetch the 1000 records missing in our example.

Has anyone experienced the same issue with BDS ?

(We do not have this issue when using postgresql based featurelayer.)

Re: BIG DATASTORE : Cannot fetch data beyond the 10 000th record using Result Offset

Anonymous User — Fri, 21 Jan 2022 15:43:46 GMT

Hello @Yann-EricBoyeau

You could try modifying the maxResultWindow parameter for the STBDS hosted feature layer from the default 10,000 to a larger value to see if this changes the behavior. I would strongly recommend doing this on a test feature service first:

Browse to the REST endpoint of the Hosted Feature Layer.
Click on the LayerID (0), or the layer we are working with for this service.
Between the breadcrumb and bold layer title, there is a small link named "JSON", click this link.
Move your mouse to the URL of this page (ex. .../server/rest/services/Hosted/...). Add "admin" between /rest/services/ like so ".../server/rest/admin/services/Hosted/..."
Use the browser to search for "maxResultWindow". It will be at the very bottom. This parameters value should be "10000".
Navigate back to or pick the browser tab sitting on the services LayerID page. Make sure "admin" is still shown in the URL: (ex. .../server/rest/admin/services/Hosted/serviceName/FeatureServer/0).
Scroll to the very bottom and click the "Update Definition" link.
Add "{“maxResultWindow”:15000}", (do not use the quotes) to the Input box. Click the "updateDefitnion" button.

I hope this helps!

Best,

Calvin

topic BIG DATASTORE : Cannot fetch data beyond the 10 000th record using Result Offset in ArcGIS GeoEvent Server Questions

BIG DATASTORE : Cannot fetch data beyond the 10 000th record using Result Offset

Re: BIG DATASTORE : Cannot fetch data beyond the 10 000th record using Result Offset