Community

BIG DATASTORE : Cannot fetch data beyond the 10 000th record using Result Offset

643
1
01-17-2022 04:52 PM
Yann-EricBoyeau
by
Occasional Contributor
‎01-17-2022 04:52 PM

Hi,

We are currently testing the queries performance of an hosted feature layer (Big Datastore) . We implemented the FAA flight geoevent and BDS tutorial available here : https://www.arcgis.com/home/item.html?id=870b1bf0ad17472497b84b528cb9af00 to create the featureservice

Our Big Data Store is now hosting more than 1 million points, here is the exact count by querying the feature layer :

YannEricBoyeau_0-1642463498930.png
YannEricBoyeau_1-1642463544964.png

YannEricBoyeau_2-1642463576298.png

We were not able to query the data beyond the 10 000th record by using the result offset. For an example if we want to retrieve the 2 000 records between 9 000-11 000, the feature layer will only fetch 1 000 records.

YannEricBoyeau_3-1642464407589.png

the service is configured by default with Maxrecordcount=10 000 but I assume using a Result Offset, I do not have to change this setting to fetch the 1000 records missing in our example.

YannEricBoyeau_4-1642465827074.png

Has anyone experienced the same issue with BDS ?

(We do not have this issue when using postgresql based featurelayer.)

0 Kudos
1 Reply
by Anonymous User
Not applicable
‎01-21-2022 07:43 AM

Hello @Yann-EricBoyeau 

You could try modifying the maxResultWindow parameter for the STBDS hosted feature layer from the default 10,000 to a larger value to see if this changes the behavior. I would strongly recommend doing this on a test feature service first:

  • Browse to the REST endpoint of the Hosted Feature Layer.
  • Click on the LayerID (0), or the layer we are working with for this service.
  • Between the breadcrumb and bold layer title, there is a small link named "JSON", click this link.
  • Move your mouse to the URL of this page (ex. .../server/rest/services/Hosted/...). Add "admin" between /rest/services/ like so ".../server/rest/admin/services/Hosted/..."
  • Use the browser to search for "maxResultWindow". It will be at the very bottom. This parameters value should be "10000".
  • Navigate back to or pick the browser tab sitting on the services LayerID page. Make sure "admin" is still shown in the URL: (ex. .../server/rest/admin/services/Hosted/serviceName/FeatureServer/0).
  • Scroll to the very bottom and click the "Update Definition" link.
  • Add "{“maxResultWindow”:15000}", (do not use the quotes) to the Input box. Click the "updateDefitnion" button.

I hope this helps!

Best,

Calvin

0 Kudos