Feature Layer query can't return all the matching records

2084
19
Jump to solution
02-19-2020 11:00 PM
simoxu
by MVP Regular Contributor
MVP Regular Contributor

I am using 1.7.0 version of the Python API.

I have a fairly large feature layer published in our ArcGIS Portal 10.7, roughly 1 million records in it.

When I do a query on this layer, and only return the count only, I'll get the count 1329 which is correct. 

But when I try to return the actual records, it only returns 99 records despite that I've set return_all_records to True.

Anyone know why?  and how can I fix it?

Many thanks.

0 Kudos
1 Solution

Accepted Solutions
simoxu
by MVP Regular Contributor
MVP Regular Contributor

BUG-000129661: Querying a feature layer from a referenced feature service with a geometry filter returns an incomplete results using the ArcGIS API for Python

View solution in original post

19 Replies
JoshuaBixby
MVP Esteemed Contributor

At this point, it is unclear whether this is a REST API or ArcGIS API for Python issue.  Have you tried directly querying the REST API to see what results you get?

simoxu
by MVP Regular Contributor
MVP Regular Contributor

Thanks for the response.

I tested the same spatial filter in REST API, it seems returning all the records.

So it is clear to me the issue is in the python API. 

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

What if you check the records in the feature set before converting to data frame?  That might tell you whether it is a feature set issue or a data frame issue.

0 Kudos
simoxu
by MVP Regular Contributor
MVP Regular Contributor

Yes, I tried that. the query does not return all the features intersecting with the input geometry.

This only happens when the geometry filter is being used. there's no problem with the attribute filter using the where parameter.

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

I wonder if the results are correct in the ArcGIS API for Python but they are based on something happening in translation of your geometry filter from Python to REST.  What is your geometry filter? 

0 Kudos
simoxu
by MVP Regular Contributor
MVP Regular Contributor

Hi Joshua

The geometry filter is a ploygon that returns all the land parcels that intersect with it.

Here is an update on what I've done:

I exported JSON of this geometry and tried it in the REST API, it works perfectly.

In Python API, the geometry works fine if only IDs, or counts are returned. so my current workaround is to return all the IDs then query the same layer again using the where parameter to return all the features, which works fine, but apparently this is not the ideal way to do it.

I also tried this in a standalone script, same issue. So it's not a Jupyter notebook bug either. so far I can pretty much isolate it to ArcGIS Python API. I just need ESRI to confirm my guessing. 

I've send ESRI a support request,  and I'll keep this post updated when I hear from them.

Cheers

Simo

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

Nice work, and sometimes a workaround is the best, most practical path forward. Can you reproduce the behavior using a public-facing service and some arbitrary polygon?  If so, please share.  I am guessing it has to do with the geometry information is getting translated from Python to REST.

I will be interested what you hear back.

0 Kudos
simoxu
by MVP Regular Contributor
MVP Regular Contributor

No worries. 

I also tried the same query on a large point layer (the centroids of the features in the original polygon layer), and it works fine. so it happens on polygon layers, I haven't tested polygon lines which I don't use very often. 

Hope we can get the attention from a member in the Dev Team? John Yaist

0 Kudos
JohnYaist1
Esri Contributor

I'll look through this today, simo xu.  Do you see the same results with 1.7.1?

0 Kudos