FeatureLayer.query() not returning a feature set

NG-COL

Hello,

I am looking to create a spatially enabled dataframe from a FeatureSet with the ArcGIS API for Python.

I have always:

1. Created a FeatureLayer object
2. Used .query(where="whatever") to return a feature set,
3. Used the .sdf property to create the dataframe.

Today the sdf property doesn't work, .query() is returning a plain dictionary instead of a FeatureSet. Passing as_df=True also returns a dict.

Right now, I am still able to create a SEDF by feeding that same dict to FeatureSet.from_dict().sdf. So it seems like .query() just isn't routing through it.

Pro 3.7.2, stock conda env. The layer is a MapServer sublayer (capabilities: Map,Query,Data) and type() confirms it's a FeatureLayer.

Has anyone seen this, do MapServer sublayers behave differently here?

from arcgis.features import FeatureLayer, FeatureSet

url = r"https://gis.example.com/arcgis/rest/services/Parcels/MapServer/0"
lyr = FeatureLayer(url)

result = lyr.query(where="CITY_NAME='EXAMPLE CITY'")
print(type(result)) # <class 'dict'>

result_df = lyr.query(where="CITY_NAME='EXAMPLE CITY'", as_df=True)
print(type(result_df)) # <class 'dict'>

sdf = FeatureSet.from_dict(result).sdf
print(type(sdf), sdf.shape) # <class 'pandas.DataFrame'> (100, 10)

Hayden Welch

Weird, Is the query valid? the arcgis module has a lot of fallback behavior and the FeatureLayer.query method seems to just be a wrapper around query.Query that calls FeatureSet.fromdict after execution. I'd try it with a known good query like 1=1 or OBJECTID = 1 (or some known good Object ID) to see if it's not erroring out early.

Here's the full logic for the final return path:

class Query:
    def _process_query_result(self, result):
        
        # Something went wrong
        if "error" in result:
            raise ValueError(result)
        
        # Requested Count
        if return_count_only:
            return result["count"]
        # Requested IDs or Extent
        elif return_ids_only or return_extent_only:
            return result
        # Requested Groupby or Statistics
        elif out_statistics or group_by_field_statistics:
            if self.as_df:
                return self._query_df(result)
            return arcgis_features.FeatureSet.from_dict(result)
        # Requested raw result OR The layer is a 3D Layer
        elif raw or is_3d_layer:
            return result

        # Normal path
        result["features"] = result.get("features", [])
        if self.as_df:
            return self._query_df(result)
        return arcgis_features.FeatureSet.from_dict(result)

One of the things that MIGHT be causing your issue is that the Query.execute method will force a raw return if you have a 3D layer OR the query takes too long. This second case triggers in the private _query method that is called by Query.query:

class Query:
    def _query(self, raw=False):
        """Returns results of the query for the provided layer and URL."""
        try:
            encoded_parameters = _encode_params(self.parameters)
            # Perform the initial query
            session = self.layer._con._session
            result: dict = self._send_request(session, encoded_parameters)
            return self._process_query_result(result, raw)
        except Exception as query_exception:
            return self._handle_query_exception(query_exception)  # <- Here be dragons

If this path is hit, it will attempt to retry your query with fewer records:

class Query:
    def _retry_query_with_fewer_records(self):
        """Retries the query with a reduced result record count."""
        max_record = self.parameters.get(
            "resultRecordCount", self._fetch_total_records_count()
        )
        offset = self.parameters.get("resultOffset", 0)

        if max_record < 250:
            raise Exception("Max record count too low; query still failing.")

        result = None
        max_rec = (max_record + 1) // 2  # Halve the record count
        i = 0

        while max_rec * i < max_record:
            self.parameters["resultRecordCount"] = min(
                max_rec, max_record - max_rec * i
            )
            self.parameters["resultOffset"] = offset + max_rec * i

            try:
                records = self._query(raw=True)
                if result:
                    result["features"].extend(records["features"])
                else:
                    result = records
                i += 1
            except Exception as retry_exception:
                raise retry_exception

        return result

Notice that this method DOES NOT check the as_df attribute/argument and instead just silently returns the raw result.