topic Converting geodataframe to spatially enabled dataframe messes the polygon geometry in ArcGIS API for Python Questions

Converting geodataframe to spatially enabled dataframe messes the polygon geometry

ASLPipeline — Mon, 20 Sep 2021 17:59:30 GMT

I use a script to get data into our ArcGIS online organization, but it seems like the GeoAccessor function messes with the vertices and outputs wrong geometry.

Source Data:

Polygon after adding to ArcGIS online using the script below:

Not sure how visible it is on the screenshots here, but it seems like the 2 inner holes of this big polygon get all mesed up when converting the geodataframe to a sedf.

This is my script:

import geopandas as gpd from arcgis import GIS import requests from arcgis.features import GeoAccessor gis=GIS("https://arcgis.com",username, password) base_url = 'https://services1.arcgis.com/Hp6G80Pky0om7QvQ/arcgis/rest/services/DoD_Sites_Boundaries_Public/FeatureServer/0/query' payload = { 'f': 'geojson', 'where': "SITE_NAME='Eglin Air Force Base'", 'outSR': 4326, 'outFields': '*' } response = requests.get(base_url, params=payload) gdf=gpd.GeoDataFrame.from_features(response.json()['features'], crs="epsg:4326") sedf=GeoAccessor.from_geodataframe(gdf, column_name="geometry") lyr = sedf.spatial.to_featurelayer('test_poly')

I have fiddled with this for hours. I tried the .buffer(0) method - didn't work. I tried the make_valid() function from shapely.validation too. Filling holes before converting to a spatially enabled dataframe worked, but it is not an acceptable solution.
Any ideas? I have thought the vertices ordering might be to blame, since ESRI uses different clockwise/counterclockwise direction for exterior/interior rings, but I don't know how I would go about to change those.
Or maybe there is a way to bypass GeoAccessor altogether when trying to publish / append to a feature layer?

Re: Converting geodataframe to spatially enabled dataframe messes the polygon geometry

DanPatterson — Mon, 20 Sep 2021 20:32:57 GMT

you might want to start an "Issue" there to narrow down the possibilities.

Re: Converting geodataframe to spatially enabled dataframe messes the polygon geometry

Webturtles — Tue, 21 Sep 2021 07:22:43 GMT

Have you tried repairing the geometry first? Admittedly I mix the API with ArcPy, but to dissolve groups of polygons I had to run arcpy.management.RepairGeometry (used OGC validation) first.
So I had sqlite - gdf - sedf - fc - repair geoms - dissolve. Maybe not 100% related, but sometimes shifting formats/APIs can bypass the problem.

Re: Converting geodataframe to spatially enabled dataframe messes the polygon geometry

ASLPipeline — Tue, 21 Sep 2021 13:35:49 GMT

I tried to repair geometries using shapely functions, thing is, by the time I would be able to use arcpy repair - the geometries are already mangled. But yes, that is my thought too.. I need to get there using different formats/APIs, I am trying to add the data to a regular dataframe instead of geodataframe at the moment, but that gives me some other errors.

Re: Converting geodataframe to spatially enabled dataframe messes the polygon geometry

ASLPipeline — Tue, 21 Sep 2021 13:40:17 GMT

Looking more at this, I am pretty sure it's a bug.. Thanks for sending that link! I don't even know if I should be glad or cry, these probably take a while to get solved.
Do you know of any other way to publish a geodataframe as a feature layer?

Re: Converting geodataframe to spatially enabled dataframe messes the polygon geometry

ASLPipeline — Tue, 02 Nov 2021 14:07:19 GMT

While my Issue on the Esri's Github is still "under consideration" (link here), I have found a workaround that does the job for us. Basically I bypass GeoAccessor.from_geodataframe() function altogether and append the data to an existing feature layer using:

Target_Layer = gis.content.get('{}'.format(layerid)) json_gdf=gdf.to_json() insights_dict=json.loads(json_gdf) # create FeatureSet from json dict and append data to the target layer fset=FeatureSet.from_geojson(insights_dict) for feature in fset: add_result = Target_Layer.layers[0].edit_features(adds=[feature])

Re: Converting geodataframe to spatially enabled dataframe messes the polygon geometry

MehdiPira1 — Wed, 03 Nov 2021 02:43:17 GMT

@ASLPipeline ,

There is also a simpler way of implementing this process. You don't need to use neither Geopandas nor GeoAccessor:

from arcgis.gis import GIS from arcgis.features import FeatureLayer gis=GIS("https://arcgis.com",username, password) Target_Layer = gis.content.get('{}'.format(layerid)) flyr = FeatureLayer("https://services1.arcgis.com/Hp6G80Pky0om7QvQ/arcgis/rest/services/DoD_Sites_Boundaries_Public/FeatureServer/0", gis=gis) features = flyr.query(where="SITE_NAME='Eglin Air Force Base'") for feature in features: add_result = Target_Layer.layers[0].edit_features(adds=[feature])

Cheers

Mehdi

Re: Converting geodataframe to spatially enabled dataframe messes the polygon geometry

ASLPipeline — Fri, 18 Feb 2022 14:34:56 GMT

Thank you for sharing this. In reality, my code was much more complicated than the snippet I posted here, and I needed to process large amounts of data, not just copy them from one source to another. The query only works for 2k features and I wasn't able to find a way to buffer a FeatureSet, so I had to use a Geodataframe.. but that's already off topic.

In the meantime, I received an email from the developers that the GeoAccessor bug was fixed.