Converting geodataframe to spatially enabled dataframe messes the polygon geometry

1849
7
Jump to solution
09-20-2021 10:18 AM
ASLPipeline
New Contributor II

I use a script to get data into our ArcGIS online organization, but it seems like the GeoAccessor function messes with the vertices and outputs wrong geometry. 

Source Data:

AirspaceLinkPipeline_0-1632154476013.png

Polygon after adding to ArcGIS online using the script below:

AirspaceLinkPipeline_1-1632154578695.png

Not sure how visible it is on the screenshots here, but it seems like the 2 inner holes of this big polygon get all mesed up when converting the geodataframe to a sedf.

This is my script:

 

 

import geopandas as gpd
from arcgis import GIS
import requests
from arcgis.features import GeoAccessor


gis=GIS("https://arcgis.com",username, password)

base_url = 'https://services1.arcgis.com/Hp6G80Pky0om7QvQ/arcgis/rest/services/DoD_Sites_Boundaries_Public/FeatureServer/0/query'

payload = {
'f': 'geojson',
'where': "SITE_NAME='Eglin Air Force Base'",
'outSR': 4326,
'outFields': '*'
}
response = requests.get(base_url, params=payload)

gdf=gpd.GeoDataFrame.from_features(response.json()['features'], crs="epsg:4326")

sedf=GeoAccessor.from_geodataframe(gdf, column_name="geometry")

lyr = sedf.spatial.to_featurelayer('test_poly')

 

 

I have fiddled with this for hours. I tried the .buffer(0) method - didn't work. I tried the make_valid() function from shapely.validation too. Filling holes before converting to a spatially enabled dataframe worked, but it is not an acceptable solution. 
Any ideas? I have thought the vertices ordering might be to blame, since ESRI uses different clockwise/counterclockwise direction for exterior/interior rings, but I don't know how I would go about to change those.
Or maybe there is a way to bypass GeoAccessor altogether when trying to publish / append to a feature layer?

0 Kudos
1 Solution

Accepted Solutions
ASLPipeline
New Contributor II

While my Issue on the Esri's Github is still "under consideration" (link here), I have found a workaround that does the job for us. Basically I bypass GeoAccessor.from_geodataframe() function altogether and append the data to an existing feature layer using:

 

Target_Layer = gis.content.get('{}'.format(layerid))
json_gdf=gdf.to_json()
insights_dict=json.loads(json_gdf)

# create FeatureSet from json dict and append data to the target layer
fset=FeatureSet.from_geojson(insights_dict)
for feature in fset:
    add_result = Target_Layer.layers[0].edit_features(adds=[feature])

 

View solution in original post

0 Kudos
7 Replies
DanPatterson
MVP Esteemed Contributor

related? spatial.from_geodataframe() mangles geometries · Issue #931 · Esri/arcgis-python-api (github.com)

you might want to start an "Issue" there to narrow down the possibilities.


... sort of retired...
ASLPipeline
New Contributor II

Looking more at this, I am pretty sure it's a bug.. Thanks for sending that link! I don't even know if I should be glad or cry, these probably take a while to get solved. 
Do you know of any other way to publish a geodataframe as a feature layer?

0 Kudos
Webturtles
New Contributor II

Have you tried repairing the geometry first?  Admittedly I mix the API with ArcPy, but to dissolve groups of polygons I had to run arcpy.management.RepairGeometry (used OGC validation) first.
So I had sqlite - gdf - sedf - fc - repair geoms - dissolve.  Maybe not 100% related, but sometimes shifting formats/APIs can bypass the problem.

0 Kudos
ASLPipeline
New Contributor II

I tried to repair geometries using shapely functions, thing is, by the time I would be able to use arcpy repair - the geometries are already mangled. But yes, that is my thought too.. I need to get there using different formats/APIs, I am trying to add the data to a regular dataframe  instead of geodataframe at the moment, but that gives me some other errors. 

0 Kudos
ASLPipeline
New Contributor II

While my Issue on the Esri's Github is still "under consideration" (link here), I have found a workaround that does the job for us. Basically I bypass GeoAccessor.from_geodataframe() function altogether and append the data to an existing feature layer using:

 

Target_Layer = gis.content.get('{}'.format(layerid))
json_gdf=gdf.to_json()
insights_dict=json.loads(json_gdf)

# create FeatureSet from json dict and append data to the target layer
fset=FeatureSet.from_geojson(insights_dict)
for feature in fset:
    add_result = Target_Layer.layers[0].edit_features(adds=[feature])

 

0 Kudos
MehdiPira1
Esri Contributor

@ASLPipeline ,

There is also a simpler way of implementing this process. You don't need to use neither Geopandas nor GeoAccessor:

from arcgis.gis import GIS
from arcgis.features import FeatureLayer

gis=GIS("https://arcgis.com",username, password)
Target_Layer = gis.content.get('{}'.format(layerid))

flyr = FeatureLayer("https://services1.arcgis.com/Hp6G80Pky0om7QvQ/arcgis/rest/services/DoD_Sites_Boundaries_Public/FeatureServer/0", gis=gis)
features = flyr.query(where="SITE_NAME='Eglin Air Force Base'")
for feature in features:
    add_result = Target_Layer.layers[0].edit_features(adds=[feature])

Cheers

Mehdi

ASLPipeline
New Contributor II

Thank you for sharing this. In reality, my code was much more complicated than the snippet I posted here, and I needed to process large amounts of data, not just copy them from one source to another. The query only works for 2k features and I wasn't able to find a way to buffer a FeatureSet, so I had to use a Geodataframe.. but that's already off topic.

In the meantime, I received an email from the developers that the GeoAccessor bug was fixed. 

0 Kudos