FeatureSet.from_geojson bug?

[DionLiddell]

When calling FeatureSet.from_geojson (ArcGIS API for Python) in an ArcGIS Pro 2.6 python command line or an advanced Juypter Notebook in ArcGIS Online, I sometimes receive the following error: 

TypeError: the JSON object must be str, bytes or bytearray, not 'Polygon'

The problem for me occurs when my geojson encodes a complex doughnut polygon and the ArcGIS API for Python appears to be attempting to utilise arcpy.  

The error seems to originate in the following python module in both ArcGIS Pro and AGOL:

lib/site-packages/arcgis/features/feature.py

More specifically, around lines 880 and 907 in the feature.py module I've noticed the following code:

geom = arcpy.AsShape(geom)
geometry = Geometry(json.loads(geom))

I think here that geom is a geometry object, and this is causing json.load to crash.

I'm guessing that what is intended here is that Geometry is expecting to be passed a geometry object, so maybe the code could be corrected as follows (not sure):

geom = arcpy.AsShape(geom)
geometry = Geometry(geom)

I'm just wondering if this issue (if it is an issue) has been reported before? And if so is there a fix forthcoming, or a work-a-round?

[JoshuaBixby]

Since the problem occurs "sometimes" and involves a "complex doughnut polygon," it is best to provide an example GeoJSON that can generate the error.

[DionLiddell]

Over the last 24 hours I've discovered an open bug report against the ArcGIS python API that highlights a possibly related issue within the same section of code in feature.py: https://github.com/Esri/arcgis-python-api/issues/600

I'll link my question and observations to that bug report.

Example GeoJSON...

I use arcpy first in both examples below to map the geojson geometry to check that it looks OK.  I then try and create a featureset from the geojson.  The 'simple' doughnut works, but the 'complex' doughnut triggers what appears to be an issue in feature.py.

Both examples assume ArcGIS Pro 2.6 has been opened and a new Map created.  The code is executed in the python window.

EDIT: the simple doughnut is a Polygon, the complex is a MultiPolygon.  If this is the trigger then I might be able to post a smaller example that causes FeatureSet.from_geojson to fall over. 

Simple Doughnut:

geojson = {'type': 'FeatureCollection', 'features': [{"type": "Feature", "geometry":{"type":"Polygon","coordinates":[[[176.08340699700011,-38.684118549999937],[176.07802846800007,-38.687596664999944],[176.08582733500009,-38.691020994999974],[176.08340699700011,-38.684118549999937]],[[176.08268986000007,-38.685767965999958],[176.08453648800003,-38.689640505999932],[176.08111215500003,-38.687112597999942],[176.08268986000007,-38.685767965999958]]]}, "properties": {"name": "doughnut"}}]}

# lets quickly map the geometry using arcpy to see if it looks right

geom = arcpy.AsShape(geojson['features'][0]['geometry'])

arcpy.management.CreateFeatureclass(arcpy.env.scriptWorkspace,"doughnut","POLYGON",spatial_reference=4326)

cursor = arcpy.da.InsertCursor('doughnut',['SHAPE@'])

cursor.insertRow([geom])

del cursor

# within the newly created 'doughnut' layer in your map, you should be able to zoom to the new feature thus confirming the validity of the geometry

# now trying with arcgis python api

import arcgis

# FeatureSet.from_geojson executes without an error with the simple doughnut:

fset = arcgis.features.FeatureSet.from_geojson(geojson)

Complex Doughnut:

# Complex Doughnut Polygon arcgis API crash example: assuming that ArcGIS Pro 2.6 has just been opened, and a new map added - execute the following in the python window

geojson = {'type': 'FeatureCollection', 'features': [{'type': 'Feature', 'geometry': {'type': 'MultiPolygon', 'coordinates': [[[[176.37030675, -38.305125654296276], [176.3695243667, -38.30498478759614], [176.3692713333, -38.305168537596316], [176.3694515833, -38.30545720429661], [176.36861981670003, -38.30585190429698], [176.3684908333, -38.30605207089717], [176.36825398329998, -38.30613775429725], [176.3680026667, -38.30635117089746], [176.3676923, -38.306532537597626], [176.3679374833, -38.30656092089765], [176.3680943333, -38.306546337597645], [176.36892496669998, -38.30619777089731], [176.36915015, -38.30590708759704], [176.36962908330003, -38.305777720896906], [176.36989879999996, -38.306098254297204], [176.3713449833, -38.307033154298104], [176.37171365, -38.30764368759869], [176.37253491669998, -38.30773952089878], [176.37289180000002, -38.3083894875994], [176.3742379167, -38.309207954300184], [176.3748589667, -38.31072793760165], [176.3754008333, -38.31147655430237], [176.37511184999997, -38.31207117090293], [176.3753932833, -38.31250388760334], [176.37485256669999, -38.31294217090377], [176.37331346669998, -38.31290938760374], [176.3717829, -38.3119377209028], [176.37053565, -38.312564587603404], [176.37043173329997, -38.312788920903614], [176.36946801669998, -38.31278613760361], [176.36868585, -38.31294655430377], [176.3679912667, -38.31382375430462], [176.3667350167, -38.314330754305104], [176.36682945, -38.31470753760546], [176.36782843330002, -38.314725770905476], [176.36843613329998, -38.31498260430572], [176.3699562833, -38.31624440430692], [176.3707647667, -38.31605138760674], [176.3725145167, -38.31730725430794], [176.37287485, -38.318000270908605], [176.3726783167, -38.31924188760979], [176.3723840333, -38.319340104309894], [176.37229036669999, -38.31951067091005], [176.3728711833, -38.31995035431048], [176.37292396670003, -38.32015307091068], [176.3731218833, -38.320525537611026], [176.37323905, -38.32059062091109], [176.37365143329998, -38.319913987610455], [176.3739065667, -38.31923197090978], [176.3744011333, -38.31743592090807], [176.3760127167, -38.31452610430528], [176.37685113330002, -38.31374102090454], [176.3773241833, -38.31304428760386], [176.3775591333, -38.31229402090315], [176.37758114999997, -38.311716220902596], [176.3774904333, -38.311376704302276], [176.3769019667, -38.31021517090116], [176.3750496667, -38.308323537599335], [176.3750027333, -38.30826125429928], [176.3748782, -38.30819117089922], [176.3718043833, -38.30659882089768], [176.37151998329998, -38.30645838759756], [176.3711886, -38.30609307089722], [176.37117810000004, -38.3060811042972], [176.37030675, -38.305125654296276]], [[176.37275173329996, -38.31448608760524], [176.3727690833, -38.31448323760523], [176.3728306333, -38.31449037090525], [176.3728823833, -38.31447723760523], [176.37294503330003, -38.31445003760522], [176.37304298329997, -38.31437365430514], [176.37310625, -38.31430297090506], [176.37318808330002, -38.31425227090503], [176.3732169167, -38.31424675430502], [176.37332743330003, -38.314243137605004], [176.37345733330002, -38.314220587604986], [176.3735003667, -38.31420775430498], [176.373588, -38.31415687090492], [176.37361260000003, -38.314126337604904], [176.37366196670004, -38.31401267090479], [176.3736947, -38.31397043760476], [176.3737536, -38.31392733760472], [176.3737764, -38.3139174376047], [176.37413171670002, -38.3138051876046], [176.37421716670002, -38.31376810430455], [176.37428140000003, -38.31371568760451], [176.37432516670003, -38.31366165430446], [176.3743546333, -38.31361267090441], [176.37440409999996, -38.313556170904356], [176.3744621667, -38.3134970876043], [176.37449306670004, -38.31347550430427], [176.37457534999996, -38.31343393760424], [176.3746039667, -38.313423854304226], [176.37470515000004, -38.313409104304206], [176.37474878330002, -38.31340767090421], [176.37477809999996, -38.31341128760422], [176.37479300000004, -38.313417670904215], [176.3748054833, -38.313433254304236], [176.37482691669996, -38.313453137604256], [176.3748619167, -38.31350917090432], [176.37491715, -38.31356223760436], [176.37496755, -38.313578887604365], [176.3749967667, -38.31358022090438], [176.37504875000002, -38.31357165430436], [176.37516254999997, -38.313519904304314], [176.37526326669996, -38.313441137604244], [176.37532035, -38.313418687604226], [176.37537235, -38.313410137604215], [176.3754366833, -38.31341488760422], [176.37544855, -38.31341907090421], [176.3754637, -38.313430004304244], [176.37551155, -38.31345358760426], [176.3755351833, -38.313459687604265], [176.3755788, -38.31345825430426], [176.37559879999998, -38.31345073760425], [176.3756460833, -38.31340803760422], [176.37565118330002, -38.3133941376042], [176.3756500833, -38.31337358760418], [176.3756431833, -38.313353237604154], [176.3755913667, -38.313254320904065], [176.3755836167, -38.31321798760403], [176.3755974333, -38.31320380430402], [176.37565380000004, -38.31316765430398], [176.37571074999997, -38.31314292090396], [176.37574518330004, -38.313132654303956], [176.3757821333, -38.31311543760393], [176.3758799333, -38.31309165430391], [176.3759061, -38.3130908043039], [176.37592671669998, -38.31309468760391], [176.37598206669998, -38.31315005430396], [176.3760748667, -38.31319732090401], [176.37617835, -38.313225954304045], [176.3763105333, -38.313246770904044], [176.37642468330003, -38.31325675430407], [176.3765391833, -38.313273587604094], [176.37656618330004, -38.31334358760416], [176.3765778, -38.31339810430421], [176.37658166669996, -38.31347115430427], [176.3765741333, -38.31349427090429], [176.3765558167, -38.31353373760433], [176.3765421, -38.31360508760441], [176.3765291333, -38.31363523760444], [176.3764643833, -38.31373340430453], [176.3764091167, -38.31379010430457], [176.37619941670002, -38.31395703760473], [176.3761543333, -38.31398595430476], [176.3760734833, -38.31405492090484], [176.37605505, -38.314092120904874], [176.3760403667, -38.314145204304914], [176.3760386667, -38.31416812090493], [176.3760216833, -38.314232704305006], [176.37602299999998, -38.31425782090503], [176.3760297833, -38.314275887605056], [176.37602526669997, -38.314410954305174], [176.37601263330004, -38.3144479543052], [176.3759857333, -38.31449000430525], [176.3755525167, -38.31483805430558], [176.3754862, -38.314906554305644], [176.3754354167, -38.314937954305684], [176.3750991333, -38.31502442090576], [176.3750454333, -38.31505590430578], [176.3750260333, -38.315074837605806], [176.3750100333, -38.315102804305845], [176.3749398, -38.315262887605996], [176.37490175000002, -38.315314437606034], [176.37483785, -38.315373704306104], [176.37464356669997, -38.31550127090621], [176.3745725167, -38.31559048760631], [176.37454596670003, -38.315639387606346], [176.37453141669997, -38.3156947543064], [176.3745323667, -38.31571300430642], [176.37454035, -38.31575390430647], [176.37455633330003, -38.31578082090647], [176.3745594833, -38.31578528760649], [176.3745674667, -38.31582620430652], [176.3745870833, -38.31586672090657], [176.37460100000004, -38.3159097209066], [176.3746169833, -38.31593663760664], [176.3746387833, -38.31596335430665], [176.3746697833, -38.316053820906745], [176.3746741333, -38.31608112090679], [176.3746664833, -38.31615683760684], [176.3746138667, -38.31620885430689], [176.3745576167, -38.316247287606934], [176.3745190667, -38.316289720906965], [176.3744738333, -38.316371237607044], [176.37446255000003, -38.316433354307115], [176.3744684667, -38.316490320907164], [176.37449015, -38.316514770907176], [176.37452975, -38.31654777090722], [176.3745367833, -38.31657042090724], [176.3745328833, -38.31660712090728], [176.3744656167, -38.316657354307324], [176.37442743329999, -38.31670663760738], [176.3744231667, -38.31673648760739], [176.37441541670003, -38.316755054307414], [176.3744167167, -38.31683503760748], [176.37441138330001, -38.316844354307506], [176.3744065333, -38.31686282090752], [176.3743363333, -38.31696802090762], [176.3743003167, -38.31700350430766], [176.37422626669996, -38.31703565430768], [176.37411150000003, -38.31706913760771], [176.37392, -38.317139437607786], [176.3738651, -38.3171480876078], [176.37383650000004, -38.317158170907796], [176.37371435, -38.31716217090782], [176.37354603330002, -38.31711965430776], [176.37348654999997, -38.317096454307745], [176.37345033330004, -38.31707247090772], [176.3734436667, -38.317056687607696], [176.3734127833, -38.31702340430767], [176.3733888, -38.31701045430766], [176.37321745, -38.31696575430762], [176.3731239167, -38.316904787607555], [176.3729829, -38.316827070907486], [176.37296218329996, -38.316820887607484], [176.3728597833, -38.31681280430746], [176.3728511833, -38.31681537090747], [176.37275745, -38.316811587607475], [176.37274620000002, -38.31681883760747], [176.3726220833, -38.3169420876076], [176.3725821667, -38.31695718760761], [176.3725350167, -38.3169473709076], [176.3724962167, -38.31693042090759], [176.3724145167, -38.31687605430753], [176.3723774667, -38.316838454307494], [176.3723560833, -38.316820887607484], [176.3723363833, -38.31667747090735], [176.3723516333, -38.3166380543073], [176.37238723329997, -38.316597937607256], [176.3724563, -38.316531504307214], [176.3724844667, -38.316514504307186], [176.37261483329996, -38.316450537607125], [176.3726926167, -38.31638380430706], [176.37272266669999, -38.316348437607026], [176.37273235, -38.316313804307], [176.37275, -38.31626515430695], [176.37276251669996, -38.316125187606815], [176.3727863167, -38.31603057090672], [176.3727650333, -38.31596268760666], [176.3727496, -38.315947204306646], [176.37268651670004, -38.31591278760661], [176.372536, -38.31577388760649], [176.3723911, -38.31568282090638], [176.3723309167, -38.31564830430636], [176.3722278667, -38.31557638760629], [176.3720654667, -38.315433320906166], [176.37201246670003, -38.315371104306095], [176.3719668833, -38.315285754306004], [176.37195651670004, -38.315256370905985], [176.3719118333, -38.31518700430591], [176.37191005000003, -38.31515505430588], [176.3719144333, -38.31512973760587], [176.37191401670003, -38.3150702709058], [176.3719228167, -38.31501963760577], [176.37198033330003, -38.31490328760565], [176.3720149, -38.314844904305595], [176.37203785, -38.31478692090553], [176.37204703330002, -38.314743154305496], [176.37206695000003, -38.31468298760543], [176.3720678, -38.314646354305395], [176.3721049, -38.31458102090534], [176.372164, -38.3145446709053], [176.37223223330002, -38.31451487090527], [176.37259108329997, -38.314475004305244], [176.3725941167, -38.31447718760524], [176.37275173329996, -38.31448608760524]]]]}, 'properties': {'name': 'complex doughnut'}}]}

# lets quickly map the geometry using arcpy to see if it looks right

geom = arcpy.AsShape(geojson['features'][0]['geometry'])

arcpy.management.CreateFeatureclass(arcpy.env.scriptWorkspace,"doughnut","POLYGON",spatial_reference=4326)

cursor = arcpy.da.InsertCursor('doughnut',['SHAPE@'])

cursor.insertRow([geom])

del cursor

# within the newly created 'doughnut' layer in your map, you should be able to zoom to the new feature thus confirming the validity of the geometry

# now for the arcgis python api

import arcgis

# executing the following line will throw an error. It appears that feature.py is incorrectly passing an arcpy geometry object to json.loads within the code

fset = arcgis.features.FeatureSet.from_geojson(geojson)

[JoshuaBixby]

It is clearly a defect in how MultiPolygons are processed, and you don't need anything complex to test it.  Using a basic MultiPolygon example from GeoJSON - Wikipedia, you can get the same result:

>>> from arcgis.features import FeatureSet
>>>
>>> geojson_multipolygon = {
...   "type": "FeatureCollection",
...   "features": [{
...     "type": "Feature",
...     "geometry" : {
...       "type": "MultiPolygon",
...       "coordinates": [
...         [[[30, 20], [45, 40], [10, 40], [30, 20]]],
...         [[[15, 5], [40, 10], [10, 20], [5, 10], [15, 5]]]
...       ]
...     },
...     "properties": {
...       "name": "Wikipedia MultiPolygon example"
...     }
...   }]
... }
>>>
>>> FeatureSet.from_geojson(geojson_multipolygon)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\feature.py", line 922, in from_geojson
    return FeatureSet.from_dict(geo_to_esri(geojson))
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\feature.py", line 826, in geo_to_esri
    esri["features"] = list(esri_features)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\feature.py", line 832, in extract
    geometry = get_geometry(feature)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\feature.py", line 880, in get_geometry
    geometry = Geometry(json.loads(geom))
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\__init__.py", line 348, in loads
    'not {!r}'.format(s.__class__.__name__))
TypeError: the JSON object must be str, bytes or bytearray, not 'Polygon'
>>> ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Using the polygon with a hole example from GeoJSON - Wikipedia , the method works fine:

>>> from arcgis.features import FeatureSet
>>> 
>>> geojson_polygon = {
...   "type": "FeatureCollection",
...   "features": [{
...     "type": "Feature",
...     "geometry": {
...       "type": "Polygon",
...       "coordinates": [
...         [[35, 10], [45, 45], [15, 40], [10, 20], [35, 10]],
...         [[20, 30], [35, 35], [30, 20], [20, 30]]
...       ]
...     },
...     "properties": {
...       "name": "Wikipedia Polygon example with hole"
...     }
...   }]
... }
>>>
>>> FeatureSet.from_geojson(geojson_polygon)
<FeatureSet> 1 features
>>> ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I recommend contacting Esri Support to log a defect and posting an issue on GitHub - Esri/arcgis-python-api: Documentation and samples for ArcGIS API for Python.

UPDATE:  I went ahead and posted a defect on GitHub:  FeatureSet.from_geojson Fails on MultiPolygons · Issue #758 · Esri/arcgis-python-api · GitHub