SpatialDataFrame - unable to save using to_featureclass

[RobW]

Hi, I am testing out SpatialDataFrame and I am trying to do a simple read-write.  I am unable to find a past post to help with my issue. 

My code (I am using ArcGIS Pro 2.5.1) is: 

import pandas as pd
from arcgis.features import SpatialDataFrame

#setup to read input feature class
AOI = r'G:\Work\pandasTest.gdb\fc_10K'
AOIname = 'fc_10K'
dfAOI = pd.DataFrame.spatial.from_featureclass(AOI)
print(type(dfAOI))

#carry out a test operation
dfAOI.columns = [x.lower() for x in dfAOI.columns]
print ("Number of original records or observations: ", len(dfAOI))

#output to a new feature class
#syntax from documentation:
# to_featureclass(out_location, out_name, overwrite=True, skip_invalid=True)
dfAOI.spatial.to_featureclass(r'G:\Work\pandasTest.gdb', AOIname + "_new")

print ('done')

The output error for the above code is: 

ValueError: 'SHAPE' is not in list

Any insight as to what is going on?

Thanks. 

[JoshuaBixby]

The SpatialDataFrame has been deprecated since August, 2018 when version 1.5 was released:  Introduction to the Spatial DataFrame | ArcGIS for Developers 

The SpatialDataFrame is deprecated as of version 1.5: Please use the Spatially Enabled DataFrame instead. See this guide for more information.

See the Accessing local GIS data section of Introduction to the Spatially Enabled DataFrame | ArcGIS for Developers 

[RobW]

Thanks for the response. The deprecated version threw me off a bit.

I read through some documentation (API 1.8, the intro to spatially enabled dataframe as you pointed out, and Apr  2020 ArcGIS for Python (YouTube) video  - 32:30 mentions SeDF). 

I modified my code as below, but still the  'ValueError: 'SHAPE' is not in list' persists.

It is as if the pd.DataFrame.spatial.from_featureclass(AOI) does not return a geo-enabled dataframe/spatially enabled dataframe (assuming they both mean the same). 

Any chance you can run a local test to give me an indication if it is isolated to my workstation?

import pandas as pd

import arcpy

#setup to read input feature class

AOI = r'G:\Work\pandasTest.gdb\fc_10K'

AOInew = AOI + '_new'

#Should return a Spatially enabled pandas.DataFrame from a feature class.

dfAOI = pd.DataFrame.spatial.from_featureclass(AOI)

print(type(dfAOI))

#carry out a test operation

dfAOI.columns = [x.lower() for x in dfAOI.columns]

print ("Number of original records or observations: ", len(dfAOI))

#output to a new feature class

#syntax from API v. 1.8 documentation:

# to_featureclass(location, overwrite=True)

# exports a geo enabled dataframe to a feature class.

dfAOI.spatial.to_featureclass(AOInew)

print ('done')

[JoshuaBixby]

You haven't loaded the spatial libraries, see the top part of the online introduction:

import pandas as pd

from arcgis.features import GeoAccessor, GeoSeriesAccessor

[RobW]

Thanks again. Actually, I tried that with the same result.

I have been running the code within a cloned workspace. In case it was isolated to my cloned workspace, I decided to try running the code within the default workspace; however I still end up with same error.  

Traceback (most recent call last):
File "<string>", line 22, in <module>
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\geo\_accessor.py", line 1925, in to_featureclass
overwrite=overwrite)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\geo\_io\fileops.py", line 555, in to_featureclass
columns.pop(columns.index(df.spatial.name))
ValueError: 'SHAPE' is not in list

Interestingly, the former SpatialDataFrame is stated to be not deprecated, so I am a little surprised why my original code did not run. 

New at version 1.5, the Spatially Enabled DataFrame is an evolution of the SpatialDataFrame object that you may be familiar with. While the SDF object is still avialable for use, the team has stopped active development of it and is promoting the use of this new Spatially Enabled DataFrame pattern. The SEDF provides you better memory management, ability to handle larger datasets and is the pattern that Pandas advocates as the path forward.

[RobW · Accepted solution]

I believe I figured it out.  In my code, the line: 

dfAOI.columns = [x.lower() for x in dfAOI.columns]

changes all fields names to lower case.  When a feature class is read into a Spatially Enabled DataFrame (SeDF), a 'shape' field is added/included and is set to all caps (SHAPE) by default. So, when the above snippet is ran and stored into a SeDF, SHAPE becomes 'shape'. Then, when 'dfAOI.spatial.to_featureclass(AOInew)' is ran, the interpreter looks for the case-sensitive field SHAPE and does not find it. 

I should not have used

dfAOI.columns = [x.lower() for x in dfAOI.columns]

as a means to test editing data.