Select to view content in your preferred language

Support a pd.NA to np.nan conversion inside the `sedf.spatial.to_featurelayer()` method

1401
2
03-10-2023 01:11 AM
Status: Open
vmarquart_max-boegl
New Contributor II

Support a pd.NA to np.nan/None conversion inside the `sedf.spatial.to_featurelayer()` method

Enhancement request:

Inside the .spatial.to_featurelayer() method check for pd.NA datatypes and replace them with np.nan or None to supress the error message.

 

Steps to recreate the problem:

1. creating a pandas df from a csv file will produce pd.NA values (if there are missing fields in the csv)

--> works without a problem (pd.NA is the default datatype for null values when setting specific types in pd, e.g. string)

2. Converting the pandas df to a spatially enabled df using arcgis python module (pd.DataFrame.spatial.from_xy())

--> works without a problem

3. Converting the SEDF to a hosted feature layer in AGOL produces following error message:

-> fails with error message: 

Could not insert the row because of error message: value #2 - unsupported type: NAType. Recheck your data

 

Steps to solve the Problem:

1. Fill NAs with numpy nas

2. Convert numpy Nans to None

df.fillna(np.nan)
df = df.replace({np.nan: None}) 

sedf = pd.DataFrame.spatial.from_xy(df=df, x_column='X', y_column='Y', sr=4326)
item = sedf.spatial.to_featurelayer("test_bug_pd_na") 

 

--> See attached Python Notebook for a example and workaround

2 Comments
feralcatcolonist

I recently ran into this issue! I wanted some logging to help identify the problem records, but if that could be abstracted away from the user I'd be all for that.

https://github.com/Esri/arcgis-python-api/issues/1494

GeorgieCassar1

I am getting a similar error with NAType when trying to convert dataframe to a File geodatabase Feature Class too.  df.spatial.to_featureclass   

(TypeError: value #5 - unsupported type: NAType)

Would be nice to have options to deal with these things within the command.