idea Support a pd.NA to np.nan conversion inside the `sedf.spatial.to_featurelayer()` method in ArcGIS API for Python Ideas

Support a pd.NA to np.nan conversion inside the `sedf.spatial.to_featurelayer()` method

vmarquart_max-boegl — Fri, 10 Mar 2023 09:11:41 GMT

Support a pd.NA to np.nan/None conversion inside the `sedf.spatial.to_featurelayer()` method

Enhancement request:

Inside the .spatial.to_featurelayer() method check for pd.NA datatypes and replace them with np.nan or None to supress the error message.

Steps to recreate the problem:

1. creating a pandas df from a csv file will produce pd.NA values (if there are missing fields in the csv)

--> works without a problem (pd.NA is the default datatype for null values when setting specific types in pd, e.g. string)

2. Converting the pandas df to a spatially enabled df using arcgis python module (pd.DataFrame.spatial.from_xy())

--> works without a problem

3. Converting the SEDF to a hosted feature layer in AGOL produces following error message:

-> fails with error message:

Could not insert the row because of error message: value #2 - unsupported type: NAType. Recheck your data

Steps to solve the Problem:

1. Fill NAs with numpy nas

2. Convert numpy Nans to None

df.fillna(np.nan)
df = df.replace({np.nan: None})

sedf = pd.DataFrame.spatial.from_xy(df=df, x_column='X', y_column='Y', sr=4326)
item = sedf.spatial.to_featurelayer("test_bug_pd_na")

--> See attached Python Notebook for a example and workaround

Re: Support a pd.NA to np.nan conversion inside the `sedf.spatial.to_featurelayer()` method

feralcatcolonist — Fri, 17 Mar 2023 00:18:59 GMT

I recently ran into this issue! I wanted some logging to help identify the problem records, but if that could be abstracted away from the user I'd be all for that.

https://github.com/Esri/arcgis-python-api/issues/1494

Re: Support a pd.NA to np.nan conversion inside the `sedf.spatial.to_featurelayer()` method

GeorgieCassar1 — Thu, 23 Mar 2023 03:30:38 GMT

I am getting a similar error with NAType when trying to convert dataframe to a File geodatabase Feature Class too. df.spatial.to_featureclass

(TypeError: value #5 - unsupported type: NAType)

Would be nice to have options to deal with these things within the command.