The field is not nullable with spatially enabled data frame

SimonWebster — Tue, 01 Sep 2020 04:49:10 GMT

I have a process where I am making use of spatially enabled data frames, and I wish to drop my dataframe back to a geodatabase (or shapefile).

What is occuring, however, is that I receive the below error message. It seems as though I am having some difficulty writing to a geodatabase (and shapefile, but that's a known issue with nulls) when my dataframe contains null values. Because I've got 50+ columns, and potentially 500,000+ rows, I do not wish to be replacing nulls. In fact, many of those nulls exist with good reason.

So, Has anyone seen this before? Is there a flag that I've missed in the documentation that will allow any field being written to permit nulls (where it might be set to notnullable by default)?

Error:

Traceback (most recent call last):
File "<string>", line 272, in execute
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\geo\_accessor.py", line 2074, in to_featureclass
has_m=has_m)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\geo\_io\fileops.py", line 688, in to_featureclass
np.apply_along_axis(_insert_row, 1, df[dfcols].values)
File "<__array_function__ internals>", line 6, in apply_along_axis
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\numpy\lib\shape_base.py", line 379, in apply_along_axis
res = asanyarray(func1d(inarr_view[ind0], *args, **kwargs))
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\geo\_io\fileops.py", line 685, in _insert_row
irows.insertRow(row)
RuntimeError: The field is not nullable. [re_id]
Failed to execute (Tool).

Here's the offending piece of code:

#Create file path variable for the Regional Ecosystem 80th Percentile Fuel Lookup Table
messages.addMessage("Loading RE Fuel data csv to dataframe")
inTableREp = parameters[3].valueAsText
dfRELUT = pd.read_csv(inTableREp)
messages.addMessage("Successfully Read: " + inTableREp)
messages.addMessage(dfRELUT.head())
messages.addMEssage(dfRELUT.dtypes)

#Read Pre-Clear Shapefile
messages.addMessage("Reading PreClear Veg table to spatially enabled dataframe")
dfPreClearVeg = pd.DataFrame.spatial.from_featureclass(tmpPreClearAOI)
messages.addMessage("Successfully Read: " + tmpPreClearAOI)
messages.addMessage(dfPreClearVeg.head())
messages.addMEssage(dfPreClearVeg.dtypes)

#Merge the datframes together
messages.addMessage("Joining PreClear df RE1 with RE Fuel Lookup Table")
left_merged = dfPreClearVeg.join(
dfRELUT.set_index(["RE"]),
lsuffix="",
rsuffix="_lut",
on=["RE1"],
)
messages.addMessage(left_merged.head())
messages.addMessage(left_merged.shape)
messages.addMEssage(left_merged.dtypes)
left_merged.spatial.to_featureclass(location=temp_gdb) #Bombs out here

Re: The field is not nullable with spatially enabled data frame

SimonWebster — Tue, 01 Sep 2020 05:44:45 GMT

The thing that slays me is that ESRIs own example data on the Introduction to Spatially Enabled Dataframes actually has null values in it.

https://developers.arcgis.com/python/guide/introduction-to-the-spatially-enabled-dataframe/

Re: The field is not nullable with spatially enabled data frame

Jose_FranciscoSánchez_Díaz1 — Fri, 23 Apr 2021 11:05:00 GMT

Same problem here! Did you find the solution?

Re: The field is not nullable with spatially enabled data frame

DarrenConly — Fri, 17 Mar 2023 18:19:16 GMT

This is a suboptimal and late answer, but you can always use Pandas' .fillna() method on the SEDF prior to exporting, which replaces nulls with a value of your choice. You can apply to single columns or to an entire dataframe.

Obviously this is suboptimal because as you said, sometimes you want nulls, but this is useful for some cases.

Also, if you found a *real* solution please post!

Re: The field is not nullable with spatially enabled data frame

KellyTaylor — Wed, 24 Jan 2024 21:56:02 GMT

I'm having the same problem. I'd rather not fill my null values, I prefer null to spaces or other fillers. I'm going to submit an idea in ArcGIS Ideas.

topic Re: The field is not nullable with spatially enabled data frame in ArcGIS Pro Questions

The field is not nullable with spatially enabled data frame

Re: The field is not nullable with spatially enabled data frame

Re: The field is not nullable with spatially enabled data frame

Re: The field is not nullable with spatially enabled data frame

Re: The field is not nullable with spatially enabled data frame