The field is not nullable with spatially enabled data frame

1034
4
08-31-2020 09:49 PM
SimonWebster
Occasional Contributor

I have a process where I am making use of spatially enabled data frames, and I wish to drop my dataframe back to a geodatabase (or shapefile). 

What is occuring, however, is that I receive the below error message. It seems as though I am having some difficulty writing to a geodatabase (and shapefile, but that's a known issue with nulls) when my dataframe contains null values. Because I've got 50+ columns, and potentially 500,000+ rows, I do not wish to be replacing nulls. In fact, many of those nulls exist with good reason.

So, Has anyone seen this before? Is there a flag that I've missed in the documentation that will allow any field being written to permit nulls (where it might be set to notnullable by default)?

Error:

Traceback (most recent call last):
File "<string>", line 272, in execute
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\geo\_accessor.py", line 2074, in to_featureclass
has_m=has_m)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\geo\_io\fileops.py", line 688, in to_featureclass
np.apply_along_axis(_insert_row, 1, df[dfcols].values)
File "<__array_function__ internals>", line 6, in apply_along_axis
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\numpy\lib\shape_base.py", line 379, in apply_along_axis
res = asanyarray(func1d(inarr_view[ind0], *args, **kwargs))
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\geo\_io\fileops.py", line 685, in _insert_row
irows.insertRow(row)
RuntimeError: The field is not nullable. [re_id]
Failed to execute (Tool).

Here's the offending piece of code:

#Create file path variable for the Regional Ecosystem 80th Percentile Fuel Lookup Table
messages.addMessage("Loading RE Fuel data csv to dataframe")
inTableREp = parameters[3].valueAsText
dfRELUT = pd.read_csv(inTableREp)
messages.addMessage("Successfully Read: " + inTableREp)
messages.addMessage(dfRELUT.head())
messages.addMEssage(dfRELUT.dtypes)

#Read Pre-Clear Shapefile
messages.addMessage("Reading PreClear Veg table to spatially enabled dataframe")
dfPreClearVeg = pd.DataFrame.spatial.from_featureclass(tmpPreClearAOI)
messages.addMessage("Successfully Read: " + tmpPreClearAOI)
messages.addMessage(dfPreClearVeg.head())
messages.addMEssage(dfPreClearVeg.dtypes)

#Merge the datframes together
messages.addMessage("Joining PreClear df RE1 with RE Fuel Lookup Table")
left_merged = dfPreClearVeg.join(
dfRELUT.set_index(["RE"]),
lsuffix="",
rsuffix="_lut",
on=["RE1"],
)

messages.addMessage(left_merged.head())
messages.addMessage(left_merged.shape)
messages.addMEssage(left_merged.dtypes)

left_merged.spatial.to_featureclass(location=temp_gdb) #Bombs out here

4 Replies
SimonWebster
Occasional Contributor

The thing that slays me is that ESRIs own example data on the Introduction to Spatially Enabled Dataframes actually has null values in it. 

https://developers.arcgis.com/python/guide/introduction-to-the-spatially-enabled-dataframe/

0 Kudos
Jose_FranciscoSánchez_Díaz1
New Contributor III

Same problem here! Did you find the solution?

0 Kudos
DarrenConly
Occasional Contributor

This is a suboptimal and late answer, but you can always use Pandas' .fillna() method on the SEDF prior to exporting, which replaces nulls with a value of your choice. You can apply to single columns or to an entire dataframe.

Obviously this is suboptimal because as you said, sometimes you want nulls, but this is useful for some cases.

Also, if you found a *real* solution please post!

0 Kudos
KellyTaylor
New Contributor III

I'm having the same problem. I'd rather not fill my null values, I prefer null to spaces or other fillers. I'm going to submit an idea in ArcGIS Ideas.

0 Kudos