topic Re: The field is not nullable in Python Questions

The field is not nullable

TonyAlmeida — Sat, 11 Dec 2021 13:54:28 GMT

I have a feature layer that i am trying to populate a field from another layer using a Dictionary/spatial join. I am getting "The field is not nullable" on line 38. Some feature are not within the polygonLayer = 'Soils", so i can see why it's not populating the Class_ID field but how do I by pass this "The field is not nullable" and keep going regardless if the point layer field is blank?

# Import arcpy module  
import arcpy
import sys
import traceback
  
# scratch spatial join feature  
sjpoints = "In_memory\SpJoin"  
  
# define the field list from the spatial join to transfer  
# you can add more field names such as: ["TARGET_FID", "Field", "AnoterField"]  
sourceFieldsList = ["TARGET_FID", "Class_ID"]  
  
# point feature that will be updated  
pointLayer = "Bld_Class"  
  
# polygon feature that will be used  
polygonLayer = "Soils"  
  
# define the field list to the original points  
updateFieldsList = ["OID@", "Class"]  
  
# Allow overwrite of join results  
arcpy.env.overwriteOutput = True  

#Run the Spatial Join tool, using the defaults for the join operation and join type  
arcpy.SpatialJoin_analysis(pointLayer, polygonLayer, sjpoints)  
  
# populate the dictionary from the polygon  
valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sjpoints, sourceFieldsList)}  

with arcpy.da.UpdateCursor(pointLayer, updateFieldsList) as cursor:      
    for Row in cursor:      
        keyValue = Row[0]
        if keyValue in valueDict:
            print keyValue
            for n in range (1,len(sourceFieldsList)):        
                Row = valueDict[keyValue][n-1]      
            cursor.updateRow(Row)

del valueDict 


arcpy.Delete_management("In_memory\SpJoin")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: The field is not nullable

JoeBorgione — Sat, 11 Dec 2021 13:54:31 GMT

Take a look at the python try/except block. It will allow you to 'try' and do something, but in the event of an exception, let's you keep going. Here are few resources to check out:

Try and Except in Python

Python Try Except

8. Errors and Exceptions — Python 3.7.1 documentation

8. Errors and Exceptions — Python 2.7.15 documentation

Here is a simple example from a script I've run:

try:
    arcpy.MetadataImporter_conversion(SourceFeatureClass,TargetFeatureClass)
    print 'Successfully Imported Metadata From {} To {}'.format(SourceFeatureClass,TargetFeatureClass)
except Exception as err:
    print err‍‍‍‍‍

Re: The field is not nullable

DanPatterson_Retired — Thu, 15 Nov 2018 19:33:33 GMT

updateRows.updateRow(updateRow)‍

careful... you have a method and an object with the same name, may not be the issue now

Re: The field is not nullable

TonyAlmeida — Sat, 11 Dec 2021 13:54:34 GMT

I did try, try and except.

IT looks like it once it gets to a records that is blank it doesn't continue. There is not error.

# Import arcpy module  
import arcpy
import sys
import traceback
  
# scratch spatial join feature  
sjpoints = "In_memory\SpJoin"  
  
# define the field list from the spatial join to transfer  
# you can add more field names such as: ["TARGET_FID", "Field", "AnoterField"]  
sourceFieldsList = ["TARGET_FID", "Class_ID"]  
  
# point feature that will be updated  
pointLayer = "Bld_Class"  
  
# polygon feature that will be used  
polygonLayer = "Soils"  
  
# define the field list to the original points  
updateFieldsList = ["OID@", "Class"]  
  
# Allow overwrite of join results  
arcpy.env.overwriteOutput = True  
  
#Run the Spatial Join tool, using the defaults for the join operation and join type  
arcpy.SpatialJoin_analysis(pointLayer, polygonLayer, sjpoints)  
  
# populate the dictionary from the polygon  
valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sjpoints, sourceFieldsList)}  

try:
    with arcpy.da.UpdateCursor(pointLayer, updateFieldsList) as updateRows:      
        for updateRow in updateRows:      
            keyValue = updateRow[0]
            if keyValue in valueDict:      
                for n in range (1,len(sourceFieldsList)):        
                    updateRow = valueDict[keyValue][n-1]      
                updateRows.updateRow(updateRow)

    del valueDict 
except Exception:
    pass

arcpy.Delete_management("In_memory\SpJoin")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: The field is not nullable

TonyAlmeida — Sat, 11 Dec 2021 13:54:36 GMT

I made some changes. Thanks for pointing that out. I grab the code from

Richard Fairhurst Turbo Charging Data Manipulation with Python Cursors and Dictionaries

with arcpy.da.UpdateCursor(pointLayer, updateFieldsList) as cursor:      
        for Row in cursor:      
            keyValue = Row[0]
            if keyValue in valueDict:      
                for n in range (1,len(sourceFieldsList)):        
                    Row = valueDict[keyValue][n-1]      
                cursor.updateRow(Row)

    del valueDict ‍‍‍‍‍‍‍‍‍

Re: The field is not nullable

JoeBorgione — Sat, 11 Dec 2021 13:54:39 GMT

You may want to put some print statements in your code and see where it's failing. You mention 'blank': blank is not null. Blank is blank. I also notice the lack of setting a workspace although you set feature class variables as if you have; perhaps you have set the workspace in your working script? Line 7 has a little problem too:you should either escape that slash or use 'r' to do it for you:

# yours:

sjpoints = "In_memory\SpJoin" 

sjpoints = "In_memory\\SpJoin" 
#or
sjpoints = r"In_memory\SpJoin" ‍‍‍‍‍‍‍

Re: The field is not nullable

RandyBurton — Sat, 11 Dec 2021 13:54:42 GMT

You reference Richard Fairhurst‌'s article which describes a process using a dictionary to avoid using a join. I notice in your code you are using a join before creating a dictionary:

#Run the Spatial Join tool, using the defaults for the join operation and join type  
arcpy.SpatialJoin_analysis(pointLayer, polygonLayer, sjpoints)  
  
# populate the dictionary from the polygon  
valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sjpoints, sourceFieldsList)}‍‍‍‍‍‍‍‍‍‍‍

First, can you verify the join works as expected. I'm not sure the TARGET_FID and OID@ are providing the link you need to make the dictionary technique work. I am making the assumption that you want to capture some attribute of the polygon feature and save it as an attribute with all the points that fall inside the polygon. Perhaps you can clarify if this is not the case.

As an additional test, you can print some or all of the valueDict to see if there are null/None values associated with the various dictionary keys. You could add a test in the update block, that if the dictionary key's value is None/null, then skip the update or substitute a default value. As written, the block is skipping any updates if the keyValue is not in the valueDict.

Hope this helps.

Re: The field is not nullable

TonyAlmeida — Sat, 11 Dec 2021 13:54:44 GMT

I have set my work space and changed sjpoints to

arcpy.Delete_management(sjpoints)‍

I have added a print statement.

{0: (u'C_ID3889701100',), 1: (u'C_ID3890500000',), 2: (u'C_ID3890200000',), 3: (u'C_ID3890001000',), 4: (u'C_ID3889900000',), 5: (u'C_ID3889301100',), 6: (u'C_ID3889400000',), 7: (u'C_ID3889600000',), 8: (u'C_ID3889500000',), 9: (u'C_ID3889800000',), 10: (u'C_ID3889801000',), 11: (u'C_ID3889701000',), 12: (u'C_ID3888500000',), 13: (u'C_ID3888601000',), 14: (u'C_ID3825200000',), 15: (u'C_ID3825100000',),  16: (None,), 17: (None,), 18: (None,)}

Runtime error 
Traceback (most recent call last):
  File "<string>", line 44, in <module>
RuntimeError: The field is not nullable. [Class_ID]‍‍‍‍‍‍

Re: The field is not nullable

TonyAlmeida — Mon, 19 Nov 2018 17:12:53 GMT

Yes i am doing a spatial join, yes the join is working I checked various times.

You are correct that i am trying to capture some attributes of the polygons features and save it as an attribute with all the points that fall inside the polygon. Not sure how to add a test in the update block that you mentioned.

Re: The field is not nullable

RandyBurton — Sat, 11 Dec 2021 13:54:47 GMT

In your dictionary, the last 3 dictionary keys contain null values:

{....  16: (None,), 17: (None,), 18: (None,)}‍

You can try something like this to test for null/None:

if valueDict[keyValue][n-1] is not None:
    Row = valueDict[keyValue][n-1]
else:
   # use substitute value or pass‍‍‍‍

Re: The field is not nullable

JoeBorgione — Mon, 19 Nov 2018 17:23:01 GMT

That's not telling us much, since we don't know what you result you are printing. The idea of using print() to trouble shoot is to plug them in along the way to see what has succeeded before the failure.

Re: The field is not nullable

TonyAlmeida — Mon, 19 Nov 2018 18:24:38 GMT

Where exactly would in try this at? Sorry python is not my expertise's

Re: The field is not nullable

RandyBurton — Sat, 11 Dec 2021 13:54:50 GMT

The lines would be inserted in this section:

with arcpy.da.UpdateCursor(pointLayer, updateFieldsList) as cursor:      
    for Row in cursor:      
        keyValue = Row[0]
        if keyValue in valueDict:      
            for n in range (1,len(sourceFieldsList)):        
                if valueDict[keyValue][n-1] is not None:
                    Row[n] = valueDict[keyValue][n-1]
                else:
                    Row[n] = 'n/a'
            cursor.updateRow(Row)‍‍‍‍‍‍‍‍‍‍‍‍‍‍

But I'm not sure this will give you the results you want. Normally, the dictionary technique that Richard describes in his blog is used to avoid creating joins. And since you are doing a spatial join anyway, I would probably use the field calculator to copy the contents of the polygon's field that you want into the point's field. The join should only join points to polygons if the point is inside the polygon, so the calculation will only make the update where there is a match.

If you do want to use the dictionary, I suspect that your dictionary key needs to be the OID from the point feature and the value needs to be the Class_ID from the polygon feature. You may want to look at the field names that the join uses as it might be something like pointLayer.OID and polyLayer.Class_ID.

Hope this helps.