topic Re: Python won't recognize NULL (or None) in Python Questions

Python won't recognize NULL (or None)

RichardThurau — Mon, 26 Sep 2011 20:11:43 GMT

Hi,
Trying to calculate a field based on another field. Want to give Contiguous a "Y" where Contig_ID is not NULL.

"Calculate Field for Contiguous"
Field Name: Contiguous

Expression: Answer(!Contig_ID!)

Code Block: 
def Answer(CI):
  if CI ==None:
    return "N"
  else:
    return "Y"

I've read plenty of posts explaining that NULL == None in python, but this code only returns Y for all numeric values in Contig_ID. Where Contig_ID == Null, no values are returned to Contiguous.

Any help would be grand!

Thanks

Rich

Re: Python won't recognize NULL (or None)

StacyRendall1 — Sat, 11 Dec 2021 21:56:39 GMT

After some extensive testing, I can confirm this behaviour. If any of the inputs are <NULL>, it won't calculate for that row... Even when you get the row to calculate on itself.

That is weird though, as I'm sure I have done this before in Arc!

An annoying workaround would be to make a selection by attributes where Contig_ID IS NULL, set that to a marker value (i.e. 99999) and use that in your second calculation:

Expression: Answer(!Contig_ID!)

Code Block: 
def Answer(CI):
  if CI == 99999: # or '99999' if string...
    return "N"
  else:
    return "Y"

Let's hope the ESRI guys have an answer!

Re: Python won't recognize NULL (or None)

RichardThurau — Tue, 27 Sep 2011 14:39:21 GMT

Thank you Stacy,
Indeed, I am hopeful as well. Unfortunately, the null values are an indicator within the dataset that the client desires to maintain. Assigning another numeric value (0 or 999999) could potentially create miscalculations down the road.
Not to mention the messiness of creating a layer, selecting, and calculating a field based on data that are already present.

We are going to wait on this to hear if there is a solution.

Thanks all.

Rich

Re: Python won't recognize NULL (or None)

RichardThurau — Thu, 29 Sep 2011 16:49:52 GMT

75 people have viewed this thread. Has anyone else had any success in ArcGIS10 calculating a field in python based on a null value??

At the least, maybe this fix will make into SP3 due out next month?!

Re: Python won't recognize NULL (or None)

MathewCoyle — Thu, 29 Sep 2011 17:57:44 GMT

Only successful test I found was using a searchcursor, not field calculator. Null values must be read as something else in field calculator, because they come back as value True.

Re: Python won't recognize NULL (or None)

MarcNakleh — Sat, 11 Dec 2021 21:56:42 GMT

Hello Richard,

I've definitely been able to reproduce your issue and don't know what's going on.
However, the task you have in mind could be done trivially using Select by Attribute, followed by a standard Field Calculator expression.

In a Python script or in the Python window, it would be as simple as:

expr = '"REFERENCE_FIELD" IS NULL'
arcpy.MakeFeatureLayer_management('FEATURE_CLASS_NAME', 'temp', expr)
arcpy.CalculateField_management('temp', 'CALCULATED_FIELD', VALUE)

I don't have the exact figures, but this approach was significantly faster than comparing values one at a time through an unfiltered Update Cursor (and, by extension the way I understand it, any call to an CalculateField Python script.)

Doesn't change the fact that you're definitely pointing out some very weird behaviour that should be fixed!
Problem NIM070782 listed for Service Pack 3 might be this, but I'm not quite sure.
If you don't get a response from ESRI or someone knowledgeable in the next while, I'd recommend posting it on ArcGIS Ideas.

As an aside, I believe that the proper format should always be:
foo is None
and not
foo == None
because of:

back-end optimization (which is also sadly why while 1 is preferred over while True); and

the ability to overload an operator in classes and change the default behaviour of an == comparison.

It's no biggie, but worth pointing out. 🙂

Cheers,
Marc

Re: Python won't recognize NULL (or None)

JasonMiller — Sat, 11 Dec 2021 21:56:45 GMT

Had this issue too... here's how got around it...

Expression: TestForNullOrEmpty(str(!FieldA!))

Code Block: 
def TestForNullOrEmpty(fieldVal):
  if len(fieldVal.strip()) == 0:
    return "FieldValueWasNullOrEmpty"
  else:
    return "FieldHadSomethingInItAlready"

The trick is to put str(!FieldA!) in as your parameter in the Expression, this will convert a Null or None value to an empty string. Then by testing the length of the fieldVal that you passed and adding on the .strip() you can then filter out all 3 of this scenarios with this code...

1. Field Value is Null (or None)
2. Field Value is empty string
3. Field Value is empty string but has some spaces in it (shouldn't happen but it does...)

Note however, that this code will not be capable of picking out "Only Null Values", because this code will treat Null's and Empty String as the same thing...

JMiller

Re: Python won't recognize NULL (or None)

LuisFragoso — Fri, 16 Mar 2018 14:36:10 GMT

All this solutions will work if you are on a edit session. You must Start editing the feature for the code to replace the value.

Re: Python won't recognize NULL (or None)

BarryHall — Tue, 23 Jul 2019 15:40:08 GMT

try

if not CI:

I was testing for None for Null field -- i.e. if CI == 'None' for field returned with UpdateCursor

Even though a print statement returned 'None', couldn't be tested for as above.

if not ___

seems to work fine.

Cheers

Re: Python won't recognize NULL (or None)

JoshuaBixby — Tue, 23 Jul 2019 16:05:12 GMT

When cursors encounter NULL they return None and not 'None', which is a very important distinction. The former is the Python None object while the latter is simply a string with the word None. The expression CI == 'None' will never evaluate true for cursors returning fields with NULL. The proper way to check for a NULL field with cursors is CI is None

Re: Python won't recognize NULL (or None)

BarryHall — Tue, 23 Jul 2019 16:21:35 GMT

Joshua -- many thanks -- I am a Python newbie -- does the Python None object equate to false in logical tests?

Re: Python won't recognize NULL (or None)

JoshuaBixby — Tue, 23 Jul 2019 16:30:51 GMT

Yes, see 2.3.1 Truth Value Testing . Since there are several other data structures that exhibit falsy behavior, e.g., an empty string, testing for false instead of for None might result in some non-NULL fields getting treated as NULL.

Re: Python won't recognize NULL (or None)

BarryHall — Thu, 25 Jul 2019 14:36:33 GMT

Why is an individual field attribute an object (re:Null is returning as a Python None object)?

Re: Python won't recognize NULL (or None)

JoshuaBixby — Thu, 25 Jul 2019 15:47:28 GMT

Anything returned from any field in a cursor is a Python object because all data types are fundamentally objects:

>>> lst = [int(), float(), str(), bool(), None]
>>> for obj in lst:
... print(isinstance(obj, object))
...
True
True
True
True
True
>>>

In terms of why NULL gets returned as None; well, None is the closest data type conceptually to SQL NULL: Built-in Constants — Python 3.7.4 documentation

None
The sole value of the type NoneType. None is frequently used to represent the absence of a value, as when default arguments are not passed to a function.