Dan_Patterson

NumPy Snippets # 6 .... much ado about nothing ... NaN stuff

Blog Post created by Dan_Patterson Champion on Sep 9, 2016

NumPy Snippets

 

Updated: 2016-09-09

 

Recently I posted about 'nothing' in None isn't...nor is 0 or 1 ... more explorations into geometry  . 

This snippet shows how to deal with nothing... errrr ... nulls.  Simply put, for most numpy functions, there is an option to account for numeric null values... NaN ... in python parlance.  Now remember, ArcMap often has to deal with null values in fields.  This is often a stumbling block for people trying to summarize their data.  Here is the snippet for you to think about then to explore. 

"""
numpy_NaN

Author:  Dan.Patterson@carleton.ca

Purpose:
Create an array using a 'seed' list, caste it as a float and then
do some sums with sums with and without a mask

"""


import numpy as np

fields = ['a','b','c','d','e']        # field names used to define columns
seed = [['1','2','3','4','5'],
        ['2','3','4','5','1'],
        ['2','3','4','5','2']]

a = np.asarray(seed,dtype='float64')  # produce the array

b = np.sum(a,axis=0)                  # sum by the columns

print("\nSum Demo... \nUsing np.sum(array,axis=0)\nUsing np.nansum(array,axis=0)")
print('\nData:\n{}\n\nColumn sum no nulls:\n{}'.format(a,b))
#
# now with nulls

null = np.NaN                         # NaN... not a number ... or is it?
seed2 = [['1',null,'3','4','5'],
         [null,'3','4','5','1'],
         [null,'3',null,'5','2']]

a2 = np.asarray(seed2,dtype='float64')

b2 = np.sum(a2,axis=0)

c2 = np.nansum(a2,axis=0)

print('\nData with nulls... :\n{}\n\nColumn sum with nulls:\n{}'.format(a2,b2))
print('\nColumn sum omitting nulls:\n{}'.format(c2))

 

Now...the reveal...

 

Sum Demo...

Using np.sum(array,axis=0)

Using np.nansum(array,axis=0)

 

Data:

[[ 1.  2.  3.  4.  5.]

[ 2.  3.  4.  5.  1.]

[ 2.  3.  4.  5.  2.]]

 

Data with nulls... :

[[  1.  nan   3.   4.   5.]

[ nan   3.   4.   5.   1.]

[ nan   3.  nan   5.   2.]]

 

Column sum no nulls:            [  5.   8.  11.  14.   8.]

Column sum with nulls:         [ nan  nan  nan  14.   8.]

Column sum omitting nulls:   [  1.   6.   7.  14.   8.]

 

So clever isn't it.. now there are other np.nan... functions to explore.

Outcomes