NumPy Snippets
Updated: 2016-09-09
Recently I posted about 'nothing' in None isn't...nor is 0 or 1 ... more explorations into geometry .
This snippet shows how to deal with nothing... errrr ... nulls. Simply put, for most numpy functions, there is an option to account for numeric null values... NaN ... in python parlance. Now remember, ArcMap often has to deal with null values in fields. This is often a stumbling block for people trying to summarize their data. Here is the snippet for you to think about then to explore.
<SPAN class="string token">"""
numpy_NaN
<SPAN>Author: </SPAN><A class="jive-link-email-small" href="mailto:Dan.Patterson@carleton.ca" rel="nofollow noopener noreferrer" target="_blank">Dan.Patterson@carleton.ca</A>
Purpose:
Create an array using a 'seed' list, caste it as a float and then
do some sums with sums with and without a mask
"""</SPAN>
<SPAN class="keyword token">import</SPAN> numpy <SPAN class="keyword token">as</SPAN> np
fields <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="string token">'a'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'b'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'c'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'d'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'e'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="comment token"># field names used to define columns</SPAN>
seed <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'1'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'2'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'3'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'4'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'5'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN>
<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'2'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'3'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'4'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'5'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'1'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN>
<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'2'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'3'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'4'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'5'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'2'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN>
a <SPAN class="operator token">=</SPAN> np<SPAN class="punctuation token">.</SPAN>asarray<SPAN class="punctuation token">(</SPAN>seed<SPAN class="punctuation token">,</SPAN>dtype<SPAN class="operator token">=</SPAN><SPAN class="string token">'float64'</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="comment token"># produce the array</SPAN>
b <SPAN class="operator token">=</SPAN> np<SPAN class="punctuation token">.</SPAN>sum<SPAN class="punctuation token">(</SPAN>a<SPAN class="punctuation token">,</SPAN>axis<SPAN class="operator token">=</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="comment token"># sum by the columns</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">"\nSum Demo... \nUsing np.sum(array,axis=0)\nUsing np.nansum(array,axis=0)"</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'\nData:\n{}\n\nColumn sum no nulls:\n{}'</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>a<SPAN class="punctuation token">,</SPAN>b<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token">#</SPAN>
<SPAN class="comment token"># now with nulls</SPAN>
null <SPAN class="operator token">=</SPAN> np<SPAN class="punctuation token">.</SPAN>NaN <SPAN class="comment token"># NaN... not a number ... or is it?</SPAN>
seed2 <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'1'</SPAN><SPAN class="punctuation token">,</SPAN>null<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'3'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'4'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'5'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN>
<SPAN class="punctuation token">[</SPAN>null<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'3'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'4'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'5'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'1'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN>
<SPAN class="punctuation token">[</SPAN>null<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'3'</SPAN><SPAN class="punctuation token">,</SPAN>null<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'5'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'2'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN>
a2 <SPAN class="operator token">=</SPAN> np<SPAN class="punctuation token">.</SPAN>asarray<SPAN class="punctuation token">(</SPAN>seed2<SPAN class="punctuation token">,</SPAN>dtype<SPAN class="operator token">=</SPAN><SPAN class="string token">'float64'</SPAN><SPAN class="punctuation token">)</SPAN>
b2 <SPAN class="operator token">=</SPAN> np<SPAN class="punctuation token">.</SPAN>sum<SPAN class="punctuation token">(</SPAN>a2<SPAN class="punctuation token">,</SPAN>axis<SPAN class="operator token">=</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN>
c2 <SPAN class="operator token">=</SPAN> np<SPAN class="punctuation token">.</SPAN>nansum<SPAN class="punctuation token">(</SPAN>a2<SPAN class="punctuation token">,</SPAN>axis<SPAN class="operator token">=</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'\nData with nulls... :\n{}\n\nColumn sum with nulls:\n{}'</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>a2<SPAN class="punctuation token">,</SPAN>b2<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'\nColumn sum omitting nulls:\n{}'</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>c2<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>Now...the reveal...
Sum Demo...
Using np.sum(array,axis=0)
Using np.nansum(array,axis=0)
Data:
[[ 1. 2. 3. 4. 5.]
[ 2. 3. 4. 5. 1.]
[ 2. 3. 4. 5. 2.]]
Data with nulls... :
[[ 1. nan 3. 4. 5.]
[ nan 3. 4. 5. 1.]
[ nan 3. nan 5. 2.]]
Column sum no nulls: [ 5. 8. 11. 14. 8.]
Column sum with nulls: [ nan nan nan 14. 8.]
Column sum omitting nulls: [ 1. 6. 7. 14. 8.]
So clever isn't it.. now there are other np.nan... functions to explore.
