Skip navigation
All People > Dan_Patterson > Py... blog > 2016 > November
2016

A stats ditty... so I don't forget and some of you may be interested in graphics using MatPlotLib

bivariate_normal.png

Code for the above...written verbosely so you get the drift.

"""
Script:    bivariate_normal.py
Path:      F:\A0_Main\
Author:    Dan.Patterson@carleton.ca

Created:   2015-06-06
Modified:  2015-06-06  (last change date)
Purpose:   To examine the affect of parameters on the bivariate distribution
Requires:  numpy and matplot lib
Notes:
  see help on (np.random.normal)
  x = numpy.array([1, 2, 3])         # put in the values for the x values
  y = numpy.array([10, 20, 30])   # ditto for y
  XX, YY = numpy.meshgrid(x, y)   # make your 'mesh' which is the result
  ZZ = XX + YY                    # in this case the sum of X and Y
  ZZ => array([[11, 12, 13],
               [21, 22, 23],
               [31, 32, 33]])
  >>> (X,Y) = meshgrid(x,y)     # actually Y
  YY, XX = numpy.mgrid[10:40:10, 1:4]  # Y,X
  ZZ = XX + YY # These are equivalent to the output of meshgrid
  YY, XX = numpy.ogrid[10:40:10, 1:4] #
  ZZ = XX + YY # These are equivalent to the atleast_2d example
  which
  XX, YY = numpy.atleast_2d(x, y)
  YY = YY.T # transpose to allow broadcasting
  ZZ = XX + YY
References: many
"""
import numpy as np
import matplotlib.pyplot as plt
np.set_printoptions(edgeitems=3,linewidth=75,precision=2,
                    suppress=True,threshold=10)
# (1) make a floating point grid and get some stats
#     100x100 grid cells numbered from top left
Xs = np.arange(0., 100.1, 1) # as floats
Ys = np.arange(0., 100.1, 1)
Xc = Xs.mean();   Yc = Ys.mean()
Xmin = Xs.min();  Xmax = Xs.max()
Ymin = Ys.min();  Ymax = Ys.max()
# (2) ....now do some work.... -----------------------------------------|
X,Y = np.meshgrid(Xs,Ys)   # X, Y as a meshgrid
XX = X**2                  # square the X
out = 2*X + Y - 3.0        # do the spatial math
Z = X**2 + Y**2            # getting it now?       
# (3) .... calculate the pdf -------------------------------------------|
#   normal pdf/ellipse
rho = -0.5
s_x = 60.0
s_y = 45.0   # 4*3 ratio Xc,Yc = (50,50) stds at 2Std level
d_x = ((X-Xc)/s_x)
d_y = ((Y-Yc)/s_y)
rho2 = (1.0-rho**2)
m_rsq = np.sqrt(rho2)
lower = 2.0*(1-rho**2)                    # rho = 0 lower = 2
upper = d_x**2 + d_y**2 - 2.0*rho*d_x*d_y # if rho= 0 drop last term
left = (1.0/(2.0*np.pi*s_x*s_y*m_rsq))
f_xy = left**(upper/lower)
# (4) .... make a figure -----------------------------------------------|
fig = plt.figure(facecolor='white')
ax = fig.add_subplot(1, 1, 1)
plt.axis('equal')
plt.axis([Xmin, Xmax, Ymax, Ymin]) # decreasing Y
plt.set_cmap('Blues')
cont = plt.contourf(Xs, Ys, f_xy, origin='upper')
plt.title("Bivariate normal distribution")
plt.xlabel("X ==>")
plt.ylabel("<== Y")
cbar = plt.colorbar(cont)
cbar.ax.set_ylabel('values')
lbl_r = "rho = {}\n2 std x\n1 std y".format(abs(rho))  # reverse rho since plotting -ve Y
plt.text(0,20,lbl_r)
#
plt.show()
plt.close()

Check out their code gallery for a multitude of options on the MatPlotLib Home Page.

So ... new interface, time to try out some formatting and stuff.  What a better topic than how to order, structure and view 3D data like images or raster data of mixed data types for the same location or uniform data type where the 3rd dimension represents time.

 

I will make it simple.  Begin with 24 integer numbers and arange them into all the possible configurations in 3D.  Then it is time to mess with your mind and show you how to convert from one arrangement to another.  Sort of like Rubic's Cube, but simpler.

 

So here is the generating script (note the new cool python syntax highlighting... nice! ... but you still can't change the brownish background color, stifling any personal code preferences).  The def just happens to be number 37... it has no meaning, just 37 in a collection of functions

def num_37():
    """(num_37) playing with 3D arrangements...
    :Requires:
    :--------
    :  Arrays are generated within... nothing required
    :Returns:
    :-------
    :  An array of 24 sequential integers with shape = (2, 3, 4)
    :Notes:
    :-----
    :  References to numpy, transpose, rollaxis, swapaxes and einsum.
    :  The arrays below are all the possible combinations of axes that can be
    :  constructed from a sequence size of 24 values to form 3D arrays.
    :  Higher dimensionalities will be considered at a later time.
    :
    :  After this, there is some fancy formatting as covered in my previous blogs.
    """

    nums = np.arange(24)      #  whatever, just shape appropriately
    a = nums.reshape(2,3,4)   #  the base 3D array shaped as (z, y, x)
    a0 = nums.reshape(2,4,3)  #  y, x axes, swapped
    a1 = nums.reshape(3,2,4)  #  add to z, reshape y, x accordingly to main size
    a2 = nums.reshape(3,4,2)  #  swap y, x
    a3 = nums.reshape(4,2,3)  #  add to z again, resize as befor
    a4 = nums.reshape(4,3,2)  #  swap y, x
    frmt = """
    Array ... {} :..shape  {}
    {}
    """

    args = [['nums', nums.shape, nums],
            ['a', a.shape, a], ['a0', a0.shape, a0],
            ['a1', a1.shape, a1], ['a2', a2.shape, a2],
            ['a3', a3.shape, a3], ['a4', a4.shape, a4],
            ]
    for i in args:
        print(dedent(frmt).format(*i))
    return a

 

And here are the results

|-----------------------------------------------------  

|

3D Array .... a 3D array .... a0
Array ... a :..shape  (2, 3, 4)
[[[ 0  1  2  3]
  [ 4  5  6  7]
  [ 8  9 10 11]]

[[12 13 14 15]
  [16 17 18 19]
  [20 21 22 23]]]

# This is the base array...
Array ... a0 :..shape  (2, 4, 3)
[[[ 0  1  2]
  [ 3  4  5]
  [ 6  7  8]
  [ 9 10 11]]

[[12 13 14]
  [15 16 17]
  [18 19 20]
  [21 22 23]]]

 

|-----------------------------------------------------
|
In any event, I prefer to think of a 3D array as consisting of ( Z, Y, X ) if they do indeed represent the spatial component.  In this context, however, Z is not simply taken as elevation as might be the case for a 2D raster.  The mere fact that the first axis is denoted with a 2 or above, indicates to me that it is a change array.  Do note that the arrays need not represent anything spatial at all, but this being a place for GIS commentary, there is often an implicit assumption that at least two of the dimensions will be spatial.

 

To go from array a to a0, and conversely, we need to reshape the array.  Array shaping can be accomplished using a variety of numpy methods, including rollaxes, swapaxes, transpose and einsum to name a few.

 

The following can be summarized:

R   rollaxis       - roll the chosen axis back by the specified positions

E   einsum       - for now, just see the swapping of letters in the ijk sequence

S   swapaxes   - change the position of specified axes

T   transpose   - similar to swapaxes, but with multiple changes

 

 

a0 = np.rollaxis(a, 2, 1)           #  a = np.rollaxis(a0, 2, 1)
a0 = np.swapaxes(a, 2, 1)           #  a = np.swapaxes(a0, 1, 2)
a0 = a.swapaxes(2, 1)               #  a = a0.swapaxes(1, 2)
a0 = np. transpose(a, (0, 2, 1))    #  a = np.transpose(a0, (0, 2, 1))
a0 = a.transpose(0, 2, 1)           #  a = np.transpose(a0, 2, 1)
a0 = np.einsum('ijk -> ikj', a)     #  a = np.einsum('ijk -> ikj', a0)

 

When you move on to higher values for the first dimension you have to be careful about which of these you can use, and it is generally just better to use reshape or stride tricks to perform the reshaping

|-----------------------------------------------------
|

3D array .... a13D array .... a2

Array ... a1 :..shape  (3, 2, 4)
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7]],

       [[ 8,  9, 10, 11],
        [12, 13, 14, 15]],

       [[16, 17, 18, 19],
        [20, 21, 22, 23]]])
Array ... a2 :..shape  (3, 4, 2)
array([[[ 0,  1],
        [ 2,  3],
        [ 4,  5],
        [ 6,  7]],

       [[ 8,  9],
        [10, 11],
        [12, 13],
        [14, 15]],

       [[16, 17],
        [18, 19],
        [20, 21],
        [22, 23]]])

|-----------------------------------------------------

3D array .... a2 to a conversion
>>> from numpy.lib import stride_tricks as ast
>>> back_to_a = a2.reshape(2, 3, 4)
>>> again_to_a = ast.as_strided(a2, a.shape, a.strides)
>>> back_to_a
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]],

       [[12, 13, 14, 15],
        [16, 17, 18, 19],
        [20, 21, 22, 23]]])
>>> again_to_a
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]],

       [[12, 13, 14, 15],
        [16, 17, 18, 19],
        [20, 21, 22, 23]]])

 

|-----------------------------------------------------

Now for something a little bit different

 

Array 'a' which has been used before.  It has a shape of (2, 3, 4).  Consider it as 2 layers or bands occupying the same space.

array([[[ 0, 1, 2, 3],
        [ 4, 5, 6, 7],
        [ 8, 9, 10, 11]],

       [[12, 13, 14, 15],
        [16, 17, 18, 19],
        [20, 21, 22, 23]]])

 

A second array, 'b', can be constructed using the same data, but shaped differently, (3, 4, 2).  The dimension consisting of two parts is effectively swapped between the two arrays.  It can be constructed from:

 

>>> x = np.arange(12)
>>> y = np.arange(12, 24)
>>>
>>> b = np.array(list(zip(x,y))).reshape(3,4,2)
>>> b
array([[[ 0, 12],
        [ 1, 13],
        [ 2, 14],
        [ 3, 15]],

       [[ 4, 16],
        [ 5, 17],
        [ 6, 18],
        [ 7, 19]],

       [[ 8, 20],
        [ 9, 21],
        [10, 22],
        [11, 23]]])

 

If you look closely, you can see that the numeric values from 0 to 11 are order in a 4x3 block in array 'a', but appear as 12 entries in a column, split between 3 subarrays.  The same data can be sliced from their respetive array dimensions to yield

 

... sub-array 'a[0]' or ... sub-array 'b[...,0]'

yields

[[ 0  1  2  3]
[ 4  5  6  7]
[ 8  9 10 11]]

 

The arrays can be raveled to reveal their internal structure.

>>> b.strides # (64, 16, 8)
>>> a.strides # (96, 32, 8)
a.ravel()...[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
b.ravel()...[ 0 12 1 13 2 14 3 15 4 16 5 17 6 18 7 19 8 20 9 21 10 22 11 23]
a0_r = a[0].reshape(3,4,-1) # a0_r.shape = (3, 4, 1)
array([[[ 0],
[ 1],
[ 2],
[ 3]],
[[ 4],
[ 5],
[ 6],
[ 7]],
[[ 8],
[ 9],
[10],
[11]]])

Enough for now.  Learning how to reshape and work with array structures can certainly make dealing with raster data much easier.

This is a sample output for a function (def) in python.  You will notice most of the code is format 'fluff'.  Why do it?  Because if you need to document something for posterity or a contract or a course, then you had better have as much information as possible..

 

So... the following is for those that have a need for documentation.  There is numerous examples of format tips therein as well.  I have also documented the function that does the documentation .... get_func ... .  I have another one called ... get_modu ... that documents whole modules.

 

Enjoy

:num_54() Producing uniformly distributed data
    :Requires:
    :--------
    :  The class numbers have to be specified and the number of repeats
    :  to give you a total population size.
    :Reference:
    :---------
    :  https://geonet.esri.com/thread/185566-creating-defined-lists
   
:Generate Data that conform to a uniform distribution.
:
:Class values: [1 2 3 4 5 6]
:Population size: 60
:Results:
:  values:
    [[3 5 2 3 4 2 4 4 1 2 5 6 3 4 1 5 1 5 2 3]
     [5 2 6 6 6 2 4 4 6 4 3 2 3 4 1 6 6 5 2 1]
     [3 6 1 3 1 6 4 2 4 1 1 6 5 5 5 2 3 3 1 5]]
:  table:
    [(0, 3) (1, 5) (2, 2) (3, 3) (4, 4) (5, 2) (6, 4) (7, 4) (8, 1) (9, 2) (10, 5)
     (11, 6) (12, 3) (13, 4) (14, 1) (15, 5) (16, 1) (17, 5) (18, 2) (19, 3) (20, 5)
     (21, 2) (22, 6) (23, 6) (24, 6) (25, 2) (26, 4) (27, 4) (28, 6) (29, 4) (30, 3)
     (31, 2) (32, 3) (33, 4) (34, 1) (35, 6) (36, 6) (37, 5) (38, 2) (39, 1) (40, 3)
     (41, 6) (42, 1) (43, 3) (44, 1) (45, 6) (46, 4) (47, 2) (48, 4) (49, 1) (50, 1)
     (51, 6) (52, 5) (53, 5) (54, 5) (55, 2) (56, 3) (57, 3) (58, 1) (59, 5)]
:  histogram: (class, frequency)
    [[ 1 10]
     [ 2 10]
     [ 3 10]
     [ 4 10]
     [ 5 10]
     [ 6 10]]
:Then use NumPyArrayToTable to get your table.

>>> print(art.get_func(num_54))

:-----------------------------------------------------------------
:Function: .... num_54 ....
:Line number... 664
:Docs:
num_54() Producing uniformly distributed data
    :Requires:
    :--------
    :  The class numbers have to be specified and the number of repeats
    :  to give you a total population size.
    :Reference:
    :---------
    :  https://geonet.esri.com/thread/185566-creating-defined-lists
   
:Defaults: None
:Keyword Defaults: None
:Variable names: ('frmt', 'st', 'end', 'vals', 'reps', 'z', 'ID', 'tbl', 'h', 'pad', 'args')
:Source code:
   0  def num_54():
   1      """num_54() Producing uniformly distributed data
   2      :Requires:
   3      :--------
   4      :  The class numbers have to be specified and the number of repeats
   5      :  to give you a total population size.
   6      :Reference:
   7      :---------
   8      :  https://community.esri.com/thread/185566-creating-defined-lists
   9      """

  10      frmt = """
  11      :{}
  12      :Generate Data that conform to a uniform distribution.
  13      :
  14      :Class values: {}
  15      :Population size: {}
  16      :Results:
  17      :  values:
  18      {}
  19      :  table:
  20      {}
  21      :  histogram: (class, frequency)
  22      {}
  23      :Then use NumPyArrayToTable to get your table.
  24      """

  25      # import numpy as np
  26      st = 1
  27      end = 7
  28      vals = np.arange(st,end)
  29      reps = 10
  30      z = np.repeat(vals,reps)
  31      np.random.shuffle(z)
  32      ID = np.arange(len(z))
  33      tbl = np.array(list(zip(ID, z)),
  34                     dtype = [('ID', 'int'), ('Class', 'int')])
  35      h = np.histogram(z, np.arange(st, end+1))
  36      h = np.array(list(zip(h[1], h[0])))
  37      pad = "    "
  38      args =[num_54.__doc__, vals, reps*len(vals),
  39             indent(str(z.reshape(3,20)), pad),
  40             indent(str(tbl), pad), indent(str(h), pad)]
  41      print(dedent(frmt).format(*args))

:
:-----------------------------------------------------------------

 

Now the function that documents the function documenting itself.

 

>>> print(art.get_func(art.get_func))

:-----------------------------------------------------------------
:Function: .... get_func ....
:Line number... 485
:Docs:
Get function (def) information.
    :Requires: 
    :--------
    :  from textwrap import dedent, indent, wrap
    :  import inspect
    :Returns:
    :-------
    :  The function information includes arguments and source code.
    :  A string is returned for printing.

:Defaults: (True,)
:Keyword Defaults: None
:Variable names:
    obj, verbose, frmt, inspect, dedent, indent, wrap,
    lines, ln_num, code, vars, args, code_mem
:Source code:
   0  def get_func(obj, verbose=True):
   1      """Get function (def) information.
   2      :Requires: 
   3      :--------
   4      :  from textwrap import dedent, indent, wrap
   5      :  import inspect
   6      :Returns:
   7      :-------
   8      :  The function information includes arguments and source code.
   9      :  A string is returned for printing.
  10      """

  11      frmt = """
  12      :-----------------------------------------------------------------
  13      :Function: .... {} ....
  14      :Line number... {}
  15      :Docs:
  16      {}
  17      :Defaults: {}
  18      :Keyword Defaults: {}
  19      :Variable names:
  20      {}
  21      :Source code:
  22      {}
  23      :
  24      :-----------------------------------------------------------------
  25      """

  26      import inspect
  27      from textwrap import dedent, indent, wrap
  28      lines, ln_num = inspect.getsourcelines(obj)
  29      code = "".join(["{:4d}  {}".format(idx, line)
  30                      for idx, line in enumerate(lines)])
  31      vars  = ", ".join([i for i in obj.__code__.co_varnames])
  32      vars = wrap(vars, 50)
  33      vars = "\n".join([i for i in vars])
  34      args = [obj.__name__, ln_num, dedent(obj.__doc__), obj.__defaults__,
  35               obj.__kwdefaults__,indent(vars, "    "), code]       
  36      code_mem = dedent(frmt).format(*args)
  37      return code_mem

:
:-----------------------------------------------------------------

 

The two modules that do the heavy lifting are the inspect and textwrap modules.  You can obtain more information on these by simply using the 'dir' and 'help' functions (ie. dir(textwrap) ) to get details of the methods used.  Textwrap allows for indentation, dendentation and wrapping of text blocks.  Pretty well anything can be passed to the builtins if they can be converted to string format.  You will notice that I make extensive use of 'dedent' because def headers are indented by 4 spaces, which I would like to remove prior to printing.

 

Learn something about formatting and worry less about reducing code length... it is not about the bytes saved when coding ... it is about code coming back to bite you when you can't remember how it works.

 

As a parting easy one, this is a decorator which you can use in your own scripts.  Edit to suit your return needs

 

def func_run(func):
    """Prints basic function information and the results of a run.
    :Required:  from functools import wraps
    :  Uncomment the import or move it to within the script.
    :Useage:   @func_run  on the line above the function
    """

    from functools import wraps
    @wraps(func)
    def wrapper(*args,**kwargs):
        frmt = "\n".join(["Function... {}", "  args.... {}",
                          "  kwargs.. {}", "  docs.... {}"])
        ar = [func.__name__, args, kwargs, func.__doc__]
        print(dedent(frmt).format(*ar))
        result = func(*args, **kwargs)
        print("{!r:}\n".format(result))  # comment out if results not needed
        return result                    # for optional use outside.
    return wrapper

 

Enjoy

Key concepts: nulls, booleans, list comprehensions, ternary operators, condition checking, mini-format language

 

Null values are permissable when creating tables in certain data structures.  I have never had occasion to use them since I personally feel that all entries should be coded with some value which is either:

  • a real observation,
  • one that was missed or forgotten,
  • there is truly no value because it couldn't be obtained
  • other

 

Null, None etc don't fit into that scheme, but it is possible to produce them, particularly if people import data from spreadsheets and allow blank entries in cells within columns. Nulls cause no end of problems with people trying to query tabular data or contenate data or perform statistical or other numeric operations on fields that contain these pesky little things. I should note, that setting a record to some arbitrary value is just as problematic as the null.  For example, values of 0 or "" in a record for a shapefile should be treated as suspect if you didn't create the data yourself.

 

NOTE:  This post will focus on field calculations using python and not on SQL queries..

 

List Comprehensions to capture nulls

As an example, consider the task of concatenating data to a new field from several fields which may contain nulls (eg. see this thread... Re: Concatenating Strings with Field Calculator and Python - dealing with NULLS). There are numerous ways to accomplish this, several will be presented here.

List comprehensions, truth testing and string concatenation can be accomplished in one foul swoop...IF you are careful.

This table was created in a geodatabase which allows nulls to be created in a field.  Fortunately, <null> stands out from the other entries serving as an indicator that they need to be dealt with.  It is a fairly simple table, just containing a few numeric and text columns.

image.png

The concat field was created using the python parser and the following field calculator syntax.

 

Conventional list comprehension in the field calculator

# read very carefully ...convert these fields if the fields don't contain a <Null>
" ".join(  [  str(i) for i in [ !prefix_txt!, !number_int!, !main_txt!, !numb_dble!  ] if i ]  )
'12345 some text more'

 

and who said the expression has to be on one line?

" ".join(
[str(i) for i in
[ !prefix_txt!, !number_int!, !main_txt!, !numb_dble!]
if i ] )

 

table_nulls_03.png

I have whipped in a few extra unnecessary spaces in the first expression just to show the separation between the elements.  The second one was just for fun and to show that there is no need for one of those murderous one-liners that are difficult to edit.

 

So what does it consist of?

  • a join function is used to perform the final concatenation
  • a list comprehension, LC, is used to determine which fields contain appropriate values which are then converted to a string
    • each element in a list of field names is cycled through ( for i in [...] section )
    • each element is check to see if it meets the truth test (that is ... if i ... returns True if the field entry is not null, False otherwise])
    • if the above conditions are met, the value is converted to a string representation for subsequent joining.

You can create your appropriate string without the join but you need a code block.

 

Simplifying the example

Lets simplify the above field calculator expression to make it easier to read by using variables as substitutes for the text, number and null elements.

 

List comprehension

>>> a = 
12345;


b = None
;

c = "some text";

d = "" ;
e = "more"


>>> " ".join([str(i) for i in [a,b,c,d,e] if i])


 

One complaint that is often voiced is that list comprehensions can be hard to read if they contain conditional operations.  This issue can be circumvented by stacking the pieces during their construction.  Python allows for this syntactical construction in other objects such as lists, tuples, arrays and text  amongst many objects.  To demonstrate, the above expression can be written as:

 

Stacked list comprehension

>>> " ".join( [ str(i)               # do this
...           for i in [a,b,c,d,e]   # using these
...           if i ] )               # if this is True
'12345 some text more'
>>>

 

You may have noted that you can include comments on the same line as each constructor.  This is useful since you can in essence construct a sentence describing what you are doing.... do this, using these, if this is True...  A False condition can also be used but it is usually easier to rearrange you "sentence" to make it easier to say-do.

 

For those that prefer a more conventional approach you can make a function out of it.

 

Function: no_nulls_allowed

def no_nulls_allowed(fld_list):
    """provide a list of fields"""
    good_stuff = []
    for i in fld_list:
        if i:
            good_stuff.append(str(i))
        out_str = " ".join(good_stuff)
    return out_str
...
>>> no_nulls_allowed([a,b,c,d,e])
'12345 some text more'
>>>

 

Python's mini-formatting language...

Just for fun, let's assume that the values assigned to a-e in the example below, are field names.

Questions you could ask yourself:

  • What if you don't know which field or fields may contain a null value?
  • What if you want to flag the user that is something wrong instead?

 

You can generate the required number of curly bracket parameters, { }, needed in the mini-language formatting.  Let's have a gander using variables in place of the field names in the table example above.  I will just snug the variable definitions up to save space.

 

Function: no_nulls_mini

 

def no_nulls_mini(fld_list):
    ok_flds = [ str(i) for i in fld_list  if]
    return ("{} "*len(ok_flds)).format(*ok_flds)

>>> no_nulls_mini([a,b,c,d,e])
'12345 some text more '

 

Ok, now for the breakdown:

  • I am too lazy to check which fields may contain null values, so I don't know how many { } to make...
  • we have a mix of numbers and strings, but we cleverly know that the mini-formatting language makes string representations of inputs by defaults so you don't need to do the string-thing ( aka str( ) )
  • we want a space between the elements since we are concatenating values together and it is easier to read with spaces

Now for code-speak:

  • "{} "  - curly brackets followed by a space is the container to put out stuff plus the extra space
  • *len(ok_flds)  - this will multiply the "{} " entry by the number of fields that contained values that met the truth test (ie no nulls)
  • *ok_flds  - in the format section will dole out the required number of arguments from the ok_flds list (like *args, **kwargs use in def statements)

Strung together, it means "take all the good values from the different fields and concatenate them together with a space in between"

 

Head hurt???  Ok, to summarize, we can use simple list comprehensions, stacked list comprehensions and the mini-formatting options

 

Assume  a = 12345; b = None ; c = "some text"; d = "" ; e = "more"

# simple list comprehension, only check for True
" ".join( [ str(i) for i in [a, b, c, d, e]  if]  )
12345 some text more

# if-else with slicing, do something if False
z = " ".join([[str(i),"..."][i in ["",'',None,False]]
              for i in [a,b,c,d,e]])
12345 ... some text ... more

a-e represent fields, typical construction

 

advanced construction for an if-else statement, which uses a False,True option and slices on the condition

def no_nulls_mini(fld_list):
    ok_flds = [ str(i) for i in fld_list  if]
    return ("{} "*len(ok_flds)).format(*ok_flds)
provide a field list to a function, and construct the string from the values that meet the condition
def no_nulls_allowed(fld_list):
    good_stuff = []
    for i in fld_list:
    if i:
        good_stuff.append(str(i))
    out_str = " ".join(good_stuff)
    return out_str

a conventional function, requires the empty list construction first, then acceptable values are added to it...finally the values are concatenated together and returned.

And they all yield..    '12345 some text more'

 

Closing Tip

If you can say it, you can do it...

 

list comp = [ do this  if this  else this using these]

 

list comp = [ do this        # the Truth result

              if this        # the Truth condition

              else this      # the False condition

              for these      # using these

              ]

 

list comp = [ [do if False, do if True][condition slice]  # pick one

              for these                                   # using these

             ]

 

A parting example...

 

# A stacked list comprehension
outer = [1,2]
inner = [2,0,4]
c = [[a, b, a*b, a*b/1.0]  # multiply,avoid division by 0, for (outer/inner)
     if b                # if != 0 (0 is a boolean False)
     else [a,b,a*b,"N/A"]    # if equal to zero, do this
     for a in outer      # for each value in the outer list
     for b in inner      # for each value in the inner list
     ]
for val in c:
    print("a({}), b({}), a*b({}) a/b({})".format(*val )) # val[0],val[1],val[2]))

# Now ... a False-True list from which you slice the appropriate operation
d = [[[a,b,a*b,"N/A"],           # do if False
      [a,b,a*b,a*b/1.0]][b!=0]   # do if True ... then slice
     for a in outer
     for b in inner
     ]
for val in d:
    print("a({}), b({}), a*b({}) a/b({})".format(*val ))
"""
a(1), b(2), a*b(2) a/b(2.0)
a(1), b(0), a*b(0) a/b(N/A)
a(1), b(4), a*b(4) a/b(4.0)
a(2), b(2), a*b(4) a/b(4.0)
a(2), b(0), a*b(0) a/b(N/A)
a(2), b(4), a*b(8) a/b(8.0)
"""

 

Pick what works for you... learn something new... and write it down Before You Forget ...