Dan_Patterson

Geometry ... Attributes actually... the other bits  # 5

Blog Post created by Dan_Patterson Champion on Apr 17, 2019

Part 5 ... The attributes attached to the geometry

 

---- As before, the inputs ----

The polygons that I will be using are shown to the right.

  1. A square, 5 points, first and last duplicates
  2. A lake on an island in a lake...
  3. A multipart with a two different shaped donut holes
  4. The letter 'C'
  5. The letter 'V'

Each part is labelled at the labelpoint rather than the centroid, hence each part gets labelled.

 

 

 

 

 

---- An Alternate Geometry Reconstructor ----
---- Arrays to Poly* Features ----

 

Never use <null> in a table.  To many posts on the forum on how to trap them, find them, replace them.

Always put in a value to represent all conditions.  Too many people use None <null> as the catchall category.  In reality all observations need to be classified exactly, there really is no such thing as <null>.  You either made and observation or you didn't.  If you didn't, your classification scheme should provide a key indicating that.

 

If an observation was made but the phenomenon/parameter/whatever was actually not there, there should be a key for that.  Similar, for 'I forgot', 'Wasn't my job' or whatever other excuses exist.  

 

  • For observations recorded as floating point numbers, that truly yielded 'nothing/None/nadda/zilch', I use 'nan' (np.nan) since it is a recognized number.
  • For text/string observations, I use None since None the object translates to 'None' the text easily in most tables.
  • For time, there is now 'not a time' (NaT), but I don't work with time, preferring to use the string incarnations of those observations
  • For integers... sadly there is no 'Nint'.  You have to provide an actual integer value to represent no value observed, although you desperately tried.   You can use the old school -999, or even 2**8, 2**16 etcetera.
  • For all of the above, anything that is truly doesn't represent an observation where no value was observed, you will have to provide alternatives

 

Making none/null/real nothingness

 

You can add to, or remove from, the list below.  These are some that I use.  I will draw your attention to the NumPy incarnations for integers.  Equivalent values exist for floats.  Any value that ensures that you will take a second look if a calculation looks weird is good.  However if your table contains <nulls> even after my lecture above, this will help mitigate your... stupidity is such a harsh word... but you get my drift

def _make_nulls_(in_fc, int_null=-999):
    """Return null values for a list of fields objects, excluding objectid
    and geometry related fields.  Throw in whatever else you want.

    Parameters
    ----------
    in_flds : list of field objects
        arcpy field objects, use arcpy.ListFields to get a list of featureclass
        fields.
    int_null : integer
        A default to use for integer nulls since there is no ``nan`` equivalent
        Other options include

    >>> np.iinfo(np.int32).min # -2147483648
    >>> np.iinfo(np.int16).min # -32768
    >>> np.iinfo(np.int8).min  # -128

    [i for i in cur.__iter__()]
    [[j if j is not None else -999 for j in i ] for i in cur.__iter__() ]
    """
    nulls = {'Double': np.nan, 'Single': np.nan, 'Float': np.nan,
             'Short': int_null, 'SmallInteger': int_null, 'Long': int_null,
             'Integer': int_null, 'String':str(None), 'Text':str(None),
             'Date': np.datetime64('NaT')}
    #
    desc = arcpy.da.Describe(in_fc)
    if desc['dataType'] != 'FeatureClass':
        print("Only Featureclasses are supported")
        return None, None
    in_flds = desc['fields']
    shp = desc['shapeFieldName']
    good = [f for f in in_flds if f.editable and f.name != shp]
    fld_dict = {f.name: f.type for f in good}
    fld_names = list(fld_dict.keys())
    null_dict = {f: nulls[fld_dict[f]] for f in fld_names}
    # ---- insert the OBJECTID field
    return null_dict, fld_names

My favorite way of getting just the attributes

 

Such nice functions, FeatureClassToNumPyArray, TableToNumPyArray, and back the other way. 

I am sure many of you have explored where it all comes from only to find it all buried in a *.pyd file

import arcpy.da as apd
apd.__file__ # ---- 'C:\\...install path...\\Resources\\ArcPy\\arcpy\\da.py'

# ---- which is actually just imports arcgisscripting
# ---- so import it directly

import arcgisscripting as ags

ags.__file__
# ---- 'C:\\...install path...\\bin\\Python\\envs\\arcgispro-py3\\lib\\
# site-packages\\arcgisscripting.pyd'

 

No bother, since you can pull out data for the attributes nicely accounting for <null> records.

def fc_data(in_fc, verbose=False):
    """Pull all editable attributes from a featureclass tables.  During the
    process, <null> values are changed to an appropriate type.

    Parameters
    ----------
    in_fc : text
        Path to the input featureclass
    verbose : boolean
        Requires ``'prn_rec' in locals().keys()`` in order to set to ``True``.
        ``prn_rec`` is imported from arraytools.frmts
    """

    flds = ['OID@', 'SHAPE@X', 'SHAPE@Y']
    null_dict, fld_names = _make_nulls_(in_fc, int_null=-999)
    fld_names = flds + fld_names
    new_names = ['OID_orig', 'X_centroid', 'Y_centroid']
    a = arcpy.da.FeatureClassToNumPyArray(in_fc, fld_names, skip_nulls=False,
                                          null_value=null_dict)
    a.dtype.names = new_names + fld_names[3:]
    if verbose:
        try: prn_rec(a)  # ** prn_rec imported from arraytools.frmts
        except: print(a)
    return np.asarray(a)

The explorers amongst us, may have discovered a few searchcursor shortcuts

fld_names = ['OBJECTID', 'Long_1', 'Short_1', 'Float_1', 'Double_1', 'Text_1']

cur = arcpy.da.SearchCursor(in_fc, fld_names, explode_to_points=False)

z = cur._as_narray()

Traceback (most recent call last):
File "<ipython-input-220-b4e724f0982b>", line 1, in <module>
z = cur._as_narray()

Sadly, the integer fields with <nulls> bring the whole shortcut down.

The searchcursor actually has enough information in it to create a structured/recarray. 

If you have a clean table with no nulls, the actual calls to _dtype and fields show that you can clearly link cursors and NumPy arrays.  Too bad, the whole integer fix isn't incorporated, but _as_narray and FeatureClassToNumPyArray and TableToNumPyArray yield the same results on a 'clean' dataset.

dir(cur)

[...snip... '_as_narray', '_dtype', 'fields', 'next', 'reset']

 

--------------------------------------------------------------------------

Coming soon

 

  • Work with the geometry... and/or … work with the attributes, then put it all back together.

 

Geometry in NumPy... # 1 

Geometry ... ArcPy and NumPy... # 2 

Geometry ... Deconstructing poly* features  # 3 

Geometry ... Reconstructing Poly Features # 4 

Outcomes