Geometry ... Attributes actually... the other bits # 5

DanPatterson_Retired · ‎04-17-2019

Part 5 ... The attributes attached to the geometry

target="_self">Part 5 ... The attributes attached to the geometry
Making none/null/real nothingness
My favorite way of getting just the attributes
Coming soon
Other links

---- As before, the inputs ----

The polygons that I will be using are shown to the right.

A square, 5 points, first and last duplicates
A lake on an island in a lake...
A multipart with a two different shaped donut holes
The letter 'C'
The letter 'V'

Each part is labelled at the labelpoint rather than the centroid, hence each part gets labelled.

---- An Alternate Geometry Reconstructor ----

---- Arrays to Poly* Features ----

Never use <null> in a table. To many posts on the forum on how to trap them, find them, replace them.

Always put in a value to represent all conditions. Too many people use None <null> as the catchall category. In reality all observations need to be classified exactly, there really is no such thing as <null>. You either made and observation or you didn't. If you didn't, your classification scheme should provide a key indicating that.

If an observation was made but the phenomenon/parameter/whatever was actually not there, there should be a key for that. Similar, for 'I forgot', 'Wasn't my job' or whatever other excuses exist.

For observations recorded as floating point numbers, that truly yielded 'nothing/None/nadda/zilch', I use 'nan' (np.nan) since it is a recognized number.
For text/string observations, I use None since None the object translates to 'None' the text easily in most tables.
For time, there is now 'not a time' (NaT), but I don't work with time, preferring to use the string incarnations of those observations
For integers... sadly there is no 'Nint'. You have to provide an actual integer value to represent no value observed, although you desperately tried. You can use the old school -999, or even 2**8, 2**16 etcetera.
For all of the above, anything that is truly doesn't represent an observation where no value was observed, you will have to provide alternatives

Making none/null/real nothingness

You can add to, or remove from, the list below. These are some that I use. I will draw your attention to the NumPy incarnations for integers. Equivalent values exist for floats. Any value that ensures that you will take a second look if a calculation looks weird is good. However if your table contains <nulls> even after my lecture above, this will help mitigate your... stupidity is such a harsh word... but you get my drift

def _make_nulls_(in_fc, int_null=-999):
    """Return null values for a list of fields objects, excluding objectid
    and geometry related fields.  Throw in whatever else you want.

    Parameters
    ----------
    in_flds : list of field objects
        arcpy field objects, use arcpy.ListFields to get a list of featureclass
        fields.
    int_null : integer
        A default to use for integer nulls since there is no ``nan`` equivalent
        Other options include

    >>> np.iinfo(np.int32).min # -2147483648
    >>> np.iinfo(np.int16).min # -32768
    >>> np.iinfo(np.int8).min  # -128

    [i for i in cur.__iter__()]
    [[j if j is not None else -999 for j in i ] for i in cur.__iter__() ]
    """
    nulls = {'Double': np.nan, 'Single': np.nan, 'Float': np.nan,
             'Short': int_null, 'SmallInteger': int_null, 'Long': int_null,
             'Integer': int_null, 'String':str(None), 'Text':str(None),
             'Date': np.datetime64('NaT')}
    #
    desc = arcpy.da.Describe(in_fc)
    if desc['dataType'] != 'FeatureClass':
        print("Only Featureclasses are supported")
        return None, None
    in_flds = desc['fields']
    shp = desc['shapeFieldName']
    good = [f for f in in_flds if f.editable and f.name != shp]
    fld_dict = {f.name: f.type for f in good}
    fld_names = list(fld_dict.keys())
    null_dict = {f: nulls[fld_dict[f]] for f in fld_names}
    # ---- insert the OBJECTID field
    return null_dict, fld_names‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

My favorite way of getting just the attributes

Such nice functions, FeatureClassToNumPyArray, TableToNumPyArray, and back the other way.

I am sure many of you have explored where it all comes from only to find it all buried in a *.pyd file

import arcpy.da as apd
apd.__file__ # ---- 'C:\\...install path...\\Resources\\ArcPy\\arcpy\\da.py'

# ---- which is actually just imports arcgisscripting
# ---- so import it directly

import arcgisscripting as ags

ags.__file__ 
# ---- 'C:\\...install path...\\bin\\Python\\envs\\arcgispro-py3\\lib\\
# site-packages\\arcgisscripting.pyd'‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

No bother, since you can pull out data for the attributes nicely accounting for <null> records.

def fc_data(in_fc, verbose=False):
    """Pull all editable attributes from a featureclass tables.  During the
    process, <null> values are changed to an appropriate type.

    Parameters
    ----------
    in_fc : text
        Path to the input featureclass
    verbose : boolean
        Requires ``'prn_rec' in locals().keys()`` in order to set to ``True``.
        ``prn_rec`` is imported from arraytools.frmts
    """
    flds = ['OID@', 'SHAPE@X', 'SHAPE@Y']
    null_dict, fld_names = _make_nulls_(in_fc, int_null=-999)
    fld_names = flds + fld_names
    new_names = ['OID_orig', 'X_centroid', 'Y_centroid']
    a = arcpy.da.FeatureClassToNumPyArray(in_fc, fld_names, skip_nulls=False,
                                          null_value=null_dict)
    a.dtype.names = new_names + fld_names[3:]
    if verbose:
        try: prn_rec(a)  # ** prn_rec imported from arraytools.frmts
        except: print(a)
    return np.asarray(a)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The explorers amongst us, may have discovered a few searchcursor shortcuts

fld_names = ['OBJECTID', 'Long_1', 'Short_1', 'Float_1', 'Double_1', 'Text_1']

cur = arcpy.da.SearchCursor(in_fc, fld_names, explode_to_points=False)

z = cur._as_narray()

Traceback (most recent call last):
 File "<ipython-input-220-b4e724f0982b>", line 1, in <module>
 z = cur._as_narray()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Sadly, the integer fields with <nulls> bring the whole shortcut down.

The searchcursor actually has enough information in it to create a structured/recarray.

If you have a clean table with no nulls, the actual calls to _dtype and fields show that you can clearly link cursors and NumPy arrays. Too bad, the whole integer fix isn't incorporated, but _as_narray and FeatureClassToNumPyArray and TableToNumPyArray yield the same results on a 'clean' dataset.

dir(cur)

[...snip... '_as_narray', '_dtype', 'fields', 'next', 'reset']‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

--------------------------------------------------------------------------

Coming soon

Work with the geometry... and/or … work with the attributes, then put it all back together.

Geometry ... Attributes actually... the other bits # 5

Part 5 ... The attributes attached to the geometry

Making none/null/real nothingness

My favorite way of getting just the attributes

Coming soon

Other links