List of unique values in multiple columns

3445
13
Jump to solution
02-08-2021 05:07 PM
JamesCrandall
MVP Frequent Contributor

I need create a list of unique values from multiple date columns ("startSurvey" and "EndSurvey")...(I guess a list of lists is fine as long as I can iterate over it and grab the 2 unique values). In the end, I am trying to build sql for a separate SearchCursor.

2021-02-08_19-55-55.jpg

The desired result of the example needs to be:

[[StartSurvey: 6/5/2020 12:21:19 AM, EndSurvey: 6/5/2020 12:35:32 AM, [StartSurvey: 6/11/2020 12:42:05 AM, EndSurvey: 6/11/2020 5:13:18 AM]]

 

It doesn't necessarily have to be list of lists like this, but in some way that I can easily iterate over it to grab the unique StartDate/EndDate valules.

 

 

0 Kudos
13 Replies
JamesCrandall
MVP Frequent Contributor

I get 

 

arr = TableToNumPyArray(fc, "*")
TypeError: long() argument must be a string or a number, not 'NoneType'

0 Kudos
DanPatterson
MVP Esteemed Contributor

TableToNumPyArray—ArcGIS Pro | Documentation

ArcGIS Pro 2.7.x  Python 3.6.whatever

You don't indicate what fc is. 

I just gave a command line example... the principle is the same.  If you prefer functions, here are two that I have used for years.

Do you have nulls in your table?  if so, there are other parameters you can specify.

I don't allow nulls in my tables, but if I get one, I fix it and use a function to call the helper

Helper script 

def make_nulls(in_fc, include_oid=True, int_null=-999):
    """Return null values for a list of fields objects.

    Thes excludes objectid and geometry related fields.
    Throw in whatever else you want.

    Parameters
    ----------
    in_fc : featureclass or featureclass table
        Uses arcpy.ListFields to get a list of featureclass/table fields.
    include_oid : boolean
        Include the `object id` field to denote unique records and geometry
        in featureclasses or geodatabase tables.  This is recommended, if you
        wish to join attributes back to geometry.
    int_null : integer
        A default to use for integer nulls since there is no ``nan`` equivalent
        Other options include

    >>> np.iinfo(np.int32).min # -2147483648
    >>> np.iinfo(np.int16).min # -32768
    >>> np.iinfo(np.int8).min  # -128

    >>> [i for i in cur.__iter__()]
    >>> [[j if j else -999 for j in i] for i in cur.__iter__() ]

    Notes
    -----
    The output objectid and geometry fields are renamed to
    `OID_`, `X_cent`, `Y_cent`, where the latter two are the centroid values.
    """
    nulls = {'Double': np.nan, 'Single': np.nan, 'Float': np.nan,
             'Short': int_null, 'SmallInteger': int_null, 'Long': int_null,
             'Integer': int_null, 'String': str(None), 'Text': str(None),
             'Date': np.datetime64('NaT'), 'Geometry': np.nan}
    #
    desc = Describe(in_fc)
    if desc['dataType'] not in ('FeatureClass', 'Table'):
        print("Only Featureclasses and tables are supported")
        return None, None
    in_flds = desc['fields']
    good = [f for f in in_flds if f.editable and f.type != 'Geometry']
    # good = [f for f in in_flds if f.type != 'Geometry']
    fld_dict = {f.name: f.type for f in good}
    fld_names = list(fld_dict.keys())
    null_dict = {f: nulls[fld_dict[f]] for f in fld_names}
    # -- insert the OBJECTID field
    if include_oid and desc['hasOID']:
        oid_name = desc['OIDFieldName']
        oi = {oid_name: -999}
        null_dict = dict(list(oi.items()) + list(null_dict.items()))
        fld_names.insert(0, oid_name)
    return null_dict, fld_names

 

Main script to read tables and fix nulls

# Featureclass table attribute data
def tbl_data(in_tbl):
    """Pull all editable attributes from a featureclass tables.

    During the process, <null> values are changed to an appropriate type.

    Parameters
    ----------
    in_tbl : text
        Path to the input featureclass.

    Notes
    -----
    The output objectid and geometry fields are renamed to
    `OID_`, `X_cent`, `Y_cent`, where the latter two are the centroid values.
    """
    flds = ['OID@']
    null_dict, fld_names = make_nulls(in_tbl, include_oid=True, int_null=-999)
    if flds not in fld_names:
        new_names = out_flds = fld_names
    if fld_names[0] == 'OID@':
        out_flds = flds + fld_names[1:]
        new_names = ['OID_', 'X_cent', 'Y_cent'] + out_flds[3:]
    a = TableToNumPyArray(
        in_tbl, out_flds, skip_nulls=False, null_value=null_dict
    )
    a.dtype.names = new_names
    return np.asarray(a)

 


... sort of retired...
by Anonymous User
Not applicable

This should give you a unique list of tuples.

date_info_list = list(set([(row[0], row[1]) for row in arcpy.da.SearchCursor(fc, ["StartSuvey", "EndSurvey"])]))

 

JamesCrandall
MVP Frequent Contributor

That does it!

0 Kudos