Select to view content in your preferred language

Text data ... NumPy moves up a notch

233
0
05-13-2025 05:36 PM
Labels (1)
DanPatterson
MVP Esteemed Contributor
4 0 233

Making text fields in Pro or any many other tabular formats requires that you specify the field width to accommodate the larges width.  If you specify a size too small, data gets truncated, if it gets accepted at all.

For example, lets make a small data set which we might want to add to a table.

vals = 'abcdefgh'  # -- an iterable to extract the values from
# -- generate the values
fld = [vals[i - 1] * i for i in range(1, len(vals) + 1)]
fld
['a', 'bb', 'ccc', 'dddd', 'eeeee', 'ffffff', 'ggggggg', 'hhhhhhhh']

Now we cleverly create an array specifying a "field width"... I will use an array dtype in the following example.

z0 = np.array(fld, dtype='U3')
z0
array(['a', 'bb', 'ccc', 'ddd', 'eee', 'fff', 'ggg', 'hhh'], dtype='<U3')

# --- notice that the array isn't correct !!!!!
# --- there is truncation of the input values to a length of 3 (unicode text)

Now add a field in a table in Pro and mis-specify the field width and see if the following holds the same as in the NumPy example (it does btw).

Soooo, what did NumPy do?  They implemented https://numpy.org/neps/nep-0055-string_dtype.html 

Here is how it works.

z0 = np.array(fld, dtype='U3')  # -- again with the input data
z0
array(['a', 'bb', 'ccc', 'ddd', 'eee', 'fff', 'ggg', 'hhh'], dtype='<U3')

z1 = np.array(fld, dtype=StringDType())  # -- notice that unicode and size not specified
z1
array(['a', 'bb', 'ccc', 'dddd', 'eeeee', 'ffffff', 'ggggggg', 'hhhhhhhh'], dtype=StringDType())
# -- data size retained!!!

This is one of the small obstacles that the arcpy people will have to deal with should they continue to move forward with supporting NumPy > 2.0.

A few of the affected...

ExtendTable—ArcGIS Pro | Documentation 

FeatureClassToNumPyArray—ArcGIS Pro | Documentation

NumPyArrayToFeatureClass—ArcGIS Pro | Documentation

NumPyArrayToTable—ArcGIS Pro | Documentation

Contributors
About the Author
Retired Geomatics Instructor (also DanPatterson_Retired). Currently working on geometry projects (various) as they relate to GIS and spatial analysis. I use NumPy, python and kin and interface with ArcGIS Pro.
Labels