# Group, split and reclass using numpy, python and arcpy

246
3
07-30-2021 02:25 AM
Labels (1)
MVP Esteemed Contributor
0 3 246

You have a gdb table and you need to group data, sort it, split it and reclass it.

Doing it outside is often the easiest.

This demonstrates the powerful arcpy.da functions of NumPyArrayToTable and TableToNumPyArray

I have a data set with a field containing unsorted values containing 1, 2 or 3.

I need to group the 1's, 2's and 3's together, then break those groups into sequential bunches of 12.

The imports....

``````import numpy as np
from numpy.lib.recfunctions import unstructured_to_structured as uts
from numpy.lib.recfunctions import structured_to_unstructured as stu
import arcpy``````

Load the table into a structured array

``````tbl = # full path to the gdb table or featureclass table in the gdb
arr_ = arcpy.da.TableToNumPyArray(tbl, ['ID_num', 'Val_'])
arr = stu(arr_)``````

Do the work  (I will elaborate soon ...

``````idx_srted = np.lexsort((arr[:, 0], arr[:, 1]))  # sort the array/field
srted = arr[idx_srted]  # sort the array
idx_split = np.nonzero(srted[:-1, 1] - srted[1:, 1])[0] + 1 # split indices
split_arrs = np.array_split(srted, idx_split)  # split the sub arrays
#
# now group into bunches of 12
groups = [np.arange(0, len(i), 12) for i in split_arrs]
cnt = 1  # default counter
out = []
groups = [np.arange(0, len(i), 12) for i in split_arrs]
cnt = 1  # default counter
out = []
for i, sub in enumerate(split_arrs):  # now do the work
bits = np.array_split(sub, groups[i])  # split each subgroup
for bit in bits:
if bit.size > 0:
zz = np.repeat(cnt, len(bit))[:, None]  # repeat the counter
comb = np.concatenate((bit, zz), axis=1)  # magic
out.append(comb)
cnt += 1
cnt = 1``````

Now send it back to Pro, make the table.

``````arr2 = np.concatenate(out, axis=0)
dt2 = np.dtype([('ID_num', 'i8'), ('Val_', 'i8'), ('New_key', 'i8')])
arr2 = uts(arr2, dtype=dt2)
tbl2 = r"C:\Arc_projects\Test_project\Test_pro28.gdb\tbl_2"
arcpy.da.NumPyArrayToTable(arr2, tbl2)``````

Open the table and compare the before and after.

Have fun with NumPy and Arcpy

Tags (4)
New Contributor III

Dear Dan,

weeks went by and I thought it would be a no-brainer.

But since I have never used Arcpy before I'm completely lost on how to work with your script.

I recreated the first table "tbl" with its attributes and saved it in a gdb. But how do I  make the transition work?

I only used Modelbuilder before...this is clearly out of my comfort zone. Can you help to make it run so I can fully focus on starting with beginner arcpy stuff again? Really wanna help my friend since he never worked with python and I seem to be his last hope.

Moreover is there a youtube playlist or book that you can recommend?

Greetings Fynn

Greetings Fynn

New Contributor III

It always says table is not defined

MVP Esteemed Contributor

tbl .... that would be the full path to the table in your geodatabase ...

liberties taken for brevity

for full details on requirements