Group, split and reclass using numpy, python and arcpy

[DanPatterson]

You have a gdb table and you need to group data, sort it, split it and reclass it.

Doing it outside is often the easiest.

This demonstrates the powerful arcpy.da functions of NumPyArrayToTable and TableToNumPyArray

I have a data set with a field containing unsorted values containing 1, 2 or 3.

I need to group the 1's, 2's and 3's together, then break those groups into sequential bunches of 12.

Follow the code.

The imports....

import numpy as np
from numpy.lib.recfunctions import unstructured_to_structured as uts
from numpy.lib.recfunctions import structured_to_unstructured as stu
import arcpy

Load the table into a structured array

tbl = # full path to the gdb table or featureclass table in the gdb
arr_ = arcpy.da.TableToNumPyArray(tbl, ['ID_num', 'Val_'])
arr = stu(arr_)

Do the work  (I will elaborate soon ...

idx_srted = np.lexsort((arr[:, 0], arr[:, 1]))  # sort the array/field
srted = arr[idx_srted]  # sort the array
idx_split = np.nonzero(srted[:-1, 1] - srted[1:, 1])[0] + 1 # split indices
split_arrs = np.array_split(srted, idx_split)  # split the sub arrays
#
# now group into bunches of 12
groups = [np.arange(0, len(i), 12) for i in split_arrs]
cnt = 1  # default counter
out = []
groups = [np.arange(0, len(i), 12) for i in split_arrs]
cnt = 1  # default counter
out = []
for i, sub in enumerate(split_arrs):  # now do the work
    bits = np.array_split(sub, groups[i])  # split each subgroup
    for bit in bits:
        if bit.size > 0:
            zz = np.repeat(cnt, len(bit))[:, None]  # repeat the counter
            comb = np.concatenate((bit, zz), axis=1)  # magic
            out.append(comb)
            cnt += 1
    cnt = 1

Now send it back to Pro, make the table.

arr2 = np.concatenate(out, axis=0)
dt2 = np.dtype([('ID_num', 'i8'), ('Val_', 'i8'), ('New_key', 'i8')])
arr2 = uts(arr2, dtype=dt2)
tbl2 = r"C:\Arc_projects\Test_project\Test_pro28.gdb\tbl_2"
arcpy.da.NumPyArrayToTable(arr2, tbl2)

Open the table and compare the before and after.

Have fun with NumPy and Arcpy

[FynnScharpen]

Dear Dan,

weeks went by and I thought it would be a no-brainer.

But since I have never used Arcpy before I'm completely lost on how to work with your script.

I recreated the first table "tbl" with its attributes and saved it in a gdb. But how do I  make the transition work? 

I only used Modelbuilder before...this is clearly out of my comfort zone. Can you help to make it run so I can fully focus on starting with beginner arcpy stuff again? Really wanna help my friend since he never worked with python and I seem to be his last hope.

Moreover is there a youtube playlist or book that you can recommend? 

Greetings Fynn

Greetings Fynn 

[FynnScharpen]

It always says table is not defined

[DanPatterson]

tbl .... that would be the full path to the table in your geodatabase ...

liberties taken for brevity 

TableToNumPyArray—ArcGIS Pro | Documentation 

for full details on requirements