Faster method than CalculateField_management?

JohnLay

I have a script that joins one table to another, then calculates one column to another. Everything works as it should, but it takes 4 hours to do so if it is scripted while it only takes a few seconds if I do it manually. The script actually does a lot more in that it loops through 4 tables and joins them each to 3 different tables. This is a process that needs to occur fairly regularly, so this is not something I really want to be doing manually every time I need to update these columns.

FieldNameList = ["CATTLE", "DAIRY", "POULTRY", "SWINE"] Iteration = 0 Iteration2 = 0 if TABLETYPE == "Animal Operations":     for JoinFile in UPDATELIST_join:         if Iteration > 2:             Iteration = 0             Iteration2 = Iteration2 + 1         layer = CENSUSLayerLIST[Iteration]         ANOPSFIELD = FieldNameList[Iteration2]         arcpy.AddJoin_management(layer, "GEOID10_1", JoinFile, "GEOID10_1", "KEEP_COMMON")         arcpy.CalculateField_management(layer, ANOPSFIELD, "!NumCount!", "PYTHON_9.3")         arcpy.RemoveJoin_management(layer)         Iteration = Iteration + 1 else:     for layer in CENSUSLayerLIST:         JoinFile = UPDATELIST_join[Iteration]         arcpy.AddJoin_management(layer, "GEOID10_1", JoinFile, "GEOID10_1", "KEEP_COMMON")         arcpy.CalculateField_management(layer, CENSUSFIELDTRANS, "!NumCount!", "PYTHON_9.3")         arcpy.RemoveJoin_management(layer)         Iteration = Iteration + 1

After my first attempt ran for 4 hours, I thought I might give da.SearchCursor / da.UpdateCursor a go. But after it ran for 6 hours I gave up.

Is there a better way to skin this cat?

Anonymous User

Now, just so that I understand what is happening here,

Code:

valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(JoinFile, readList)}

In this part, you are building a python [url=https://docs.python.org/2/tutorial/datastructures.html#dictionaries]dictionary[/url] in which the first field in the readList is the key, and the rest of the field values are the values that are matched to each key.  So the dictionary looks like this for each pair:

valueDict = { field_1 : [ field_2, field_3, field_4] }

The important thing to note here is that you want to treat the first field like a join field.  This field needs to have the values that match up to a field in the other table.

is building an array from which

Code:

with arcpy.da.UpdateCursor(layer, updateList) as cursor:     for row in cursor:         GEOIDval = row[0]         if GEOIDval in valueDict:             row[1] = valueDict[GEOIDval][0]             cursor.updateRow(row)

the current cursor row searches for a match. And

Code:

row[1] = valueDict[GEOIDval][0]

means the value of the update cursor row's second field (row[1]) now equals the matched object's next value from the matched part (valueDict[GEOIDval][0]). Correct?

And yes, you hit the nail on the head here.  Dictionaries and the da cursors are a very powerful combo.