I have just run out of memory with too large a dictionary containing large strings (132 chars) to update a table.The symbology classification using quantile intervals gave me an idea of the way to split using the key into equal batches of records.Here is a more useful Python function that did the job to partition the data splendidly.quantile functionI was able to use this to create a list of breakpoints which then limited the dictionary size and updaterow counts to 250K each.Each step ran in a few seconds for a total time of 3 minutes for 3 million records, instead of over an hour for smaller sets.I needed to trap missing code errors.
dBreaks = {"parcel":[3272715, 3542277, 3812689, 4079535, 4350267, 4620430, 4890728, 5160184, 6770930],
"pother":[6845367, 6940090, 7017134, 7077541, 7135010, 7190427, 7248224, 7301079, 7359295]}
# ....
breaks = dBreaks[fcSource]
print breaks
for step in range(len(breaks)):
if step == 0:
expr = "PAR_ID < "+str(breaks[step])
elif step == len(breaks)-1:
expr = "PAR_ID >= "+str(breaks[step])
else:
expr = "PAR_ID >= "+str(breaks[step-1])+" and PAR_ID < "+str(breaks[step])
print "step",step,expr
e = 0
dApp = dict([(row[0],row[1]+" "+row[2]) for row in arcpy.da.SearchCursor(wsSource+"/"+fcLabel,["par_id","legal1","legal2"],expr)])
print "recprds",len(dApp)
with arcpy.da.UpdateCursor(fcTarget,["PAR_ID",fldApp],expr) as cur:
for row in cur:
try:
row[1] = dApp[row[0]]
except:
e+=1
cur.updateRow(row)
if e > 0: print "errors",e
[/URL]This only works if the key is numeric. When the key is not numeric and there is no other field to split the source and target featureclasses I copy one of the tables into the same geodatabase so that I can use the MakeQueryTable tool and then CopyFeatures immediately to a new featureclass. (You don't seem to be able to do any further processing on this strange layer.) Further processing might be FeatureToFeature to rename fields because MakeQueryTable prefixes the table name even if QualifedFieldNames is set to False.