I'm attempting to write a script that will extract specific records from another very large table (29 million records) so that I can run a QC on the extracted data.
My plan was to create a set object containing record ID’s I'm interested in. I would then perform a da.SearchCursor to match each of the 29 million records to the values in my set object. Any matching row would then be copied into a new table with a da.InsertCursor instance.
It didn’t work and I have no idea why. I only have issues if I try to use a conditional statement to insert rows into a new table. Any other 'then' statement works just fine. Can someone help me figure out where my problem is? Or suggest a work around?
SUMMARY_Set = ([CID, or, a, bunch, of, Community, IDs]) #interested in the ‘CID’ field FGDB = r'D:\~Working\Developement\RISK\QC_Test.gdb' BIG_TABLE = r'Database Connections\NC_RISK.sde\NC_RISK.sql.L_DAMAGE_SUMMARY_JURISDICTION' new_table = os.path.join(FGDB, "L_DAMAGE_SUMMARY_JURISDICTION") arcpy.CreateTable_management(FGDB, "L_DAMAGE_SUMMARY_JURISDICTION", BIG_TABLE) FieldNameList =  fieldnames = arcpy.ListFields(BIG_TABLE) for field in fieldnames: FieldNameList.append(field.name) InsertTable = arcpy.da.InsertCursor(new_table, FieldNameList) with arcpy.da.SearchCursor(BIG_TABLE, FieldNameList) as searchRows: for row in searchRows: if row in SUMMARY_Set: InsertTable.insertRow(row)
I used the same code without the conditional statement and the rows copied into a new table just fine. it took a while, but all the records were there.
with arcpy.da.SearchCursor(SUMMARY_LAYERS, FieldNameList) as searchRows: for row in searchRows: InsertTable.insertRow(row)
I modified the code again to only print the matching rows instead of inserting them into a new table. It worked just like I had intended it to.
>>> FieldNameList.index("CID") 0 >>> with arcpy.da.SearchCursor(BIG_TABLE, FieldNameList) as searchRows: ... for row in searchRows: ... if row in SUMMARY_Set: ... print row + " is in SUMMARY_Set" ... 370042 is in SUMMARY_Set 370042 is in SUMMARY_Set 370042 is in SUMMARY_Set 370042 is in SUMMARY_Set 370042 is in SUMMARY_Set