Insert missing features from one layer to another

[CCWeedcontrol]

I need to insert missing features from one layer to another. I don't want to use the append tool because I don't want a new dataset, I just want to insert the missing features. Both are polygons and have pretty much the same fields except for 3 fields ('Field1','Field2', 'Field3'). I have the following but I get no error when I run it.

import arcpy

fc1 = "lyTB" 
targetFC = "lyA_2"

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]

keepList = []
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row[0])
        #print(keepList)
del cursor

#list of values from fc1 to inject into targetFC
ids = []

with arcpy.da.SearchCursor(fc1, fieldnames) as cursor:
    for row in cursor:
        #print (row)
        if row[0] not in keepList:
            ids.append(row)
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insertCursor.insertRow(rows)

print ('Done')

[DanPatterson]

Append doesn't create a new output featureclass... merge does

Append (Data Management)—ArcGIS Pro | Documentation

[CCWeedcontrol]

I guess I mixed up append and merge. Append will append all feature in one layer to the other making duplicates, which is what I am trying to avoid.

[AlfredBaldenweck]

Not if you have the records you want selected, and drag the layer into the Append dialog

[AlfredBaldenweck]

It looks like you're checking fc1 twice with the search cursor.

The first round adds all of the objectIDs to the list, then the second round checks the same table to see if it's somehow missing the objectIDs you just added to it.

I'd check using something other than object ID if I were you, probably by checking the contents of the row, reason being that if one of the missing records is in the middle,  objectIDs will not match up.

[CCWeedcontrol]

I made the following changes.

line 24  was

if row[0] not in keepList:

should be;

if row not in keepList:

Line 32 was

InsertCursor .insertRow(rows)

Should have been

insCursor.insertRow(rows)

fc1 = "lyTB" 
targetFC = "lyA_2"

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]

keepList = []
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row[0])
        #print(keepList)
del cursor

#list of values from fc1 to inject into targetFC
ids = []

with arcpy.da.SearchCursor(fc1, fieldnames) as cursor:
    for row in cursor:
        #print (row)
        if row not in keepList:
            ids.append(row)
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insCursor.insertRow(rows)

print ('Done')

How can I check using two other fields other than just the OID?

[AlfredBaldenweck]

So, first off, I owe you an apology. I made some assumptions about your code without fully understanding your fields variables or testing it myself. As it turns out, you weren't checking ObjectID at all.

In the code below, I've made keepList from the targetFC, and ids from the difference between it and fc1, checking by the entire row. Since those rows don't actually include ObjectID, if row[0] is, in fact, a unique field like a site number or something (and you know it's unique), I think that'd be fine?

I only was able to test through the second search cursor, but the sum of keepList and ids came to the total of records in fc1, so I think this is at least a good start.

aprx = arcpy.mp.ArcGISProject("CURRENT")
mp = aprx.activeMap
fc1 = mp.listLayers()[0] # Copy in the Edit GDB, contains 25096 records
targetFC = mp.listLayers()[1] #Copy in the Published GDB, contains 11396 records

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]
#print(out_fields)
#print(fieldnames)

keepList = []
# Switched from fc1 to targetFC
# You want to check the final product first
with arcpy.da.SearchCursor(targetFC, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row) #Appended row, not row[0]. 
                             #Otherwise they might not check for the same thing
del cursor

#list of values from fc1 to inject into targetFC
ids = []
#Check for missing values comparing by row in total.
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor: 
    for row in cursor:
        if row not in keepList:
            ids.append(row)
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insCursor.insertRow(rows)

print ('Done')

Also maybe check out Feature Compare (Data Management)—ArcGIS Pro | Documentation and see if that's helpful?

[CCWeedcontrol]

The current codes insert all the features in fc1 into the targetFc, at least for me. It doesn't appear to only insert the missing ones. If I run the code again, it just adds the total number of features in fc1again. I appreciate your input.

[AlfredBaldenweck]

I'm confused; I just tested it with no problems?

I have a my raw (target) table, which is purple and has 5196 records.

My edited table (fc1) is brown and has 5198 records. (I don't have these two records selected when I run the code)

Running the code:

aprx = arcpy.mp.ArcGISProject("CURRENT")
mp = aprx.activeMap
fc1 = mp.listLayers("Edited")[0] # Copy in the Edit GDB, contains 25096 records
targetFC = mp.listLayers("Raw")[0] #Copy in the Published GDB, contains 11396 records

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]
#print(out_fields)
#print(fieldnames)

keepList = []
# Switched from fc1 to targetFC
# You want to check the final product first
with arcpy.da.SearchCursor(targetFC, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row) #Appended row, not row[0]. 
                             #Otherwise they might not check for the same thing
print("Keeplist length: ", len(keepList))
del cursor

#list of values from fc1 to inject into targetFC
ids = []
#Check for missing values comparing by row in total.
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor: 
    for row in cursor:
        if row not in keepList:
            ids.append(row)
print("ids: ", ids, len(ids))
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insCursor.insertRow(rows)



print ('Done', arcpy.management.GetCount(targetFC))

My two new records are added to the target (ignore the OBJECTIDs; I've run this a few times already).

[CCWeedcontrol]

I was running this outside Pro and thought maybe that was why I was getting different results but no. I put the info in Pro and ran the code.

Here are my results

I thought maybe I had something wrong in the code, so I copied the code you posted and ran it.

aprx = arcpy.mp.ArcGISProject("CURRENT")
mp = aprx.activeMap
fc1 = mp.listLayers("Source")[0] # Copy in the Edit GDB, contains 25096 records
targetFC = mp.listLayers("target")[0] #Copy in the Published GDB, contains 11396 records

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]
#print(out_fields)
#print(fieldnames)

keepList = []
# Switched from fc1 to targetFC
# You want to check the final product first
with arcpy.da.SearchCursor(targetFC, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row) #Appended row, not row[0]. 
                             #Otherwise they might not check for the same thing
print("Keeplist length: ", len(keepList))
del cursor

#list of values from fc1 to inject into targetFC
ids = []
#Check for missing values comparing by row in total.
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor: 
    for row in cursor:
        if row not in keepList:
            ids.append(row)
print("ids: ", ids, len(ids))
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insCursor.insertRow(rows)



print ('Done', arcpy.management.GetCount(targetFC))