Select to view content in your preferred language

Insert missing features from one layer to another

1807
18
03-29-2023 02:00 PM
CCWeedcontrol
Regular Contributor

I need to insert missing features from one layer to another. I don't want to use the append tool because I don't want a new dataset, I just want to insert the missing features. Both are polygons and have pretty much the same fields except for 3 fields ('Field1','Field2', 'Field3'). I have the following but I get no error when I run it.

 

import arcpy

fc1 = "lyTB" 
targetFC = "lyA_2"

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]

keepList = []
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row[0])
        #print(keepList)
del cursor

#list of values from fc1 to inject into targetFC
ids = []

with arcpy.da.SearchCursor(fc1, fieldnames) as cursor:
    for row in cursor:
        #print (row)
        if row[0] not in keepList:
            ids.append(row)
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insertCursor.insertRow(rows)

print ('Done')
0 Kudos
18 Replies
DanPatterson
MVP Esteemed Contributor

Append doesn't create a new output featureclass... merge does

Append (Data Management)—ArcGIS Pro | Documentation


... sort of retired...
0 Kudos
CCWeedcontrol
Regular Contributor

I guess I mixed up append and merge. Append will append all feature in one layer to the other making duplicates, which is what I am trying to avoid.

0 Kudos
AlfredBaldenweck
MVP Regular Contributor

Not if you have the records you want selected, and drag the layer into the Append dialog

0 Kudos
AlfredBaldenweck
MVP Regular Contributor

It looks like you're checking fc1 twice with the search cursor.

The first round adds all of the objectIDs to the list, then the second round checks the same table to see if it's somehow missing the objectIDs you just added to it.

I'd check using something other than object ID if I were you, probably by checking the contents of the row, reason being that if one of the missing records is in the middle,  objectIDs will not match up.

 

0 Kudos
CCWeedcontrol
Regular Contributor

I made the following changes.

line 24  was

if row[0] not in keepList:

should be;

if row not in keepList:

Line 32 was

InsertCursor .insertRow(rows)

Should have been

insCursor.insertRow(rows)

 

 

fc1 = "lyTB" 
targetFC = "lyA_2"

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]

keepList = []
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row[0])
        #print(keepList)
del cursor

#list of values from fc1 to inject into targetFC
ids = []

with arcpy.da.SearchCursor(fc1, fieldnames) as cursor:
    for row in cursor:
        #print (row)
        if row not in keepList:
            ids.append(row)
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insCursor.insertRow(rows)

print ('Done')

 

How can I check using two other fields other than just the OID?

 

0 Kudos
AlfredBaldenweck
MVP Regular Contributor

So, first off, I owe you an apology. I made some assumptions about your code without fully understanding your fields variables or testing it myself. As it turns out, you weren't checking ObjectID at all.

In the code below, I've made keepList from the targetFC, and ids from the difference between it and fc1, checking by the entire row. Since those rows don't actually include ObjectID, if row[0] is, in fact, a unique field like a site number or something (and you know it's unique), I think that'd be fine?

I only was able to test through the second search cursor, but the sum of keepList and ids came to the total of records in fc1, so I think this is at least a good start.

aprx = arcpy.mp.ArcGISProject("CURRENT")
mp = aprx.activeMap
fc1 = mp.listLayers()[0] # Copy in the Edit GDB, contains 25096 records
targetFC = mp.listLayers()[1] #Copy in the Published GDB, contains 11396 records

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]
#print(out_fields)
#print(fieldnames)

keepList = []
# Switched from fc1 to targetFC
# You want to check the final product first
with arcpy.da.SearchCursor(targetFC, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row) #Appended row, not row[0]. 
                             #Otherwise they might not check for the same thing
del cursor

#list of values from fc1 to inject into targetFC
ids = []
#Check for missing values comparing by row in total.
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor: 
    for row in cursor:
        if row not in keepList:
            ids.append(row)
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insCursor.insertRow(rows)

print ('Done')

 

Also maybe check out Feature Compare (Data Management)—ArcGIS Pro | Documentation and see if that's helpful?

0 Kudos
CCWeedcontrol
Regular Contributor

The current codes insert all the features in fc1 into the targetFc, at least for me. It doesn't appear to only insert the missing ones. If I run the code again, it just adds the total number of features in fc1again. I appreciate your input.

0 Kudos
AlfredBaldenweck
MVP Regular Contributor

I'm confused; I just tested it with no problems?

I have a my raw (target) table, which is purple and has 5196 records.AlfredBaldenweck_1-1680195677972.png

My edited table (fc1) is brown and has 5198 records. (I don't have these two records selected when I run the code)

AlfredBaldenweck_2-1680195721016.png

Running the code:

aprx = arcpy.mp.ArcGISProject("CURRENT")
mp = aprx.activeMap
fc1 = mp.listLayers("Edited")[0] # Copy in the Edit GDB, contains 25096 records
targetFC = mp.listLayers("Raw")[0] #Copy in the Published GDB, contains 11396 records

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]
#print(out_fields)
#print(fieldnames)

keepList = []
# Switched from fc1 to targetFC
# You want to check the final product first
with arcpy.da.SearchCursor(targetFC, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row) #Appended row, not row[0]. 
                             #Otherwise they might not check for the same thing
print("Keeplist length: ", len(keepList))
del cursor

#list of values from fc1 to inject into targetFC
ids = []
#Check for missing values comparing by row in total.
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor: 
    for row in cursor:
        if row not in keepList:
            ids.append(row)
print("ids: ", ids, len(ids))
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insCursor.insertRow(rows)



print ('Done', arcpy.management.GetCount(targetFC))

AlfredBaldenweck_4-1680195943510.png

My two new records are added to the target (ignore the OBJECTIDs; I've run this a few times already).

AlfredBaldenweck_3-1680195851638.png

 

0 Kudos
CCWeedcontrol
Regular Contributor

I was running this outside Pro and thought maybe that was why I was getting different results but no. I put the info in Pro and ran the code.

 

Here are my resultsBeforeRunningCode- ArcGIS Pro.pngAfterRunningCode- ArcGIS Pro.png

 

I thought maybe I had something wrong in the code, so I copied the code you posted and ran it.

 

aprx = arcpy.mp.ArcGISProject("CURRENT")
mp = aprx.activeMap
fc1 = mp.listLayers("Source")[0] # Copy in the Edit GDB, contains 25096 records
targetFC = mp.listLayers("target")[0] #Copy in the Published GDB, contains 11396 records

dsc = arcpy.Describe(targetFC)
fields = dsc.fields
out_fields = [dsc.OIDFieldName, dsc.lengthFieldName, dsc.areaFieldName, 'Field1','Field2', 'Field3']
fieldnames = [field.name if field.name != 'Shape' else 'SHAPE@' for field in fields if field.name not in out_fields]
#print(out_fields)
#print(fieldnames)

keepList = []
# Switched from fc1 to targetFC
# You want to check the final product first
with arcpy.da.SearchCursor(targetFC, fieldnames) as cursor:
    for row in cursor:
        keepList.append(row) #Appended row, not row[0]. 
                             #Otherwise they might not check for the same thing
print("Keeplist length: ", len(keepList))
del cursor

#list of values from fc1 to inject into targetFC
ids = []
#Check for missing values comparing by row in total.
with arcpy.da.SearchCursor(fc1, fieldnames) as cursor: 
    for row in cursor:
        if row not in keepList:
            ids.append(row)
print("ids: ", ids, len(ids))
del cursor

#import all fields
#create insert cursor variable
with arcpy.da.InsertCursor(targetFC,fieldnames) as insCursor:
    for rows in ids:
        insCursor.insertRow(rows)



print ('Done', arcpy.management.GetCount(targetFC))
0 Kudos