Append only ids that don't exist in another class

[ThomasHamming]

I have 2 points classes with the same attributes, and I am looking to append data from one to the other. This is simple enough, but the first class has a lot of data that is already in the second class. I am looking to append only the rows/ids that don't exist in the other class already, and I can't seem to find an easy way to do this in pro. Any idea on how to do this?

For example, if this were a program it might look something like this:

for row in class1:

if row.id not in class2:

class2.append(row)

[JayantaPoddar]

A workaround. How about Append followed by Delete Identical?

[ThomasHamming]

Thanks, and this is a solution but not an ideal one. The class I am appending from has over 10,000 points, and a normal append operation would take hours to complete. Theoretically only adding missing points would take much less time.

[Robert_LeClair]

I wonder if a Pairwise Erase workflow would accomplish this task.  The input features parameter is FC1.  The erase features parameter is FC2.  The output feature class is those features that are not coincident.  Then use the output feature class to append the data to FC2.  

[JohannesLindner · Accepted solution]

target_class = "path:/to/class_2"
append_class = "path:/to/class_1"
id_field = "RowID"

# read the ids that are already in the target class
existing_ids = [row[0] for row in arcpy.da.SearchCursor(target_class, [id_field])]

# create an SQL where clause "IdField NOT IN (1, 2, 3)"
if isinstance(existing_ids[0], str):
    id_list = ["'{}'".format(i) for i in existing_ids]
else:
    id_list = [str(i) for i in existing_ids]
where_clause = "{} NOT IN ({})".format(id_field, ", ".join(id_list))

# create a layer of all append features that are not in the target class, append
append_layer = arcpy.management.MakeFeatureLayer(append_class,  "append_layer", where_clause)
arcpy.management.Append(append_layer, target_class)

[ThomasHamming]

Thank you, this worked!

[JonathanWhite1]

I am trying to get that script to work in python notebook but no luck. Just copied above and added paths for target_class, append_class and specified the id_field. I'm super new to this so its probably something simple I am not understanding.

SyntaxError Traceback (most recent call last) File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\ast.py, in parse: Line 35: return compile(source, filename, mode, PyCF_ONLY_AST) SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape (<string>, line 2)

[JohannesLindner]

Sorry it took so long to answer, I was on vacation... If you haven't found a soultion yet:

It's probably because you used backslashes instead of forward slashes in your paths. Single backslashes are special characters in Python strings, they are used to encode characters that you can't simply type, e.g. "\n" for a linebreak.

Use one of these methods:

# forward slashes
"path:/to/your/fc"
# double backslashes
"path:\\to\\your\\fc"
# define it as raw string
r"path:\to\your\fc"

[JonathanWhite1]

No matter what I do it just gives me this error below. I have tried just about everything. The only thing I can think of is perhaps this will not work with AGOL hosted tables? The target layer is in an AGOL relationship class tables while the append layer is just in AGOL feature set tables. I have also attempted to use a local file geodatabase table as the append class but I get the same error. 

Traceback (most recent call last):
File "S:\GISDataActive\ParcelMap\AutomatedUpdates\Appendaddress.py", line 24, in <module>
append_layer = arcpy.management.MakeFeatureLayer(append_class, "append_layer", where_clause)
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\management.py", line 9395, in MakeFeatureLayer
raise e
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\management.py", line 9392, in MakeFeatureLayer
retval = convertArcObjectToPythonObject(gp.MakeFeatureLayer_management(*gp_fixargs((in_features, out_layer, where_clause, workspace, field_info), True)))
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 512, in <lambda>
return lambda *args: val(*gp_fixargs(args, True))
arcgisscripting.ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000732: Input Features: Dataset https://services2.arcgis.com/wEula7SYiezXcdRv/arcgis/rest/services/Parcel_Updates/FeatureServer/1 does not exist or is not supported
Failed to execute (MakeFeatureLayer).
Failed script AddressAppend...
Failed to execute (AddressAppend).

[JohannesLindner]

Oof. Yeah, arcpy (and in extension the desktop geoprocessing tools) don't play well with web content.

You can interact with hosted layers using the ArcGIS API for Python, but I don't know anything about that.

So, you have two options:

1. Download the hosted layers, run my script in the desktop environment, overwrite the hosted layer with the result
2. Figure out how to do it in the Python API. There are probably multiple community members who can help you.