topic Re: Should Append tool perform Update and Add at the same time in ArcGIS Pro Questions

Should Append tool perform Update and Add at the same time

LindsayRaabe_FPCWA — Wed, 18 Sep 2024 07:20:04 GMT

Hi All. I'm using the Append tool in ArcGIS Pro, and in arcpy to try and both ADD new features to a feature service in ArcGIS Online (from a feature class), as well as UPDATE existing features if they match (Upsert).

From reading the documentation here I understand that the tool should look for features with identical attributes within the fields listed under the "Update Options" section. If a feature in the Input Dataset matches a feature in the Target Dataset based on those fields, then it will update all the fields listed in the Field Map (as well as the Geometry if that option is enabled at the bottom of the pane). If no matching feature is found, then those features in the Input Dataset should be Appended as new features.

However, the behavior I'm getting doesn't seem to match that. When I have no Update fields listed, the features in my Input Dataset are being appended as New features as expected. When I do have fields listed under the Update Options section, I am seeing matching features update but new features are not being added as I would expect.

Can someone clarify if there is a problem somewhere, or if I have my understanding wrong and the Append tool is either Add or Update, not both?

Re: Should Append tool perform Update and Add at the same time

LindsayRaabe_FPCWA — Wed, 18 Sep 2024 07:43:21 GMT

Quick video showing a typical addition of new features using the Append tool (no issues).

(view in My Videos)

2nd video shows the Append tool with Update Options included failing to correctly update the Geometry of a matching feature (Bowelling 3), and also adding 2 features that should not have been added (Bowelling 3 again which should have been updated and McAlinden 2 was identical so should have been skipped). It did correctly update the missing Tenure attribute against Bowelling 5.

(view in My Videos)

The 3rd video showing the Append tool with Update Options included failing to correctly update the Geometry (Bowelling 3) and Attributes (Bowelling 3 & 5) of matching features, instead adding Bowelling 3 again, skipping Bowelling 5 completely but at least adding 2 features correctly.

(view in My Videos)

Re: Should Append tool perform Update and Add at the same time

DanPatterson — Wed, 18 Sep 2024 08:44:02 GMT

Appends to, or optionally updates, an existing target dataset with multiple input datasets.

The opening line suggest OR, but as you point out, your clip and paragraph screengrab suggests AND

You should really contact Tech Support for clarification and to get the documentation clarified at least.

Re: Should Append tool perform Update and Add at the same time

RhettZufelt — Wed, 18 Sep 2024 18:55:40 GMT

Do the target fields that aren't getting updated have the same name as the input fields?

Documentation isn't really clear, and somewhat contradicts itself, but:

You can use this tool to insert new records and update existing records in a target dataset in the same operation (equivalent to an UPSERT database operation). Use the Matching Fields for Update parameter to specify the fields from the input dataset to match to the fields in the target dataset. For records with matching field values, any fields in the target record will be updated with values from fields with the same name in the input record. Geometry will be updated if the Update Geometry parameter is checked.

This tends to suggest that it will only update fields that have the same name. Almost as if the Field Map is ignored for the upsert operation.

Have not tested, but a thought....

Re: Should Append tool perform Update and Add at the same time

ChristopherCounsell — Wed, 18 Sep 2024 22:16:38 GMT

What version of ArcGIS Pro?

Upsert was introduced in 3.1. there was a bug for this exact behaviour fixed in 3.2

https://support.esri.com/en-us/bug/using-the-append-tool-to-upsert-data-does-not-insert-ne-bug-000161623

Re: Should Append tool perform Update and Add at the same time

LindsayRaabe_FPCWA — Thu, 19 Sep 2024 00:30:48 GMT

So we're still on 3.1.2, so it's quite possible this is the problem, though @RhettZufelt and @DanPatterson also make good points. Looks like I'm going to have to reach out to Tech Support to get some clarification on the capabilities at least, and also find out if this bug is at play in my situation. Thanks all for you prompt feedback. Will update back here when I get an answer.

Re: Should Append tool perform Update and Add at the same time

LindsayRaabe_FPCWA — Thu, 19 Sep 2024 00:35:17 GMT

A follow-up question, in lieu if getting the update through, does anyone have a workaround suggestion? I'm happy to work in python as thats where the final process will actually be executed from. I resorted to Pro to help write and test arcpy steps in my process. So I'm not bound by ArcGIS Pro, or arcpy. Those are just what I'm most familiar with and I had no luck with Copilot and testing alternative methods.

Re: Should Append tool perform Update and Add at the same time

ChristopherCounsell — Thu, 19 Sep 2024 01:14:23 GMT

I think you'll be wasting your time contacting support. They will point to
the documentation and ask you to upgrade to 3.2 and see if the issue
persists. The functionality was introduced at 3.1, bug lodged at 3.1.x and
fixed at 3.2. I used to work tech support.

https://www.esri.com/arcgis-blog/products/arcgis-pro/analytics/upsert-your-datasets-using-the-append-tool-in-arcgis-pro-3-1/

You should be able to upsert in one operation. Blog above clearly outlines
this.

Input field(s) is for matching values. I.e. a unique ID field, or multiple
fields that could uniquely identify exisiting records to updated.

Field maps and geometry is for what attributes get pushed.

You can probably work around the issue by running the append tool twice,
for each method.

Re: Should Append tool perform Update and Add at the same time

DanPatterson — Thu, 19 Sep 2024 02:26:55 GMT

Nothing gets fixed unless it gets reported.

Also, there is a link on the help topic page that you can supply comments for the help topic.

I find regularly that any lack of clarity in a help topic is address by correcting an error, or by that team contacting the appropriate section within esri to confirm what is written.

Could just be me, but it would be worth reporting, especially since beta 3.4 will be coming up

Re: Should Append tool perform Update and Add at the same time

ChristopherCounsell — Thu, 19 Sep 2024 02:31:48 GMT

The exact issue that Lindsay is describing was reported as a defect in 3.1 and fixed in 3.2.

Lindsay is on 3.1.

There's no benefit to reporting it as they will ask you to update and test the issue on the version where the issue has been fixed. May as well do that first.

I'm also not seeing any issues with the documentation, it seems like the user hit a known defect.

Re: Should Append tool perform Update and Add at the same time

SFM_WillBrewer — Tue, 14 Jan 2025 00:10:11 GMT

This issue does not appear to have been fixed. I am experiencing it in 3.2..

Re: Should Append tool perform Update and Add at the same time

Mondi_GIS — Mon, 24 Mar 2025 12:06:22 GMT

Hi Christopher,

I am on 3.3 and I have the same problem - the update functionality is not doing anything and it is just appending all the records (identical records) to the database again. Definitely not fixed in 3.2 😞

Re: Should Append tool perform Update and Add at the same time

ChristopherCounsell — Mon, 24 Mar 2025 12:31:52 GMT

The context of this thread is:

User seeing defect on 3.1
Known issue on 3.1
Fix released on 3.2

My advice was that for this user on 3.1 it's a known issue. They will have no luck troubleshooting it and technical support should rightly direct them to upgrade to 3.2 and see if the behaviour persists.

If you are seeing this behaviour on 3.3 I would recommend exploring some of the advice in this thread around the requirements. If you are confident you are seeing an issue I would recommend a new thread and contacting tech support. It may be workflow related, and you'll get more engagement on a new post, or if there are newer / repeat defects tech support should be able to pick it up pretty quick.

Re: Should Append tool perform Update and Add at the same time

AaronKoelker — Wed, 17 Sep 2025 18:03:56 GMT

Seeing a similar issue in 3.4.3

Append tool "Update (Upsert) Options" being ignore... - Esri Community