See What Changed and Where It Changed

[BruceHarold]

Dataset management in ArcGIS has plenty of supporting tools and workflows, but when you don't have control for any reason you may be the person who has to figure out what data changed, and where.

This blog is about a tool published in the ArcGIS Online sample galleries for bulk change detection between pairs of feature classes.

My first example datasets are two parcel feature classes, where one has been revised with survey and subdivision work, but without any edit tracking fields - the data is not managed in ArcGIS.  The maps are named for their content, Original has the old data, Revised has the new data.

The two datasets have about 650,000 features each over a huge area, so visual comparison is impossible, especially as I need to compare attributes too.  The Feature Compare geoprocessing tool is an option if my data has a unique key field to sort on (it does) but its output is a table, I want features.

The Pro Change Detector tool delivers flexible change detection between two feature classes with your choice of attribute and geometry comparison, and outputs feature classes of Adds, Deletes, Updates and NoChanges (Updates are only detectable if the data has a unique key field separate to ObjectID; without a key field updates are output as spatially overlapping deletes and adds).

The tool requires the ArcGIS Data Interoperability extension, but you don't have to learn to drive the Workbench application delivered with Data Interoperability, this sample is just a normal Python script tool.

For my parcel data I chose all the attributes to be considered as well as geometry:

Then 7 1/2minutes later after comparing ~650,000 features per input I had my change sets:

You can compare any geometry type but if you are going to do change detection of multiple pairs of feature classes be sure to change the output objects names as the tool will overwrite its outputs.  Alternatively, keep your data in separate project databases (see below).

For a second example I decided to 'go big' and compare two street address datasets each with about 2 million features and a lot of attributes:

Now its 22 minutes to find a couple of thousand changes to 2 million features:

...and in the map it is easy to find a locality where subdivision has resulted in new addresses being created - see the extra address points in the Revised map:

To use the tool your data must be in a single File Geodatabase, here is how my Catalog pane looks, note to preserve my change sets I used two separate databases in the Project.

The tool was created with ArcGIS Pro 2.5 beta 2 software (sharp eyed people will see the new style geoprocessing Details view above) but works in Pro 2.4.  You will need ArcGIS Data Interoperability installed and licensed, and you'll need permission to copy a file into the install of your Pro software, please see the README file in the download.

Now go detect some changes and comment in this blog how you get on!

Edited 2/8/2021 to replace the .pth file with one suitable for Python 3.7 in Pro 2.7+

ProChangeDetector.zip

[BrettRosso]

Hi Bruce,

I am trying to get this tool to work but cannot get past this error:

I projected both parcel layers to the same coordinate system so I do not understand how the spatial references do not agree. I am using a trial copy of ArcGIS Pro 2.4.3 because our enterprise agreement does not come with data interoperability extension. 

Could you please advise further?

Brett Rosso

City of Longmont

brett.rosso@longmontcolorado.gov

[BruceHarold]

Hi Brett, sorry about the delay responding, I totally missed your comment.

The check throwing the error is coded as a string comparison comparing the two spatial references.

In your Pro session's Python window key in code like below, replacing the inputs with your layer names.

If the spatial references differ you may see where by inspecting the strings.

A good way to force agreement in spatial references is to put both data sets into a feature dataset.

old = "OldAddress"
new = "NewAddress"
newSR = arcpy.Describe(new).spatialReference.exportToString()
oldSR = arcpy.Describe(old).spatialReference.exportToString()
newSR != oldSR
# False

[SaloniRajput]

Hi Bruce, @BruceHarold 

I am trying to use your tool for our monthly updates to a web application that we have built with ArcGIS API for JavaScript, I do this by overwriting a feature service every month once I have updated the point layer in the map service. I want to compare the the previous month file to the new file, using FacilityID, to know all the  facilities that have been discontinued( to be deleted in the old file) and add new facilities that are in the new file. I have been doing this manually using joins and the table to table attribute check in the Data  Reviewer. I want to automate the process by using your tool in a model builder.

I have both the point layers are in a feature dataset that is in the default File Geodatabase for the project. I have copied the ArcGISProDataInteroperability.pth in the site-packages folder. I am using  ArcGIS Pro version 2.7.1.
I am getting this error when I run the tool with the two point layers and comparing FacilityID field in both the layers.

Thanks.

[BruceHarold]

Hi, please check you have Data Interoperability extension installed and licensed, the import error looks like a missing install or license issue.

As your real goal is to detect changes in a feature service it may be more efficient to use a Spatial ETL tool directly in a model.  If you are able to share some sample data I can show you how that would work.

[SaloniRajput]

Hi Bruce,

Thanks for the response, I checked the  Data Interoperability extension installation, looks like it is installed 

and I can see the extension under the Analysis Tab in ArcGIS pro

I am not detecting changes in feature service,  after I update the data in ArcGIS desktop I republish the feature service by overwriting it. 

[BruceHarold]

I replaced the .pth file with one that expects Python 3.7 in Pro 2.7, please download the zipfile again, your process should work.

[SaloniRajput]

Thanks, Bruce, it worked perfectly, got the exact results that I was expecting.

[BruceHarold]

Good to hear, but by the way, your screen shot of 10.8.1 for Desktop above relates to the ArcMap install, but I could see the Pro install was OK as the Analysis ribbon commands are active.

[SaloniRajput]

Bruce, 

I checked and the ArcGIS pro install looks fine as well.

Thanks for the tool!