topic Re: Union tool error in ArcGIS Pro 3.2.0 in ArcGIS Pro Questions

Union tool error in ArcGIS Pro 3.2.0

TomaszBartus — Sun, 26 Nov 2023 20:15:39 GMT

I think I've noticed a major bug with ArcGIS Pro 3.2.0.
In the attachment I am sending the project with data (Test.zip).
There are 3 of feature classes in the geodatabase:
- polygon standsCD,
- StreamsCD_Buffer_Union buffer symbolized in unique values of the S_Distance attribute,
- output feature class from the Union geoprocessing tool.

The Union tool cannot handle a simple union of the first two feature classes. Some objects are assigned wrong values for the S_Distance attribute. In the connected polygons located outside surface stream buffers, the S_Distance attribute value should be 0, and for a many of features the tool incorrectly assigns an attribute value of 50.

It seems to me that this is a very serious error of the application, making it virtually impossible to use it in analyses.

Tomasz Bartuś

StandsCDStreamsCD_Buffer_UnionUnion effect with the errors

Re: Union tool error in ArcGIS Pro 3.2.0

DanPatterson — Sun, 26 Nov 2023 21:22:49 GMT

It would have been useful to have to have the two inputs for the first buffer since

StreamCD_Buffer_Union

is full of self-intersection, invalid ring order errors,

I would check that rile and run a Repair Geometry

I would also do a check geometry on the two files in the input for the first union to make sure they are fine.

The data are projected, which is good (nad27 utm)

If you are processing things in a file geodatabase with 3.,2 and you still get the same issues, then you should forward your inputs to data to Tech Support.

Re: Union tool error in ArcGIS Pro 3.2.0

TomaszBartus — Mon, 27 Nov 2023 06:06:30 GMT

Thank you for your answer.

All data comes from the official ESRI tutorial for ArcGIS (Learning ArcGIS Desktop (for ArcGIS 10.0)), so I assume it has been topologically checked. They were used there in the model for exactly the same purpose and nothing happened. I think the data problem can be dismissed a priori.

StreamCD_Buffer_Union is a class resulting from the Union process of two buffers (which is visible).

The Union process was performed using the "Allow gaps" option.

The data is processed in a file geodatabase.

The case has been reported to ESRI Polska support.

Regards

Tomek

Re: Union tool error in ArcGIS Pro 3.2.0

DanPatterson — Mon, 27 Nov 2023 11:41:08 GMT

Tomek, I used the Esri validation method in Check Geometry if you need to report that.

Keep us posted if you get a case/bug number

Re: Union tool error in ArcGIS Pro 3.2.0

KenHartling — Wed, 29 Nov 2023 19:12:57 GMT

Thanks Tomek and Dan. Definitely issues with this input data that will impact the analysis results that need taking care of. Bad data in, bad data out.

Once I repair all the issues... spatial reference, bad geometries... it appears to work as it should.

Spatial Reference set incorrectly:

Checking Spatial Reference...

XYTOLERANCE ISSUE - NOT set to default. Current: 1.9384789092521745e-05 |  Default for this SR: 0.001

XYRESOLUTION ISSUE - NOT set to default. Current: 2.423098636565218e-06 |  Default for this SR: 0.0001

And as mentioned lots of geometry issues:

(300+ bad geometries reported)

Thx. Ken

Re: Union tool error in ArcGIS Pro 3.2.0

KenHartling — Wed, 29 Nov 2023 19:16:44 GMT

If you are wondering how to fix the spatial reference and geometries... this article gives you the steps to follow to rebuild your feature class and repair the geometries - https://www.esri.com/arcgis-blog/products/arcgis-pro/analytics/geoprocessing-resolution-tolerance-and-hair/

Re: Union tool error in ArcGIS Pro 3.2.0

DanPatterson — Wed, 29 Nov 2023 19:39:27 GMT

Ken, Tomek indicated that the data came from the ...

official ESRI tutorial for ArcGIS (Learning ArcGIS Desktop (for ArcGIS 10.0)

can you flag internally the tutorial people to get them to check their posted data to ensure it is correct.

Re: Union tool error in ArcGIS Pro 3.2.0

KenHartling — Wed, 29 Nov 2023 19:48:15 GMT

I don't know who that would be... but maybe I can find someone that does know. Stay tuned...

Re: Union tool error in ArcGIS Pro 3.2.0

TomaszBartus — Thu, 30 Nov 2023 13:11:24 GMT

ESRI Polska checked the data from the StreamsCD_Buffer_Union set. These are data from two buffers (from the ESRI tutorial) combined with the Union tool. The Check Geometry (Data Management) tool showed geometry errors. So either the Buffer or Union tool generated some errors. These errors were corrected with the Repair Geometry (Data Management) tool. After this procedure, the analysis showed no errors.

It appears that either the ESRI training data or the geoprocessing tools are not error-free.

Thank you all for your comments.
best regards from Kraków

Tomasz Bartuś

Re: Union tool error in ArcGIS Pro 3.2.0

KenHartling — Mon, 11 Dec 2023 20:47:48 GMT

Our folks responsible for the tutorials had this to say about this particular data - "The course, Learning ArcGIS Desktop (for ArcGIS 10.0) was retired back in 2017. This data should no longer be used in other Esri Academy content and should not appear in any newer Pro courses. "

Re: Union tool error in ArcGIS Pro 3.2.0

DanielTomasetti — Sun, 07 Apr 2024 19:32:25 GMT

I just went through a very similar issue while using the Union tool. I was performing a simple union of two polygon feature classes and the result should have been a new feature class with about 600 features. Instead, it was producing a feature class that had 6000+ records. Turns out that each unique geometric feature had 10+ records associated with that unique geometry and tons of erroneous values in the attribute fields.

Ultimately this post led me to repairing geometry on my input feature class. 1 of the inputs had 12 or so errors related to ring ordering. After repairing the geometry and running the tool again, the tool ran properly and I got the correct output with ~600 features.

I frequently use public datasets from states and counties for geospatial analysis and I have never run into this one. The data I am using is the exact same data I used 1 year ago for the same process. I'm vetting last year's lab exercise before assigning it to this year's class. Last year I was using ArcGIS Pro 3.1 or 3.1.1 and I did not experience any issues.

I have a few questions for an ESRI representative:

Are there higher data integrity requirements built in to 3.2?
Have ring ordering errors always affected the Union tool?
The operation was completed with no warnings or error messages. Isn't there some sort of QA function that detects when several records with identical geometry are created?

-Dan

Re: Union tool error in ArcGIS Pro 3.2.0

KenHartling — Mon, 08 Apr 2024 20:31:23 GMT

Hi Dan.

There are no new requirements as far as the correctness of input data geometry. The same high standards apply.

Bad geometry always has the possibility of affecting the processing of data, regardless of tool.

For your third question it's not clear to me what you are asking. Identical geometry can be a valid output to an overlay operation. There are tools (Find/Delete Identical) that can detect and remove identical geometry if they are not desired. If you are referring to whether we have code in place to detect when bad geometry is being used and could potentially cause an issue... we do not. Checking and Repairing all input geometry during analysis is performance adverse and many issues may require manual intervention (In particular, bad Spatial Reference properties can cause 'good' geometry to be 'bad' and 'bad' geometry to be worse once applied during analysis. Repairing a Spatial Reference requires recreating of the feature class. See https://www.esri.com/arcgis-blog/products/arcgis-pro/analytics/geoprocessing-resolution-tolerance-and-hair/ ). If errors are thrown while using 'bad' data we catch and report the errors. From the the doc I provide below:

The onus is on the data's consumer to ensure that the feature class contains valid geometries before the data is used for projects or analysis.

Please see https://pro.arcgis.com/en/pro-app/3.0/tool-reference/data-management/checking-and-repairing-geometries.htm for the documentation on checking and repairing geometry in preparation for analysis.

If you have a copy of the data prior to repair that I can take a look at it would be appreciated. I can confirm a few things.

Thx. Ken

Re: Union tool error in ArcGIS Pro 3.2.0

DanielTomasetti — Thu, 11 Apr 2024 01:27:52 GMT

Hi Ken,

I appreciate the response. Attached are the two inputs I used for the union operation (no repair performed). I am curious to see if you're able to recreate the issue. Also, I would love to hear your thoughts afterwards.

To clarify my third question, I was wondering why Pro would not recognize an erroneous output. I assumed the application would flag something if a union operation results in a feature class with 10+ records for each unique polygon.

Regards,
Dan

Re: Union tool error in ArcGIS Pro 3.2.0

KenHartling — Tue, 23 Apr 2024 18:59:33 GMT

Thanks Dan.

Sorry for the delayed response.

I was able to reproduce the bad output with the bad inputs... I see the same bad output using 3.1, 3.1.1, 3.2 and the soon to be released 3.3.

We'll do some due diligence to make sure there were no internal errors that we missed. We will not be repairing bad geometries each time we encounter an input geometry. All we can do is raise any errors that occur if the bad geometry causes a failure.

To reiterate what I've already posted above... always ensure the input data is good before using in analysis. Checking the inputs for issues should be a step in every workflow prior to analysis. It should be a part of every data providers workflow as well.

Bad input = errors, crashes or worst case scenario bad output with no error.

Thx. Ken