topic Re: Finding Duplicate lines or lines that intersect. in Data Management Questions

Finding Duplicate lines or lines that intersect.

JayKappy — Mon, 06 Aug 2012 14:49:29 GMT

I have a single Feature Class of Pipes....I noticed that somehow along the way Duplicates were created.
They are basically exact duplicates right on top of one another.

Is there a tool to run to find the lines that intersect or duplicate in a single Feature Class?

THanks

Re: Finding Duplicate lines or lines that intersect.

DanLee — Mon, 06 Aug 2012 15:37:18 GMT

Try the Find Identical or Delete Identical tool and check the 'Shape' field.
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//001700000054000000
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Delete_Identical/001700000053000000/

Re: Finding Duplicate lines or lines that intersect.

JasonKleps — Tue, 11 Dec 2012 17:45:12 GMT

I have set of duplicate lines as well (they are spatially identical) but the Find Identical tool is not detecting them. I've tried numerous variations all to no avail.

Re: Finding Duplicate lines or lines that intersect.

denisedavis — Tue, 11 Dec 2012 20:22:51 GMT

If you are using GIS10 you could try "Delete Identical" under Data Management in the toolbox. You may want to give coordinates (Calculate Geometry) to each line so you don't delete lines that are the same length.

Re: Finding Duplicate lines or lines that intersect.

NobbirAhmed — Tue, 11 Dec 2012 20:49:25 GMT

Hi Jason,

Could you please share part/all of your data? I have posted some questions to the other thread (on same issue) for you - please respond to the questions - that will help me refine my tests and find a good answer for you.

Thanks, Nobbir
Geoprocessing Team

Re: Finding Duplicate lines or lines that intersect.

JasonKleps — Thu, 13 Dec 2012 12:11:19 GMT

Hi Jason,

Could you please share part/all of your data? I have posted some questions to the other thread (on same issue) for you - please respond to the questions - that will help me refine my tests and find a good answer for you.

Thanks, Nobbir
Geoprocessing Team

Hi Nobbir,

Thanks for getting back to me so quickly. I'm sending you a folder ("Data_For_ESRI") that contains a personal geodatabase ("Data"). Inside there are 2 Feature Classes and a table. "Original_Data" is a FC I created to test the procedure. I started an editing session, set the snapping environment so that I could "Snap to Sketch" and then drew some legitimate lines (with no self-intersections) and various self-intersecting lines. I then broke these features apart using "Split Lines At Vertices" to create "Data_Split_Lines_By_Vertices". I then ran the "Find Identical" tool (selecting the "Shape" field in the dialog box) to create "Find_Identical_Table"). As you can see from the table it did not identify any of the self-intersecting segments. These tests were run in both geographic (NAD-83) and projected coordinate systems ("Albers Equal Area Conic"). As I said, I have tried it on a computer running SP4 as well as one with SP5.

Thanks,

Jason

Re: Finding Duplicate lines or lines that intersect.

NobbirAhmed — Thu, 13 Dec 2012 15:36:12 GMT

Thanks for the data 🙂

I guess your input to Find Identical is Data_Split_Lines_By_Vertices, right? Here is the screenshot for one of the self-intersecting lines (feat# 5 of your Test_Lines data):

[ATTACH=CONFIG]19907[/ATTACH]

After splitting this line by vertices, we get 17 short lines.

[ATTACH=CONFIG]19909[/ATTACH]

None of these 17 lines lie exactly one upon another - that is, there is no spatially duplicate feature among these 17 lines. Could you share your spatially identical lines?

Re: Finding Duplicate lines or lines that intersect.

JasonKleps — Thu, 13 Dec 2012 17:50:16 GMT

Thanks for the data 🙂

I guess your input to Find Identical is Data_Split_Lines_By_Vertices, right? Here is the screenshot for one of the self-intersecting lines (feat# 5 of your Test_Lines data):

[ATTACH=CONFIG]19907[/ATTACH]

After splitting this line by vertices, we get 17 short lines.

[ATTACH=CONFIG]19909[/ATTACH]

None of these 17 lines lie exactly one upon another - that is, there is no spatially duplicate feature among these 17 lines. Could you share your spatially identical lines?

Wouldn't a line that starts at vertex 0, goes out to vertex 1, then returns to vertex 0 and closes as vertex 2 (and vertex 2 lies on top of vertex 0 by setting the snapping environment appropriately) be identical by definition? And wouldn't splitting by vertices create 2 lines of the exact same length right on top of one another? Or am I missing something?

For example, take a look at OBJECT IDs 15 & 16 in Data_Split_Lines_By_Vertices. Are those not spatially identical? Or OBJECT IDs 7 & 8 to the north?

Re: Finding Duplicate lines or lines that intersect.

NobbirAhmed — Thu, 13 Dec 2012 19:21:17 GMT

Wouldn't a line that starts at vertex 0, goes out to vertex 1, then returns to vertex 0 and closes as vertex 2 (and vertex 2 lies on top of vertex 0 by setting the snapping environment appropriately) be identical by definition?

Yes, the answer is "should be". However, in 10.0 (including SP5), digitizing direction mattered. So, the above two lines are considered non-identical 😞 We have enhanced the tool's behavior by ignoring the digitizing direction in 10.1.

Let me know if you need a workaround. If you want to try - just look the documentation for ArcPy Geometry classes. There is a method called equals - try using that.

Earlier, I was unable to reproduce the case as I was using a 10.1 setup. 😞

Re: Finding Duplicate lines or lines that intersect.

JasonKleps — Thu, 13 Dec 2012 19:29:13 GMT

Yes, the answer is "should be". However, in 10.0 (including SP5), digitizing direction mattered. So, the above two lines are considered non-identical 😞 We have enhanced the tool's behavior by ignoring the digitizing direction in 10.1.

Let me know if you need a workaround. If you want to try - just look the documentation for ArcPy Geometry classes. There is a method called equals - try using that.

Earlier, I was unable to reproduce the case as I was using a 10.1 setup. 😞

Yes, if you have a workaround that would be great. I'll also explore the Arcpy Geometry class as well.

Re: Finding Duplicate lines or lines that intersect.

NobbirAhmed — Sat, 11 Dec 2021 10:11:40 GMT

Hi Jason,

Try this code. Set the path of your input data to in_fc. I haven't done anything (other than printing) with the duplicate features - you'll need to use either an insertcursor (with a new empty feature class) or updatecursor (to work with current feature class) to carry over the unique features.

import arcpy

in_fc = r"C:\mytools\findidentical\geom_equals.gdb\trailsfew"

dsc = arcpy.Describe(in_fc)
sr = dsc.spatialReference
oid_field_name = dsc.oidFieldName

# get a cursor on the input features
rows1 = arcpy.SearchCursor(in_fc)

# exclude features already compared once
exclude = []

# iterate through the first coursor
for row1 in rows1:
    
    oid1 = row1.getValue(oid_field_name)
    shp1 = row1.shape
    
    # get a second cursor on the same input features
    rows2 = arcpy.SearchCursor(in_fc)
    
    # add the feature to be compared to exclude list
    exclude.append(oid1)
    
    # create a set to hold duplicate features
    group = set()
    
    # iterate through the second cursor
    for row2 in rows2:
        
        oid2 = row2.getValue(oid_field_name)
        shp2 = row2.shape
        
        # ignore features already compared
        if oid2 in exclude:
            continue
        
        # test equality
        if shp1.equals(shp2):
            
            # add both feature ids to the set of identical features
            group.add(oid1)
            group.add(oid2)
            
            # add the feature just compared to the exclude list
            exclude.append(oid2)
            
    if group:  # if the group is not empty
        print group

Note: I have used two separate cursors on the same feature class. This is due to some issues at 10.0 - at 10.1 one cursor is enough.

Let me know if you find any issue or if you need help with creating an output.

Thanks, Nobbir

Re: Finding Duplicate lines or lines that intersect.

JasonKleps — Thu, 27 Dec 2012 16:16:24 GMT

Hi Jason,

Try this code. Set the path of your input data to in_fc. I haven't done anything (other than printing) with the duplicate features - you'll need to use either an insertcursor (with a new empty feature class) or updatecursor (to work with current feature class) to carry over the unique features.

Note: I have used two separate cursors on the same feature class. This is due to some issues at 10.0 - at 10.1 one cursor is enough.

Let me know if you find any issue or if you need help with creating an output.

Thanks, Nobbir

Thanks Nobbir - This is a great Christmas present!

- Jason.