topic Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric in ArcGIS Parcel Fabric Questions

Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

RobertChaney — Wed, 10 Apr 2024 21:28:38 GMT

I've run the Upgrade ArcMap Parcel Fabric GP tool with the option to delete identical overlapping lines checked. I read that identical overlapping lines will be deleted even if the directions are reversed by 180 degrees, and if the Distances in the Distance field are the same out to four decimal places.

The Distances in the Distance field is where I'm seeing the possible problem. A lot of my lines are essentially the exact same but differ in distance at the fourth decimal place. So, I believe this is why they're not being deleted. Some duplicate lines are being deleted and come into the Pro fabric as a single line because the lines in the ArcMap Fabric are the same out to the fourth decimal place.

How can I solve this issue? Would I need to do some sort of recalc on the Distance field, or would changing the field properties of the ArcMap Fabric Distance field to three decimal places fix the issue?

Thanks,

Robert.

ArcPro ver. 3.2.2

Fabric ver. 5, fgdb

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

JasonCamerano — Wed, 10 Apr 2024 23:44:07 GMT

@RobertChaneyYou could always run Delete Identical on each of the Lines layer after the Upgrade is complete. The Upgrade Tool runs that GP tool during that stage of the upgrade. You can configure the parameters however you want in this case.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

RobertChaney — Thu, 11 Apr 2024 14:59:08 GMT

As a test, I went ahead and copied the ArcMap Fabric to a fgdb, set the properties of the distance field on the lines to 3 decimal places.

ran the Upgrade ArcMap Parcel Fabric tool and got the same result as before in the Pro Fabric.

It would seem the upgrade tool is not honoring the change to the field properties.

I think I'll just have to use the Delete Identical tool as Jason described, but choose the Shape_Length field.

Robert

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

JasonCamerano — Thu, 11 Apr 2024 17:39:08 GMT

That is correct that the GP Script tool doesn't look at map properties for field formatting as the tool can be run without any maps open.

One other thing you can do is modify the script.

Here are the steps:

1. Go to your Pro install location and browse to this fodler: C:\Program Files\ArcGIS\Pro\Resources\ArcToolBox\Scripts
Then find the FabricForArcMapToArcGISPro.py file

2. Copy that file to another location, like your desktop.

3. Right click on the file and Open it with Notepad or any other editor of your choice.

4. Once the file is open scroll all the way to the bottom and you will find a variable called IDENTICAL_ROUND_VALUE. It is currently set to 4. Change the 4 to what ever rounding value you decide

5. Save you changes and close the editor

6. Copy and paste the .py file back to the original directory in step one and overwrite the existing file.

Let me know how this works out for you.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

RobertChaney — Tue, 16 Apr 2024 03:08:20 GMT

Jason,

I've followed the steps you outlined and modified the script so the variable, IDENTICAL_ROUND_VALUE = 3.

However, I'm still getting the same results, the resulting Pro Fabric still has the same duplicated lines. So, it looks as though the modification to the script is not being honored. Could this be a potential Bug? Any other suggestions, or would you recommend running the Delete Identical on the Shape_Length field?

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

JasonCamerano — Tue, 16 Apr 2024 21:03:07 GMT

Is there a way you could send me a small subset of your data so I can run it on my end? I can take a look.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

RobertChaney — Tue, 16 Apr 2024 21:53:42 GMT

Jason,

What would be the best way to export out a portion of the ArcMap fabric in ArcMap, while keeping it in the fabric model?

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

JasonCamerano — Tue, 16 Apr 2024 22:10:35 GMT

In ArcMap there is a GP Tool called Copy Parcel Fabric https://desktop.arcgis.com/en/arcmap/latest/tools/parcel-fabric-toolbox/copy-parcels.htm

You can select some parcels and run this tool. Then send me that FGDB with the newly copied parcel fabric.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

RobertChaney — Tue, 16 Apr 2024 22:21:37 GMT

Jason,

I was able to find that tool. I sent you a direct message with the attachment that contains the sample data.

Thanks.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

JasonCamerano — Tue, 16 Apr 2024 22:28:09 GMT

Thanks, I got it. I'll take a look.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

JasonCamerano — Wed, 17 Apr 2024 22:40:27 GMT

Hey @RobertChaney

So I did some investigating of the data and here is what I found. It doesn't look like the Distance rounding is the primary culprit. It looks like it's the Direction field.

Let me start with what I think might be path forward for your data then I'll get into the nitty gritty of what is happening behind the scenes.

I would suggest not running the Delete Identical option during the Upgrade. Instead, run it manually after upgrade is complete. The Upgrade ArcMap tool simply runs the GP Delete Identical so running it after will be basically the same except you can control the fields it uses to compare more specifically. If you do not care about the Direction and Distance field values, you can just run the Delete Identical tool and use SHAPE as the field it checks against.

However, if you are concerned about the Distance fields you can run the delete identical against: SHAPE, DISTANCE, ARCLENGTH, and even CREATEDBYRECORD. However you might need to round your distance fields to a lower decimal place, and if you don't want to modify the surveyed data you can create a new field and compute the Distance values over there and round the values in that field. Then use that newly created field in the Delete Identical.

Let me know if that will be a good workflow for you!

Ok... now for some nerdy numbers...

When looking at the Direction field I noticed the decimal values went over 10 places. And during the Upgrade ArcMap tool what happens if you choose to Delete Identical is that I will create two new fields; one that stores the Direction field minus 180 if it's a angle that is greater than 180 and two a distance field that is rounded to 4 decimal places. I then use those two fields, plus the CREATEDBYRECORD, RETIREDBYRECORD and SHAPE field in the parameter for Delete Identical.

With that being said looking at the Direction Field, rounding to 6 decimal places the values still differed between the two lines. Use these lines as an example

You can see on the left the UpgradeDirectionMinus180 fields differ between the two lines. And the Direction values that are stored have huge precision. Seeing this makes me think that these full raw values aren't necessarily important to you so running Delete Identical simply against the shape of the line features might be a viable option for you.

Hopefully all this makes some sense. Let me know if you have any other questions.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

RobertChaney — Thu, 18 Apr 2024 22:54:24 GMT

Thanks for your explanation and suggestion Jason. I've went ahead and tried the upgrade without checking the box for deleting identical lines. Instead after the upgrade, I went ahead and ran the delete identical on the lines choosing the shape field your mentioned. This seems to be a method to use for the final run. However, I'm noticing on some curves with duplicates the lines aren't being deleted when using the shape field. I can choose these duplicates manually and run the delete identical tool using the Arc_Length tool, but I hesitate to do this on all curves at once for fear of removing lines that shouldn't be deleted. If I choose the Shape and Arc_Length fields none of them get deleted. Any suggestions on handle these duplicate curves? I'm going to test out your suggestion of creating a second distance field as well.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

JasonCamerano — Thu, 18 Apr 2024 23:16:16 GMT

@RobertChaney

Ah interesting. I ran the delete identical on your data you provided me right now with SHAPE only. I looked for a curve and luckily there was one and it has one curve that is still overlapping after.

I used the tool Highlight Gaps and Overlaps tool available on the Quality Tab in Pro

I notice there are alot of gaps and overlaps along these curves

Zooming to that curve that has the duplicate lines you'll notice there is a gap between the parcels so the lines technically aren't overlapping.

Examining that ArcMap Fabric these gaps and overlaps are all present there too.

I took a look at one of the areas where there are yellow gaps at the top and I noticed the parcel polygon geometry wasn't true curves along the line, they were still densified arcs. I toggled on the Parcels Vertices layer and can see there are alot of extra verticies on the polygon.

So what I did to fix this selected all the parcels in the cul-de-sac and clicked Shrink To Seed

Once they were shrunk to seed I manually selected one of those 'duplicate lines' and deleted it

I then ran Reconstruct From Seeds which rebuilt all those parcels I shrunk to seed (note: nothing happened to any of the attributes of that parcel)

I ran the Highlight Gaps and Overlaps tool again and everything was all fixed. No more gaps or densified arcs on the polygons.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

RobertChaney — Fri, 19 Apr 2024 00:11:51 GMT

It wasn't something I did on the sample data I sent you, but after the upgrade I'm running a merge collinear boundaries and the SLACA tool to clean up some of these issues.

At the north end of the data I sent there's the issue

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

JasonCamerano — Fri, 19 Apr 2024 00:30:13 GMT

Well, some good news regarding that type of curve...

At ArcGIS Pro 3.3 we have enhanced the Merge Collinear Lines tool to work with curves. I selected those lines and ran the GP tool on them and it created a curve

Only problem is you'll have to wait just a bit until 3.3 comes out in order to use the tool with the new functionality.

Re: Identical Overlapping Lines not being deleted after upgrading ArcMap Fabric to Pro Fabric

RobertChaney — Tue, 30 Apr 2024 02:04:55 GMT

Thanks to Jason for providing some great options for upgrading my ArcMap parcel fabric and providing some additional methods for cleaning up the data after the upgrade. On the final run, I used Jason's suggestion to upgrade the ArcMap fabric without choosing to delete identical overlapping lines. Instead running the delete identical GP tool using the SHAPE field after the upgrade. I will instruct the end users to use the available fabric tools to clean up the left over duplicates as well as fixing the gaps and overlaps.

Robert.