Pre UN migration data cleanup workflows

alexccruz2

Hello all,

Are there any documented, proven workflows to help check water utility data for errors before the UN migration, such as floating and stacked points, overlapping and dangling lines, etc.?

Thank you.

-Alex C

RobertKrisher

Look at the first few videos and articles on this reading list for an explanation of how to approach this: ArcGIS Utility Network migration reading list

If you use the migration wizard to create an initial model, it will show you all the topology errors you need to fix in your source data. Once you've done that, you can then rerun the migration, create a controller for your system, and look for disconnected elements. There are quite a few articles that talk about how to do quality assurance on topology errors in and subnetworks, you can find them in industry specific reading lists.

You can find the collection of reading lists in the Getting started with the utility network page.

alexccruz2

Thank you for the response!

Do any workflows exist that can enable an analyst to clean the data before creating an initial model?

Alex

RobertKrisher

It should take you less than an hour to set up the wizard to migrate your data to a generic utility network (I've seen it done in as little at 10 minutes). Once you've done this you will get a list of topology errors from the utility network that enforce all its constraints with your data (stacked point features, self-intersecting lines, etc). You can then take these errors, overlay them on your source data, and correct them there.

Sure you could create a workbench or some data reviewer checks to look at your data based off some assumptions about how your data might be migrated, but if there's already a quick and easy tool that can do most of this for you, included in ArcGIS Pro, why not start with that?

Once you've worked through all the initial set of topology errors reported by the tool, then you could consider whether you want to migrate your data into one of the industry standard models to see how well your data matches those standards, or whether you want to build out your own configuration based on your existing model. This is covered in the resources I shared above.

gis_KIWI4

@alexccruz2 - Robert's given you some direction about the identification of errors.
I'll just make a comment about the fixing of the errors.
The number of errors can be daunting but I would suggest looking into what types of errors exist. You might find that 3 errors make up 90% of all the errors.

Once you know what the errors are there are multiple ways of fixing them. The approach I recommend is finding a balance of automation and manual fix. Then decide how you are going to fix them FME, ArcPy, or UNTools.

There are some UN tools available out of the box - https://pro.arcgis.com/en/pro-app/3.5/tool-reference/utility-networks/apply-error-resolutions.htm

https://pro.arcgis.com/en/pro-app/3.5/help/data/utility-network/analyze-and-address-errors.htm#ESRI_...

I would also recommend getting some hands-on time fixing these for yourself and the team. It's a good training exercise to better understand these errors.

Also think about when the errors are fixed - pre-migration/post-migration? You might find some fixing some errors before the migration makes the process a lot smoother and the remaining can be fixed in post.