UN Enable Topology performance

892
10
10-31-2019 02:36 AM
mrasiukevicius
Esri Contributor

We are in a way of migration utility data from Geometric Network to UN.

To have some understanding about data amounts and performance:

over 720k linear feature assets

over 110k kilometers of utilities

AssetPackage takes ~40min to load initial data + 3hrs to ImportAssociations (~500k records)

Initial data load and performance is acceptable, overall environment (Portal, Server, DB) performance is good.

But we see a performance issue during Enable Topology operation witch takes days, not hours. And performance degrades if subsequently Disable / Enable topology operations are performed, because _DIRTYAREAS, _ERRORS, etc. table data grows dramatically.

We understand that initial data has lots of validation errors generated. But we have some concern when thinking about such problem in production environment. For example to add extra asset types, rules, etc. means UN services will be unavailable during period of Enable Topology, while UN changes itself will take some minutes.

Is there any recommendations from ESRI regarding Enable Topology performance?

0 Kudos
10 Replies
AnthonyRyanEQL
Occasional Contributor III

Mindaugas,

What coordinate system is your data in? There is an issue if the coordinates are in geographic (eg. Lat/Lng) because the dirty area becomes approx. 10,000 sq km in size.

0 Kudos
mrasiukevicius
Esri Contributor

It is Projected Corrdinate System: LKS 1994 Lithuania TM, WKID = 3346

and Vertical Coordinate System: LAS07 height, WKID = 105704

0 Kudos
AnthonyRyanEQL
Occasional Contributor III

I was hoping you were going to say 'Yes we are using geographic coordinates'. Sorry I have no ideas relating to this now. I'll be watching to see what other people say.

Good luck

0 Kudos
KevinKirkwood
New Contributor III

I don't have anything to add about the state of the topology tools but if you worked on the migration with smaller subsets of the data it may help you. Maybe migrate a single substation at a time to control the number of dirty areas and topology errors generated, clean up those dirty areas and errors, verify the data is clean, then start on the next substation. The validate tool is probably only taking that long because you have an astronomical amount of errors in your data that it needs to generate. 

ManojPatil
Esri Contributor

I agree with Kevin's approach, and we had followed the same approach when I was with a large utility company, start with migrating data substation wise and with "COMPLETE" connectivity of feeder by the feeder outgoing from those substations.

Based on your network voltage hierarchy and complexity, you can take any approach -1) Bottom to Up or 2)Up to the Bottom. 

0 Kudos
PaulLeBlanc1
Esri Contributor

How many error features do you have?

0 Kudos
mrasiukevicius
Esri Contributor

Error statistics below. As you can see, most of them are missing rules, or missing correct connectivity elements between features. Initially we need to load all data to generate and analyze all possible errors. And migration from GN to UN feeder by feeder is not acceptable in our case. UN needs to be completely operational just after migration process.

But from current situation we observe Enable Topology performance. So we have some concern regarding this. Will performance still be the same event if most of error will be corrected? How it will perform in production systems when UN model will be updated and topology needs to be rebuild?

GEOMETRYERR COUNTERR CODEERR DESC
line3120Self-intersecting line.
line10813Midspan connectivity not allowed.
line19936Line feature has invalid terminal.
line29238Devices with multiple terminals cannot be midspan.
line75431Association record is missing one of the endpoints. 
line87121Duplicate vertices.
line10429Invalid connectivity - More than one junction edge rule applicable.
line5108410Invalid connectivity - The edges are different subtypes and cannot connect.
line1585028Invalid connectivity - No junction edge rule.
point2131Association record is missing one of the endpoints. 
point10813Midspan connectivity not allowed.
point14336Line feature has invalid terminal.
point29238Devices with multiple terminals cannot be midspan.
point10429Invalid connectivity - More than one junction edge rule applicable.
point6764225Stacked point features.
point1585028Invalid connectivity - No junction edge rule.
0 Kudos
PaulLeBlanc1
Esri Contributor

I'd reccomend you enable topology on un-versioned dataset. It will be faster and you can easily fix the data and/or rules without publishing.

Performance will be improved if you clean up these errors. With almost 450k errors, the majority of time spent during enable / validate will be error generation. I'd suggest starting with the missing rules and stacked points.

mrasiukevicius
Esri Contributor

ERROR 001992: Dataset must be versioned.
 Failed to execute (EnableNetworkTopology).