UN Enable Topology performance

[mrasiukevicius]

We are in a way of migration utility data from Geometric Network to UN.

To have some understanding about data amounts and performance:

over 720k linear feature assets

over 110k kilometers of utilities

AssetPackage takes ~40min to load initial data + 3hrs to ImportAssociations (~500k records)

Initial data load and performance is acceptable, overall environment (Portal, Server, DB) performance is good.

But we see a performance issue during Enable Topology operation witch takes days, not hours. And performance degrades if subsequently Disable / Enable topology operations are performed, because _DIRTYAREAS, _ERRORS, etc. table data grows dramatically.

We understand that initial data has lots of validation errors generated. But we have some concern when thinking about such problem in production environment. For example to add extra asset types, rules, etc. means UN services will be unavailable during period of Enable Topology, while UN changes itself will take some minutes.

Is there any recommendations from ESRI regarding Enable Topology performance?

[AnthonyRyanEQL]

Mindaugas,

What coordinate system is your data in? There is an issue if the coordinates are in geographic (eg. Lat/Lng) because the dirty area becomes approx. 10,000 sq km in size.

[mrasiukevicius]

It is Projected Corrdinate System: LKS 1994 Lithuania TM, WKID = 3346

and Vertical Coordinate System: LAS07 height, WKID = 105704

[AnthonyRyanEQL]

I was hoping you were going to say 'Yes we are using geographic coordinates'. Sorry I have no ideas relating to this now. I'll be watching to see what other people say.

Good luck

[KevinKirkwood]

I don't have anything to add about the state of the topology tools but if you worked on the migration with smaller subsets of the data it may help you. Maybe migrate a single substation at a time to control the number of dirty areas and topology errors generated, clean up those dirty areas and errors, verify the data is clean, then start on the next substation. The validate tool is probably only taking that long because you have an astronomical amount of errors in your data that it needs to generate. 

[Anonymous User]

I agree with Kevin's approach, and we had followed the same approach when I was with a large utility company, start with migrating data substation wise and with "COMPLETE" connectivity of feeder by the feeder outgoing from those substations.

Based on your network voltage hierarchy and complexity, you can take any approach -1) Bottom to Up or 2)Up to the Bottom. 

[PaulLeBlanc1]

How many error features do you have?

[mrasiukevicius]

Error statistics below. As you can see, most of them are missing rules, or missing correct connectivity elements between features. Initially we need to load all data to generate and analyze all possible errors. And migration from GN to UN feeder by feeder is not acceptable in our case. UN needs to be completely operational just after migration process.

But from current situation we observe Enable Topology performance. So we have some concern regarding this. Will performance still be the same event if most of error will be corrected? How it will perform in production systems when UN model will be updated and topology needs to be rebuild?

GEOMETRY	ERR COUNT	ERR CODE	ERR DESC
line	31	20	Self-intersecting line.
line	108	13	Midspan connectivity not allowed.
line	199	36	Line feature has invalid terminal.
line	292	38	Devices with multiple terminals cannot be midspan.
line	754	31	Association record is missing one of the endpoints.
line	871	21	Duplicate vertices.
line	1042	9	Invalid connectivity - More than one junction edge rule applicable.
line	51084	10	Invalid connectivity - The edges are different subtypes and cannot connect.
line	158502	8	Invalid connectivity - No junction edge rule.
point	21	31	Association record is missing one of the endpoints.
point	108	13	Midspan connectivity not allowed.
point	143	36	Line feature has invalid terminal.
point	292	38	Devices with multiple terminals cannot be midspan.
point	1042	9	Invalid connectivity - More than one junction edge rule applicable.
point	67642	25	Stacked point features.
point	158502	8	Invalid connectivity - No junction edge rule.

[PaulLeBlanc1]

I'd reccomend you enable topology on un-versioned dataset. It will be faster and you can easily fix the data and/or rules without publishing.

Performance will be improved if you clean up these errors. With almost 450k errors, the majority of time spent during enable / validate will be error generation. I'd suggest starting with the missing rules and stacked points.

[mrasiukevicius]

ERROR 001992: Dataset must be versioned.
 Failed to execute (EnableNetworkTopology).