We are in a way of migration utility data from Geometric Network to UN.
To have some understanding about data amounts and performance:
over 720k linear feature assets
over 110k kilometers of utilities
AssetPackage takes ~40min to load initial data + 3hrs to ImportAssociations (~500k records)
Initial data load and performance is acceptable, overall environment (Portal, Server, DB) performance is good.
But we see a performance issue during Enable Topology operation witch takes days, not hours. And performance degrades if subsequently Disable / Enable topology operations are performed, because _DIRTYAREAS, _ERRORS, etc. table data grows dramatically.
We understand that initial data has lots of validation errors generated. But we have some concern when thinking about such problem in production environment. For example to add extra asset types, rules, etc. means UN services will be unavailable during period of Enable Topology, while UN changes itself will take some minutes.
Is there any recommendations from ESRI regarding Enable Topology performance?
Mindaugas,
What coordinate system is your data in? There is an issue if the coordinates are in geographic (eg. Lat/Lng) because the dirty area becomes approx. 10,000 sq km in size.
It is Projected Corrdinate System: LKS 1994 Lithuania TM, WKID = 3346
and Vertical Coordinate System: LAS07 height, WKID = 105704
I was hoping you were going to say 'Yes we are using geographic coordinates'. Sorry I have no ideas relating to this now. I'll be watching to see what other people say.
Good luck
I don't have anything to add about the state of the topology tools but if you worked on the migration with smaller subsets of the data it may help you. Maybe migrate a single substation at a time to control the number of dirty areas and topology errors generated, clean up those dirty areas and errors, verify the data is clean, then start on the next substation. The validate tool is probably only taking that long because you have an astronomical amount of errors in your data that it needs to generate.
I agree with Kevin's approach, and we had followed the same approach when I was with a large utility company, start with migrating data substation wise and with "COMPLETE" connectivity of feeder by the feeder outgoing from those substations.
Based on your network voltage hierarchy and complexity, you can take any approach -1) Bottom to Up or 2)Up to the Bottom.
How many error features do you have?
Error statistics below. As you can see, most of them are missing rules, or missing correct connectivity elements between features. Initially we need to load all data to generate and analyze all possible errors. And migration from GN to UN feeder by feeder is not acceptable in our case. UN needs to be completely operational just after migration process.
But from current situation we observe Enable Topology performance. So we have some concern regarding this. Will performance still be the same event if most of error will be corrected? How it will perform in production systems when UN model will be updated and topology needs to be rebuild?
GEOMETRY | ERR COUNT | ERR CODE | ERR DESC |
line | 31 | 20 | Self-intersecting line. |
line | 108 | 13 | Midspan connectivity not allowed. |
line | 199 | 36 | Line feature has invalid terminal. |
line | 292 | 38 | Devices with multiple terminals cannot be midspan. |
line | 754 | 31 | Association record is missing one of the endpoints. |
line | 871 | 21 | Duplicate vertices. |
line | 1042 | 9 | Invalid connectivity - More than one junction edge rule applicable. |
line | 51084 | 10 | Invalid connectivity - The edges are different subtypes and cannot connect. |
line | 158502 | 8 | Invalid connectivity - No junction edge rule. |
point | 21 | 31 | Association record is missing one of the endpoints. |
point | 108 | 13 | Midspan connectivity not allowed. |
point | 143 | 36 | Line feature has invalid terminal. |
point | 292 | 38 | Devices with multiple terminals cannot be midspan. |
point | 1042 | 9 | Invalid connectivity - More than one junction edge rule applicable. |
point | 67642 | 25 | Stacked point features. |
point | 158502 | 8 | Invalid connectivity - No junction edge rule. |
I'd reccomend you enable topology on un-versioned dataset. It will be faster and you can easily fix the data and/or rules without publishing.
Performance will be improved if you clean up these errors. With almost 450k errors, the majority of time spent during enable / validate will be error generation. I'd suggest starting with the missing rules and stacked points.
ERROR 001992: Dataset must be versioned.
Failed to execute (EnableNetworkTopology).