Improving Subnetwork Validation Performance for Large Electric Networks

[VenkataKondepati]

We’re validating subnetworks with 500k+ features and seeing long batch validation times. Has anyone optimized validation performance without breaking propagation accuracy or controller association?

[JohnAlsup]

Can you provide a bit more detail?  For example, are you using partitioned or hierarchical?  Are you Transmission or Distribution?  If distribution, balanced or unbalanced?  500K features per subnetwork seems high.  Did you break conductor in to spans?

[RobertKrisher]

@VenkataKondepati 

First, make sure you have a case logged with support on this.

Second, are you saying that your entire dataset has 500k features? or that you have certain subnetworks that have hundreds of thousands of features?

Third, can you give a screenshot of the tool that is running slow and the parameters you used to run it? "batch validation" isn't a core product or tool so I'm trying to figure out exactly which tool you are referring to. Is this is a workflow or script you've created?

[VenkataKondepati]

Thanks for the questions. I should clarify that I no longer have access to the system or tools, as I’ve since left the company. Because of that, I can’t provide screenshots or exact parameter values.

From memory, this was a Distribution Utility Network (unbalanced, hierarchical) with conductors segmented into spans. The 500k+ figure referred to the overall dataset size, not a single subnetwork. Validation was being run via a scripted workflow calling Validate Network Topology across multiple dirty areas, where individual runs were fine, but cumulative validation time became an issue.

I’m mainly looking for general design or configuration guidance others have used to improve validation performance at scale, rather than troubleshooting a specific implementation.

[RobertKrisher]

@VenkataKondepati 

I'm going to focus on the guidance portion of your question. I have concerns about modeling an unbalanced electrical network using a hierarchical electric domain, but since you are no longer with that company it doesn't seem appropriate to discuss it here.

If you maintain an error free network, you should be able to validate network topology using the full extent on a nightly basis for a network of that size as long as you have reasonable editing workflows in place that aren't causing a large number of dirty areas to be created.

If you carry a large number of topology/subnetwork errors in default that will make things more challenging. This is why it is important to have a plan to resolve errors during the implementation where they can be addressed through automation, configuration, etc. In particular if your data is not clean and you have Manage IsDirty set to true on your subnetworks, this can introduce significant cost to the validate network topology operation as it attempts to discover and mark subnetworks as dirty.

If you do have a large volume of dirty areas, then you could use some of the tools in the utility data management support tools to create quad trees and validate the topology using those polygons.