Understanding Subnetworks: Status
When the utility network was first released, it introduced a new concept to the Esri ecosystem and vocabulary, the Subnetwork. This abstraction was created to describe the different ways that users separate and manage their networks into network zones without using industry-specific terms like ‘Circuit’ or ‘Pressure zone’.
But why do we use the term “subnetwork”?
All the data used to manage the connectivity for one set of resources for a utility is stored in a utility network (e.g. the network). So, when this network is subdivided into smaller, topologically contiguous areas (circuits, pressure zones, etc.) then each of these zones can be accurately described as a sub-network. With this abstraction in place, a framework was created to manage each subnetwork as its own GIS object that can be visualized, analyzed, and even used for reporting purposes. An important component of this framework is the ability to track metadata on each subnetwork so users can understand how their system is changing over time.
This metadata provides basic information like editor tracking fields or can be expanded to incorporate user-defined values such as the number of customers, number of protective devices, etc. Metadata also allows users to answer the most common, and important question that a GIS person gets asked: is this data correct?
This is a surprisingly tough question because correct means different things to different people. However, breaking the original question down into more specific questions it becomes much easier to answer:
- When was the subnetwork last updated?
- When was the subnetwork last extracted to another system?
- Is this subnetwork up to date?
- Have features in the subnetwork been modified since it was last updated?
- Are there any known errors that need to be fixed in this subnetwork?
The first two questions are easy to answer through several date fields maintained on the subnetwork, namely the Last Update Subnetwork and Last Ack Export Subnetwork fields. The last three questions can all be answered by looking at the Status field of a subnetwork (in earlier models this was called the ‘Is Dirty’ field) which includes the status values of Clean, Dirty, and Invalid. Here’s what these values mean:
- Clean – A subnetwork that has no known errors and is ready to be used for analysis.
- Dirty – A subnetwork that has been modified and that needs to be updated.
- Invalid – A subnetwork that has one or more known errors that need to be resolved.
With the foundation established, the rest of this article will focus on how the system manages the Status field and how to interpret each of the different status values.
Marking Subnetworks Dirty
The key to managing the status of subnetworks in the utility network is that the system needs to be able to determine when a subnetwork is affected by an edit so it can be marked as dirty. While there are many ways to accomplish this, the software currently uses the validate network topology operation to discover and mark subnetworks as dirty. Because determining which subnetwork was affected by an edit requires tracing, it would introduce too much overhead to the editing process if the system performed this analysis after every edit. Finally, because all editing workflows affecting the network must be validated, this allows the system to ensure the state of subnetworks cannot get out of sync with the data as it is edited.
But how does the system determine which subnetworks were modified?
The system performs a subnetwork controller trace for all the validated features to discover which subnetworks were affected by the edits. The exact behavior differs slightly depending on whether the validated edits belong to a partitioned or hierarchical utility network. What do these terms mean and why does this matter? We’ll discuss this below.
In a partitioned subnetwork, each feature can belong to at most a single subnetwork. This means that the system will analyze each tier of the network to determine if any of the edited features belong to that tier. Once a feature has been matched to a specific subnetwork it can be removed from analysis in subsequent tiers, and once all features are matched the analysis is complete, even if all tiers have not been analyzed. You can see a simplified example of a partitioned network can be seen below:
In the case of a hierarchical network, each feature can belong to multiple subnetworks because the tiers of the network are nested in each other. As a result, the system must always analyze each tier in the network for all the edited features. You can see a simplified example of a hierarchical network below:
Now that we’ve looked at the different topology types at a high level, we will examine several editing scenarios for each of these topology types and make note of how the system behaves.
Partitioned
When the network topology is validated in a partitioned network, the system needs to identify the subnetwork affected by each edit. To do this it identifies all the tiers in the network that have subnetworks and analyzes each tier to determine which subnetworks in each tier, if any, were affected by the edit. The system repeats this process with each tier until it has discovered network sources for all the edited features, or all the tiers in the network have been analyzed. Let’s look at a few examples of this behavior in action.
In the first example below, a single edit is made to the utility network (purple hash polygon). When validate network topology runs it finds a single network source for the edit on the first tier it analyzed, the distribution tier. This allows validate to skip doing any analysis on the transmission tier since the system has already discovered a subnetwork controller that accounts for all the edits.
In this second example, multiple edits in several different areas in the network are validated. The system must perform multiple traces to discover sources for all the edits because the edits occurred in multiple tiers of the network.
In the third example, the system validates an edit made to a feature that is not connected to any subnetworks. In this case, the system must trace all the tiers in the network to confirm that no subnetworks were affected.
As you can see, the validate network topology operation can more quickly identify the modified subnetworks in partitioned networks when the edits are limited to a single tier and when the subnetworks that were modified are smaller. As the number of tiers and the size of the subnetwork increase, the system will take longer to identify the modified subnetworks during validate network topology because it must perform more traces and the traces will take longer.
Hierarchical
Because every feature in a hierarchical network can belong to multiple tiers, the system must consider every tier in the network that has subnetworks when trying to identify subnetworks affected during validate network topology.
Below you can see an example of a simple hierarchical network that has a single system subnetwork that contains two smaller subnetworks.
In this first example below, a feature belonging to one of the smaller subnetworks is edited. The system first identifies the smaller subnetwork that the edit belongs to. However, because this is a hierarchical subnetwork the system must analyze all the remaining tiers in the network, and in this case, it identifies a second subnetwork in a different tier to mark as dirty.
In the second example, a feature that belongs exclusively to the larger subnetwork is edited. In this case, the lower-ranking subnetworks do not get marked as dirty. This is because the edit didn’t affect any of the lower-ranking subnetworks. This logic is the same regardless of whether the network is source-based or sink-based because the highest-ranking network is always the outermost container in the hierarchy (the ultimate source or ultimate sink of the network).
In the third example, a feature that does not belong to a subnetwork is edited. In this case, no subnetworks are marked as dirty. However, the utility network must trace each tier of the network to confirm that the feature doesn’t belong to any subnetworks.
Because system subnetworks are so large, they can add a larger performance cost to validate network topology than smaller subnetworks. Also, because these subnetworks contain so many features, they are more likely to be marked as dirty during validate, and as a result, each system subnetwork often needs to be updated many times per day to stay clean. Because of this, it is common for the highest-level tiers in a hierarchical network to be set to not manage the status field and instead just update them on a nightly basis. In addition to reducing how often these subnetworks are updated, this configuration also improves the performance of validate network topology because it allows the utility network to skip this tier during validation. The next section of this article discusses how you can determine whether a tier is configured to manage the status field.
Configuration
Even though maintaining the Status field is the default behavior of the subnetworks in the utility network, there are some users, and entire industries, that do not make use of subnetwork status as part of their workflows. To support this configuration the Update Subnetwork Policy section of the Set Subnetwork Definition tool contains an option e to determine whether the corresponding tier in the subnetwork should ‘Manage IsDirty’.
- When this option is checked, state management is enabled the system will behave according to the ‘normal’ behaviors outlined in this article.
- When this option is unchecked, state management is disabled, and the system will no longer manage the status field during validate network topology.
- One of the side effects of disabling state management is that every subnetwork in the corresponding tier will always appear dirty, this is because the system has no way of identifying which subnetworks are modified during validation so it must always assume the network is dirty.
Users who choose not to opt into this behavior (disabling state management) typically do so for one of two reasons:
- Their editing workflows result in subnetworks that are always dirty, or...
- Performance impacts
Remember that this setting is configured separately for each tier in your network, so you may choose to leave this setting enabled for some tiers in your network (distribution, pressure zones, etc.) while disabling this for other, larger tiers (transmission, system, etc.).
Regardless of how your network is currently configured, you can always change this setting later. If your model currently has this setting enabled, this option can be disabled if you don’t find managing the status field useful. If instead, you have a model that does not manage the status field, but you later decide that you want to take advantage of the status field in your workflows, it can be enabled.
Validate Consistency
The entire discussion up to this point has looked at how, when, and which subnetworks get marked as dirty when a dirty area is validated. This of course raises the question; how does the system respond or identify subnetworks that contain edits that haven’t been validated? This is where the idea of validating consistency during analysis comes into play.
The default behavior when you perform a trace in the utility network is to validate the consistency of your trace result. What this means in practical terms is that the system checks to see if there are dirty areas associated with your trace results. If there aren’t any dirty areas associated with your results, then your results are considered consistent.
However, if there are dirty areas associated with your trace results, the trace will fail, and you will receive an error letting you know that one or more dirty areas were discovered during the trace. If you want to ignore the dirty areas and see the trace results, potentially producing incorrect results, you can uncheck the option to Validate Consistency.
How does this apply to subnetwork status? If dirty areas are encountered during update subnetwork, the subnetwork will be marked as dirty. However, a clean subnetwork can be consistent or inconsistent, depending on whether it has outstanding, unvalidated edits since it was last updated. If there are unvalidated dirty areas in your database, the system doesn’t know which subnetwork (if any) to mark as dirty until the edit is validated. Once all your dirty areas are validated, the corresponding subnetwork(s) are marked as dirty, and all traces will be consistent.
What impact does this have on managing subnetworks? It means that you can’t necessarily look at the status of a subnetwork and know whether it is consistent. However, you can be confident that if you trace a subnetwork, you will receive an error if you attempt to analyze or export an inconsistent subnetwork, even if the subnetwork says it is clean.
Conclusion
Now that you’ve finished reading this article you should have a better understanding of the benefits and workflows associated with subnetwork management, and how the status field helps guide you through those workflows. You should also understand how the system manages this field and why certain industries may choose to only manage the status field for some of the tiers in their utility network.