Latest Contributions by RobertKrisher

In this article, we will examine some of the more advanced configurations that can be applied to gravity-based networks to answer more sophisticated questions and to improve data quality. We will look at how the utility network addresses these challenges through its use of subnetworks, terminals, and rules. Is everything connected? Being able to run traces to identify the flow direction in our system is a useful analytical tool, but a common question we need to answer is whether all our features are properly connected. The simplest, but least efficient, way to determine this is to place a trace location in your map and run a connected trace. This trace will identify all the features that are traversable to that location. This works well for small networks where everything is connected, but if your dataset is very large this trace will take more time and if your data is not all connected to a single connected system you need to provide multiple start locations to cover your entire network. Below you can see an example of a stormwater dataset. Because only the catchment areas are modeled, and not the rivers and channels that connect them, a single connectivity trace cannot be used to identify disconnected features. One of the ways the utility network can help identify disconnected features is by configuring subnetworks. A subnetwork represents a named subset of our utility network with a set of devices that are responsible for controlling the resources in that area. In the case of a stormwater network this is typically the outfalls that are controlling each catchment area. If we were to turn all the outfalls for our catchment areas into subnetwork controllers we would be able to run a trace to find everything that was connected to our network. We can simulate this by running a connected trace using each outfall in our network as a starting location. If we were to create a single subnetwork called “Stormwater System”, with each of these devices as a subnetwork controller, the selected features in the graphic above are what the subnetwork would look like. While this is useful from an initial quality assurance perspective, it would be a lot more useful if we modeled each collection of outfalls that governed an area as controllers for a specific catchment area. The engineers who rely on GIS data to maintain planning and engineering models already track this information outside GIS. By modeling catchment areas inside GIS, we can validate the changes we make are not only topologically correct, but that they also validate the information required by engineers or planners to produce their models. Depending on the quality, complexity, and volume of data you maintain you may decide to just model a single subnetwork for your entire system, or you may decide to create separate subnetworks for each catchment area. We will be discussing the more difficult task of configuring separate subnetworks. These instructions will assume that when you ran the Migrate to Utility Network tool and you identified one or more layers as having controllers in them, if not, you will need to perform additional configuration to allow a feature to be a subnetwork controller. Rules and Terminals By default, the Migrate to Utility Network tool will configure subnetwork controllers in a sink-based network to only connect to features using their Upstream terminal. If you don’t model anything downstream of your outfalls, then you can move onto the next section where you see how to create your subnetwork controllers. The first thing we should do is review the rules in our network using the network properties dialog. Below we can see a subset of the rules for our discharge points. If we look closely we can see discharge points have been configured to connect to every type of line in our model using their upstream terminal. We can refine these rules so they more accurately reflect the way that our discharges should be connected to other features in the network. Outfall – These features allow a pipe to drain into an open drain Overflow – These features allow a pipe or virtual drain line to overflow into an open drain Standard Outlet – These features allow a pipe to discharge into a virtual drain line Terminal Discharge – These features allow a pipe or open drain to exit the system We need to run the Add Rule and Delete Rule tools to get the rules configured to match these requirements. Once we’ve configured the rules and enabled our network topology, we are likely to see connectivity errors (the red lines in the graphic below) in our database for many of our discharge points. Depending on your data and how you adjust your rules you will see two different kinds of errors: Invalid connectivity – There is not a rule that allows the two features to connect. Ambiguous connectivity. – There is more than one rule that allows the features to connect. While we could review each error individually and determine how to fix them one at a time, if there are more than a few errors it is more efficient to use the Analyze Network Data tool to give us a summary of all the types of errors. In addition to creating a layer file that can be used to review our errors, the tool also outputs a RuleCandidates.csv file. This file contains any rules that could be added to the network to resolve connectivity errors. You will want to carefully review the list of rules before importing and only import rules for features that should be allowed to be connected. You may need to check with an engineer or field worker to determine what is appropriate. You can learn more about this process in the Refining connectivity rule article. Once you’ve determined which rules you want to add, use the Import Rules tool to add the rules to your utility network. Remember that before you can add rules you must disable the network topology. Once you’ve added the missing rules and enabled the network topology again, you will then need to use the Modify Terminal Connections tool to resolve any ambiguous connectivity. This is only required if you have a line that is allowed to connect to more than one terminal on a device. If you want more examples of working through these kinds of connectivity errors you can find three tutorials to help guide you through this decision making process in the Editing and connectivity learning series. Once our network topology is enabled and error free, we’re ready to move forward with creating our subnetworks. Create a simple subnetwork The first thing we need to do is identify the outfalls that act as subnetwork controllers for each catchment area of our network. This can be done by running a connectivity trace for an area of our network and stopping whenever we reach an outfall, which have been configured as subnetwork controllers. While we could manually select and add barriers for these features, it is often easier to use a condition barrier to automatically identify these features when tracing. To do this we add a condition barrier to our trace for features with a Category of Subnetwork Controller. This will cause the trace to stop for any feature whose asset type has a category of Subnetwork Controller, in this case outfalls. This network category is initially populated by the Migrate to Utility Network tool for any mappings you designated as being controller. You can also adjust this later using the Set Network Category tool. We can see the results of such a trace below. In this instance, we have an area that is all connected to a single outfall. Use the Modify Subnetwork Controller tool to set the upstream terminal of the outfall, the side that receives materials, a subnetwork controller to a new catchment area. The subnetwork controller needs a unique name, and this area needs its own unique name as well. If we discuss our catchment areas and outfalls with engineering or operations, they may already have specific identifiers that they want us to use. Using the same unique identifiers as other departments will also facilitate collaboration and communication. After validating the change with the network, we can now run a subnetwork trace to see all the features connected to this outfall. We can also use the update subnetwork tool to store the name of the catchment area on these features. This makes it easy for us to identify which features belong to a catchment area, and which do not. After running this tool, we can see that all the features in this subnetwork have their subnetwork name populated. And we can see that there is now a subnetwork line feature that represents all the lines in this catchment area. Once you’ve used the Update Subnetwork tool to create a subnetwork line, that subnetwork will now appear in the Find Subnetwork pane when that line is in the current extent. Now that we’ve seen how to create a simple subnetwork with a single controller, let’s look at how to handle a catchment with multiple outfalls. Subnetworks with multiple controllers Not every subnetwork has a single controller. Stormwater networks often have multiple outfalls that are responsible for discharging water from a catchment area, depending on how much water is in the system at a given time. We begin the process the same way; by performing a connectivity trace in our network and treating subnetwork controllers as a barrier, this will cause the trace to stop at outfalls. Consider creating and using a trace configuration for this trace to help make the process easier. In this instance we have a catchment area with three outfalls. We repeat the same process as above, uniquely naming each outfall, but we give each of these outfalls the same subnetwork name. This is because they are all sharing the responsibility of discharging water from the same area. Note: If you do not provide a subnetwork controller name, the tool will automatically use the global id of the feature. Once again, validate the network topology and run update subnetwork to finish creating the second catchment. Repeat this process until all the features in the network belong to a subnetwork. This sounds easy, but let’s look at some of the most common problems that can arise. Missing Controllers Using this process provides a reliable, but manual process for identifying all your subnetworks. The most common problem you will run into is that the subnetwork controllers may be missing from your data, or data problems can cause a catchment area to appear much larger than it should be. Looking at the example below we can see what should have been a small catchment area is appearing as a much larger area. If we zoom into the region called out in the image above, we can see that the problem is there isn’t an outfall between the pipes of the catchment and the open channel it drains into. This is called out in the graphic below. To correct this error we would work with an engineer or field crew to ensure the GIS matches what is currently installed in the field. In this case we would create an outfall between the pipe and the river channel. We would then make this new outfall a subnetwork controller for our third catchment area. If we look closely at the above screenshot, we can see that there is still a potential problem in this area. Some of the pipes north of this catchment don’t have any form of outlet. If we zoom in we can see that this area likely should be connected to the catchment area. We would need to confirm with an engineer or field crew that this is indeed the case, but once we’ve determined how/if it is connected, we can then update our GIS data to reflect this. Conclusion In this article you learned how to use subnetworks, rules, and terminals to improve your data quality. You saw how this allowed you to identify missing features, improperly connected features, and how to identify locations in your network without any outlets. If you’re interested in more advanced analysis tasks like watershed or sewershed management or managing sub-basins and catchments, let us know! If you have any questions about the utility network, be sure to ask them on the Esri Community site. If you want to know more about using the utility network to manage gravity-based networks, like sewer and stormwater data, please explore the Learn ArcGIS Utility Network for Sewer and Stormwater learning series. This learning series includes tutorials and articles that demonstrate how to address the needs of the sewer and stormwater industry using the utility network. ... View more

In our previous article we saw how you can configure a utility network to perform isolation traces for gas and water networks. In that article we saw how the system can identify equipment that can be used for isolation, along with how to configure our network to use a normal position field to identify whether a valve is open or closed. In this article we will show you how to configure a utility network to manage pressure zones. The examples outlined in this article are for a gas network, but the techniques and concepts are applicable to any pressurized network. What is a pressure zone? An important part of managing the distribution of resources to customers in a pressurized system is the creation and analysis of pressure zones. The engineers at a utility rely on complex formulas and engineering software to ensure the system operates as intended, however, they often rely on the network model stored in a GIS to build these models. It is the responsibility of a GIS analyst to ensure that the data in the GIS is kept up-to-date and accurate so engineers, planners, and network operators can make informed decisions using GIS. Pressure zones play an important role in the daily responsibilities of operators and engineers. For operations and engineering to use GIS, they must have confidence that it contains an accurate accounting of the features regulating and belonging to each pressure zone. Read the Understanding Pressure Zones article to learn how to use GIS to manage and analyze pressure zones. Historically, customers have maintained pressure zone information as an attribute on pipes, or by using a polygon layer in their GIS that shows the extent of each zone. This information may look good on a map but doesn’t work in cases where there are multiple pressure zones on the same street or when a user inadvertently creates a connection between pipes in different pressure zones. Scenarios such as this are where a utility network provides value, since it can be configured to model and validate the extents of pressure zones. The utility network created using the Migrate To Utility Network tool can include a single tier for managing distribution systems by default. These large groupings of pipes all share a common source of gas, water, or energy. To assign a pressure zone to a pipe, we need to configure a tier to represent the pressure zones. If your domain network is configured to be partitioned each feature can only belong to a single tier, so you must choose between tracking the system tier or pressure zones. Gas, water, and district heating domain networks are typically modeled using a hierarchical network for this reason. Hierarchical networks allow each feature to participate in multiple tiers in the network. This is not limited to systems and pressure zones either. Some customers also use GIS to track isolation zones, cathodic protection structures, or even district metered areas. When adding a new tier to a utility network, you need to ask yourself the following questions: What is the purpose of this tier? What features govern or regulate (sources or sinks) this tier? What features are allowed to belong to this tier? Are there any statistics I want calculated for the subnetworks in this tier? With this information in hand, you are ready to start configuring a new tier. The first step is to configure certain features to act as the sources or sinks in the tier. In the utility network, these features are called subnetwork controllers. Subnetwork Controller Configuration Before a feature can act as a subnetwork controller, its asset type must be configured to act as a subnetwork controller in a tier in the network. You can find a full list of the steps on the set a subnetwork controller page in the online help, but we will cover them briefly here. Assign a terminal configuration (optional) Delete previous rules Add rules Assign a network category (optional) Add a tier Set the subnetwork definition If you have already identified the equipment that impacts flow or pressure as subnetwork controllers when you ran the Migrate To Utility Network tool, then your asset types are already configured to act as subnetwork controllers, and you can skip ahead to adding a tier. To start, we need to assign a terminal configuration to the asset types we want to be assigned as subnetwork controllers. When a line connects to a device with terminals, it must specify which terminal it is connected to. The ability to differentiate between connections is required when creating subnetwork controllers. As an example, without the use of a terminal configuration and terminal connections, a regulator would be unable to distinguish between the pipes connected to its inlet and the pipes being regulated connected to its outlet. The Migrate To Utility Network tool includes three generic terminal configurations: Directional Source – This terminal configuration is used when a device in a source-based network needs upstream and downstream terminals. Directional Sink – This terminal configuration is used when a device in a sink-based network needs upstream and downstream terminals. BiDirectional – This terminal configuration is used when there is no explicit upstream or downstream terminal. It can be used in either source or sink-based networks. Picking the right terminal configuration is important to ensure that tracing works properly. In a pressurized system, most network controllers have an explicit upstream and downstream terminal configuration. This means that water, gas, etc. can only flow from the pipe on the upstream terminal to the pipe on the downstream terminal. However, if you have equipment that can be configured to not regulate flow in the field you should use the BiDirectional terminal configuration which will allow pressure to be unregulated as it flows through the device. In our example, we will be configuring our device to be BiDirectional because several of our regulator stations maintain a high-pressure loop in our distribution system. Note: If this tool is run using ArcGIS Pro 3.5 or later, you will need to remove any existing rules associated with the asset type using the Delete Rule tool before you can set its terminal configuration. Once you have changed the terminal configuration of the device, you then need to create rules that allow you to define which types of pipes are allowed to connect to either terminal. In this example regulator stations have distribution pipes on both the inlet and outlet side of the device, so you will add rules for each side. Note: When a pipe is allowed to connect to more than one terminal on a device, each pipe must specify which terminal it is connected to on the device. If it does not, this creates an ambiguous connectivity error that must be resolved using the Modify Terminal Connections tool. If you were adding rules for town border stations or custody transfer meters, you would only allow transmission pipe to be on the upstream terminal and distribution pipe on the downstream terminal. If you are uncertain of which rules to add, consult with an engineer or someone else with an understanding of your system to help you make the right decision. In the case of water models, pumps or pressure reducing valves would have distribution pipes on both the upstream and downstream terminals. Now that the regulator in our example has terminals which allow it to differentiate between the pipe connections on either side, you can begin to configure it as a subnetwork controller. The next step is to assign a network category to the asset type to allow it to serve as a subnetwork controller. To do this, use the Set Network Category tool: The next step is to define what kind of subnetworks the device is allowed to act as a subnetwork controller for using the Set Subnetwork Definition tool. The device should serve as a subnetwork controller for pressure zone subnetworks, but first a tier must be created to represent the pressure zones first. This is done using the Add Tier tool. Adding a pressure zone tier Now that your asset types have the network category, terminals, and rules to act as network controllers you’re ready to add your tier to the network. Use the Add Tier tool to add a new tier to the network. You can learn more about these settings on the Tiers topic in the online help. Because our utility network already contains a tier for the distribution system, which has the default rank of 1, we set the rank of pressure zones to be 2. This means that pressure zones are deeper in the network hierarchy than our system subnetworks. Because we already have a field that stores our system subnetwork name, we specify a new field called PressureSubnetworkName to store the name of the pressure zone. Don’t worry about creating this field ahead of time, the tool will automatically add the field for you. Now that we’ve added a pressure zone tier to our network, we need to define the features that act as subnetwork controllers and participate in the subnetwork, along with the rules of how the subnetwork trace should be performed. To do this we use the Set Subnetwork Definition tool. This tool has quite a few parameters, so it is recommended that you read the subnetwork definition page in the online help before running this tool on your network. For the Valid Features and Objects section, we specify all the asset types in our network with just a few exceptions. For the Valid Subnetwork Controllers parameter, we only select our regulator stations and any other equipment we configured to act as subnetwork controllers for pressure zones. The Aggregated Lines for Subnetline Feature Class parameter is used to identify which lines are used to create the subnetwork line geometry when the update subnetwork operation is run. Because this line is used to visualize pressure zones when zoomed out to small scales, we don’t want to include any asset types that contain many small lines. In the case of our pipe system, we don’t want to include any services, laterals, or lines used exclusively for cathodic protection. Note: Features excluded from the subnetwork line are still considered when calculating summaries for the subnetwork. Because your network doesn’t include structures or nonspatial content, you will want to disable, or uncheck the options to include containers, content, and structures. You will also want to treat open devices as barriers using a condition barrier, just as you did with the system tier. Don’t worry about populating summaries when first setting your subnetwork definition. You can change your subnetwork definition later to include summaries about things like pipe length, volume, and number of service connections. The aggregated lines for subnetline feature class parameter is used to identify which lines are used to create the subnetwork line geometry when update subnetwork is run. Because this line is used to visualize pressure zones when zoomed out to small scales, we don’t want to include any asset types that contain many small lines. In the case of our pipe system, we don’t want to include any services, laterals, or lines used exclusively for cathodic protection. Update Subnetwork Policy The last major section to configure for the subnetwork definition is the update subnetwork policy. This gives you control over how subnetwork information is updated within your utility network. Making changes to these settings is not a simple decision for many customers, as it involves making tradeoffs between convenience and performance. We will give a brief overview of these decision points here and refer you to more thorough discussions where available. If you chose to include containers, content, and structures in your network, additional options will be displayed for you to specify whether to update structure/domain network containers. This determines whether the Subnetwork name and Supported subnetwork name attribute fields on your structures and containers will be updated when they contain or support a feature that belongs to a subnetwork. This allows you to use the Select By Attributes tool to identify features that support a subnetwork without running a trace. While this is convenient for reporting purposes, it also means that the update subnetwork operation will take longer to run because there may be hundreds of thousands of additional features that need to be updated. The next option to discuss is whether the tier should manage the IsDirty (Status) field, you can find a deep dive on state management on the Esri Community site. This field is used to indicate whether there have been edits validated in your utility network that have impacted a particular subnetwork. This option is set to False by the Migrate To Utility Network tool because it can have performance impacts. When this property is enabled, the utility network must perform one or more traces to identify which subnetworks are impacted every time you validate edits, The performance cost is relative to the size of your largest pressure zone. If you have pressure zones that contain hundreds of thousands of features (pipes, junctions, and devices), you should keep this parameter disabled. However, if all your pressure zones are smaller than that, you may consider waiting a few additional seconds during the validate network topology operation to be a relatively small cost compared to the benefits to quality assurance. If you are not sure which option to choose, you should leave this property disabled until you are confident about the size and performance impact of your pressure zones. This option should almost always be set to false for system tiers, since they can easily contain your entire dataset. Having this property enabled allows you to focus your quality assurance efforts on just the subnetworks that have been modified and allows you to easily identify which circuits have been clean and ready to be extracted to an external system like an OMS. The most performant configuration is to leave this option disabled. The configuration that is most beneficial for quality assurance and integrations is to enable state management. The last option to consider is which eventing mode to use for your default and for named versions. This decision affects the performance of update subnetwork and whether the subnetwork field is populated during certain workflows. There is a deep dive on eventing modes on the Esri Community site. A greatly simplified version of the discussion follows. When a subnetwork is updated without events it will run faster. There are several factors that contribute to this, but one reason is that attribute rules and editor tracking are not triggered. The largest drawback to this behavior is that when updating subnetwork in a named version, not all features are guaranteed to have their subnetwork name updated to match the subnetwork they belong to. When a subnetwork is updated with events, the first time the update subnetwork operation is run it will take longer than those which follow.  Subsequent updates may also take longer to update if you have attribute rules configured and are updating large numbers of features. The benefit of updating subnetworks with eventing enabled is that when update subnetwork is run in a version you can guarantee that the subnetwork name feature is correctly populated for all the features belonging to that subnetwork. Note: Starting with ArcGIS Enterprise 11.4 and ArcGIS Pro 3.4, the performance cost of triggering attribute rules can be mitigated using the new Triggering Fields behavior of attribute rules. The most performant configuration is to leave the eventing mode to update without events. The most useful configuration for quality assurance is to enable the eventing mode when in named versions and to ensure you have properly configured triggering fields on all your attribute rules. When you are working in a mobile or file geodatabase the recommended setting is to update without eventing, since this allows you the first and subsequent runs of update subnetwork to run quickly as you work through your connectivity and quality assurance issues. Enabling subnetwork controllers Once you’ve done this, the last remaining step is for you to identify the subnetwork controllers for each pressure zone in your system. To do this you will need to visit every subnetwork controller in your network (in this example regulator stations) and use the Modify Subnetwork Controller pane to associate each terminal on the device with the respective pressure zone on either side. If you have ambiguous connectivity errors on your device, you need to use the Modify Terminal Connections pane before enabling the subnetwork controller to ensure you are enabling the correct terminal as a subnetwork controller. When starting this process, it can be difficult to know where to start, since you may have dozens or hundreds of pressure zones. This is further complicated when one pressure zone is nested inside another pressure zone. The temptation is often to begin as the highest-pressure pressure zones at the center of your system and work your way outwards, the problem with this approach is that as soon as you create your first pressure zone it will consume all downstream and nested pressure zones because those subnetwork controllers haven’t been configured to regulate pressure yet. Instead, it is often easier to begin creating controllers at the edges of your system and work your way inwards. In this way you don’t need to worry about nested zones, and if you make a mistake, you likely only need to investigate the connections with one or two neighboring pressure zones. If you zoom into the regulator station on the western side of the service territory you can see that the inlet/outlet of this regulator isn’t immediately obvious. However, deciphering this can be made easier by turning on labels for the set pressures of each pipe. Similarly, it helps to look at the terminal each line is connected to. In fact, when creating subnetwork controllers, you should always verify that the terminal connections between your controller and device are correct. If they are not properly set this will cause problems when you trace your subnetworks. You can see the terminal connection using the Modify Terminal Connections pane on each connected line or, if you’re particularly clever, you can create a label class on the layer to show this information. Because you assigned a BiDirectional terminal configuration, the terminals are specified as  Side 1 and Side 2. If you had a directional terminal configuration you would want to see the 720 psi pipe on the upstream terminal and the 60 psi pipe on the downstream terminal. Now that you are confident in the terminal connections of our regulator, you can create the subnetwork(s). Use the Modify Subnetwork Controller pane on the regulator to create a subnetwork for the 60 psi pipe. Ensure the tier is set to Pressure Zone and select the terminal connected to the 60 psi pipe. Because a subnetwork can have multiple controllers, you need to uniquely identify this subnetwork controller. If you have a unique name field you can use, put it here, otherwise you can leave this blank and the system will use the feature’s global id. Finally, put the name of the pressure zone in the subnetwork name field. Once the terminal has been enabled as a subnetwork controller, you must validate the network topology for the feature before it can be used. After validating the topology, you can then use the trace tool to trace the subnetwork and validate the correct result is returned. Once you’ve verified it traces correctly, use the Update Subnetwork tool to create the subnetwork for the first time. Once the subnetwork has been successfully updated, you can use the Find Subnetworks pane to visualize the subnetwork. This pane can also be used to trace, update, and review the status of your subnetworks.   Once you’ve repeated this process for all your pressure zones you should repeat the quality assurance processes you followed for systems to ensure that every feature is associated with the correct pressure zone. Conclusion In this article you learned how to configure your utility network to model subnetwork controllers for pressure zones. You learned about the configuration required to allow a feature to be a subnetwork controller, along with how your subnetwork definitions affect the behavior of your system. Now that you’ve created pressure zones for your data you can use them for many kinds of analysis. If you’re looking for inspiration, check out the Understanding Pressure Zones article. Your configuration journey doesn’t have to end here. If you’re looking for more things to configure, consider the following: Run isolation traces using the new pressure zones Create trace configurations using your pressure zones Add summary functions to your subnetwork definitions (pressure or system tier) Review the eventing and state management configurations of your pressure tier If you want to know more about using the utility network to manage pressurized networks, like gas and water systems, please explore the Learn ArcGIS Utility Network for Water Utilities and Learn ArcGIS Utility Network for Gas and Pipeline learning series. This learning series includes tutorials and articles that demonstrate how to address the needs of these industries using the utility network. As always, if you have any questions or comments be sure to ask them in the Esri Community site! ... View more