Tips for tracing and quality assurance
- General tips
- Named Trace Configurations
- Aggregated Geometry
- Including Barriers
- Persisting trace locations
- Non-subnetwork-based tracing
- Connected Trace
- Shortest path trace
- Loops Trace
- Subnetwork-based tracing
- Update Is Connected
- Propagation
General tips
This section provides general tips about how you can use the different parameters of a trace configuration to help with analysis.
Named Trace Configurations
Named trace configurations allow you to save the parameters of a trace and re-use them. If you have a particular set of parameters that you use for quality assurance, you should consider using the Add Trace Configuration tool to create a named trace configuration to make use of them. This has several benefits:
- Allows you to perform traces using the Trace pane, which eliminates the heavy overhead of running a geoprocessing tool for tracing.
- Reduces the amount of time required spent configuring traces.
- It ensures your quality assurance processes use a consistent set of parameters.
Aggregated Geometry
The Aggregated Geometry result type allows you to store the results of your trace as an aggregated multipart geometry, with separate layers for points, lines, and polygons.
The primary use cases for using aggregated geometry when performing quality assurance are:
- Partial edges
- Nonspatial objects
- Comparing results
Partial edges
The default result type of a trace is a selection set. With the Selection result type, if a trace stops partway along the length of a line feature, the entire line feature must be selected. This can be confusing when reviewing results for datasets that have barriers that are drawn midspan on lines, or when the trace uses a function barrier that can stop the trace midspan along an edge.
Selecting the aggregated geometry result type will create an aggregated line that includes only the portion of the line that is included in the trace result. Below you can see an example of a trace (yellow) that stops after 500 feet using a selection set (top) and aggregated geometry (bottom).
Nonspatial objects
Another common use case for aggregated geometry is to represent trace results which include nonspatial objects and connectivity through junction-junction connectivity associations. In the Trace tool, if you uncheck the Include containers and Include structures options, select the Aggregated Geometry result type, and choose the Synthesize Geometries option, the aggregated geometry result will include lines representing the path of nonspatial objects and connectivity through associations.
Comparing results
When you need to compare the results from different traces, you may also want to consider using the Aggregated Geometry output. This result type is especially useful when combined with the use of named trace configurations. When configuring the output, you should consider creating layers that are named uniquely to identify the nature of the quality assurance you are performing (i.e. disconnected features).
Should you decide to store the results of multiple traces in a single layer, you should uniquely name each trace using the trace Name parameter and ensure that you check the Clear All Previous Trace Results option when you run the first trace to clear all previous results.
Including Barriers
When running a trace, it can be unclear if a trace stopped because it encountered a barrier, or because the last feature traced is not connected to anything. An easy way to determine this is to uncheck the Include Barriers Features option and re-run the trace.
You can then compare the results to identify whether the trace is stopping because of a barrier, or because it has traversed all the connected features.
If the trace is stopped because of propagation the feature stopping the trace will never be included in the result. Techniques for evaluating these conditions (sometimes referred to as propagation barriers) are described later in this article.
Persisting trace locations
If you plan to repeat the same analysis on a regular basis using the same collection of starting points, consider using the Add Trace Locations tool to persist them to a feature class in a local geodatabase. This feature class can then be used as the input to the Starting Points parameter when you run the trace tool. To create a class that contains many features, use the following instructions.
1. Use the Select by Attributes (or other tool) to create a selection of features you want to be starting points.
2. Open the Add Trace Locations tool
3. Set the Input Utility Network parameter to the utility network in the map
4. Set the Output Feature Class to a new feature class in a local geodatabase, like your project geodatabase
5. Check the box next to the Load Selected Features parameter
6. Click Run
You now have a feature class you can use as starting points when running traces
Non-subnetwork-based tracing
Now that you understand how to take advantage of some of the parameters available with the trace tool to help with quality assurance, let’s look at how specific trace types can be used to answer different questions. Some of the most common questions when performing quality assurance with tracing are:
- Is it connected?
- How is it connected?
- Are there loops?
When performing traces that do not rely on subnetwork controllers, there are a common set of constraints you need to keep in mind, and may need to work around in your trace configuration:
- If the trace does not rely on subnetwork controllers, the directionality of terminals is ignored
- Because the trace is not subnetwork aware, it will not stop at subnetwork controllers.
- By default, the trace doesn’t include any condition barriers.
You can work around some of these limitations by adding condition barriers to your trace.
Connected Trace
The most common way of answering the question “Is it connected?” is to run a connected trace. This allows you to identify all the features that are traversable from the starting point(s) of your trace. Depending on your needs, you may want to add condition barriers to match the trace configuration of your subnetwork definitions or stop at certain devices, such as devices with terminals or those that act as subnetwork controllers. Below you can see an example connected trace of a stormwater system originating at all the outfalls.
Think carefully about your data about how you will use the results before running a Connected trace. If the intention is to identify disconnected features, you should consider creating an aggregated geometry result instead of returning a selection set.
If you have multiple, disjoint systems you will need to add a starting point to each system to ensure you are capturing the correct results.
If one or more of your systems contains millions of features the trace may take some time to complete and will select many features in your map. If you have subnetworks, you may want to consider using the Update Is Connected tool discussed below.
Shortest path trace
Sometimes when troubleshooting strange behaviors with subnetwork-based traces (upstream/downstream traces or inconsistent subnetwork names) you need to understand how connectivity is made from one part of the network to another. Because the subnetworks themselves may be compromised you need to rely on a non-subnetwork based trace to answer the question. A Connected trace can show you what is traversable/connected, but the Shortest path trace can show you the specific path between two locations.
Shortest path traces are useful for identifying a path between controllers for subnetworks that are inconsistent, or when you’re trying to find the shortest path from a location to a subnetwork controller in a looped network. In these scenarios you cannot rely on a subnetwork trace to identify the path or directionality of flow in the network. However, remember that the shortest path trace, just like a connected trace, ignores the directionality of terminals because of their non-reliance on subnetworks.
An example of this is discussed in the inconsistent subnetwork names section later on in this article.
Loops Trace
One often overlooked component of quality assurance with tracing is the analysis of indeterminate flow in a network. This is often referred to as identifying loops. The Loops trace can be used to identify areas of indeterminate flow in a network. In the case of a radial network (electric, sewer, stormwater, or communication) there should be very few parts of the network that are looped. Mesh networks (gas and water) on the other hand will be predominantly looped.
Below is an example of a loop identified by running the Loops trace.
T
It is important to remember that when an upstream or downstream trace originates in, or passes through, an area of indeterminate flow in the network it can produce what appear to be incorrect results. Knowing how to run a Loops trace or exclude indeterminate flow from trace results can allow you to correctly identify an unexpected result from a test as a data quality issue instead of a configuration or software issue.
Subnetwork-based tracing
When working with a dataset that has subnetwork defined there are additional steps you should take while performing quality assurance on your data, but also additional tools and techniques you can use to assist with this validation.
Update Is Connected
The question of whether a feature is connected takes on new significance when subnetworks are defined. Each feature that can participate in a subnetwork has an IsConnected field that defines whether it is traversable to one or more subnetwork controllers. This field is used entirely for quality assurance, and its value does not control any tracing, analysis, or other behaviors in the utility network.
When a new feature is created its IsConnected field is set to Unknown, and it should be populated by either running the Update Is Connected tool (as part of an initial data load) or by running Update Subnetwork. Customers who regularly run Update Subnetwork should not worry about running Update Is Connected, since the field is being managed by Update Subnetwork for the features in each subnetwork and the Update Is Connected tool can take a long time to run.
Once the IsConnected field is well populated you can use symbology or charts to track which features are connected to a subnetwork controller (IsConnected=True) and which ones are disconnected (IsConnected=False).
An additional check you should consider doing is which features are connected to a subnetwork controller but do not belong to a subnetwork controller (IsConnected=True, SubnetworkName=Unknown). These are indications that there is a problem with the subnetworks in that area or condition barriers of your network are causing features that should be connected to a subnetwork to become disconnected. The most common cause of this is inactive features of the network causing sections of the network to become disconnected, but other common causes include devices with the improper open/closed status, flipped terminal connections, or issues with propagation (electric only).
This can be verified by running a subnetwork trace in the affected area and removing different condition barriers, running a connected trace with condition barriers present (to ignore any directional terminal issues), or by running a trace with propagation turned off. Overriding the propagation behaviors in a subnetwork definition requires additional steps and is described below.
Propagation
Within the context of subnetworks and tracing, propagators are used to calculate a value along each path of a subnetwork using an attribute, function, and comparison operator. When the conditions of the propagator are no longer satisfied (no valid phasing, not enough pressure, etc) then the feature that failed the condition, and all subsequent features traversable from that location, are not considered part of the subnetwork.
This means that when a subnetwork belongs to a tier that is configured for propagation, and features within a subnetwork are de-energized, one of the checks that can be performed is whether or not some or all of the de-energized features are caused by propagation.
Additionally, in some cases a tier may not be configured for propagation yet because of data quality concerns, but you may want to see what the effect of enabling propagation would be without needing to maintain two databases with separate subnetwork definitions.
The solution to both problems is to configure a trace (or create a trace configuration) that overrides the propagators on the trace configuration associated with a tier in your network. Using this technique, you can confirm whether propagation is causing features to be disconnected from their subnetwork, or you can look at the effects propagation would have on the number of features connected to each subnetwork.
The challenge with implementing this is that the Propagators parameter is not exposed via the user interface. The solution to this problem is to use Model Builder, Python, or ArcGIS Pro SDK to run a trace (or create a trace configuration) with the desired propagation configuration. Below you can see an example of the propagators defined in the Electric Utility Network Foundation:
To run the trace without Propagators using Python, you would copy the snippet of the trace and remove the parameter associated with the Propagators:
Another option would be to create a Model for the trace, expose the Propagators as a parameter, and remove the value.
Likewise, if the trace configuration doesn’t contain Propagators, you can use the same techniques to run a trace that includes Propagators. Using Python you can set the parameter to be: “'E:Phases Propagated' 'E:Phases Substitution' PROPAGATED_BITWISE_AND INCLUDES_ANY 7” (replacing the field and network attributes with your own values).
You could also use Model Builder to populate the Propagators parameter with your own configuration:
For the sake of convenience, you should save these trace configurations in the database so you can easily compare your trace results with Propagators and without Propagators. Refer to the section on Aggregated Geometry earlier in this article for effective techniques for comparing trace results.
