While attending the 2017 GeoConX conference and after presenting on both implementing the Utility Pipeline Data Model (2016) UPDM for the geometric network and the gas utility network, I wanted to share insights, graphics, and guidance to pipeline operators considering their own GIS implementations now and in the future.
In this post, I’m discussing the Esri geodatabase UDPM designed for hazardous fluids and liquids such as oil, water or natural gas. In continuing my pursuit of understanding Esri’s new Network Management Extension, I also delve into relating the UPDM 2016 version to the 2017 version in Network Management’s. This Esri geodatabase template solves the gas/water/hazardous liquid pipeline operator’s regulatory requirement to track and trace integrity, inspections, and maintenance of the pipe. However, when your GIS department considers upgrading its system to Network Management, you might be asked: “Why not transition to the utility network now?” Let me answer that question.
Considering Upgrading to a Utility Network in Network Management?
Moving to Esri’s Network Management Extension to use utility networks is a complete paradigm shift spinning in new architecture, workflows, and data structures. You really need to plan all pieces of the puzzle for upgrading your existing infrastructure, applications, and data models to handle the new requirements for the ArcGIS Enterprise platform of ArcGIS' Server, Portal, Web Adapters, Data Stores, and Pro. To set the stage for upgrading to Network Management, you also need to understand the basic technical functionality available through the new Network Management Extension. All editing will be completed through feature services published to your Portal for ArcGIS (or ArcGIS Online in future releases).
This simple graphic depicts the existing geometric network for the 2016 UPDM and how it reduces into the new utility network (a compressor would be the better analogy, but I'd already added the reducer...oh well). Please note that the below graphic of the utility network structure is NOT the final UPDM 2017 model. It's only a sample provided with the latest Network Management Beta, with customizations I added. I wanted to share my insights and graphics for explaining the UPDM migration.
Benefits of the new model with Network Management
At a high level, here are the basics of Network Management and the new utility network model:
- Improved Performance: Having a compressed model means fewer database queries.
- Improved Connectivity Model: Having physical and logical associations such as Junction-Junction (new), Junction-Edge, Edge-Junction-Edge enforces specific connections offers the precision of defining exact connection types. In addition, the model has required subtypes (in the AssetGroup field) with associated domains (in the field AssetType field), which are unique combinations of connections configured to define how features connect.
- Containment Model: The Assembly and StructureBoundary feature classes can contain other equipment for simpler map display and easily viewing the internals of a system or structure, such as a regulator station.
- Improved Transaction Versioning: The previous Adds and Deletes tables are gone and replaced with only one child version off of the DEFAULT, which uses time stamps in place of the state model. See my post on workflows related to the new versioning system.
- Improved Data Quality: Out-of-the-box validation tools in ArcGIS Pro with the utility network allow you to validate all new feature edits against connectivity (both logical, attribute, and geometric connectivity) rules.
- Out-of-the-Box Tracing: There are six default traces with immensely powerful options for calculations, filtering, summarizing, setting where traces stop to replace your previous customized tracing options in the geometric network with configured trace options in the utility network. I talk more about these trace options in my Tracing the Utility Network post.
- Schematic Diagrams: Taking the trace results, you can build schematic diagrams that represent logical associations and connections for designing and conceptualization of the network. This replaces ArcMap’s Schematics extension.
Utility Pipeline Data Model (2016 & 2017 version) Basics
In its current beta, Esri is still adding all components to the Network Management Extension for tracking and traceability. However, the latest beta release doesn’t account for all the regulatory requirements of the tracking and traceability through the distribution and transmission pipeline integrity yet. I’m excited after speaking with Esri experts and gas gurus that in upcoming releases, the UPDM 2017 (utility network version) will provide the similar and improved functionality that the UPDM 2016 model currently provides in the geometric network in ArcMap, except in the utility network in ArcGIS Pro!
The UPDM 2016 is the current release of an Esri schema database template for utilities on ArcMap 10.2.1 through 10.5+. It’s designed for utilities that manage any liquids or gases transported through pipelines. Esri has released similar database models as templates for the last 15 years for free. Such organizations include utilities, government agencies (federal, state, tribal, municipal, and local), businesses, and universities.
The UPDM 2016 contains structures, rules, connectivity, feature classes, fields, domains, and subtypes that industry experts from the oil, gas, water, and other hazardous liquid industry have provided best practices for data structures and modeling pipeline-related utility systems. It’s built on modeling and equipment standards and industry knowledge foundations. The UPDM models each piece of physical equipment within a pipe geometric network. For example, gas, liquid, or water systems are modeled from wellhead to a customer meter (for gas) or from the wellhead to water terminal or delivery point (for water).
The UPDM 2017 model, although not yet released, will provide similar functionality and connectivity rules as the 2016 version except with more connectivity rules and new containment and attribute rules. I'll touch on this in a later post, but rest assured it's coming down the proverbial pipeline soon! For utilities choosing to upgrade to the Network Management Extension in the near future, it might make sense to upgrade their data first to the UPDM 2016 if they need a solution NOW. Otherwise, you could consider postponing upgrades until the UPDM 2017 is ready for prime-time. Let RAMTeCH be your guide with any of these decisions.
UPDM Tracks Integrity
The US Dept. of Transportation-Pipeline and Hazardous Material Safety Administration (PHMSA) mandates that pipeline operators monitor the condition of the pipes (i.e., ensure its integrity). This includes both gas and hazardous liquid transportation in distribution and transmission pipes (1, 2). Tracking pipeline integrity adds another layer to utilities for federal regulatory requirements. For instance, gas distribution utilities are required to ensure gas reaches customers at certain flow rates and that the pressure stays below the maximum allowable operating pressure (MAOP) for that pipe segment.
These requirements can be tracked in the UPDM as out-of-the-box (OOTB) functionality. Generally, the UPDM structures inspection and integrity management feature classes and tables for managing inspections, surveys, leaks, maintenance schedules, and any consequence areas affected by leaks. Specifically, feature classes are grouped as System Tables, Physical Components, Assemblies, Flow Control Devices, Inline Junction Devices, Structures, Cathodic Protection Systems, Legacy Abandoned Assets, and Activity and Integrity. Physical equipment feature class groups encompass all lines, point controllable and non-controllable devices and other asset components. These lines, devices, and structure features naturally migrate into the Network Management utility network structure.
UPDM Has Linear Referencing Using Stationing
Another reason to move to the UPDM takes aim at pipeline operators. Those who historically managed asset anomalies during pigging runs with stationing can continue managing their assets using linear referencing provided through ArcGIS Pipeline Referencing (APR) tools. Linear referencing allows operators to use stationing of equipment to measure reference locations when specific events occur, with or without GPS locations to verify location accuracy of equipment or events.
How Do Pipeline Models Compare?
Most GIS Managers might be thinking, “My existing model works great in [APDM, Smallworld, PODS, PODS Spatial…etc].” As an answer to that question: yes, existing models such as the APDM, GE’s Smallworld, or PODS (Pipeline Open Data Standard) account for this functionality too. However, the UPDM provides the comprehensive Esri solution for all systems and apps for all systems of gathering from the field to high pressure transmission down to distribution to your customers. PODS does not account for the system (transmission or distribution) in which they live and only manages the transmission side of data. On the upside though (and benefit to those using the APDM now), the APDM is used as a basis for the UPDM (3). GE’s Smallworld environment and tools provide a great solution as a completely stand-alone solution. I love using these too! It still holds its own for developing the containment model years ago! The downside to Smallworld is in its lack of crosswalk to or simple integration with existing Esri apps, maps, or data. Yet, these non-UPDM solutions require that pesky customization, which again, leads to maintenance nightmares for future updates, upgrades and knowledge transfer. They lack OOTB integration with Esri’s plethora of apps, code, services, and support. They also lack the scalability that was built into the UPDM. In other words, no matter what size company, from a few thousand meters, up to 3 million meters, this model handles your data and applications.
What’s the Migration to Network Management for the UPDM?
I’m not going into details in this post about the migration pattern for going from UPDM 2016 to UPDM 2017 (more to come later on migration!), but here’s a sample graphic of understanding of which UPDM 2016 feature classes need to migrate into the UPDM 2017 utility network feature classes going forward. Each AssetGroup (the new subtype) is represented with colors for each UPDM 2016 feature class that migrates into the utility network feature class. I call out one example of an AssetType (domain associated with each AssetGroup) as an example.
Wrapping UpThe UPDM offers the comprehensive solution to managing liquid and gas in distribution or transmission systems. Operators will easily meet the demands of the DIMP and TIMP regulatory standards for tracking pipeline integrity without exhaustive customizations to extend the UPDM’s OOTB data model schema. Pipeline operators have options to measure distance and events using linear referencing and location.
As always, let me know if you have questions I can help answer. We're all learning the new functionality and awesomeness with the new network. See how RAMTeCH can help you test it out and plan for an implementation in the future!
1. US Dept. of Transportation. (2017) "Pipeline and Hazardous Material Safety Administration." (2017). https://www.phmsa.dot.gov/
2. US Dept. of Transportation Pipeline and Hazardous Material Safety Administration. (2017). “Pipeline Technical Resources.” https://primis.phmsa.dot.gov/dimp/resources.htm.
3. Wiley, Clarke. (2016, January 10). “What Are The Pipeline Transmission Database Options?” http://sspinnovations.com/blog/2016/01/10/what-are-pipeline-transmission-database-options#.WWTZTIjyvmE. (Thanks for your help Clarke!)