Latest Contributions by TomDeWitte

Finding those key buried devices and paths   By Tom Coolidge and Tom DeWitte A gas utility or pipeline typically transports natural gas or hazardous liquids to customers through a large and complex network of interconnected pipes.  In addition to pipe, these networks are comprised of an even larger number of other components, including fittings, valves, regulators and many more, some of which can affect the flow of the fluid through the pipes.  Modeled properly, ArcGIS enables you to create a “digital twin” of all this complexity.  This is key as many solutions require that you be able to determine a path directionally from a location in your connected network to a separator or separators that bound it.  The utility network provides this capability. It all starts with location.  I find that as I get older, I am more frequently asking myself questions such as; where did I leave my glasses, or where is my phone.  Resolving these questions usually entails me wandering about the house until I find those misplaced glasses or phone.  Finding these items is not that difficult because I can see my glasses sitting on a table or I can see my phone as it sits on the kitchen counter where I left it. Now imagine you work for a natural gas or hazardous liquids pipe organization, and all of the assets you are looking for are buried three or more feet below the surface.  How do you go about finding a specific valve, fitting or cathodic protection anode?  The short answer is maps.  But, maps like traditional paper maps have their limitations in that when looking for a specific valve you must have a pretty good idea of where the valve is located in order to know what map sheet to look at, and where on that busy map sheet to look. Digital maps are better, in that they allow you to search for a characteristic of the valve such as its assetID, manufacturer, size or type.  But, a digital map also assumes you have some knowledge already about the valve you are looking for. So, what do you do when your question is about the pipe network, and how a specific asset participates in the pipe network?  This is where tools which understand how the assets connect to form the pipe network are required.  This is where you need tracing tools to know your pipe system. What type of questions can be answered with a trace? When managing a pipe system there are many questions that get asked everyday which require an understanding of how the pipe system works.  During an emergency, a very common and important question is: what valves do I need to close to isolate a section of the pipe network where damage or a leak has occurred? A common question asked by cathodic protection technicians is where is the nearest CP test point from my current location on the pipe system? Gas engineers who are evaluating a pressure zone ask the question; what are the regulator stations providing gas to this location? What do I need to do to configure my gas system for tracing? For a software system to be able to answer these common types of pipe system questions, an understanding of how the components of a pipe system connect is required.  It is not enough to simply draw a digital representation of the asset on a map, such as is commonly done with CAD software. In addition to drawing the digital representation of the asset on a map, there also needs to be an understanding that the two polyethylene pipe segments which have been butt fusioned together are connected.   This software understanding of connectivity is network topology.  Within the Esri ArcGIS platform, our latest version of network topology for utility systems is what we call the utility network. Can I perform a trace in ArcGIS Pro? Yes. Tracing your network can be performed within ArcGIS Pro version 2.1 or later.  Additionally, with the utility network being a service based solution, tracing can also be done with web applications, and eventually will be able to be performed by mobile applications. Within ArcGIS Pro, the options for configuring a trace have been significantly enhanced when compared to the ArcMap geometric network tools.  It is now possible to dynamically answer questions by simple configuration of the properties of the trace tool.  For example, if you are trying to determine the amount of gas or liquid lost due to a break in the pipe, you need to know the volume of the portion of the pipe network which was isolated.  There is now a function property to the trace tool to allow you to summarize the total pipe volume of the trace selected pipe segments. If you need to ask the question, what portion of my pipe system is upstream of a specified location, but only trace on those assets which are in production, and are open to allow the gas or liquid to pass through.  The ArcGIS Pro trace tool now supports the ability to use designated asset attributes such as LifeCycleStatus, DeviceStatus, Pincheable, and Insulator Device to dynamically constrain which assets the trace can traverse. This, too, is a simple configuration of the tools parameters. Since the trace tool is a geoprocessing tool, your preferred configuration properties can be saved as a model and shared across the organization. How do I configure a trace to find the nearest asset? Being able to find the nearest type of asset such as a regulator, valve, or CP test point, is another useful new addition to the capability of the trace tool. Simply checking a box within the filter options will constrain the trace output to the specified features which are closest based on the distance traversed across the pipe network. How do I configure the trace tool to find the sources feeding a gas subsystem? The new trace tool within ArcGIS Pro contains some new trace options, such as subnetwork, subnetwork controller, shortest path, and loops.  When a planner or engineer needs to find the regulators feeding a specified location, the subnetwork controller option makes this an easy question to ask of the pipe network. Tracing with the new utility network solution provided by Esri, is unique in its ability to allow gas and hazardous liquids pipe companies to easily ask questions of their pipe networks.  Databases alone cannot answer these questions.  CAD systems cannot answer these questions. Even GIS systems which do not include network topology cannot answer these questions.  Only a complete GIS system which includes network topology can answer these everyday questions about your pipe network. Only a network topology specifically built for management of utility systems such as a gas or hazardous liquids pipe network can provide the intelligent tools to help you know your system. PLEASE NOTE: The postings on this site are my own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

This is the official release of the 2018 Edition of the Utility and Pipeline Data Model (UPDM).  It is designed to support data management for the natural gas and hazardous liquids industries.  Supporting the diverse needs of these industries means supporting multiple implementation patterns.  Specifically a network topology with the Utility Network implementation pattern, and a linear referencing implementation pattern with the ArcGIS Pipeline Referencing solution.. If you have questions, please post them to geonet, so everyone can see and share the information. Thank you Tom DeWitte Esri Technical Lead – Natural Gas Industry ... View more

Gas Network Complex Facilities Can Be More Fully Defined in the Utility Network By Tom Coolidge and Tom DeWitte   What if it is really greater goodness that is to be found in having the details of your complex gas network components consolidated in one system of record along with other network data?  Think about the improvements to efficiency and productivity that can result when you have all needed data organized in one place and included in your network models.  Yes, it’s your angel that we believe lies ahead in the consolidated details.  That angel can help you more fully realize the benefits available to you by unlocking the potential of the ArcGIS platform. Bringing data from disparate sources together on a common geographic basis for visualization and analysis is one of the hallmarks of the Web GIS pattern exemplified by the capabilities of the ArcGIS platform.  Increasingly, though, while this advance means that they do not have to, many gas utilities are looking to move a broader range of data now stored elsewhere into today’s modern ArcGIS platform. Historically, data has been siloed in different systems for many reasons.  In the case of detailed data needed to define complex gas components, one of those is that GIS did not provide the functionality needed to define and store that detailed data.  That is changing now.  This has many benefits for gas utilities.  Among those are making it easier to see a holistic view of the network in whatever level of detail is desired and to analyze it, and facilitating interoperation with other application software that rely upon a published definition of the detailed network.  At the same time, it reduces the total cost of ownership by consolidating into one system of record what previously was in multiple systems.  In a sense, this advance brings GIS closer to operating just as you operate. That brings us to the Utility Network.  One of the Utility Network capabilities getting the most buzz is the capability to more fully define and manage, in real geographic space, the design and operational details of complex gas components, such as regulator stations and compressor stations. The Utility Network opens up new capabilities in leveraging these details to better understand and operate your gas system. Rest Easy Before we explore this new capability, let us quickly emphasize that the capability to more fully define complex components does not mean that you ever must!  So, rest easy.  The choice is yours as to whether you elect to take advantage of the new capability and, if you do, when and to what extent.  Moreover, if you do elect to add more detail, you can do it incrementally if you wish, rather than in one big project. Before Now Before the release of the Utility Network, complex gas components typically were represented in ArcGIS as a point feature.  Historically, CAD software often has been used to create precision drawings of gas network complex components.  With CAD, what you see is basically what you get – just a picture, not usable data.  ArcGIS delivers much more.  You still can see the same thing, but what you see is just a representation of the data behind it which you can use in many powerful ways. Now instead of looking at a picture of the internals of a gas facilities you can interact and ask questions of the gas facility details.  What kind of questions might you be asking of the details?  In an emergency operation instead of an isolation trace stopping at the simple representation of a gas facility such as a regulator station, it can now identify the specific critical valves within the facility which need to be closed.  If a recall is issued for a specific manufacturer device or fitting, and that device or fitting was installed within a gas facility, those gas facility components can now be identified and reported.  Answering these types of questions is not possible when the gas facility internals are just a picture. How Much Detail We increasingly are asked how detailed should the definition in ArcGIS be of a gas network? There is no one answer to that question.  With the ArcGIS platform increasingly supporting the mapping and spatial analytics needs of a growing number of users in a broader range of gas utility functional areas and roles, the answer to that question is evolving. Here is one way to look at it.  The answer to what needs to go into your geodatabase largely depends on what you want to get out of it.  That is because each software application has its own specific data requirements.  If you want your geodatabase to support one application, then only that application’s data requirements need to be accommodated.  If you want your geodatabase to support two applications, then the data requirements of both need to be accommodated.  And, so on.  While some applications share data requirements, generally as the number of applications to be supported increases, so, too, does the breadth and depth of data requirements.  The key to your answer lies in understanding the number of applications to be supported and their combined data requirements. Remember the title of this blog, “Your Angel Is In The Details?”  As a rule, erring on the side of more detail is a good thing.  It is easier to simplify a more detailed definition than it is to add detail to less detailed one. An Example One of the functional areas now more fully exploiting ArcGIS capabilities is gas operations. Analytics for gas operations often require more granular data than analytics for other functional areas. Let’s consider regulator stations.  Gas networks typically include multiple sub-networks, each operating at up to a different maximum pressure, with pipe sizes and maximum pressures reducing the closer gas gets to delivery points.  Regulators control the safe reduction of pressure or flow from a higher pressure sub-network to a lower one.  A regulator station can be simple, with a single regulator on a single path.  Or it can very complex, with multiple regulators and other devices on multiple paths.  It also likely contains safety devices.  These safety devices may include additional regulators, relief valves, and remote monitoring equipment. In ArcGIS, a regulator station traditionally has been defined as a point feature.  In reality, a regulator station is a complex facility.  Now, in the Utility Network you can define those design and operational details.      Technical Discussion                 Since gas systems were originally mapped in a GIS several decades ago, GIS professionals have struggled to get the balance right between needed detail and desired cartography.  One example of this struggle is the need to manage geographically condensed features like those contained within a gas facility. These details can create very cluttered and hard-to-understand map displays. As already stated, these details need to be more than just a picture.  They need to be asset records.  These asset records need to be spatially reportable so gas companies know where gas devices and fittings are located.  Then, add to these gas facility data management requirements the gas operations requirement that these gas facility details be traceable.  This is to aid gas ops staff during emergency operations to not only know that a valve in a gas facility needs to be closed, but to identify which critical valve(s) in the gas facility need to be closed.  The solution to these problems is the new Utility Network and its container capability.          So what is a container A container is an association between the individual features representing the assets internal to a gas facility with the single point feature representation of the gas facility. Once the container association has been established, the contained assets are hidden from the standard map display.  Similar to the legacy picture representation, users are able to click on the simple gas facility representation and see the internals of the gas facility in a separate map window.              What reporting can I do                 The assets contained within a container are geospatial features stored in geodatabase featureclasses.  Standard database reporting tools, whether ArcGIS-based or Business Intelligence-based can be used to query, summarize and report these on features.  But, what about spatially querying these featureclasses with a standard ArcGIS tool like “Select Features by Location”? This is an additional type of supported reporting, because the Utility Network provides the ability to precisely place the internal assets at their true geographic location within the gas facility.  This is a key point, so let me repeat.  Internal assets are placed at their true geographic location!          Can I trace these gas facility assets With the Utility Network, all container contained assets participate in the overall pipe system’s network topology.  This means, for instance, that during a gas emergency operation, an isolation trace task can be performed to identify the critical valves within the gas facility which need to be closed for the emergency.  This improvement in modeling complex gas facilities additionally provides a better understanding of cathodic protection areas, pressure zones, and system zones.                 The Utility Network Management extension container capabilities provides a solution to the gas industries growing needs for better management of the details of gas facilities.  This ability to manage gas facility internal assets as features instead of pictures, allows gas organizations to provide clear and concise maps, without sacrificing the ability to model individual assets. Containers provide the angel that gas organizations have been looking for to solve the problem of managing a gas facilities details.     PLEASE NOTE: The postings on this site are my own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

For the latest updates to the UPDM 2016 version, please download this release.  The zip file includes: Change Log Data Dictionary File Geodatabase of the UPDM schema To learn more about the Utility and Pipeline Data Model, please reference the ArcGIS for Gas solution site. ... View more