Skip navigation
All Places > Electric and Gas > Blog
1 2 Previous Next

Electric and Gas

24 posts

By Tom Coolidge and Tom DeWitte

 

Today’s Collector and ArcGIS Enterprise provide new enhancements and capabilities. These enhancements include; improved user interface, better GPS antenna support, direct capture of barcode via the mobile device camera, and allow for a more streamlined workflow for field users.  In addition to those important enhancements, the enterprise geodatabase capability of attribute rules allows for the automatic decoding of the barcode and the derived barcode data to be automatically written to the appropriate attributes.  This automatic decoding and attribute population provides significant productivity gains for field users and allows for a simpler deployment pattern for administrators. In this blog we will take a deeper dive into how to configure and deploy the ArcGIS platform and collector to address the industry need of Tracking and Traceability.

 

For an introductory explanation of how the ArcGIS platform addresses Tracking and Traceabiliy, please read the first blog of this 2 blog series: Tracking & Traceability – Part 1

 

Like any good recipe for success, we need to know the required ingredients.  The Tracking and Traceability solution requires the following software:

  • Collector for ArcGIS
  • ArcGIS Enterprise 10.6.1 or higher

 

Additionally, we will need arcade scripts which provide the logic of how to decode the ASTM F2897 barcode 16-character string and use the derived data to automatically populate the appropriate attributes.

 

Though not required, most deployments also include a GPS Antenna to improve spatial accuracy.  

 

The Basic deployment steps

Deploying the ArcGIS Platform to meet the needs of Tracking and Traceability can be broken down into 5 steps.  These steps are:

  1. Preparing the enterprise geodatabase
  2. Creation of staging geodatabase layers
  3. Application of attribute rules
  4. Publication of staging geodatabase layers as a feature service
  5. Creation of web map for Collector

 

The overall data flow process for Tracking and Traceability is to have Collector post the field collected features directly to the staging geodatabase.  There is NO translation or conversion of the field collected data.  Once the field collected data is submitted to the staging geodatabase a GIS mapping technician can review the new features and append them into the enterprise geodatabase.

Preparing the Enterprise Geodatabase

The first step to setting up this workflow is ensuring your Enterprise Geodatabase has the required feature classes, feature class attributes and coded value domains to store the information collected in the field.

 

If you are starting a new enterprise geodatabase, it is recommended that you use the Esri provided pipe system data model called Utility and Pipeline Data Model (UPDM). The 2019 edition of UPDM includes everything needed to store the information collected in the field. You can download this data model with this link:

 UPDM 2019 Edition download

 

 If you have an existing enterprise geodatabase, then you need to make sure the asset feature classes have the correct attributes to store the field collected data. Examples of assets captured by field staff include, fittings, valves, and pipe segments. Here is a specific listing of the minimally required attributes:

 

Point Asset Featureclasses

Field Name

Field Definition

Coded Value Domain

barcode

Text(16)

 

manufacturer

Text(2)

Pipeline_ASTM_Manufacturer

manufacturerlotno

Long Integer

 

manufacturedate

Date

 

manufacturecomponent

Text(2)

Pipeline_ASTM_Manufacture Component

material

Text(2)

Pipeline_ASTM_Material

diameter

Double

Pipeline_Fitting_Diameter

diameter2

Double

Pipeline_Fitting_Diameter

wallthickness

Double

 

wallthickness2

Double

 

 

Line Asset Featureclasses

Field Name

Field Definition

Coded Value Domain

barcode

Text(16)

 

manufacturer

Text(2)

Pipeline_ASTM_Manufacturer

manufacturerlotno

Long Integer

 

manufacturedate

Date

 

manufacturecomponent

Text(2)

Pipeline_ASTM_Pipe_Manufacture Component

Material

Text(2)

Pipeline_ASTM_Material

nominaldiameter

Double

Pipeline_Pipe_Diameter

wallthickness

Double

Pipeline_Pipe_Wall Thickness

 

After your Enterprise Geodatabase is ready to accept the decoded barcode values and the appropriate ASTM F2897 coded value domains have been assigned, you are ready to create the staging geodatabase.

 

Creating the Staging GDB

This step involves setting up your staging geodatabase layers.  These layers should be a schema duplicate of the enterprise geodatabase asset layers. Being a schema duplicate will simplify the appending of data from the staging geodatabase to the enterprise geodatabase.

 

The simplest approach to setting up the staging geodatabase is to create schema duplicate feature classes in the enterprise geodatabase.  I recommend creating a new feature dataset to store these duplicate layers.  If using the UPDM 2019 edition data model the feature classes to duplicate are:

  • PipelineDevice
  • PipelineJunction
  • PipelineLine

To help keep the staging layers uniquely separate from the production layers I like to rename the layers as follows:

  • StagingDevice
  • StagingJunction
  • StagingLine

These duplicate layers should not have any features/records.

 

To properly support disconnected field capabilities, you should use the “Add GlobalID” tool to add a GlobalID field to every staging feature class.

 

Additionally, though not required, it is recommended that you enable “Editor Tracking” to allow all edits to have a date/time stamp and the ArcGIS platform user ID of who created and last updated the feature/record.

 

A final step not to be overlooked is to decide whether you want to include photos as part of the new construction data collection process.  It the answer is “yes” then remember to “Enable Attachments” for each of the layers you want to have field staff capturing photos.

 

With the staging geodatabase layers now created it is time for attribute rules.

 

Application of attribute rules

With ArcGIS Enterprise 10.6.1 the attribute rule capability has evolved to provide a robust automation capability for managing attributes. For Tracking and Traceability, attribute rules provide the ability to automatically read the barcode value, decode the barcode and automatically populate the derived attribute fields (manufacturer, manufacture lot #, manufacture component type, manufacture date, material, diameter, and wall thickness). When this capability is applied to the staging geodatabase layers, the auto-population occurs when Collector submits the new feature.  This means a connected mobile device running Collector to capture new construction will be able to see the decoded information while they are documenting the new assets in the field.

 

The following link provides the arcade attribute rule scripts and detailed documentation on how to apply them.

ASTM F2897 barcode decode attribute rules 

 

The way attribute rules work is to assign them to a single attribute field. This means the decoding of the barcode is broken out into 9 separate arcade scripts.  Here is a breakdown of how the arcade scripts are applied to the staging geodatabase layers. 

 

StagingDevice Featureclass

Attribute Fields

Arcade attribute rule script

manufacturer

Device_Manufacturer.txt

manufacturerlotno

Device_Manufacturelotno.txt

manufacturedate

Device_ManufactureDate.txt

manufacturecomponent

Device_ManufactureModel.txt

material

Device_Material.txt

diameter

Device_Diameter.txt

diameter2

Device_Diameter2.txt

wallthickness

Device_Wallthickness.txt

wallthickness2

Device_Wallthickness2.txt

  

StagingJunction Featureclass

Attribute Fields

Arcade attribute rule script

manufacturer

Junction_Manufacturer.txt

manufacturerlotno

Junction_Manufacturelotno.txt

manufacturedate

Junction_ManufactureDate.txt

manufacturecomponent

Junction_ManufactureModel.txt

material

Junction_Material.txt

diameter

Junction_Diameter.txt

diameter2

Junction_Diameter2.txt

wallthickness

Junction_Wallthickness.txt

wallthickness2

Junction_Wallthickness2.txt

 

StagingLine Featureclass

Attribute Fields

Arcade attribute rule script

manufacturer

Line_Manufacturer.txt

manufacturerlotno

Line_Manufacturelotno.txt

manufacturedate

Line_ManufactureDate.txt

manufacturecomponent

Line_ManufactureComponent.txt

material

Line_Material.txt

nominaldiameter

Line_NominalDiameter.txt

wallthickness

Line_Wallthickness.txt

 

Once the attribute rules are successfully applied to your enterprise geodatabase staging layers you are ready to publish the staging layers as a feature service.

 

Publication of staging geodatabase layers as a feature service

Publishing the staging layers from ArcGIS Pro is a very straight forward process. The steps are as follows:

  1. Create a new Map
  2. Add staging gdb layers to map
  3. Symbolize layers as desired
  4. Publish map as a feature service

After the map is created and the staging geodatabase layers are added to you map you will have a ArcGIS Pro map which looks like the following:

I find using ArcPro for defining the symbology to be easier and quicker than using the ArcGIS Enterprise Portal map viewer tools.  Additionally, I can use more advanced symbology such as the UPDM2019_Symbols style set that is included in the UPDM 2019 Edition download.  When the layers are symbolized as desired, remove the basemaps and prepare to publish.

To publish the staging layers as a feature service, use the sharing ribbons’ web layer – Publish Web Layer tool to create the feature service.

With the feature service now published your staging geodatabase layers are ready for the final step which is to create the web map for Collector.

 

Creation of web map for Collector

Creating a web map for Collector is the opportunity to fine tune the interface your field staff will use for documenting the new construction.  Items to think about when creating the web map are:

  • Scale Constraints of layers
  • Which data fields will be exposed to the field staff
    • Which fields will be exposed during editing
    • Which field will be exposed during viewing

Both the ArcGIS Enterprise portal map viewer or the ArcGIS Pro desktop tool can be used to accomplish this task.

 

When the web map is defined and saved you are now ready to take Collector to the field to being collecting your new gas pipe construction.

 

Summary

With the latest enhancements to Collector and the new attribute rule capability for enterprise geodatabases. Deploying the ArcGIS platform to address the needs of tracking and traceability is easier than ever. Five basic steps  are all that it takes to enable your field staff to efficiently capture new construction digitally and retire the time consuming and inefficient historical paper based process.

  1. Preparing the enterprise geodatabase
  2. Creation of staging geodatabase layers
  3. Application of attribute rules
  4. Publication of staging geodatabase layers as a feature service
  5. Creation of web map for Collector

 

PLEASE NOTE: The postings on this site are my own and don’t necessarily represent Esri’s position, strategies, or opinions

By Tom Coolidge and Tom DeWitte

Tracking and Traceability is now a well-established practice in the natural gas distribution industry supported by ArcGIS®.

 

ArcGIS mobile app advances over the last three years have helped adoption of Tracking and Traceability activity grow. Collector for ArcGIS has evolved to now include the ability to use a mobile device’s camera to read the ASTM F2897 barcode. Collector also now includes the capability to run arcade scripts in the pop-up window while the device is disconnected from the network.  Not to be overlooked, Esri also released a new enterprise geodatabase capability called attribute rules.

 

Those three new capabilities have enabled many gas utilities, and increasingly gas pipe installation contractors; to use Collector to capture the location, barcode, and other information about the newly-installed pipe and its related components. These new capabilities and lessons learned from the many organizations actively using Collector for the digital as-builting portion of the Tracking and Traceability workflow have resulted in a more efficient and streamlined process for performing these tasks.

 

The purpose of this blog is to give an overview of how the current version of Collector, when combined with an ArcGIS 10.7 or higher enterprise geodatabase, can result in a simpler and more efficient Tracking and Traceability workflow. A second blog article will follow with a detailed explanation of the new attribute rule arcade scripts which completely automate the decoding of the ASTM F2897 barcode and the automatic population of the derived attributes.

 

A quick review of Tracking and Traceability

PHMSA proposed rules in May of 2015 to 49 CFR part 192 to address the need for operators to better ‘track’ the details and location of assets after their delivery from the manufacturer or supplier.  The rule also speaks to the need for better ‘traceability’ of assets; meaning the ability to locate assets by material, size, manufacturer, model, or other attribute.

 

The ASTM F2897 standard, developed collaboratively by the natural gas industry and its leading suppliers, specifies a 16-digit alphanumeric barcode format that embodies identification of a pipeline component’s manufacturer, lot number, production date, model, material, diameter, and wall thickness.  This barcode standard is now a common piece of the manufacturer provided information for plastic pipe and its plastic components.  Additional efforts spearheaded by the Gas Technology Institute are currently underway to define a more advanced barcode standard which can be applied to both steel and plastic pipe and their components.  This barcode “thing” is not going away.  Just the opposite, it is going to expand significantly in the years to come.

 

Pattern Overview

The ArcGIS deployment pattern for Tracking and Traceability is comprised of four steps, as illustrated here:

 

 

Step 1: Digital as-builting

The recent improvements to Collector have made this process easier than it was just a few years ago.  The first enhancement was the revamping of the interface to simplify data entry. The second enhancement was to increase the certification of GPS vendors and their devices. Here is a link to the list of GPS receivers which can be used with Collector: https://doc.arcgis.com/en/collector/ipad/help/high-accuracy-prep.htm

The third enhancement is the native ability of Collector to use the mobile device’s camera to capture the ASTM F2897 barcode.

 

With these enhancements, field staff can go into the field and capture the as-built information of the new construction using a smart device running Collector. The smart device is Bluetooth-connected to a high precision GPS antenna.  The field staff use the high accuracy GPS antenna to capture the location of the newly installed assets. The collected location data is directly streamed into Collector as native ArcGIS features.  No translation or conversation is required.  The field staff then manually input into Collector a minimal amount of information, such as Installation Date, and installation method.  The field staff then uses the device’s camera to capture the barcode and automatically populate the BARCODE attribute of the GIS feature.  The BARCODE value contains information about the asset, such as size, material, manufacturer and manufacture date.  Once the BARCODE value is captured, the field staff no longer need to manually enter this information.

 

The recent enhancement to Collector supporting the ability to run arcade scripts in the pop-up window, provides the ability to immediately display the decoded data to the field staff even when the device is disconnected.

 

An additional capability of an Esri mobile app on a smart device or tablet is the ability to capture photos of the newly installed assets.  These photos are automatically associated to the GIS feature.

 

When the field staff have completed the collection of the newly installed assets, the GIS features are submitted to the staging geodatabase.

 

Step 2: Contractor/crew assessable storage

A fundamental challenge of Tracking and Traceability is how to correctly integrate high precision GPS geospatial data, with less accurate legacy geospatial data.  A key component to overcoming this challenge is the staging geodatabase.  A staging geodatabase can be either hosted in ArcGIS Online as hosted feature layers or stored on premise with a local ArcGIS Enterprise implementation. The key purpose of the staging geodatabase is to provide an easily accessible data repository for the field crews to submit their collected construction information too.  The staging geodatabase only holds the newly collected construction information.  The construction data sits in the staging geodatabase until a mapping professional using ArcGIS Desktop accesses and downloads it to the enterprise geodatabase.

 

With the new enterprise geodatabase capability of attribute rules, it is possible to have the captured barcode value automatically read and used to auto-populate the derived attributes manufacturer, lot number, production date, model, material, diameter, and wall thickness.  If the digital as-builting described in step 1 happens while the device is connected to the enterprise geodatabase, then Collector will automatically decode the barcode, auto-populate the derived attributes and display the decoded information immediately after the new/updated GIS feature is submitted by Collector. In the second blog, we will provide links to these arcade scripts and describe how to apply them to an enterprise geodatabase.

 

Step 3: Append to enterprise geodatabase

One of the time saving capabilities of ArcGIS Desktop is the ability to interact with data from both the staging geodatabase and the enterprise geodatabase at the same time.  This allows the mapping professional to easily select the staging geodatabase features and append them into the final enterprise geodatabase feature classes. 

 

If the staging geodatabase layers are stored in ArcGIS Online, the previously described attribute rule arcade scripts can be applied to enterprise geodatabase layers. 

 

NOTE: Attribute rules only work with ArcGIS Enterprise 10.7 or higher. Additionally, ArcGIS Pro is the only desktop tool to understand attribute rules.  If using ArcMap and a geometric network, it is important that the staging geodatabase layers be stored in an enterprise geodatabase and the attribute rules are applied to the staging geodatabase layers.

 

The standard arctoolbox geoprocessing append tool can be used to copy the newly collected GIS features from the staging geodatabase layers to the final enterprise geodatabase feature classes.

 

Step 4: Mappers connect digital as-built with gas system

With the new construction data now appended from the staging geodatabase into the enterprise geodatabase and the barcode value decoded, the mapping professional now needs to determine how to connect the high precision geospatial features with the less accurate geospatial features. The outcome of this process needs to honor two data requirements:

  • Connecting the new features with the legacy features to create a single topologically connected gas pipe system.
  • Preserving the high precision GPS collected geospatial coordinate data.

 

The recommended best practice for accomplishing this seemingly disparate set of requirements is for the enterprise geodatabase point features such as Meters, Excess Flow Valves, and Non-Controllable Fittings to have the following attributes added: SPATIALACCURACY, GPSX, GPSY, GPSZ.  Here is another example where attribute rules can streamline the population of these GPS fields.  If using ArcMap and the geometric network, then a configuration of Esri’s Attribute Assistant tool or ArcFM’s AutoUpdater capability can be used to automatically populate these fields.  This will preserve the original GPS location values, which can be used later to rubbersheet all features (legacy and GPS) to the more accurate GPS location preserved in the GPSX, GPSY, and GPSZ attributes.  With the GPS location preserved, the mapper can adjust the new construction features as required to connect to the legacy gas pipe system.

 

Business value of using ArcGIS platform

This approach to Tracking and Traceability provides an opportunity for the GIS department to once again show the greater gas organization that not only can the GIS Department provide a solution which addresses this new common industry practice, but it can do so in a manner that improves the operational efficiency of the gas organization.  This pattern improves the operational efficiency of the gas organization and their contractors as follows:

  • Using Collector to collect construction data improves location accuracy and attribute quality by eliminating translation to paper and interpretation of paper based information.
  • Bluetooth integration with high precision GPS antennas improves the speed at which data is collected.
  • Capturing the barcode value reduces the amount of information the field staff manually collects. Material, diameter, manufacturer, manufacture model, manufacture data, manufacture lot number are all automatically populated by the decoding of the barcode.
  • Digitally collected data is immediately available for GIS department to process into enterprise geodatabase. This eliminates the historical latency problem of the GIS department waiting for the inter office mail transmittal of the construction packet.
  • The GIS department mapping professional task of updating the as-built representation of the gas pipe system is simplified. The mapper is no longer manually transposing paper based red-line drawings, but instead appending field collected geospatial features.  This improves the speed at which a mapper can complete the task of updating the as-built representation of the gas pipe system.
  • Safety of field operations staff is improved by providing the new construction data in a timelier manner.

 

This deployment pattern not only provides the ability to improve the efficiency of the field data collection, it improves the productivity of the mapping professional, and provides new construction updates to locators and field operations staff in a timely manner.

 

Next blog

In our next blog, we will dig into how to configure and deploy the arcade scripts for this solution to Tracking and Traceability.

 

PLEASE NOTE: The postings on this site are my own and don’t necessarily represent Esri’s position, strategies, or opinions.

***This workflow works with etlsolutions v0.5.2***

Click Here to learn more about benefits and how to get started with the Data Translation Tools

 

We have had requests for additional documentation and samples at the UC. A zip pro package has been attached to the thread.

 

We want to bring data from one file geodatabase to another file geodatabase. This is also known as Extract, Transfer, and Load (ETL). Let me set the stage.

 

We have a file geodatabase or fgdb for short. Let's call this fgdb, our source.  Inside this geodatabase, we have a feature class named Cars.  We have three subtypes in this feature class: Red, Blue and Green. There are domains assigned to each of these subtypes. I want all my data in this feature class to shuttle over to my target file geodatabase. This target fgdb has a feature class called Trucks with different schema. Schema is a term that includes attributes, data types, domains, and other data management concepts. The Trucks feature class has three subtypes as well, but they are different than those found in Cars. They are One, Two, and Three rather than Red, Blue and Green.

 

There are three ways to translate data with the workbook. The first method does a straight field mapping without any domains. The second method uses a sheet as a lookup table for domains. Finally, the last method uses a hard coded value to ‘burn in’ values. I will cover each method in this blog.

 

 

 

The first tool in our Data Translation toolbox is Create Mapping Workbooks. 

  • Open the GP Tool Create Mapping Workbooks tool within Data Translation toolbox.
  • Point each one of your source feature classes to its corresponding target feature class.
  • Specify the output folder for your mapping workbooks.
  • (Optional) Check the box to Calculate feature count statistics to generate information on what fields have populated data.

 

 

 

Once the tool has run, you will see a Points folder that contains a mapping workbook called Cars and a mapping.xlsx file which will be used later when using the Load Data from Workbook tool. Let's go into how to populate information into these excel workbooks.

 

Method One: Straight Field Mapping

 

Step 1:

Open the Cars workbook and navigate to the Mapping sheet.  We have columns for targetField, sourceField, and fieldType which are all system derived. The columns, expression, sheet, sheetKeys, and sheetValue are used for methods two and three. This default workbook is configured to transfer data to your new file geodatabase without any translations.

 

 

 

What this means is the domains will be mapped using their codes. For example, Red will not be mapped as anything since there isn't a 0 code in the domain "Type" of our target feature layer. Blue will be mapped as One since they share the domain code of 1. Green will be mapped as Two since they share the domain code of 2.

 

Step 2:

Before we use the Load Data from Workbook tool to Extract, Transfer and Load our data, it's time to inspect our mapping.xlsx workbook.

 

Each row in this workbook directs to the tool to: set the source database, set the target database, and set the lookup workbook. This first row was created when we pointed our Cars feature class to our Trucks feature class during the Create Mapping Workbooks section at the beginning of this workflow.

 

 

Step 3:

Now that I have inspected everything, it's time to run Load Data from Workbook tool.

  • Open the Load Data from Workbook tool in Data Translation toolbox.
  • Specify the location of the mapping workbook created from Create Mapping Workbooks tool.
  • (Optional) Truncate the target geodatabase before loading.

 

 

 

When we run this tool, we return values of 0, One, and Two.

 

 

Method Two: Field Mapping with Lookup

 

Step 1:

Open the Cars workbook and navigate to the Mapping sheet. In this sheet, we will define what sheets and columns to use as a lookup table. Since I am not using the sourceField for a straight mapping, I will remove Type from the sourceField column In this sheet, the values I have entered are highlighted. In the sheet column, "Type", highlighted in yellow, is used by the Load Data from Workbook tool to locate the correct excel sheet when mapping to Type.  In the sheetKeys column, "Type", highlighted in yellow, is used by the Load Data from Workbook tool to locate the correct column(s) on the Type tab when mapping to Type. In the sheetValue column, "NewType", highlighted in orange, is used by the Load Data from Workbook tool to locate which column to use as a lookup table on the Type tab.

 

 

Navigating to the Type tab, I have highlighted the Type column in yellow to show the relationship to the user-defined column in keys on the mapping tab previously mentioned.  I have also highlighted the user-defined column and values in orange. I entered the domain values 1, 2, and 3 in column C to map to Red, Blue, and Green, respectively.  I have also included their domain descriptions in column D. What I have accomplished in this workbook is creating a lookup table for old domain values to new domain values. Where Red Car was using a domain value of 0 to describe Red, Truck One uses a domain value of 1 to describe One.

 

 

Repeat steps 2 and 3 above from Method One

 

Here is our final output.

 

 

 

Method Three: Hard Code Burn-in

 

Step 1.

Open the Cars workbook and navigate to the Mapping sheet. In this sheet, remove all the values you entered in Method 2. We will simply enter in a hard-coded value of "3" into the expression column. This will burn-in the value of 3 for Red, Blue, and Green.

 

 

Repeat steps 2 and 3 above from Method One

 

Here is our final output.

 

In his book The Road Ahead, Bill Gates said, "We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten. Don't let yourself be lulled into inaction." We are already seeing increased complexity in electric distribution circuits. If Gates is correct, we will see much more in the coming years.

Electric utilities historically operate their distribution systems in sections called circuits. This blog series looks at some important characteristics of the circuits of the future and how they may differ from those of the past and the present. These differences are acting to convert our familiar circuits into a network that relies on electronics and data for routine operation. Are you setting your system up for success in the face of these changes?

Part 1 of this blog introduced some fundamental differences between circuits and future networks. Part 2 examined why networks must be capable of being split into smaller parts. This final segment will consider the network's greater complexity and its need to handle rapid changes.

Greater Complexity

The sheer volume of electric system devices is going through the roof—microgrids, distributed generators, smart inverters, sensors, and automatic switches all bring greater complexity. These devices are more sophisticated than most of the circuit devices commonly in use today.

Arguably the most common device on a circuit today is a fuse—a skinny piece of wire that, as its sole operation, burns up! An important insight is to realize that most circuits, on their journey to become a network, are starting from a very low level of sophistication.

Sophisticated devices have more connection points, bypass functions, and test provisions. In addition, they are often configurable, integrate with communication systems, and exchange parameters with other devices and systems. These parameters help govern equipment settings, price signals, and protection from harmful conditions. Much of this complexity is linked to modern electronics that consume data in real time.

Smaller network pieces and sophisticated devices complicate routine operating decisions. Formerly simple manual operations, like opening an overhead pole switch, will be initiated remotely with the use of a new and vastly more sophisticated switch. Instead of simply verifying adequate electrical capacity and switching from one circuit to another, operators will need to understand how each of these changes affects the entire network.

I investigated several high-voltage accidents while working for utilities. Two of the worst injury accidents had their root cause in misidentified energized equipment. In tight spaces, like substations or underground structures, complexity brings the need for more equipment which takes up precious working space. More equipment and less space makes safe operation that much more challenging. To work safely, the data and information systems supporting these new networks must also accommodate their greater complexity and detail.

Rapid Changes

Traditional circuit layouts tend to be static, changing only for specific tasks or between summer and winter configurations. Dispatch and field personnel often have them nearly memorized. A gray-haired supervisor may confidently tell a new apprentice, "That transformer is on circuit number 121—it feeds from Buckingham substation up on the hill," speaking on the assumption that the circuit's characteristics remain constant.

Self-healing capabilities such as reverse power flow and the use of automatic switches and microgrids can all change networks rapidly and without much warning. They can alternate in response to different conditions in a short period of time. Traditionally, to alert employees, such changes are announced over the operation's two-way radios mounted in work trucks. Advanced network changes may occur with little or no human interaction, and without radio announcement. This real-time operating paradigm sparks different work procedures and safety concerns because such rapid changes were not normal in the past.

Staff are not used to their circuits changing quickly. They are accustomed to referencing their relatively static maps. Historically, a period of weeks to apply map updates was acceptable. But now, last month's map products from the Maps and Records division will be simply inadequate to meet the real-time operating needs of new networks. All users, in the office and the field, will need more detailed information in near real time.

Wrap-Up

Is there a time coming when we won't even think of circuits at all? Probably, but not in the immediate future. For decades, circuits were the only source of power to the distribution system. Today, every rooftop solar installation is another source to consider. The circuit at the substation may not be the only source on the network, but it will certainly remain important for quite some time.

Many of the standards necessary to implement a smarter network are still under development. Given all the forces acting to change circuits into networks, prepare for a continuous evolution of equipment and capability. When you don't know exactly what will be required, flexibility is a key strategy.

New functions will continue to be added, improving our ability to optimize distribution operations for power quality, cost, and reliability. Grid modernization and circuit evolution also mean a great deal of physical work, building networks and systems to support them.

Because networks of the future will be controlled with electronics and data, the underlying information models and systems will be foundational to success. Like Bill Gates said, "Don't let yourself be lulled into inaction." The ArcGIS platform is specifically designed to help utilities model and operate these new complex and rapidly changing networks.

For more information on how the ArcGIS platform helps electric utilities manage advanced networks, visit our site.Advertisement

In electric utilities, we are really attached to our circuits. Get ready- those circuits are going to change, and as we progress, may become unrecognizable! Circuits are so embedded in our culture that a colleague once remarked that we are fixated on them!  He was right. We are fixated on circuits, and with good reason.

Electric utilities typically operate their distribution systems in pieces called circuits or feeders. We map by circuit, patrol facilities, trim trees, and report statistics by their circuit name or number.  As the utility industry changes, circuits will need to morph into more of an interconnected network. “Network” is a much better term than “circuits” to describe the future state. Is your utility preparing to successfully operate the network of the future?  

Part I of this series introduced some fundamental differences between traditional circuits and future networks.  This part II will examine the first difference – smaller pieces.

Smaller Pieces

In the future, electric networks will need to break down into smaller pieces for greater operating flexibility. Utilities established the sections of today’s circuits to reduce the customer impact of power outages, perform maintenance tasks, and supply large blocks of customer load. Utilities now need additional flexibility to accommodate different types of both customer usage and power generation. Distributed generation, including solar and wind, is steeply on the rise.  These variable resources bring constantly shifting power flows to circuits that were only designed for one-way power flow. Networks need smaller pieces with flexibility to handle variable power flows.

Electric vehicles can plug-in anywhere moving their electricity demand around like a big 2-story house on wheels! Yesterday’s large stable blocks of customer load are becoming less consistent and are now driving from one place to another.  Networks must be more configurable than circuits to meet the needs of tomorrow’s customers.

Smart-grid technologies too will drive networks to operate in smaller sections. Self-healing networks use smart switches to sense real-time conditions. In the blink of an eye, they communicate with other devices and compare observations. Together they determine when a power problem occurs and limit customer impact with instant automatic switching.  As utilities implement more self-healing capability, the pieces of the network will get smaller enhancing customer value by improving reliability.

It’s clear the operating pieces of the network will be smaller than those of a typical circuit today, delivering sorely needed operational flexibility.

Wrap up

New devices will split the coming network into smaller pieces. Because networks will be controlled with electronics and data, the backbone of this evolution will be a central data model of the entire system. This feature-rich model may be called a digital twin.  A fully functional digital twin, adequate to support disparate utility roles, is a big step from the straightforward facility mapping models of the past.

A digital twin should be sophisticated enough to represent each device accurately at its precise location. Device location on the network will guide its every operation.Smart utility operations begin and end with this location intelligence.

We are already seeing the writing on the wall as pilot projects adapt existing circuits to accommodate new unconventional devices. Common distribution circuits will evolve into a more robust network.  This flexible network must consist of smaller pieces, include numerous new complexities, and change quickly in response to system and customer needs.

The ArcGIS platform gives all stakeholders the ability to access and share critical network information. Advanced network information will have to become embedded in our “circuit culture” as it evolves.   The ArcGIS Utility Network Management extension is specifically designed to handle the smaller pieces and greater detail of the advanced electric networks now on the horizon.  

For more information on how the ArcGIS platform helps electric utilities manage advanced networks, visit our site.

One afternoon in engineering school, my professor boldly proclaimed, “Nothing has changed in power engineering since the 1930s!”  He knew that even the most advanced electrical devices of the time were controlled by very simple things like springs and magnets.  He wanted us to be solidly grounded in the underlying laws of physics, and my resulting education served me very well.  In contrast, the circuit networks of the future will be controlled with electronics and data, often managing the springs and magnets inside devices.

Can you imagine a professor making that statement today? Such a professor would certainly be out of touch with the utility business. In his book, What Got You Here Won’t Get You There, Marshall Goldsmith describes the necessity for successful people to make changes to further their success.  Successful utilities, and their circuits, will need to adapt to a dynamic world to be relevant and successful in the future.

Electric utilities typically operate their distribution systems in pieces called circuits or feeders. This practice served the industry well for many years. However, more than a few changes occurred since my university days in the early 1980s.  At that time, I didn’t have a computer or cell phone, and wrote term papers on an electric typewriter. This blog series will look at some important characteristics of the circuits of the future and how they may differ from the past and present. Like never before, data and information systems will be an integral part of operating future circuit networks.   How is your utility preparing to successfully operate the electric system of the future?

Given the changes in our business, the very term “circuit” could become confusing.  For clarity, I’ll call the future arrangement a network, rather than circuits.  The interconnected electric system is sometimes referred to as the largest and most complex machine on Earth. Historically operated as circuits, the network may become a utility’s most important asset, capable of enabling new business models and greater customer value. Rather than a one-way circuit that delivers electricity to customers, the network will have to become a market place for many more participants.

To avoid disappointment and become a market enabler, the network of the future must have some fundamental key differences – it must be divided into smaller pieces, have greater complexity, and change more rapidly than we are used to.

1 - Smaller Pieces

Sections of circuits are optimized simply to isolate system problems and supply large blocks of customers. Utilities, and in fact all users, will need the increased flexibility of smaller pieces to accommodate an exploding range of possible operating conditions brought about by all types of additional network devices.

2 - Greater Complexity

The sheer number of devices is increasing dramatically -  micro-grids, solar panels, sensors, electronic controls, all bring greater complexity.   A simple fuse, in service and undisturbed for 30 years, will be replaced by a sophisticated switch with an electronic controller communicating with the network. Its complex operation is based on the actual operating conditions as they exist right now! The complexity will usher in new challenges in recordkeeping, workforce development, maintenance, operation, and troubleshooting.

3 - Rapid Changes

Self-healing capabilities, reversing power flow, automatic switches, and smaller pieces, will all require networks to change rapidly.  Changes may occur with little or no human interaction, and may bounce between multiple states in a short period of time.  When things change quickly, they become a safety concern.  A pile of paper circuit maps is wholly inadequate to safely operate a more complex and rapidly changing network. Engineers and line workers alike will require near real-time information to operate the network safety and effectively.  

Near real-time information drives and enables better optimization of the entire system to reduce costs, enhance reliability, and improve power quality. A customer’s equipment, needs, and choices will also affect familiar circuits in many new ways.

Wrap up

Industry changes will require our beloved distribution circuits to be much different in the future.  The array of new utility devices offered in the marketplace is dizzying and trade articles regularly detail pilot projects dramatically altering the traditional distribution circuit layout. 

Because networks of the future will be controlled with electronics and data, information systems will be the foundation to operating these networks. The electronic model of the network will be central to all core business functions.

All the change drivers relate to the new network based on their exact connection points and location.   The ArcGIS platform gives all stakeholders the ability to access and share critical information, including the network model. The ArcGIS Utility Network Management extension is specifically designed to address the needs of a more complex and variable electric network. 

My professor, with his 1930s thinking, would be shocked how much power engineering has changed. He would be surprised at how inadequate classic circuits will be to safety support utility operations in a few years. How long will it take before circuits change into networks?  In some locations, the shift has already begun. The ArcGIS platform is designed to help utilities operate these new networks.

For more information on how the ArcGIS platform helps electric utilities model advanced networks, visit our site.

Gas Outage Management Part 3 of 3

The Gas Relight Process

By Tom Coolidge and Tom DeWitte

Natural gas distribution utilities have a proven track record of high reliability, even during extreme weather events.  But, in addition to smaller gas outages, exceptions resulting in large gas outages do infrequently occur.  These exceptions continue because of such things as planned pipe replacements, unplanned errors during routine maintenance activities, and transmission pipeline supply issues.  As examples, just in the last year or so:

  • 2,800 customers in Dallas, TX experienced a large gas outage,
  • 7,600 customers in Merrimack Valley, MA were out for some time,
  • 2,200 customers in Dayton, OH experienced a large gas outage, and
  • 4,900 customers in Ashland, WA experienced a large gas outage.


A Better Way

While these large gas outages are generally like those of earlier years, the expectation of how a gas relight event is best handled has been changing. I experienced this first hand in 2018.  During that gas outage event, I saw state and local elected officials arrive at the gas utility every morning for an in-person update.  Later in the day, they expected to be provided with a video update on the progress of gas service restoration.  Not only did the elected officials expect to be given an up-to-date summary of the overall outage event, they also had questions and expected answers to the status of individual customers. These same elected officials expected that their local gas utility be able to inform them on demand on the status of any customer. Further, they expected that any customer could contact their local gas utility and receive the same information.

 

Most current gas relight processes in use in the gas industry are not capable of meeting these expectations.  They lack the real-time communication capability between multiple gas field staff and they lack the real-time communication capability to inform office staff, executives, and local communities seamlessly with this same information. ArcGIS software leverages today’s telecommunication system to enable the gas relight information to be collected and shared in near real-time. The key to a successful gas outage solution based on the ArcGIS software is in knowing how. In this blog, we will answer the question of how to enable gas field staff to collect and share an individual customer’s gas relight status in near real-time and how office executives can monitor and share the event status in near real-time.

 

 

In the first blog of this series, we described how the ArcGIS desktop software can be used to perform a gas isolation trace which will accurately identify the customer meters impacted, the isolating devices or pinch locations and the extent of the outage.  In the second blog, we addressed how the ArcGIS platform can be leveraged to transmit this large amount of information from the office to the correctly assigned gas field staff. In this blog, the third of the series, we will walk through a hypothetical gas relight event and see how this information is streamed between office to field and field to field.

 

Gas Relight Initiated

When the gas relight list of impacted customer meters is uploaded and assigned to the gas field staff, the office staff and executives have immediate visibility into the event.  The office dashboard shown below shows an example of a gas relight dashboard at the initiation of the gas relight event.  At this point, all 15 impacted customer meters can be individually seen on the map, and their status is communicated by symbol color (red = out, orange = off, green = relit, and purple = no entry).

In the field, the gas field staff can see the same near real-time information on their mobile devices (smart phone, tablet, laptop). This is the first view of the outage event provides a view of the entire event.  Every gas field technician can see the status of every impacted customer meter impacted.

 

The second view is a presentation of those impacted customer meters assigned to the field technician.

Together, these two views provide gas field technicians with a complete understanding of the gas relight event.

 

Gas Relight Meter Turn-off

As the gas field staff begins the process of turning off the individual impacted customer meters, everyone in the field and the office will see the meter status change. Having the status of a meter seamlessly communicated to everyone in the field and the office allows for improvements to the gas relight process itself.

Gas field technicians no longer need to stop working the outage to deliver their documentation to a gas operations supervisor.  If a gas field technician finishes his or her assigned meters, he or she can see the real-time status of the other gas field technicians.  This means that a technician who has just completed the left side of the street, can simply walk across the street to start working those meters which have not yet been done by the technician working on the right side of the street.

 

Gas Relight Meter Turn-On

The turn-on process is a multi-step process that includes turning the valve at the meter to open the customer to the flow of natural gas.  It also includes going into the customer building to relight the gas appliances.  If the customer is not available to provide access to the appliances within the building, the customer is marked as “no access”. If all turn-on tasks can be completed, then the customer’s meter status is changed to “relit”.

Remember the office is seeing the status change as soon as the field technician updates the meter and submits the record change. Gas executives and customer representatives can see the status of a meter within seconds, as quickly as the telecommunication system makes possible.  This allows customer service representatives to confidently inform customers when they call asking for information.  It also allows gas executives to easily communicate to an elected official the status of the overall event and any individual meter.

 

Conclusion

Today’s ArcGIS presents gas utilities with the opportunity to greatly improve on how they execute the gas outage restoration process.  Modern gas service restoration at its best is an enterprise-wide activity with workers in the field and office working together collaboratively in real-time on the same data. This blog, and the two that preceded it, together described how the core capabilities of the ArcGIS platform enable a gas utility to implement a modern gas service restoration process.  This process is accurate, efficient, and timely.  It is a process that will provide customer service representatives, gas operations supervisors, and gas management with real-time clarity on the progress of each customer through the gas service restoration process.

Communicating identified customers to Field Gas Operations

By Tom Coolidge and Tom DeWitte

There’s no authoritative record of the date of the first sizable gas outage in the United States, but a candidate for that distinction is June 14, 1837.  If the Gas Light Company of Baltimore had a control room then, the first alarm likely would have sounded shortly after 9 p.m. The Baltimore Sun reports it was around then that a second powerful thunderstorm dumped an enormous amount of rain in a short period, leaving the Jones’ Falls stream “incapable of retaining its boundaries.”  The resulting flooding caused loss of life, loss of houses, and vast destruction of other property– including partial inundation of the Gas House sufficient to prevent the manufacture of gas for days.  Restoring service was a formidable challenge.

When this first outage occurred how do you think the staff of the Gas Light Company of Baltimore determined which customers were impacted by the outage. Did they simply test each gas lamp to see it lit or not?  Once they figured out the extent of the outage in the field, how was that information shared back to the office?  Most likely someone got on horseback or climbed into a horse-drawn buggy and rode the information from the field to the office. If the office had additional feedback for the field on how to isolate the outage and restore service, that information would have also been delivered back to the location of the outage on horseback.  How much longer was the duration of the outage extended while the gas utility staff waited for the information to be delivered?

 

Innovation and communication

The speed of communication has always been a limitation on the speed with which gas outages can be resolved. For over 100 years from the creation of that first pipe system in Baltimore, the speed of information was limited to the speed a person could be transported from one place to another.  Innovation in the second half of the 19th century enabled small amounts of information to be transmitted between staff in the office and the field by telegraph and the telephone. Further enhancements in the early 21st century have enabled large amounts of information to be transmitted between the office and the field staff. These technologies broke the limitation of the speed of information being constrained by the speed of humans.

 

A Better Way

Today’s telecommunication system provides the capability to communicate large amounts of information, such as a list of impacted customer meters, between the office and the field in near real time.  The ArcGIS software leverages today’s telecommunication system to transmit that list of impacted customer meters. The key to a successful gas outage solution based on ArcGIS is in knowing how. In this blog, we will answer the question of how to get the list of impacted customer meters from the office, and to the assigned field staff.

In the first blog of this series, we described how the ArcGIS desktop software can be used to perform a gas isolation trace which will identify the customer meters impacted, the isolating devices or pinch locations and the extent of the outage.  In this blog, we will address the next major step which is to get this large amount of information from the office to the correctly assigned gas field staff.

Prepping Data for the Field

The first step in accomplishing this is to use a geoprocessing model to upload the selected data and append it to existing feature layers. These feature layers can be hosted in ArcGIS Online, or they can be hosted on an ArcGIS Enterprise Portal.

In addition to appending the customer meters impacted, the isolating devices, pinch locations, and the extent area, a geoprocessing model provides the opportunity to prepare the data for field use.  Here is a list of commonly added attributes and their purpose:

  • TRACEID to customer meters point features, pinch location point features, isolating valves point features, and extent area polygons. This will allow all data associated with the outage event to have a common ID.
  • RELIGHTSTATUS to the customer meters point features. This will allow gas field staff to track each meter/customer point feature through their gas light cycle (unassigned, assigned, out, off, relit, no entry,). Default value is “unassigned”.
  • TIMEOUT to the customer meters point features. This will allow gas field staff to document the date and time when the meter lost service.
  • TIMEOFF to the customer meters point features. This will allow gas field staff to document the date and time when the meter was turned off.
  • TIMERELIT to the customer meters point features. This will allow gas field staff to document the date and time when the meter was turned back on.
  • NUMBEROFPASSES to the customer meters point features. This will allow gas field staff to document the number of attempts to gain access to the premise to relight the gas appliances.
  • OUTAGETYPE to the outage event area polygon features. This will allow office staff to identify the type of event which caused the gas outage.

 

The geoprocessing model when run will take impacted customer meters and upload them to the feature layers.  A minute or two after the model has finished running, the data is available for gas operations staff. No more waiting for the horse-drawn buggy to arrive with the information.

 

Assigning Impacted Meters/Customers

The last step is to assign the impacted customer meters to individual gas operations field staff.  To perform this step, we will use Workforce for ArcGIS. Workforce is comprised of two applications; a dispatcher web application and a smart device mobile application.  The Worforce web application provides the ability to view the newly uploaded list of impacted customer meters.  Because the individual records can be viewed on a map, it is very easy to use geography to assign them to field staff.  For example, in the screen shot below the customer meters on the west side of the street can easily be selected and assigned to a single gas field technician.  This will improve the efficiency of the gas relight process by clustering the assigned meters.

When the Workforce Dispatcher web application assigns impacted customer meters, the field staff are immediately notified.  The mobile app will show the gas field technicians their assigned customer meters.  No more waiting for information.

In the third and final blog of this blog series, the issue of working the gas relight process will be addressed.

 

Conclusion

ArcGIS today is deployed worldwide at many gas organizations, providing the ability to replace and improve upon non-spatial legacy processes.  Identifying impacted customers, whether they are connected by steel pipe or pinchable plastic pipe, can be accomplished in just a few minutes.  Using ArcGIS tools enables information to be prepared, transmitted, assigned, and viewed by field staff in a matter of minutes. No more waiting for the horse-drawn carriage, telegraph message, or telephone message to arrive with the information.  

 

PLEASE NOTE: The postings on this site are my own

and don’t necessarily represent Esri’s position, strategies, or opinions.

Getting Started By Identifying Customers Impacted

By Tom Coolidge and Tom DeWitte

 

News of a gas outage can arrive from various sources.  It can come from a sensor indicating an abnormal condition.  Maybe it comes from a customer calling into customer service.  Or, a contractor calling operations after an excavation mishap.  Another possibility is a citizen calling in to report gas odor at a location.  Regardless of the source of the outage news, confirmation of an outage triggers one of a gas utility’s priority processes – restoring safe and reliable service to customers.

 

As important and critical a task as gas outage management is to a gas organization and to the community it supports, this process has changed little over the last 100 years.  For many gas organizations, it can take several hours to identify which customers have been impacted.  Once the customers are identified, getting the list of impacted customers to the field gas operations staff is still primarily a paper process. Someone literally must get into a vehicle and drive the list of customers to the location of the gas outage event.  As the field gas operations staff begins the gas relight process, they too still tend to use paper to document the status of each customer.  This means that management will always have a delayed understanding of the progress of restoring gas service.  When the mayor or governor calls asking for an update, gas executives are often get caught with little current information to pass on.

 

There has got to be a better way.

 

And, there is.  In fact, most of the gas industry already possesses the software to resolve these issues and significantly improve a gas organization’s response to a gas outage event.  The software I am referring to is the ArcGIS software currently widely used by gas organizations around the world.  This blog is the first in a series of three blogs explaining how the standard capabilities of the ArcGIS software can be deployed to address these common gas outage management challenges.  All functionality described in these blog articles are standard capabilities available today.  No customization or coding is needed. 

 

This first blog addresses the issue of identifying the customers impacted by a gas outage event.  This task often takes several hours when it needs to be accomplished in minutes. Additionally, the historical processes have had problems with accurately identifying the impacted customers and communicating precisely where those customer meters are located. 

 

The second blog will address the issue of communicating the list of impacted customers to the gas operations field staff.  The typical paper process takes too much time, causing delayed field operations and lower customer satisfaction. 

The third blog will address the gas relight process.  This process is also typically performed with paper.  The use of paper to track and communicate progress adds difficulty and inefficiency to this process.  The use of paper not only engrains a delay in relaying the update status to gas management and other interested parties, it also inputs a delay in relaying the status of individual meters between deployed field staff.

 

Identifying Impacted Customers

Current methods used by many gas organizations are lacking in accuracy and timeliness when identifying the customers impacted by a planned or unplanned gas outage.  One common method is to use the Customer Information System (CIS) for identifying impacted customers.  Since a CIS typically lacks an understanding of the connectivity of the pipe system, it is forced to rely on street address ranges.  The use of address ranges is inaccurate.  At every street intersection are four corner parcel lots.  Whether they are included in the address range is dependent on what street the house is listed under. This inaccuracy often requires a time-consuming manual process of having someone review the list, identify all crossing streets within the address ranges, determine the address ranges of those crossing streets, identify the corner lot addresses, then determine for each corner lot, whether it gets its gas from the impacted line, or from the gas line running down the cross street.  

 

Another common method is to use flow analysis systems to perform an isolation trace to identify the impacted customers.  This process is quicker, but it too is imprecise. The imprecision is due to the flow analysis software’s requirement to cluster groups of customers onto the gas pipe system at a singular location even though they each have individual service lines connecting to the gas main at discrete locations. In today’s gas pipe systems, the majority of gas mains are constructed of pinchable polyethylene plastic pipe. A gas event can be isolated or pinched at nearly any point along the plastic gas main.  The clustering of customer locations along the pipe system creates an inherent conflict between where gas operations places a clamp to pinch the pipe, and where the flow modeling engineer chose to aggregate the cluster of customers. This conflict creates an inaccuracy in the identification of impacted customers.

 

Accurately and quickly identifying impacted customers

The solution to addressing this problem is to use a system that understands the connectivity of the entire pipe system from its source, such as a town border station, to its end destination at the customer meter. ArcGIS provides the ability to maintain a connected representation of the entire pipe system, and the ability to perform a gas isolation trace to identify the meter or meter sets impacted by a gas outage. To perform this trace, you will require the following software:

  • ArcGIS 10.2.1 or higher, with a geometric network

or

  • ArcGIS Pro 2.3 or higher, ArcGIS Enterprise 10.7 or higher with a utility network

 

Additionally, your ArcGIS representation of the gas pipe system will need to model the following gas system assets:

  • mains
  • services
  • isolation valves
  • regulator stations (if regulator station valves are not individually mapped)
  • town border stations (if town border station valves are not individually mapped)
  • meters or meter sets

 

NOTE: If using meter sets you will need a link to a table identifying all meters contained within the meter set. This table is often an extraction of information from the Customer Information System

 

Your mains and services will at a minimum need to include the material of the pipe, so pinchable pipe can be differentiated from non-pinchable pipe.

 

The Gas Isolation Trace

The gas isolation trace is a more complex trace algorithm than simply identifying those pipes connected to the location of the pipe system failure, which are also between isolating valves.  With most gas pipe systems, the network is deliberately looped, to provide multiple sources of gas to any given location in the pipe system.  If this were true for every location on the pipe system, a simple connected trace defined to stop at barriers such as isolating valves or pinch points would be all that is needed.  But, there are portions of most gas systems where locations have only one source of gas.  Think of a gas pipe running along a dead-end street or a cul-de-sac.

 

If there is an isolating valve or pinch point at the location where the single feed pipe subsystem integrates with the larger looped pipe system, then the simple connected trace would ignore the customers on the downstream side of the barrier.  A more intelligent trace algorithm is required.  This more intelligent trace algorithm is generally referred to as the gas isolation trace.  A gas isolation trace is a multi-trace trace.  This means that the isolation trace runs a series of traces.  The first trace is the connected trace to identify the barriers (isolating valves and specified pinch locations).  Then a second round of traces is performed for each selected barrier.  This second round of traces is checking to verify that there is a source of gas feeding the barrier from the opposing side of the barrier.  This is to identify those dead-ends which do not have access to another source of gas.  Those customers downstream of the barrier on the dead-end need to be included in the list of customers impacted by the outage.

 

Gas Isolation Trace tools

The ArcGIS gas user community is fortunate in that there are multiple options for tools which can perform this industry specific type of trace.

 

One option is to download the free Gas utility editing tools provided by Esri. This ArcMap Add-In is available from the following Esri web site: http://solutions.arcgis.com/utilities/gas/help/as-built-editing/

Another option is to leverage ArcMap Add-In tools from one of our business partners, such as Schneider Electric or Magnolia River.

 

For the ArcGIS Pro environment leveraging the utility network, this trace is a base capability as of the ArcGIS 10.7 release.

 

Identifying Impacted Customers

Operating the gas isolation trace tool is not complicated.  Simply identify the estimated location of the pipe system failure on the map.  In GIS speak this is called placing the flag to identify the start location of the trace.

 

When the isolation trace is run it will select all customers within the impacted area.  In my screen shot below you can see that this initial run selects over 100 impacted customers.

 

Identifying the location of pinch points

The prevalence of pinchable polyethylene plastic pipe enables the additional capability to reduce the number of impacted customers, by applying a gas clamp to pinch the pipe and stop the flow of gas to the location of the pipe system failure.  To represent this field capability in the GIS system, place a barrier at the location being considered for the pipe clamp.

 

With the proposed location(s) of the pipe clamp(s) now identified, the isolation trace is run a second time.  This time the resultant list of impacted customers has been reduced to less than 20.

The person running the analysis for both traces has so far only invested a few minutes of their time.  In that short time an accurate list of impacted customers has been created.

 

Defining the extent of the gas outage event

In today’s always connected, smartphone world, gas executives and managers expect to be able to access critical information that is easy to understand.  They generally do not need to see the list of individual customers impacted, often all they want to know is “where is the outage”, and “how many customers are impacted.”

By identifying the list of impacted customers with the ArcGIS tools, it is very easy to run an additional step to generate a polygon to define the boundary of the event.  In the GIS, a tool such as the Minimum Boundary Geometry geoprocessing tool will perform this task.

The creation of an event area feature provides a clear visual understanding of where this outage is occurring.  Having this singular feature representation also provides an intuitive means for managing event summary information, such as duration, and count of impacted customers.  The Esri-provided gas isolation tools automatically generate this polygon as part of the operation of the isolation trace.  In addition to the automatic generation of the polygon, a of every meter is generated and assigned an event ID to automatically relate the impacted customers to this specific event.

With the list of impacted customers defined and created, as well as the event bounding polygon, this information is ready to be electronically shared to gas operations field staff.

 

In the next blog, the 2nd blog of this blog series, the issue of delivering this list of impacted customers will be addressed.

 

Conclusion

ArcGIS today is deployed worldwide at many gas organizations, providing the ability to replace and improve upon non-spatial legacy processes.  Identifying impacted customers, whether they are connected by steel pipe or pinchable plastic pipe, can be accomplished in just a few minutes.  Using the ArcGIS tools can provide a more accurate list of impacted customers than is available via legacy methods.  This list not only identifies who has been impacted, it also clearly and accurately identifies where those impacted customers are located.   

 

PLEASE NOTE: The postings on this site are my own

and don’t necessarily represent Esri’s position, strategies, or opinions.

tdewitte-esristaff

Gas Outage Response

Posted by tdewitte-esristaff Employee Mar 15, 2019

Similarities and Dissimilarities Among Electric and Gas Outages

By Tom Coolidge and Tom DeWitte

 

After a storm with strong winds and rain heavy enough to cause flooding, one neighbor away at the time may ask another still there if there are any outages in the neighborhood.  It’s understandable if the first response back is about whether the area’s electricity is still on or not.  That’s because outage is a term more commonly associated with electricity than natural gas, for good reason.  Electric outages are much more common.  A survey by the American Gas Association, reported by the Natural Gas Council, revealed in one recent year that Americans experienced 8.1 million power outages and fewer than 100,000 natural gas outages!

 

One obvious reason for that sizeable difference is that electric distribution networks predominantly are above ground, while gas pipe networks predominantly are underground, free from most hazards on and above the surface.  That difference makes it rare for an event to be severe enough to impact the ability of a natural gas distribution network to safely deliver gas to customers.  But events of a magnitude sufficient to cause many customer outages at one time do occur.  For instance, entire areas of a gas utility service territory can be affected by flooding from a hurricane or pipe breaks from an earthquake.  And, likewise, they can be affected by pipe dig-ins at a critical location during construction activities or failure of pipe metal due to corrosion or other cause.

 

Beyond temporary inconvenience, gas outages caused by pipe damage releasing natural gas may be dangerous.  The release of natural gas represents an imminent threat to people and property.  That makes resolving this threat as rapidly as possible critical.  A gas outage not only presents a threat to the safety of people and the preservation of property, it also presents a negative impact to the local economy.  Restaurants that rely on gas to heat their grills and ovens, cannot operate. Hotels are unable offer their rooms when they are unable to heat their rooms or provide hot water for bathing.  Manufacturers which rely on natural gas to run their operations must close and send their workers home.

There is another significant difference between electric and gas service.  Restoring gas service is more difficult than restoring electric service.  That is because electric distribution systems are designed to be shut down under abnormal conditions and natural gas pipe networks aren’t.  Restoring natural gas service following an event that causes many outages is a multi-step process involving multiple parties, many workers, and lots of time and effort.  For this blog, I will combine the many steps into three groups.  These groups are: identification of impacted customers, assignment and transmittal of impacted customers to field staff, restoring gas service to customers.

 

These three primary steps are the same steps followed back in the horse and buggy days when gas distribution systems were initially implemented.  Back then the best technology for enabling this process was paper.  Building a gas outage process on paper is problematic.  The process of identifying impacted customers is inaccurate and time consuming.  It takes some gas organizations hours to generate the list of customers impacted by a gas outage.  Having humans manually review lists of customer addresses to determine who is connected to the impacted portion of the pipe system is time consuming and inaccurate. No customer on a cold January day wants to be told that the gas utility is still reviewing customer lists to determine who is impacted.  Using paper in the field to track the relight process for restoring gas service is inefficient.  Field staff and office management are both blind to the progress of restoration until someone stops working and manually shares their information.  These problems are not new, they have been around since the first gas distribution pipe systems were constructed in the early 1800s.

 

Technology has changed dramatically since the early 1800s.  Alexander Graham Bell invented the telephone in the 1870s, greatly improving the speed of communication.  John Atanasoff, while teaching at Iowa State University in the 1930s, created the first electronic digital computer, setting the stage for advanced data storage and analytics.  Frank Canova of IBM created the first smartphone in the early 1990s.  Steve Jobs, of Apple, Inc, would later improve upon the idea of mobile communication and computing with the IPhone.  Jack Dangermond, the founder of Esri, Inc., created ArcGIS providing the means to leverage these major technology advances with geography.  Geography is core to understanding the connectivity of a pipe system and where along the pipe system impacted customers of a gas outage are located.  

 

With all these amazing advances in communication, digital computing, mobile computers, and Geographical Information Systems (GIS), why are some gas utilities still using a predominantly paper-based solution to gas outage?

There may well be multiple reasons for that, but those possible reasons no longer include the unavailability of GIS capable of supporting a totally computer-based approach to supporting the gas outage restoration process.

Today’s ArcGIS presents gas utilities with the opportunity to greatly improve on how they execute the gas outage restoration process.  Modern gas service restoration at its best is an enterprise-wide activity with workers in the field and office working together collaboratively in real-time on one source of the truth.

 

Next week we will release the first in a series of three blogs on modern gas service restoration.

This first blog addresses the issue of identifying the customers impacted by a gas outage event.  This task often takes several hours when it needs to be accomplished in minutes. Additionally, the historical processes have had problems with accurately identifying the impacted customers and communicating precisely where those customer meters are located. 

 

The second blog will address the issue of communicating the list of impacted customers to the gas operations field staff.  The typical paper process takes too much time, causing delayed field operations and lower customer satisfaction. 

The third blog will address the gas relight process.  This process is also typically performed with paper.  The use of paper to track and communicate progress adds difficulty and inefficiency to this process.  The use of paper not only engrains a delay in relaying the update status to gas management and other interested parties, it also inputs a delay in relaying the status of individual meters between deployed field staff.

 

These three blogs together will describe how the core capabilities of the ArcGIS platform, enables a gas utility to implement a modern gas service restoration process.  A process that is accurate, efficient and timely.  A process that will provide customer service reps, gas operations supervisors, and gas management with the real-time clarity on the progress of each customer thru the gas service restoration.

 

Dramatic enhancements in communication and computation have occurred since the first gas distribution systems were built in the early 1800’s. Industry pioneers such as Bell, Atanasoff, Canova, and Dangermond have given the world incredible enhancements.  Isn’t it time these enhancements were put to use?

 

PLEASE NOTE: The postings on this site are my own

and don’t necessarily represent Esri’s position, strategies, or opinions.

When the power goes out, it is all hands on deck at a utility company. Everybody from all corners of the organization come together.  They assign damage assessment trucks to finance and transmission trading employees.  Lineman are working around the clock to get critical assets back up. With all this calamity, having real-time information to make decisions is key to success. Communicating information within the utility effectively, utilities can shave minutes to hours off of their recovery time.  Every minute matters when you have many customers without power and some of those customers require electricity to live.

Utility companies face significant challenges when the lights go out.  They need to be able to simultaneously manage their assets, their workers, and their customers. Using ArcGIS to store, analyze, and visualize data, employees from planning to operations to customer care can all work harmoniously from the same sheet of music. The ArcGIS Platform figuratively lights up the utility with GIS, allowing the utility to literally light up the community. 

Here is a new video showing Operations Dashboard for ArcGIS.  This easy to configure dashboard effectively tells the current scenario of where are the utility's damaged assets, where are their workers, and how many customers is the outage affecting.  Sharing information freely across any organization will improve operational efficiency and customer satisfaction.  Outage Management System - Real-Time Monitoring - YouTube 

 In recent years, severe storms have crippled major cities across the United States. These storms, consisting of events such as hurricanes, heavy rain and flooding, winter storms, and tropical storms, have left hundreds of thousands of people without power. Communicating information about power outages to a connected public, the media, and various agencies is a top priority for most utilities. This transparency promotes better communication and collaboration with your constituents.

Utility companies face significant public safety and public relations challenges when a network outage occurs. To meet regulatory requirements and address business needs, they must provide outage information to customers in a timely and accessible manner. When an outage occurs, customers typically want to know if the utility is aware of the outage and when will their power be restored. This information needs to be available even when the power is off.

Traditionally, most utility companies manage their public-facing outage viewers internally because they are a critical part of creating a positive customer experience. Trusting an outside company to manage these types of critical apps is often viewed as impractical and expensive. However, this type of implementation approach requires a significant infrastructure (hardware, software, network bandwidth) investment and related IT support staff during the worst possible storm event. In reality, this infrastructure may only be needed less than 20 percent of the time in any given year. As a result, this dedicated environment often sits idle during clear, blue sky days.

Esri's Outage Viewer offering, combined with our managed cloud services capabilities, makes it possible for utility companies to leverage the cloud. We can configure and host your outage viewer in a secure, scalable, and reliable environment, which will reduce your total cost of ownership (TCO) and enable you to maintain positive customer sentiment during storm events.

Many utilities are engaging their customers with successful implementations of Esri's Outage Viewer.  These online solutions are live 24/7/365 and can be viewed with the following links:

 

Additional information is attached.

Finding those key buried devices and paths

 

By Tom Coolidge and Tom DeWitte

 

A gas utility or pipeline typically transports natural gas or hazardous liquids to customers through a large and complex network of interconnected pipes.  In addition to pipe, these networks are comprised of an even larger number of other components, including fittings, valves, regulators and many more, some of which can affect the flow of the fluid through the pipes.  Modeled properly, ArcGIS enables you to create a “digital twin” of all this complexity.  This is key as many solutions require that you be able to determine a path directionally from a location in your connected network to a separator or separators that bound it.  The utility network provides this capability.

 

It all starts with location.  I find that as I get older, I am more frequently asking myself questions such as; where did I leave my glasses, or where is my phone.  Resolving these questions usually entails me wandering about the house until I find those misplaced glasses or phone.  Finding these items is not that difficult because I can see my glasses sitting on a table or I can see my phone as it sits on the kitchen counter where I left it.

 

Now imagine you work for a natural gas or hazardous liquids pipe organization, and all of the assets you are looking for are buried three or more feet below the surface.  How do you go about finding a specific valve, fitting or cathodic protection anode?  The short answer is maps.  But, maps like traditional paper maps have their limitations in that when looking for a specific valve you must have a pretty good idea of where the valve is located in order to know what map sheet to look at, and where on that busy map sheet to look.

 

Digital maps are better, in that they allow you to search for a characteristic of the valve such as its assetID, manufacturer, size or type.  But, a digital map also assumes you have some knowledge already about the valve you are looking for.

So, what do you do when your question is about the pipe network, and how a specific asset participates in the pipe network?  This is where tools which understand how the assets connect to form the pipe network are required.  This is where you need tracing tools to know your pipe system.

 

What type of questions can be answered with a trace?

When managing a pipe system there are many questions that get asked everyday which require an understanding of how the pipe system works.  During an emergency, a very common and important question is: what valves do I need to close to isolate a section of the pipe network where damage or a leak has occurred? A common question asked by cathodic protection technicians is where is the nearest CP test point from my current location on the pipe system? Gas engineers who are evaluating a pressure zone ask the question; what are the regulator stations providing gas to this location?

 

What do I need to do to configure my gas system for tracing?

For a software system to be able to answer these common types of pipe system questions, an understanding of how the components of a pipe system connect is required.  It is not enough to simply draw a digital representation of the asset on a map, such as is commonly done with CAD software. In addition to drawing the digital representation of the asset on a map, there also needs to be an understanding that the two polyethylene pipe segments which have been butt fusioned together are connected.

 

This software understanding of connectivity is network topology.  Within the Esri ArcGIS platform, our latest version of network topology for utility systems is what we call the utility network.

 

Can I perform a trace in ArcGIS Pro?

Yes. Tracing your network can be performed within ArcGIS Pro version 2.1 or later.  Additionally, with the utility network being a service based solution, tracing can also be done with web applications, and eventually will be able to be performed by mobile applications.

 

Within ArcGIS Pro, the options for configuring a trace have been significantly enhanced when compared to the ArcMap geometric network tools.  It is now possible to dynamically answer questions by simple configuration of the properties of the trace tool.  For example, if you are trying to determine the amount of gas or liquid lost due to a break in the pipe, you need to know the volume of the portion of the pipe network which was isolated.  There is now a function property to the trace tool to allow you to summarize the total pipe volume of the trace selected pipe segments.

 

 

If you need to ask the question, what portion of my pipe system is upstream of a specified location, but only trace on those assets which are in production, and are open to allow the gas or liquid to pass through.  The ArcGIS Pro trace tool now supports the ability to use designated asset attributes such as LifeCycleStatus, DeviceStatus, Pincheable, and Insulator Device to dynamically constrain which assets the trace can traverse. This, too, is a simple configuration of the tools parameters.

Since the trace tool is a geoprocessing tool, your preferred configuration properties can be saved as a model and shared across the organization.

 

How do I configure a trace to find the nearest asset?

Being able to find the nearest type of asset such as a regulator, valve, or CP test point, is another useful new addition to the capability of the trace tool.

 

Simply checking a box within the filter options will constrain the trace output to the specified features which are closest based on the distance traversed across the pipe network.

 

How do I configure the trace tool to find the sources feeding a gas subsystem?

The new trace tool within ArcGIS Pro contains some new trace options, such as subnetwork, subnetwork controller, shortest path, and loops.  When a planner or engineer needs to find the regulators feeding a specified location, the subnetwork controller option makes this an easy question to ask of the pipe network.

 

Tracing with the new utility network solution provided by Esri, is unique in its ability to allow gas and hazardous liquids pipe companies to easily ask questions of their pipe networks.  Databases alone cannot answer these questions.  CAD systems cannot answer these questions. Even GIS systems which do not include network topology cannot answer these questions.  Only a complete GIS system which includes network topology can answer these everyday questions about your pipe network. Only a network topology specifically built for management of utility systems such as a gas or hazardous liquids pipe network can provide the intelligent tools to help you know your system.

 

PLEASE NOTE: The postings on this site are my own

and don’t necessarily represent Esri’s position, strategies, or opinions.

Responding to notifications from 811 one call centers is a crucial function of any utility company’s business operations.  When a ticket from a one call center is received, utility companies have two full working days to send a locator to the excavation site to mark or locate their underground facilities.  The end-to-end workflow for an incoming ticket typically includes the following steps:

  • Prioritize the ticket
  • Pinpoint the location on a map
  • Assign to a locator
  • Track the status
  • Manage any associated notes and photos
  • Communicate the response with the excavator

Managing this workflow can pose as a challenge to utility companies who often receive several tickets from a one call center throughout the day, all while carrying out many other daily operations. Based on numerous engagements with utility companies seeking to automate and streamline the workflow outlined above, Esri Professional Services has created a package of services to configure and deploy a set of Python scripts and a mobile app for managing the lifecycle of tickets received from 811 one call centers.

 

See the attached PDF for further details.

Network Management News

for Utilities and Telecommunication Companies

We have exciting news to share.

  • New ArcGIS platform release: On January 18, Esri released ArcGIS Enterprise 10.6 and ArcGIS Pro 2.1, representing a significant step forward in the ArcGIS platform, especially for utilities. Read about the new features here.
  • New ArcGIS extension for utilities: An integral part of the ArcGIS 10.6 platform is the new ArcGIS Utility Network Management extension. We call this technology, simply, the utility network, and it represents the future of network management. We showcased the power of this new extension at DistribuTECH 2018.
  • Upcoming utility release for ArcMap: This summer we will release ArcGIS 10.6.1. This release will also include the next desktop utility release for ArcMap and the geometric network, and it will be supported until January 2024. This gives you plenty of time to plan and prepare for your migration to the utility network.

The Utility Network: A Quantum Leap in Network Management

The utility network launches a brand-new way for you to manage utility and telecom data. You can:

  • Share your network model with real-time systems
  • Create schematic diagrams automatically and make them accessible in the platform
  • Visualize your network assets in 3D
  • Build non-spatial network connectivity
  • Model devices inside other devices
  • Edit your data with ArcGIS Pro, the most modern GIS desktop software available
  • Create attribute rules and enforce rich business logic to prevent errors
  • Support annotation across the platform
  • Author ArcGIS Pro projects and share them using your ArcGIS portal

Esri will continue to expand the capabilities of the utility network over the next months and years.

In the spring, Esri will offer data migration tools to help you move to the new utility network.

Our summer release of the utility network will include:

  • Attribute rule enhancements
  • Client-side evaluation of rules
  • SDK developer enhancements
  • We have many other features coming in later releases including:
  • Mobile geodatabase support
  • Partial posting
  • Custom validation tools
  • A telecom model that includes fiber cables
  • Domains dependent on the value of other fields
  • Support for dimensioning
  • Better data export ability, including support for the IEC standard Common Information Model (CIM)
  • Support for disconnected editing

Continued Support for ArcMap and the Geometric Network

To give you time to prepare for a migration to the new utility network, we will continue to support ArcMap and the geometric network, including a new release this summer.

In 2015, we created a special utility and telecom release, 10.2.1, which we promised to support until June 2021. Once on this release, you did not have to upgrade your desktop technology. You only had to install utility patches as they become available. This made keeping up with the latest technology simple and easy. However, you have made requests related to security and platform support that we cannot accommodate with the simple patch process.

So, we are providing another option.

In the summer, you can upgrade to our 10.6.1 release, which will continue to support ArcMap and the geometric network. This is not a migration—it is a straightforward upgrade, similar to the patches you’ve already been installing. This release will allow us to offer necessary security and platform improvements. It will also be supported until January 2024, giving you more than two years of additional support with these familiar tools. In addition, we are working closely with Schneider Electric, who will be certifying ArcFM on ArcGIS 10.6.1.

We will continue to offer patches to both the 10.2.1 utility release and the 10.6.1 utility release.

The Road Ahead

The road ahead for network management is the utility network.

  • We have updated the Road Ahead for Network Management white paper.
  • We have posted our vision for the utility network on the ArcGIS blog.
  • We have also posted new blogs with information specifically for electric and gas
  • Our partners have also been preparing to offer their solutions as enhancements to the utility network. We urge you to talk to them about their plans and how their solutions will further enrich the utility network.

We look forward to your feedback. If you have questions or concerns please reach out to your account manager. They will bring the right resources to you.

Stay tuned for more updates on our exciting new technology.