Tracking and Traceability 2019 - Part 1

07-25-2019 01:33 PM
Labels (2)
Esri Regular Contributor
2 0 3,279

By Tom Coolidge and Tom DeWitte

Tracking and Traceability is now a well-established practice in the natural gas distribution industry supported by ArcGIS®.


ArcGIS mobile app advances over the last three years have helped adoption of Tracking and Traceability activity grow. Collector for ArcGIS has evolved to now include the ability to use a mobile device’s camera to read the ASTM F2897 barcode. Collector also now includes the capability to run arcade scripts in the pop-up window while the device is disconnected from the network.  Not to be overlooked, Esri also released a new enterprise geodatabase capability called attribute rules.


Those three new capabilities have enabled many gas utilities, and increasingly gas pipe installation contractors; to use Collector to capture the location, barcode, and other information about the newly-installed pipe and its related components. These new capabilities and lessons learned from the many organizations actively using Collector for the digital as-builting portion of the Tracking and Traceability workflow have resulted in a more efficient and streamlined process for performing these tasks.


The purpose of this blog is to give an overview of how the current version of Collector, when combined with an ArcGIS 10.7 or higher enterprise geodatabase, can result in a simpler and more efficient Tracking and Traceability workflow. A second blog article will follow with a detailed explanation of the new attribute rule arcade scripts which completely automate the decoding of the ASTM F2897 barcode and the automatic population of the derived attributes.

A quick review of Tracking and Traceability

PHMSA proposed rules in May of 2015 to 49 CFR part 192 to address the need for operators to better ‘track’ the details and location of assets after their delivery from the manufacturer or supplier.  The rule also speaks to the need for better ‘traceability’ of assets; meaning the ability to locate assets by material, size, manufacturer, model, or other attribute.


The ASTM F2897 standard, developed collaboratively by the natural gas industry and its leading suppliers, specifies a 16-digit alphanumeric barcode format that embodies identification of a pipeline component’s manufacturer, lot number, production date, model, material, diameter, and wall thickness.  This barcode standard is now a common piece of the manufacturer provided information for plastic pipe and its plastic components.  Additional efforts spearheaded by the Gas Technology Institute are currently underway to define a more advanced barcode standard which can be applied to both steel and plastic pipe and their components.  This barcode “thing” is not going away.  Just the opposite, it is going to expand significantly in the years to come.

Pattern Overview

The ArcGIS deployment pattern for Tracking and Traceability is comprised of four steps, as illustrated here:



Step 1: Digital as-builting

The recent improvements to Collector have made this process easier than it was just a few years ago.  The first enhancement was the revamping of the interface to simplify data entry. The second enhancement was to increase the certification of GPS vendors and their devices. Here is a link to the list of GPS receivers which can be used with Collector:

The third enhancement is the native ability of Collector to use the mobile device’s camera to capture the ASTM F2897 barcode.

With these enhancements, field staff can go into the field and capture the as-built information of the new construction using a smart device running Collector. The smart device is Bluetooth-connected to a high precision GPS antenna.  The field staff use the high accuracy GPS antenna to capture the location of the newly installed assets. The collected location data is directly streamed into Collector as native ArcGIS features.  No translation or conversation is required.  The field staff then manually input into Collector a minimal amount of information, such as Installation Date, and installation method.  The field staff then uses the device’s camera to capture the barcode and automatically populate the BARCODE attribute of the GIS feature.  The BARCODE value contains information about the asset, such as size, material, manufacturer and manufacture date.  Once the BARCODE value is captured, the field staff no longer need to manually enter this information.


The recent enhancement to Collector supporting the ability to run arcade scripts in the pop-up window, provides the ability to immediately display the decoded data to the field staff even when the device is disconnected.


An additional capability of an Esri mobile app on a smart device or tablet is the ability to capture photos of the newly installed assets.  These photos are automatically associated to the GIS feature.


When the field staff have completed the collection of the newly installed assets, the GIS features are submitted to the staging geodatabase.

Step 2: Contractor/crew assessable storage

A fundamental challenge of Tracking and Traceability is how to correctly integrate high precision GPS geospatial data, with less accurate legacy geospatial data.  A key component to overcoming this challenge is the staging geodatabase.  A staging geodatabase can be either hosted in ArcGIS Online as hosted feature layers or stored on premise with a local ArcGIS Enterprise implementation. The key purpose of the staging geodatabase is to provide an easily accessible data repository for the field crews to submit their collected construction information too.  The staging geodatabase only holds the newly collected construction information.  The construction data sits in the staging geodatabase until a mapping professional using ArcGIS Desktop accesses and downloads it to the enterprise geodatabase.


With the new enterprise geodatabase capability of attribute rules, it is possible to have the captured barcode value automatically read and used to auto-populate the derived attributes manufacturer, lot number, production date, model, material, diameter, and wall thickness.  If the digital as-builting described in step 1 happens while the device is connected to the enterprise geodatabase, then Collector will automatically decode the barcode, auto-populate the derived attributes and display the decoded information immediately after the new/updated GIS feature is submitted by Collector. In the second blog, we will provide links to these arcade scripts and describe how to apply them to an enterprise geodatabase.

Step 3: Append to enterprise geodatabase

One of the time saving capabilities of ArcGIS Desktop is the ability to interact with data from both the staging geodatabase and the enterprise geodatabase at the same time.  This allows the mapping professional to easily select the staging geodatabase features and append them into the final enterprise geodatabase feature classes. 


If the staging geodatabase layers are stored in ArcGIS Online, the previously described attribute rule arcade scripts can be applied to enterprise geodatabase layers. 


NOTE: Attribute rules only work with ArcGIS Enterprise 10.7 or higher. Additionally, ArcGIS Pro is the only desktop tool to understand attribute rules.  If using ArcMap and a geometric network, it is important that the staging geodatabase layers be stored in an enterprise geodatabase and the attribute rules are applied to the staging geodatabase layers.


The standard arctoolbox geoprocessing append tool can be used to copy the newly collected GIS features from the staging geodatabase layers to the final enterprise geodatabase feature classes.

Step 4: Mappers connect digital as-built with gas system

With the new construction data now appended from the staging geodatabase into the enterprise geodatabase and the barcode value decoded, the mapping professional now needs to determine how to connect the high precision geospatial features with the less accurate geospatial features. The outcome of this process needs to honor two data requirements:

  • Connecting the new features with the legacy features to create a single topologically connected gas pipe system.
  • Preserving the high precision GPS collected geospatial coordinate data.


The recommended best practice for accomplishing this seemingly disparate set of requirements is for the enterprise geodatabase point features such as Meters, Excess Flow Valves, and Non-Controllable Fittings to have the following attributes added: SPATIALACCURACY, GPSX, GPSY, GPSZ.  Here is another example where attribute rules can streamline the population of these GPS fields.  If using ArcMap and the geometric network, then a configuration of Esri’s Attribute Assistant tool or ArcFM’s AutoUpdater capability can be used to automatically populate these fields.  This will preserve the original GPS location values, which can be used later to rubbersheet all features (legacy and GPS) to the more accurate GPS location preserved in the GPSX, GPSY, and GPSZ attributes.  With the GPS location preserved, the mapper can adjust the new construction features as required to connect to the legacy gas pipe system.

Business value of using ArcGIS platform

This approach to Tracking and Traceability provides an opportunity for the GIS department to once again show the greater gas organization that not only can the GIS Department provide a solution which addresses this new common industry practice, but it can do so in a manner that improves the operational efficiency of the gas organization.  This pattern improves the operational efficiency of the gas organization and their contractors as follows:

  • Using Collector to collect construction data improves location accuracy and attribute quality by eliminating translation to paper and interpretation of paper based information.
  • Bluetooth integration with high precision GPS antennas improves the speed at which data is collected.
  • Capturing the barcode value reduces the amount of information the field staff manually collects. Material, diameter, manufacturer, manufacture model, manufacture data, manufacture lot number are all automatically populated by the decoding of the barcode.
  • Digitally collected data is immediately available for GIS department to process into enterprise geodatabase. This eliminates the historical latency problem of the GIS department waiting for the inter office mail transmittal of the construction packet.
  • The GIS department mapping professional task of updating the as-built representation of the gas pipe system is simplified. The mapper is no longer manually transposing paper based red-line drawings, but instead appending field collected geospatial features.  This improves the speed at which a mapper can complete the task of updating the as-built representation of the gas pipe system.
  • Safety of field operations staff is improved by providing the new construction data in a timelier manner.


This deployment pattern not only provides the ability to improve the efficiency of the field data collection, it improves the productivity of the mapping professional, and provides new construction updates to locators and field operations staff in a timely manner.


Next blog

In our next blog, we will dig into how to configure and deploy the arcade scripts for this solution to Tracking and Traceability.

PLEASE NOTE: The postings on this site are my own and don’t necessarily represent Esri’s position, strategies, or opinions.

About the Author
Technical Lead for Natural Gas Industry at Esri