This is the second of two blogs from the Esri Utility Team on the use of the ArcGIS Platform to address the proposed Tracking and Traceability regulations published in May 2015, by the Pipeline and Hazardous Materials Safety Administration (PHMSA) of the United States Department of Transportation.
The first blog can be found here: Tracking & Traceability – Part 1
In this second article we will take a closer look at how to configure and deploy the ArcGIS platform to address this proposed rule.
The Tracking and Traceability Solution Pattern Download
Along with this blog, we have provided a packaged download that contains all of the scripts, MXD’s, and documentation that is required to test and deploy the solution. In the .ZIP file you will find:
- TrackingTraceability Toolbox
- Staging MXD (GasEditing_TrackingFieldGDB_104.mxd)
- Production MXD (GasEditing_TrackingOfficeGDB_104.mxd)
- Configuring the Append Staging GDB Data Tool.pdf
- Sample Enterprise GDB (UPDM_Production.gdb)
- Sample Staging GDB (UPDM_Staging.gdb
Configuring the Tracking and Traceability Solution
The rest of the blog will run through the full Tracking and Traceability workflow, highlighting all of the resources listed above. With all of these resources, you’ll have everything you need to get started.
Preparing the Enterprise Geodatabase
The first step to setting up this workflow is ensuring your Enterprise Geodatabase has the required attributes and coded value domains to decode and parse the barcode values.
To prepare your existing Enterprise Geodatabase for Tracking and Traceability, all assets represented will require the following attributes:
Point Asset Featureclasses
Line Asset Featureclasses
The most recent update to the Utility and Pipeline Data Model (UPDM) contains all of the required attributes, their specific schema definitions, and coded value domains of the ASTM F2897 barcode standard for the barcode decoding script. The most recent UPDM schema can be downloaded from the ArcGIS Solutions site at this link: https://community.esri.com/docs/DOC-8884-updm-2016-pre-release
The Production MXD included in the Tracking & Traceability .ZIP file contains a group layer for the hosted Staging Geodatabase layers (Tracking Field Geodatabase), which contains broken links because this data has not yet been published.
After your Enterprise Geodatabase is ready to accept the decoded barcode values and the appropriate ASTM F2897 coded value domains have been assigned, you are ready to start collecting data in the field.
Creating the Staging GDB
The next step involves getting your as-built feature service ready for field users. The Staging MXD (GasEditing_TrackingFieldGDB_104.mxd) is used to publish your hosted feature service, which will enable users to capture data in the field.
Opening up the Staging MXD (GasEditing_TrackingFieldGDB_104.mxd) file, you will find all of the symbology for your layers, but no data. This is simply a blank data set that contains the full UPDM schema. By publishing this blank schema and symbology, it will allow you to collect all of their field data in the same format as the enterprise geodatabase, to ensure that it can transfer over seamlessly during the decoding process.
Now, simply log in to your ArcGIS Online or Portal account and publish this MXD as a feature service with editing enabled.
Deploying Collector for ArcGIS
After publishing out the ‘StagingGDB’ feature service, you can now create a web map to be used inside of Collector for ArcGIS. Open a new web map and add the Staging feature service. The full UPDM layers will contain many more attributes than your field user will collect. Use the configuration properties of the web map to hide all attributes which will not be edited by the field staff.
Additionally, since the BARCODE attribute contains information about the asset, you can hide values such as ‘material’ and ‘size’ as well, to avoid having field users enter data which will be derived from the barcode value. After saving your map, ensure that the feature service and map are shared to the proper groups and that syncing is enabled if you want users to take the map and data offline.
Collector for ArcGIS also enables you to capture location using a high-accuracy GPS device, along with the barcode using an external barcode scanner. The location of the assets, the full bar code, and photos can all be collected in one record (Figure 1).
With Apple devices, you can use an external barcode scanner connected via Bluetooth (we have tested the Socket Mobile CHS 8Ci). Android devices give you the ability to use the on-board camera to scan barcodes. For more detail on pairing your device with an external scanner or high-accuracy receiver, please visit these links: Do more with Collector and Use high accuracy GPS.
When loading up the map in Collector, you will see the filtered schema that was configured in the web map. You have the ability to place a new project point that can be used to manage each projects status, to ensure that management has a clear understanding of progress. Next, you can then fill out the required attributes and scan each piece of equipment at the point of installation, providing both a highly accurate GPS reading and the full barcode. A picture can also be captured at the time of installation that is attached to the record.
If using Collector for ArcGIS in a connected mode, the data and attachments are synced directly with the hosting server. If you are using the application in a disconnected mode, you can sync edits as soon as you regain connectivity.
As soon as a project is completed, you can change the status of the project location point to ‘Completed’. Thus, the project status can be tracked along with capturing all of the as-built location, attribute, and photos in one feature service.
Construction Project Dashboard
As contractors or field crews are conducting their work, management can easily monitor and track progress using the project status features.
Using Operations Dashboard for ArcGIS, you can configure a dashboard that consumes the same ‘StagingGDB’ feature service that field crews are editing. Using the configurable widgets, you can track project status, view photos, and get any summary information required (Figure 2). All of this information will update in real-time as changes are made, or as soon as edits are synced with the server.
Data Loading and Barcode Decoding
After the data is collected in the field, it can now be quickly and easily brought into the Enterprise GIS.
After opening up the Production MXD (GasEditing_TrackingOfficeGDB_104.mxd), replace the Tracking Field GDB group layers, with the StagingGDB hosted feature service. The map should now contain all of the enterprise GIS data along with the hosted, as-built feature service.
Using the Catalog window in the ArcMap session, navigate to the Tracking and Traceability project folder and locate the TrackingTraceability geoprocessing toolbox. Running the ‘Append Selected Staging GDB Data’ tool will prompt users to select their project area and hit OK (Figure 3). The tool then copies the hosted features over to the Enterprise Geodatabase and decodes all of the barcode values in the process.
Operators now have the high-accuracy location of as-built devices in the field, along with all of the attributes decoded from the ASTM F2897 specification, along with any photos or attachments.
As soon as the need for locating (tracing) assets by installation date, manufacturer, material type, etc., arises, you can quickly locate those assets with confidence using the query tools.
Connecting New Features to Legacy Gas Pipe System
With the field GPS collected features now in the Enterprise Geodatabase you will most likely need to adjust the location of these features to correctly connect to the existing gas pipe system. To preserve the precision of the originally collected features, add the following attributes to your Enterprise Geodatabase:
The AttributeAssistant tool available for download from the ArcGIS Solutions site, provides the ability to have the GPSX, and GPSY attributes automatically populated as part of the download and converted from original projection to Latitude/Longitude. This will preserve the original GPS data for future use to conflate the entire gas system to the more accurate GPS locations.
The AttributeAssistant tool can be downloaded at:
Tracking and Traceability is a great example of how the ArcGIS Platform can be utilized to not only help gas organizations address their regulatory needs, but also accomplish them in a workflow that can also improve the efficiency and effectiveness of the organization.