Latest Contributions by TomDeWitte

  By Tom DeWitte and Tom Coolidge The natural gas industry’s drive to a lower or net zero carbon footprint is focused on actions that generally can be organized in three groups: (1) reducing methane emissions from operations, (2) reducing methane emissions from customer consumption, and (3) embracing new sources and uses of alternative lower carbon fuels. This blog focuses on a key activity central to reducing methane emissions from operations – finding and fixing leaks. Managing methane emissions is without a doubt as critical a set of tasks as any every natural gas pipe organization performs.  For the owners and operators of the pipe networks which transport this critical energy source, managing methane emissions starts with surveillance programs to identify leaks. Class 1 leaks are immediately fixed. But low-level leaks are studied via engineering analysis to rank the risk and consequence of those leaks. Once analyzed, construction activities can be scheduled to repair the identified and qualified leaks. This very straightforward set of tasks to “find the leak” and to “fix the leak” have been going on for decades. So, what’s new in the world of methane emission reduction? What’s new for methane emission reduction is the availability of new sensor platforms to “find the leak”. Many Platforms for Detecting Methane Finding the leak has long been a human-intensive process of walking the pipe network. While walking, field technicians will carry methane emission detectors. Once a leak is found, a gas leak report is created to document the location, quantity, and likely source of the leak. Then came the ability to mount methane sensors on cars and trucks. These vehicles could cover more miles of pipe than the walkers. Advancements by manufacturers of these devices have further enhanced these vehicle- mounted sensors to better estimate the location of the source of the leak. These potential leaks are then investigated by field technicians to verify and complete the gas leak report.   More recently, methane emission sensors have been mounted on aerial platforms ranging from drones to helicopters to planes. In addition to aerial mounted sensors, there are now satellites circling our planet in low earth orbit with methane sensors. These satellite-based sensors sell their data and analytics, providing gas pipe operators with yet another capability to survey the pipe network and “find the leak”. Still Need Those Humans These newer tools to surveil the pipe network do not replace the regulatory requirements of Leak Survey. But they do provide a new affordable capability for higher frequency monitoring. Then when a methane emission is detected, a field technician can be sent out to the emission area to validate the sensor reading and fill out the gas leak report with the details needed for the organization to “fix the Leak”. As powerful as these additional methane detections are, humans are still required to verify and document every leak. Need a Common Repository Adding additional platforms of methane detection opens new opportunities to reduce methane emissions.  It also creates a new, but manageable, data management issue. With these new sensor platforms to identify potential methane emissions, a common data management system is needed to capture, track, and document the status of these potential leaks. With location being critical to each potential leak, A geographical information system (GIS) is the best data management system for aggregating and managing this information.   Critical Information for Gas Pipe System Management Combining the information from these many methane detection platforms into a single data management system is not the finish line.  It is at best mid span. This data still needs to flow into analytical programs such as DIMP and TIMP. This data still needs to be monitored with dashboards, summarized with reports, and communicated to gas executives, regulators, and the public. This data needs to flow to construction and mobile viewing devices enabling the gas organization to “fix the Leak”. That is the finish line. The natural gas industry is enjoying a technological revolution. New technologies are continuously being applied to better solve gas industry problems.  And the gas industry is embracing and implementing these new capabilities at a faster pace than ever before.  Placing your GIS at the center of your methane emission management ensures the data collected by all these new sensor platform technologies is fully leveraged throughout your gas organization. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

By Tom DeWitte and Tom Coolidge What keeps pipe network utility executives up at night? One common cause is the worry about the consequences of a failure that results in negative impacts to people, property, or the environment. One-way utilities pro-actively seek to pre-empt pipe network failures is investment in repairing or replacing those parts of their network thought most likely to possibly fail. Every year utilities determine which sections of their pipe system will be replaced as part of their capital construction budget. How do utilities know which sections of the pipe system need to be replaced? What are the criteria that should be used to identify and prioritize the sections of pipe for replacement? To many pipe engineers, the answer is to perform a pipe risk analysis. Pipe risk analysis is a method of identifying criteria which will be used to rank which sections of pipe should be replaced. These criteria are quantified and weighted against each other using an equation to tally the total risk score for each section of pipe. What criteria do you use to measure risk? When dealing with pressurized pipe systems, such as natural gas, hazardous liquids, district energy, and water the initial criteria list typically starts with these items:                 -pipe material                 -pipe age                 -pipe installation method                 -pipe insulation/coating materials                 -leak history These items are valid criteria that contribute to the likelihood a pipe system may fail. These types of data are easy to put into a spreadsheet and tabulate. But none of these measures the consequence of those failures and the risk to the organization. Understanding the risk to the organization can only be accomplished by including a measurement of the consequences. Understanding Risk to the Organization Risk to the organization is typically summarized as a cost, with the unit of measure being monetary ($). This cost is much more than simply the expense to the utility to have the construction group replace the identified deficient section of pipe.  When a pipe section fails, it impacts the people and facilities near the location of failure. This consequence cost to the organization often greatly exceeds the cost to replace.  The recent situation of Pacific Gas & Electric declaring bankruptcy after the involvement of its electric transmission lines in forest fires is the exclamation point example.  Every pipe utility industry has multiple examples they can point to which validate that the consequence cost of a failure greatly exceeds the cost to replace. Property damage, loss of life, loss of business revenue, civil lawsuits, and government regulatory fines are just of few of the consequences of failure and its resultant cost to the organization. To measure the risk to the organization, we need spatial tools     Quantifying the consequence of failure begins with understanding “where.”                 -Where are the properties and facilities nearby pipe segments?                 -Where are the persons who reside or congregate near the pipe segments?                 -Who are the customers downstream of the pipe segment who will be impacted by the failure? Understanding the relationship between your pipe sections in relation to critical facilities, persons, and customers, requires spatially aware-analysis tools. Spatially aware tools, such as those provided with Esri’s ArcGIS products, enable pipe utilities to identify, aggregate, calculate, and quantify these consequences. Understanding Who is Nearby Every person who resides or congregates near a pipe section is at risk of being impacted by the failure of that pipe section. As stated previously, this is key to quantifying the overall risk to the organization. Spatial analysis methods, such as buffer, can define the area near the pipe segment to look for persons and places of gathering. More advanced spatial analysis methods, such as least cost path analysis, enable pipe utilities to identify the direction in which spilled liquids will flow. Once these spatial areas near the pipe segment are identified, it is a simple intersect analysis against data sets which identify the number and type of persons who reside or congregate within the now identified areas of impact.  With this analysis capability, pipe utility organizations can quantify the consequence of the failure of a pipe segment to persons. Understanding What is Nearby The cost to the pipe organization of the failure of a pipe section includes not only the impacted persons, but also the impacted facilities. Having your own facilities damaged is bad enough, but also having to pay to repair the facilities of other utilities can be much more expensive. Understanding the consequence of failure to facilities goes beyond electric, telecommunications, and other pipe utilities, it includes transportation network systems such as road and railroad. No pipe utility wants to be the one that caused a major road route to be closed due to the damage caused by the pipe section failure. The same ArcGIS analysis tools used to identify persons impacted by a pipe section failure (Buffer, Least Cost Path, Intersect, Identity) are the tools used to identify and quantify what facilities are impacted.   This understanding of the consequence of failure to nearby facilities further clarifies the true risk to the organization. But there is still one more critical consequence to be understood and quantified.  That is the consequence to the pipe utilities customers who are downstream of the pipe section which has failed. Understanding Customers Downstream When a pipe section fails, customers downstream of that point of failure will be without the transported commodity until repairs are completed.  This means that natural gas and district heating customers could be without heating in the middle of a winter deep freeze. District cooling customers could be without air conditioning during a summer heat wave. Both consequences of failure can literally result in death to customers. Programmatically understanding and quantifying the impact to a pipe utility’s customers requires a different set of spatial analysis tools. These different spatially aware analysis tools need to understand how the pipe network of pipes, valves, and fittings connect to create a pipe network. Only when a software application understands how a commodity flows through the pipe network can the analysis quantify the consequence of failure to the customers. This pipe network aware tool which is critical to measuring the consequence of failure to customers is the downstream trace.  The downstream trace is a configuration of the Trace tool provided with ArcGIS Utility Network. With this type of spatially aware analysis, it is now possible to quantify how many critical facilities such as hospitals, senior living complexes, refineries, power generation plants, and major manufacturing facilities are impacted by the failure of a pipe section. It’s a Spatial Thing Understanding the total risk to a pipe organization requires spatial awareness.  This spatial awareness empowers pipe organizations to improve their capital improvement plans. Engineers leveraging spatially aware tools such as those provided with ArcGIS can quantify the multiplier impact consequences have on threats of failure. Knowing how your customers will be impacted by a pipe section failure is another type of spatial analysis that is also key to measuring risk to the organization. As you can see, pipe risk analysis is spatial. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions.     ... View more

Part 4 of 4 By Tom DeWitte and Tom Coolidge Talking to someone about Tracking and Traceability is typically a discussion centered on a checklist of functional capabilities. Although this is a very important discussion to have, too often the discussion ends when the checklist of desired functionality ends. What is missing is the discussion of how much productivity can the organization gain by replacing legacy processes with a digital process. The discussion of how much productivity can be gained is the base question that every field user, supervisor, and executive should be asking.  The measurement of productivity will be different for each of these gas organization personas.  The field user for example, measures productivity based on time.  How fast can I capture the required information, so I can move on to the next task? The manager has a broader view of the workflow. This person realizes that documenting new construction is a multiple-person activity, involving both field and office staff.  From the manager’s perspective, a measure of productivity would be how fast can the information get from the field construction group, through an office-based quality review and approval process, then posted back out to the entire organization. The executive understands both productivity measurements and adds in the financial cost to the organization. The executive wants to make sure the overall workflow does not include redundant data entry, and has a low IT cost of ownership, as these require the involvement of additional employees. Automating the Field Task The field user responsible for documenting the new pipe construction is always in a hurry to move on to the next task.  Therefore, spending time entering information into the mobile application is not on their top ten list of things they enjoy doing.  This is where ArcGIS, with its attribute rule ability to automate data entry for users of the ArcGIS Field Maps mobile application, can significantly reduce manual data entry. In previous blogs we have presented examples of how attributes rules can decode scanned barcodes and automatically populate the derived data fields. This automation has been extended to handle situations where there are no barcodes. This situation occurs with steel pipe/components and plastic pipe/components where the barcode is unreadable. In this situation, attribute rules are used to auto-populate many data fields by asking the field user to select a value from a picklist on a single data field. The attribute rule then reads a lookup table to retrieve the selected component information and auto-populate the derived data fields. The use of attribute rules for automation can also be used to automatically populate information without the field user entering any information.  A good example is the automatic population of construction project information. Construction projects tend to have a name and a uniqueID.  These construction UniqueIDs and names are the digital glue which link all the field collected data together. With an ArcGIS-based solution, this tagging of assets, pressure tests, exposed pipe inspections, and the many other construction documents is automatic. A configuration of relationship classes and attribute rules is used to free the field user from having to enter this information. Reducing Time to Post State regulators are increasingly pushing gas organizations to reduce the time it takes an organization to post construction changes to field users such as locators.  The simple reason for this push is to improve safety for the field workers and the community to which they provide service. The legacy paper based as-builting workflow is a process that commonly measures itself in weeks, and sometimes months.  A lot of excavation can occur in this lengthy duration between when the construction is completed and when updated documentation is available to field staff to be aware of this new construction. Companies that have migrated to a digital field as-builting workflow are seeing their overall construction to posting times reduced from months/weeks to days.  For example, Spire Energy out of St Louis, MO, has been using ArcGIS mobile apps for capturing their construction changes for many years. Over that time their time to post has decreased from weeks to less than 8 days. And it is still decreasing! Eliminating Redundancy How did companies like Spire Energy reduce their construction to posting time down to less than 8 days?  The answer is that they eliminated a fundamental redundancy in the workflow.  Paper based as-builting has a built-in very expensive duplication.  That duplication is the redline documentation in the field of the construction changes, and then having an office person read, interpret, and recreate the documentation in the final as-built documentation system. With a digital field as-built workflow, this redundancy is eliminated. The field worker can configure ArcGIS Field Maps to integrate with other field devices, such as GNSS receivers, laser range finders, and the mobile device’s native camera for barcode scanning. These integrations allow the field user to digitally capture the new construction data and have this data sent directly to the ArcGIS data repository upon submittal. The office worker is no longer recreating the field collected construction data.  Instead, they verify the completeness and quality of the data, then post it for access by the entire organization’s users.  Low IT Cost of Ownership Often hidden in the as-builting workflow is the time required to get the documentation from the field to the office and after posting, back to the field. In the legacy paper-based redline as-built workflow, the hidden cost is the time for the construction packet to get to the office. In the digital as-built workflow, there is another hidden cost.  That is the IT environment required to get the data from the field to the office. Some digital as-builting solutions require that the data must pass through multiple data environments. Those multiple data environment solutions often add another hidden cost. This undesirable hidden cost is the need for IT to create and maintain extraction, translation, and load (ETL) processes to get the data through these multiple data environments and into the hands of the office staff. The ArcGIS platform eliminates this inefficient hidden IT cost of ownership. With the ArcGIS mobile application, ArcGIS Field Maps, the field collected data is transmitted directly to the ArcGIS data store. There are no duplicate data stores, no expensive ETL process. As you can see, the choices a gas organization makes in determining how to digitally document new construction can result in a significant difference in time and cost. Selecting a solution which is functionally complete, and automates much of the data entry, results in field staff completing documentation faster. Selecting a solution which is functionally complete, and eliminates duplicate data entry, results in significantly faster workflows, such as documented by the existing ArcGIS implementation at Spire Energy. Selecting a solution which is functionally complete, and eliminates hidden IT administration, results in a lower cost to IT administration.   The ArcGIS platform with its ArcGIS Field Maps mobile application is the unique solution configurable to provide the functionality and productivity field users, managers, and executives are looking for. If you are interested in deploying this configuration of ArcGIS Field Maps, we have posted all the scripts, data model templates and instructions for this digital field as-builting solution. You can download this configuration from the Esri Community site for free. Here is the link. This blog article is the fourth in a series of four blogs articles explaining how to deploy ArcGIS Field Maps for Digital Field As-builting.  If you missed our previous blog articles on digital field as-builting, here are the links to those articles. Part 1: Digital Field As-Builting with ArcGIS Part 2: Digital Field as-Builting with ArcGIS: No barcode No Problem Part 3: Digital Field As Builting with ArcGIS: Pressure Tests PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

Part 3 of 4  By Tom DeWitte and Tom Coolidge Digitally transforming the gas and pipeline industry construction packet initially has been focused on digitally transforming the redlined paper maps into digital data. Most of the initial Tracking and Traceability solutions on the market have been almost exclusively focused on this part of the construction packet.  What about the rest of the construction packet? A pipe construction packet contains more than marked-up paper maps. It contains pressure test results, exposed pipe inspections, checklists, daily reports, and more.  What about digitally transforming those documents? In this blog article we will discuss how to configure and automate the data capture of the pressure test.  Pressure test documentation has historically been a critical piece of information that too often is relegated to the bottom of the document pile, filed at the back of the cabinet, boxed, warehoused, and eventually misplaced never to be found again. If you are responsible for managing or determining a pressure zone’s maximum allowable operating pressure (MAOP), then this frustrating reality most likely results in a palm slap to your forehead. A key benefit of digitally transforming the documentation of pressure tests is that this information is immediately tied to the tested assets. As a related record to the asset, it is a simple click to retrieve this information. No more searching through warehoused boxes of documents to find this critical information. What is a Pressure Test? Pressure testing is an industry standard practice. The purpose is to identify issues which would result in system failure and the release of gas prior to natural gas flowing through these new assets. Simply put, it is to insure the newly installed components are part of a safe and reliable pipe system.   The physical action of performing a pressure test is not complicated.  You insert an inert substance, such as air or water, into the newly installed portion of the pipe system. The new subsystem is then pressurized to the desired test pressure. Once pressurized, the subsystem is then monitored for a specified duration to verify the pipes, valves, and fittings are holding pressure and not leaking. The final step is to document the test to persist this important information for future analysis and engineering of the pipe system. Historical Issues Documenting a pressure test has two primary components.  The results of the test itself, and the identification of which pipe system components were tested. Historically this was all done on paper. It was time consuming. Field crews had to sketch the portion tested and identify the components which were included in the tested subsystem. Not only was this prone to error and data omission issues, it was redundant data documentation. Field crews were being asked to redraw the new pipe system for the pressure test after separately drawing those components for the redline as-built documentation. Early attempts to convert the paper documentation into digital documentation often focused on eliminating the redundant sketch creation. For example, one legacy solution asked the field user to manually click on each unique asset which participated in the pressure test.  This eliminated the redundant creation of the sketch, but at a cost of significant loss of productivity.  Imagine how long it would take you on your phone, tablet, or laptop to manually select every fitting, valve, and pipe segment for the pressure tested subsystem.  This could easily include over 100 unique items.  This legacy digital approach, in addition to taking more time to complete, was still prone to data omission issues.  Easier, Faster, and More Accurate This industry problem of how to efficiently and accurately document pressure tests is a great example where a spatially aware mobile software application can uniquely solve this problem.  There is a solution that provides the field user with an easier process for documenting pressure tests in a manner which is faster and more accurate. So, what exactly is the secret sauce that a geospatial mobile solution can provide?   The answer is polygons. Our Secret Sauce A geospatially aware mobile application, such as ArcGIS Field Maps, understands which of the newly installed pipe segments, valves, and fittings are contained within a polygon’s extent.  This geospatial understanding eliminates the need for a field user to manually select the pipe system components.   Using a polygon to represent the pressure test extent simplifies the documentation of the pressure test down to two steps. Step 1: Draw a polygon around the components of the pipe system which were tested. Step 2: Document the results of the test itself. The two key configurations to ArcGIS Field Maps to enable this automation is the polygon layer and an attribute rule. The Pressure Test Polygon The pressure test polygon feature is the persisted record of the pressure test.  It is enabled with attachments to allow photos of the pressure test wheel to be stored as part of the pressure test record.  The specific information a utility wishes to capture about the pressure test, duration, pipe test medium, who performed the test, etc., is the schema of the polygon. This polygon pressure test record fully captures where, when, who, how, and what was tested. Documenting Pressure Test Results For most utilities, defining the schema is a onetime process of converting the questions on the paper pressure test form into a digital smart form. The schema can include picklists, default values, and date pickers to eliminate typos and speed-up the data entry process. Persisting Link Between Asset and Test A key part of the automation of the pressure test is to tag all newly installed assets which were tested.  This is where an attribute rule is implemented. The attribute rule will compare the extent of the polygon against the pipe system components which were documented as the initial part of the digital field as-builting workflow. Here is the attribute rule arcade script to automate the assigning of the pressure test id for the staging line featureclass. //Rule Name:  StagingPressureTest_PressureTestID_StagingLine //Description: Push StagingPressureTest attributes to StagingLines //Type: Calculation //Subtype:  All //Field:  pipetestpressure //Editable:  checked (true) //Trigger:  Insert, Update //Error Code:  7 //Error Message:  Couldn't push StagingPressureTest attributes to StagingLines //Evaluate from application evaluation:  checked (true) // Get pressure test id from the polygon feature var retVal =  $feature['pipetestpressure']; var pressureTestId = $feature['PressureTestID']; var feature_set = FeatureSetByName($datastore, 'StagingLine', ['OBJECTID'], false); // Find line features that intersect the polygon var intersected_features = Intersects($feature, feature_set); //If need to exclude certain types of features tweak the next line and change the "for" loop to use: filtered_features //var filtered_features = Filter(intersected_Features, 'assetgroup not in (10, 12)') //add features to the update dictionary var updates = []; var i = 0; for (var feat in intersected_features) {     updates[i++] = {'objectid': feat.objectid,                     'attributes':  {                         'PRESSURETESTID': $feature.pressureTestID                     }                 } } //return the dictionary return {     'result': retVal,     'edit': [{         'className': 'StagingLine',         'updates': updates               }] } This attribute rule is automatically initiated as soon as the field user submits the pressure test polygon. Geospatially Aware Mobile Applications This configuration of ArcGIS Field Maps shows how the needs of documenting pipe construction information, such as a pressure test can be uniquely addressed. Geospatially aware mobile applications provide the secret sauce for automating the documentation with the ease of use and productivity field users are looking for.   It is also important to note that automating the tagging of the pressure tested assets with the pressure test unique ID improves data quality. This mobile application writing of the pressure test ID onto each of the assets, means that this critical data will no longer be lost to the stack of boxes in the warehouse. If you are interested in deploying this configuration of ArcGIS Field Maps, we have posted all the scripts, data model templates, and instructions for this pressure test configuration and the entire digital field as-builting solution. You can download this configuration from the Esri Community site for free. Here is the link.This blog article is the third in a series of four blogs articles explaining how to deploy ArcGIS Field Maps for Digital Field As-builting.  If you missed our previous blog articles on digital field as-builting, here are the links to those articles. Part 1: Digital Field As-Builting with ArcGIS Part 2: Digital Field as-Builting with ArcGIS: No barcode No Problem Next blog article will continue leveraging the productivity enhancer geospatial awareness.  Blog #4 will divulge the secret sauce of configuration and automation for managing the construction project, and how to automatically link all construction project digital documentation together. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

The attached  zip file contains the template for ArcGIS for Digital As-Builting.  This is an advanced configuration of ArcGIS Field Maps for the purpose of enabling field crews to digitally capture new pipe construction documentation. The capabilities configured into ArcGIS Field Maps are the following: Decoding the ASTM F2897 barcode and auto-populating the derived attribute fields Feature templates for automating non-barcoded asset attribution such as steel pipes Automatically assigning Project information to each asset Automatically assigning Pressure Test ID to each pressure tested asset The purpose of these configured capabilities is to streamline the field data capture process, by automating the population of much of the UPDM 2020 data fields describing the asset and required for accommodation of the Tracking and Traceability guidelines. ... View more

Part 2 of 4   By Tom DeWitte and Tom Coolidge Our first blog of this series provided an overview of how ArcGIS Field Maps is the secret sauce for a cost-effective digital field as-builting solution. This is a solution that integrates with a high accuracy GPS antenna and a mobile computing device. If you missed it, you can access it here.. In this blog article, we will divulge how to configure and automate asset data capture.  This is a configuration of ArcGIS Field Maps which can be applied to the documentation of both plastic and steel assets across the gas pipe network. Pipe organizations need to maintain an ever-increasing amount of information about the assets they install into the pipe network.  No longer is it good enough to only know the material, size, and installation date. Today’s data management, integrity management, cathodic protection sizing, and hydraulic modeling systems need to know more. Historically the solution to capturing this additional information has been to simply ask field crews and office mappers to manually enter more information.  This is not exactly the ideal solution for an organization that needs to operate efficiently. Digitally capturing the geospatial location and characteristics of an asset with sub-foot spatial accuracy and high data quality attribution requires a software application that is capable of a diverse set of data capture methods which all work in a similar workflow. The gas industry is unique across the utility spectrum due to it having industry standards which only apply to a portion of the pipe network. When installing plastic pipe, there is the ASTM F2897 barcode standard. When working with steel pipe, there is no industry barcode format. In a single workday it is not unrealistic to expect that a field crew could be documenting installed plastic pipe with barcode, installed plastic pipe with unreadable barcodes, and steel pipe with no barcodes. The gas and pipeline construction industry therefore needs a single application with an easy to use interface which is capable of automated data capture for these different situations. Capturing Plastic ASTM F2897 Barcodes ArcGIS Field Maps is a mobile application which has seen several enhancements to specifically address the needs of the gas and pipeline industry. This includes the ability to use the mobile device’s camera to easily capture the ASTM F2897 barcode.  Simply point the mobile devices built-in camera at the barcode. As soon as the Field Maps application can successfully read the barcode, it will automatically write the barcode to the designated field.  You do not even need to click the photo button! The second ArcGIS Field Maps enhancement was to be able to run arcade scripts in the pop-up configuration. This allows the barcode information to be immediately decoded and presented to the user. This real-time decoding of the ASTM barcode data even works when the mobile device running ArcGIS Field Maps is not connected to the network. This ArcGIS configuration for capturing newly installed plastic pipe and plastic components includes server-based scripts called attribute rules to automatically populate the plastic pipe/component data fields.  No manual entry of pipe or component characteristics is necessary. For more on the specifics of this part of the ArcGIS Field Maps configuration, please see the following previously published article; Tracking and Traceability 2019 – Part 2. Capturing Steel Asset Characteristics When installing steel pipe or steel components, a different approach is required. Currently there is no industry barcoding standard for steel. So, how can field users avoid manually entering every unique characteristic? The short answer is to give the field user a list of asset configurations to choose from. This configuration list represents the valid list of sizes, models, and manufacturers the gas organization uses as part of its standards for current pipe system construction. Legacy company configurations, which are no longer used, need not apply.  We only want a list of valid configurations for today’s company compliant pipes and components. The field user selects the desired configuration from the picklist. ArcGIS Field maps will then read the characteristics of that asset from the look-up table and automatically write the information to the newly installed asset’s data fields. There is no manual entry for the field user. It is a simple matter of selecting a configuration from a picklist. The software will do the rest. Implementing this capability not only addresses the issue of how to enable field users to quickly and accurately document the characteristics of newly installed steel pipe or a steel component, this same configuration can be used for situations where the plastic barcode is unreadable due to being smeared, scratched, or otherwise obscured. To deploy this part of the solution requires the following:                -Picklists (Subtype specific coded value domains)                -Lookup tables containing the characteristics of a specific configuration                -Scripts (Attribute rules) to automatically populate the asset’s data field with the configuration characteristics Need a List The picklist is defined with subtype specific coded value domains. This simplifies the field user experience by shortening the list of configurations to those which apply to a specific type of asset. Service Pipe, Distribution Pipe, Couplings, Elbows, and Controllable Valves are each separate coded value domains. The domain is defined as a field type of short integer. The short integer value is the primary key to the configuration lookup table.  The coded domain description is the text which you want the field user to see in the picklist. Configuration Look-Up tables With the subtype specific coded value domain now defined, there needs to be a source which contains the characteristics (pipe diameter, material, manufacturer, wall thickness, etc) for each configuration. This lookup table is a standard geodatabase table. It must reside in the same enterprise geodatabase as the staging layers which will store the field captured assets. To simplify the integration with the staging featureclasses and the final production featureclasses, the schema of this table should match the data fields into which this information will be inserted. Automated Data Entry Attribute rules are the scripts which provide the automation logic to read the user specified configuration value. The selected configuration code value is then used to query the configuration table to retrieve the asset’s characteristic information. With the asset characteristic information retrieved, it can then be written to the appropriate data fields of the same record from which the user selected a configuration type. //Rule Name:  StagingPipelineLine_Configuration //Description: Set multiple fields based on Configuration value //Subtype:  All //Field:  Leave blank.  Setting multiple fields. //Editable:  unchecked (false) //Trigger:  Insert, Update //Error Code:  72 //Error Message:  Couldn't set values based on ConfigurationID //Evaluate from application evaluation:  checked (true)   //Get Configuration ID value var configurationId = $feature.configuration; if (configurationId == null) {     return; } // If update operation, only fire if configuration value changed if ($editcontext.editType == 'UPDATE'){     if ($originalFeature.configuration == $feature.configuration) {         return;     } } //Query Configuration_Lines table for configuration var cuTable = FeatureSetByName($datastore, 'Configuration_Lines', [‘assettype’,'nominaldiameter','manufacturer','pipespecification','pipegrade','wallthickness','outsidediameter','primarycoatingtype','seamtype’], false); //Use the first record returned from the query var cuAttribute = First(Filter(cuTable, 'configuration  = @configurationId')); if (cuAttribute == null) {     return; } else {     return {         'result': {             'attributes': {                 'assettype': cuAttribute.assettype,                 'nominaldiameter' : cuAttribute.nominaldiameter,                 'manufacturer' : cuAttribute.manufacturer,                 'pipespecification' : cuAttribute.pipespecification,                 'pipegrade' : cuAttribute.pipegrade,                 'wallthickness' : cuAttribute.wallthickness,                 'outsidediameter' : cuAttribute.outsidediameter,                 'primarycoatingtype' : cuAttribute.primarycoatingtype,                 'seamtype' : cuAttribute.seamtype             }         }     }; } When these three configurations are added to ArcGIS Field Maps, the field user experience is simplified. The user selects the configuration from the picklist, such as 4” Steel by J-M Manufacturing. The software runs the script, which retrieves the configuration information from the look-up table, and inserts it into the new record. The pipe record now has its characteristics automatically populated.  No barcode, no problem. Documenting New Construction Requires Adaptability This ability to use the advanced configuration capabilities of ArcGIS to address the issue of how to capture pipes/components with no barcodes demonstrates how important adaptability is to a digital field as-builting solution.  Just like gas and pipeline industry field crews are constantly adapting to unexpected issues during construction, a digital field as-builting solution needs to provide a wide range of data capture capabilities to overcome unexpected construction documentation issues. The adaptability of ArcGIS Field Maps extends beyond the previously described ability to scan plastic ASTM F2897 barcodes, and to select a pre-defined configuration from a picklist when no barcodes are available. It includes the ability to copy the information from a previously placed and documented asset. Lastly, there is always the ability to manually enter the asset characteristics.  Even in this last option, there are still picklists to help the field user to quickly and accurately enter this information. This blog article is the second in a series of four blogs articles explaining how to deploy ArcGIS Field Maps for Digital Field As-builting.  Next blog article will divulge the secret sauce of configuration and automation for simplifying the documentation of pressure tests. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions.   ... View more

Part 1 of 4   By Tom DeWitte and Tom Coolidge The reality is until recently as-built data capture capabilities didn’t really meet the modern workflow needs of high-performing gas utilities. Then came breakthrough technological improvements. And, with them, a benefits trifecta for gas utilities that take advantage of them – higher quality, improved business processes, and lower cost.   The technological breakthroughs we are referring to are low cost - high accuracy GPS antennas, mobile computing devices, and a new generation of mobile data collection applications.  Each of these technological breakthroughs incrementally has improved the speed, accuracy, and process streamlining of capturing new gas pipe construction.  But, it is the recent advances and the ease with which these technologies can be combined that is the paradigm shift which is just beginning to reshape how gas pipe construction information is gathered, populated, and managed.   So, what is the magic sauce which is providing this leap in cost efficiency, data quality, and business process for capturing new construction information?   The answer is ArcGIS Field Maps. A Brief History of As-Built Documentation For over one hundred years gas pipe organizations have used paper to document the construction of the gas pipe system. For most in the industry, this collection of information is referred to as the “construction packet.”  This was a very literal description of the collection of documents which are gathered, populated, redlined, and then stuffed into a large envelope for sharing with the many departments which need to process its contents. Vermont Gas example of hand drawing At the beginning of this millennium, the industry began incorporating digital technologies to this historically paper process. For many in the industry, the 1st generation of solutions was focused on replacing the redlined paper construction drawings with a digital sketch over a digital representation of the construction drawing. Vermont Gas example of redline sketch The 2nd generation of solutions incorporated high accuracy GPS units and barcode scanners to more accurately document the new construction. This was a major improvement over 1st generation solutions. But, it was almost exclusively focused on documenting the newly installed assets. This left the rest of the construction packet in a paper format - a paper format that someone in the office would have to read, interpret, and enter into digital data management systems. Collector documenting a new service The 3rd generation of solutions provides easily configurable maps, forms, and business rules, which in addition to digitally capturing the newly installed assets, also provide the capability to capture the rest of the documentation, such as pressure tests, exposed pipe inspections, and as-built checklists. This digital transformation of the construction packet is what is today referred to as Digital Field As-builting. ArcGIS Field Maps Documenting a Pressure Test As more gas organizations digitally transform their processes for documenting new construction, it is clear that this transformation is critical to cost-effectively maintaining a safe and reliable gas pipe network. So, what is it about Digital Field As-Builting that is so transformative as to positively impact a gas organization’s ability to maintain a safe and reliable gas pipe network? There are so many improvements provided by digital field as-builting that they need to be grouped into three categories: higher quality, improved business process, lower cost. Higher Quality Implementing a Digital Field As-builting solution for documenting new pipe construction addresses many of the industry issues associated with incomplete and inaccurate data. Whether that be incorrect locating due to poor mapping or incorrect main replacement prioritization analysis due to incomplete data, the root cause is the same, poor data quality. EOS Arrow Gold GPS Receiver The incorporation of high-accuracy GPS antennas solves the spatial location accuracy portion of the data quality issue. Today’s GPS antennas provide sub-foot location accuracy. These devices connect to multiple satellite positioning systems (U.S., European, Russian, Chinese) to increase the number of satellites in view for triangulation of position. Simultaneously they connect to base stations which are providing real-time kinematic (RTK) location correction. Once the sub-foot location is determined, the GPS antenna uses Bluetooth communication to stream the coordinate location data directly to ArcGIS Field Maps running on a mobile device such as a tablet or phone.  In ArcGIS Field Maps, this information is stored as native ArcGIS features. These field collected ArcGIS features eliminate the traditional need for office staff to interpret and manually redraw the field documented new construction. Today’s mobile tablets and phones contain high definition cameras. When coupled with the ArcGIS Field Maps mobile application, these mobile device cameras can be used to capture the ASTM F2897 barcodes on the plastic components. ArcGIS Field Maps decodes the barcode and automatically writes the barcode encoded information into the appropriate data fields. This example of ArcGIS Field Maps automation consistently and accurately captures the nominal diameter, wall thickness, material, manufacturer, and manufacture date information which is critical to integrity analysis and main replacement prioritization analysis. Improvements in quality of the data captured extends to the other types of construction documentation such as pressure tests.  For pressure tests, a common problem is accurately documenting which pressure test covered which asset. The ArcGIS Field Maps ability for automation allows the inspector to simply draw a polygon encompassing the portion of the new pipe system pressure tested, the software then automatically assigns a pressure test ID to the components contained within the pressure test polygon. ArcGIS Field Maps Improved Business Processes The legacy industry processes for using paper to document the new construction are fraught with duplication. For example, an exposed pipe inspection paper form is handwritten by the field technician. A person in the office must interpret the handwritten information and then enter this same information into a geospatial digital data management system, such as ArcGIS. Once in ArcGIS, integrity analysis programs can use this information as part of their Transmission Integrity and Distribution Integrity programs. Using ArcGIS Field Maps to enable the field technician to digitally document the exposed pipe inspection eliminates the office duplicate data entry as Field Maps data is directly written to the ArcGIS Enterprise Geodatabase. ArcGIS Field Maps Smart Form example Using ArcGIS Field Maps for collecting form centric information such as a pressure test or as-built checklist reduces the time it takes to collect the new construction data. Its smart form capabilities simplify often complex and cluttered forms. The implementation of business rules automates data entry and reduces the amount of information entered by the field technician. All these process improvements lead to a significantly reduced time lapse from when construction is completed to when the digital data is posted for use by the entire organization. For example, Spire Energy out of St Louis, MO has reduced their time from construction to posting to an impressive average of 8 days.  Most gas organizations using a paper-based process struggle to complete this process in less than 3 months. Lower Costs Time is money and deployed construction crews are expensive. This has been true since the beginning of gas pipe system construction. This current generation solution stack for documenting new construction looks to be more expensive.  Afterall the original equipment used 100 years ago was paper, a red color pencil, and a tape measure. The current generation equipment list includes a high-accuracy GPS antenna, mobile device, and ArcGIS Field Maps software.  How can it have a lower cost? The current generation digital field as-builting is cheaper, because of its significant improvement in reducing the time required to collect this information. It improves the business process and eliminates many of the data entry duplications that have been built into the legacy processes. And let’s not forget that every incident avoided by the availability of improved data quality protects the safety of people and property, and saves the gas utility money – potentially large amounts of money! The Secret Sauce – Configuration and Automation Deploying a Digital field As-built solution is not as easy as shopping on Amazon and ordering a few items. It requires knowledge in GPS, mobile devices, software, and most importantly how to configure them to work together. All of this is done through configuration. The secret sauce for a successful deployment is knowing how to perform these configurations. This blog article is the first of a series of four blogs articles explaining how to deploy ArcGIS Field Maps for Digital Field As-builting.  Upcoming blogs will divulge specific examples of the secret sauce of configuration and automation to lower costs, streamline the process and improve the data quality of construction collected data. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

By Tom DeWitte and Tom Coolidge Technological changes now offer the gas utility industry the opportunity for a paradigm shift in how it manages the locational accuracy of existing buried pipe systems. These new innovative solutions allow existing pipe to be mapped to sub-foot spatial accuracies without taking the pipe out of service. They also either minimize or even eliminate the need to dig to expose the pipe.  In other words, it is now possible to accurately and cost effectively remap existing buried pipe without shovels. To understand the business value of these technological breakthroughs requires of little history of how buried pipe was mapped through the 20th century, and the scale of the problem pipe organizations are facing. Mapping of Buried Pipe in the 20th Century Currently in the United States there are nearly five million miles of buried pressurized pipe in service. Drinking water accounts for 2.2 million, natural gas 2.6 million, and petroleum 0.19 million miles.  Most of this pipe was installed decades ago. During the first half of the 20th century, accurate mapping of newly installed pipe to many organizations meant it was on the correct side of the street. In natural gas distribution pipe systems across the U.S., there are still over 50,000 miles of active pipe which was installed before 1950. During the second half of the 20th century, the United States, like the rest of the world, built out most of its buried pipe infrastructure. For natural gas distribution pipe systems in the United States, 797,359 miles of that pipe is still in service. The location accuracy of this buried pipe improved. By the beginning of the 21st century, many organizations were documenting new pipe construction to within 10 feet of its true location. Why is this a problem Every day in the United States, someone inadvertently strikes a buried pipe while excavating.  According to U.S. Department of Transportation, Pipeline and Hazardous Materials Safety Administration between 2001 and 2020 there were 12,506 reported pipeline incidents. These incidents resulted in 283 deaths, 1,180 injuries, and $9,949,689,660 in costs. Excavation Map from PHMSA of reported excavation damage since 2000 For many of these incidents, the root cause was inaccurate spatial location of the pipe from the mapping system.  As previously noted, the lack of spatial accuracy of the documentation of the 20th century installed pipe, is a major contributor to this root cause issue. In addition to excavation damage exposing the issues with inaccurately mapped inservice pipe, there are changes occurring today and in the near future which also need to be discussed. Today, new pipe construction is collected with high accuracy GPS receivers such as the Eos Arrow Gold. The Arrow Gold, like other GPS receivers in its class, is capable of centimeter location accuracy.  This is a huge improvement over knowing what side of the street the pipe was installed on as was common in the early 20th century. Soon, field technicians such as locators, will be using Augmented Reality software on their phones and HoloLens to see where the pipes are buried. Both the highly accurate new pipe data, and the technology emerging needs for more accurate pipe data are exposing the lack of location accuracy with 20th century mapped pipes. Pipes Not Connecting What is being exposed is that the sub-foot mapped newly installed pipe, does not match up with the legacy mapped pipe of years past. That newly installed service pipe tee which was mapped with high accurate GPS could literally be 10, 20, 50, even 100 feet away from the main to which it is supposed to be connected. How Do I Fix my Legacy Pipe Data Five years ago, remapping existing pipe systems to the sub-foot level of accuracy needed for today’s pipe organizations required digging spot holes to physically expose the buried pipe and validate its location.  This is very expensive, and required digging up roads and sidewalks, which is not appreciated by the local community. Thankfully there are now other options. One option is to insert a sensor probe into the active gas pipe.  The company REDUCT has worked with the energy research group, GTI, to bring this innovative approach to market. Another option is to integrate high accuracy GPS with a Utility Locator Receiver and ArcGIS. The company Eos, in partnership with Esri, has brought to market their Eos Locate for ArcGIS solution. The Eos approach works on any pipe or cable which can be accurately located with a Utility Locator Receiver. Both of these recent-to-market solutions provide the ability to very accurately map the location of existing pipe without taking the pipe out of service. Locating and Mapping At the Same Time The Eos Locate for ArcGIS is a game changer. It allows a locator who is already performing a required task, i.e., the call before you dig locate, to accurately map and directly load into ArcGIS the location of the existing pipe or cable. No digging, data prep, or data conversation is required for this new type of data collection. Remapping Without Shovels Improving the location of nearly five million miles of existing in-service buried pipe is an overwhelming challenge to pipe organizations not only in the United States but around the world. Recent technological breakthroughs in both in-pipe sensors and above-ground sensor integration are providing solutions to this challenge. These new solutions and their integration with ArcGIS are enabling those responsible in pipe organizations to cost effectively meet the pipe location accuracy needs of today and tomorrow, while leaving their shovels in the truck. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

By Tom DeWitte and Tom Coolidge Part 3 of 4 There is nothing worse than having someone waiting impatiently for a requested piece of information, and you simply cannot find it.  If you are responsible for your organization’s compliance program, the thought of having a regulator impatiently waiting for you to find the survey records for a specific leak survey area may be the kind of thought that gives you indigestion. Worrying about whether you have all your leak survey inspection records ready and easily accessible is a stressor no one wants.  So, how can a gas organization keep their leak survey area records organized, easy to manage, and accessible on demand? The answer is ArcGIS with a simple configuration for Leak Survey. In this third blog of a four-blog series, we will look at how to configure ArcGIS to enable gas organizations to more easily manage, organize and access their Leak Survey records. This builds on the seven of ten top traits of modernized gas leak survey that we discussed in the previous two parts of this blog series. 8 - Easily Accessing A Leak Survey Area’s Full History To meet the needs of gas compliance, both the current leak survey inspection information and the previous leak survey inspection records need to be easily accessible. This is accomplished with a configured web application to allow a user to simply click on an individual leak survey area to see its current state. A second click on the embedded link in the pop-up opens the list of all the leak survey history records for the specified leak survey area. You now have a listing of each completed leak survey inspection for the leak survey area for the specific regulatory type of leak survey. The listing includes the dates when those inspections were performed, which often is all that a user wants to know. To know more about a specific historical leak survey inspection, simply click on the desired record. With three clicks a user can see the current leak survey record, a list of all previously completed leak survey inspections, and see the details of a specific historical leak survey record. This easy, interactive ability to view leak survey records should prevent a gas compliance staff from never being able to find a record. Current Inspection Vs History So, what is the difference between the current Leak Survey record and the historical leak survey records for a Leak Survey Area.                 -Current Record: This is the recorded information about the Leak Survey Area for the current inspection cycle. The information stored is organized and accessed to understand the leak survey area’s status within the current inspection cycle. Included in this record is the results of the previous inspection.                 -Historical Records: This is every previously completed leak survey inspection. The organization of this information is very straightforward within the Geodatabase.  The current record is stored in a polygon featureclass. The Historical records are stored in a table which is related by a relationship class to the polygon featureclass. The relationship class connects the two sets of information together. The GlobalID field in the Leak Survey Area featureclass is the primary key field. The leaksurveyglobalID field in the Leak Survey History table is the foreign key field. 9 - Automating Management of Leak Records The last piece of configuring your ArcGIS system to provide these capabilities is to simplify the management of the information that will be flowing from the leak survey featureclass to the leak survey table when a survey inspection is completed. The desired behavior of the ArcGIS system is to have a leak survey record in the polygon featureclass to be automatically written to the leak survey history table when a leak survey inspection is completed. The mechanism within ArcGIS to implement this desired business rule behavior is called attribute rules. Here is a screenshot of the arcade script used for this attribute rule. With this attribute rule in place, the leak survey history table is automatically updated. For GIS administrators responsible for maintaining this information, this is significant.  It means that the leak survey data does not need to be archived or prepped at the end of an inspection cycle. The ArcGIS system is automatically managing these otherwise manual tasks. Where can I get This Schema and Attribute Rules The ArcGIS for Leak Survey schema and attribute rules are provided as a free download on the Esri Community site (formerly called Geonet). You can directly download the zip file containing the scripts, schema and installation instructions with this link. This simple configuration of ArcGIS provides gas compliance staff with the ease and organization they need to manage this regulatory required compliance information. The data automation provided by the recent enhancement of attribute rules automates data management insuring a consistent and reliable flow of information. No longer does the gas compliance staff have to worry about not finding a requested inspection record during a regulator audit. Additional Information If you missed our previous blog articles on improving Leak Survey, here are the links to those articles. Those two blog articles explained how ArcGIS can improve other aspects of a gas compliance Leak Survey program.  Part 1: Modernizing Leak Survey Part 2: Automating Management of Inspection Dates We have now covered 9 of the top ten traits of a modernized leak survey. The fourth and final article for this blog series will cover the tenth trait, which is how to manage digital breadcrumb trails with ArcGIS. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

  By Tom DeWitte and Tom Coolidge Part 2 of 4 Our first blog of this series provided an overview of how ArcGIS can improve the efficiency of managing a gas organization’s leak survey compliance program. If you missed it, you can access it here. In this first part of this blog series we talk about the first five of ten top traits of modernized gas leak surveys. “You’re Late”. We all cringe when we hear these words.  I am sure there are many gas compliance managers who have nightmares about a regulator saying these words during a regulatory audit. Being on time or late is all about management of dates and times. For a Leak Survey program, the dates of importance are the “Next Inspection Date” and the “Next Compliance Date.” Managing these two dates would not be too difficult if all types of leak surveys used the same intervals across all types of pipe subsystems. But that is not how the regulations work. So, how can a gas organization, whether large or small, implement a leak survey data management solution that allows these numerous varieties of leak survey intervals to be easily and accurately applied across the numerous subsystems of a pipe network? And in doing so, give gas compliance managers the confidence they need to sleep well and avoid nightmares of being told they were late.  The answer is ArcGIS and attribute rules. 6 - Easy Management of Different Survey Intervals The first step is to have the leak survey record data organized and maintained in a manner that does not require programmers or IT Administrators to implement a new type of leak survey or a new inspection interval for an existing survey. Within ArcGIS, this is accomplished by using subtypes and subtypes specific default values within the Leak Survey Area featureclass. The Leak Survey Area featureclass is a polygon featureclass which stores all leak survey areas for a gas organization. This includes distribution business districts, distribution area surveys, distribution atmospheric surveys, cast iron surveys, as well as the different types of transmission and gathering leak surveys. Each type of leak survey is defined as a subtype within the leak survey featureclass. This allows each type of leak survey to have unique default values and coded value domains. This configuration of the leak survey data allows each unique type of leak survey to have its own next inspection date and next compliance date intervals.  Should these intervals ever change, a simple edit by the data owner of the leak survey data is all that is needed to implement the change. 7 - Consistent Calculations Consistently and accurately calculating the next inspection date and the next compliance date for a leak survey area when it is completed is mission critical to the leak survey program. Within ArcGIS, this can now be easily implemented with attribute rules.  Attribute rules are arcade scripts stored within the leak survey featureclass to bind a specific business rule behavior to the featureclass. In this case it is the calculation of the current completed inspection date with the inspection interval.                 COMPLETED DATE + INSPECTION INTERVAL = NEXT INSPECTION DATE Here is what the attribute rule with some documentation looks like for calculating the next compliance date. Because this attribute rule uses each leak survey area’s individual values for CompletedDate, NextComplianceDate, and ComplianceInterval, the script never needs to be modified when the regulations change.  Only the subtype and record data need to be modified.  Consistent Experience in Field and Office Using attribute rules to bind the calculation of the inspection dates to the leak survey feaureclass in the enterprise geodatabase assures a consistent experience in the office and the field.  Regardless of what desktop, web, or mobile application updates the leak survey record to a status of completed, the same calculation will be performed to update the inspection dates. Where can I get This Schema and Attribute Rules The ArcGIS for Leak Survey schema and attribute rules are provided as a free download on the Esri Community site (formerly called Geonet). You can directly download the zip file containing the scripts, schema, and installation instructions with this link. It’s an Automated Calculation Automating the management of a leak survey’s inspection dates is one of the streamlining configurations which can be implemented into your ArcGIS system. Putting these configurations into your ArcGIS system will go a long way into helping your gas compliance management from hearing the words: “you’re late”. We have now covered 7 of the top ten traits of a modernized leak survey. In the next blog we will look at how, with a minor adjustment to the organization of the Leak Survey History data, accessing, understanding, and managing a Leak Survey’s inspection history can be greatly improved. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions.   ... View more