Latest Contributions by TomDeWitte

Taking Your Maps Offline: Publishing the Data By Tom DeWitte, Kevin Ruggiero, Mike Hirschheimer Part 3 of 5 Organizations large and small need to keep their mobile workers informed. Whether it is a blue-sky day or storms are brewing overhead, mobile workers need current information on the utility system they spend every day maintaining. This information informs the mobile worker about which conductors are energized, pipes are pressurized, and cables are live. Critical information that helps to keep the mobile worker safe and the utility system reliable. Consistently and reliably transmitting new and updated information to a utilities entire mobile workforce is not easy. The paper maps deployed for the initial 150 years of the utility industry were unable to receive incremental updates. The day after the paper maps were put into the hands of the mobile worker, the information on the map was already out-of-date. The same issue applied to early mobile map viewer applications, like ArcReader. The snapshot of information stored locally on the mobile device was unable to be incrementally updated. It too became more and more outdated with every passing day after the snapshot of information was created. These lessons learned from previous efforts to provide timely and current information to the mobile worker, tell us that a mechanism is needed to update a mobile worker’s information incrementally and consistently. In today’s mobile world, this equates to maintaining a local copy of the geospatial representation of the pipes, conductors and cables on the mobile workers phone, tablet, or laptop. The mechanism of transmitting this information to the mobile device when it is connected to a network is web services. What Are Web Services A web service is a method of communication between your mobile device and your data server. When a web service is created it is an always-on process, that is listening for mobile client applications to send requests. There are many different types of web services. Within ArcGIS Enterprise there are specialized types of web services for geocoding, geoprocessing, sharing image data, and sharing vector data.  The sharing of vector data is how feature and table records are passed between the mobile device and the centralized data repository. ArcGIS Enterprise supports multiple types of vector data web services. These include, KML, WFS, ArcGIS Feature Services and Hosted Feature Layers. ArcGIS Feature Services and Hosted Feature Layers are the only vector data sharing web services which support the synchronization of changes between the mobile device and the centralized data repository. Put another way, to keep our mobile workers informed with data changes being made by office staff and data changes being made by other mobile workers, there must be either a Feature Service or a Hosted Feature Layer to handle the communication. A feature service is how ArcGIS Enterprise Geodatabase managed data such as Utility Network data is shared to mobile devices for synchronization. A hosted feature service is how ArcGIS Portal hosted feature layer data is shared to the mobile devices for synchronization. Organizing Data with Feature Services In the example we have been describing in this blog series, the data that needs to be made available to the mobile device for local caching is coming from four different data repositories. Two of the data repositories are Enterprise Geodatabases, one data repository is the Portal’s hosted feature layers, and the fourth is an ArcGIS Online registered vector tile basemap configured for export. Each Enterprise Geodatabase will contain multiple featureclasses and tables which need to have their records synchronized to the mobile devices. A single feature service can be a grouping of 1 or more featureclasses and tables. When trying to determine how to organize your data into feature services, it is important to know that each individual feature service can only connect to one data repository.  This means that all featureclasses and table must be accessing the same enterprise geodatabase thru the same relational database connection. Publishing data for offline use has some additional constraints on the organization of content in the feature service.                   -A featureclass or table can only be referenced once.                 -Subtype group layers are not a supported layer type        Publishing a Feature Service for Syncing Creating a feature service can also be referred to as publishing the data. Publishing the data is the 2nd of the four primary steps in creating offline map areas. The “Publish the Data” step is typically accomplished with the desktop tool, ArcGIS Pro. The “Share as Web Layer” tool is the specific tool to use to publish the feature service. By default, a feature service does not support synchronization. To activate synchronization, check the box “Enable Sync” within the feature properties configuration panel. Many of the featureclasses in enterprise geodatabases have their geometries configured to support the storage of elevation (Z) and distance along a line (M). Field editors of this Z-enabled and M-enabled data may not have this information available at the time of data capture. To allow field editors to collect this information without a complete geometry definition, default values and behavior need to be defined. For the Z value check the box to turn on the “Apply default to features with Z-values”. Then set the default z-value to the desired value. For the M value check the box to turn on the “Allow geometry updates without m-value”. This will set the M value to NULL when a new feature is created.       Publishing Your Utility Network Key to keeping mobile workers safe is providing current information about their utility assets. When these assets are stored in an enterprise geodatabase and managed with the utility network capabilities, the publishing step is a matter of modifying the properties of an already published feature service. For the office mappers to create and maintain the digital connected representation of your utility assets, requires that the data be configured as branch versioned and published as a feature service. The already published utility network feature service may not be fully configured for syncing with offline map areas. To support syncing to offline map areas requires two configurations of the feature service that will need to be set. The first is to modify the already mentioned feature service properties to enable syncing. The 2nd configuration is to define the role that branch versioning will have during the synchronization process.  There are two options for branch versioning during sync: -None -Create a version for each downloaded map To know which option to set starts with an understanding of what your mobile workers will do with this utility asset data. Version Creation = None If the answer to this question is that mobile workers will only view and query the utility network data, then the default setting of “None” is the correct setting. With this configuration no additional versions are created or required to successfully sync. If you set the Version Creation option to “None” and allow your mobile workers to edit the utility assets, those edits will be posted directly to the default version. Syncing to the default version will share these edits immediately to all other office and field users. Version Creation = Version for each downloaded map This option is recommended when your mobile workers are editing the utility network managed utility assets, and those edits need to be reviewed and quality control checked prior to sharing with the rest of the organization. When this option is chosen, a branch version is created within the enterprise geodatabase when the mobile worker downloads the offline map area for the first time. All field edits will be synchronized to the created branch version. A more detailed explanation is available within the Esri online documentation. Version Creation = Create a version for each user This option is not valid for branch versioned feature services. With these two configurations set, the utility network feature service is ready to support offline map area synchronization. Publishing Your Landbase Unlike the Utility Network, a feature service may not already exist for sharing the landbase information. Office mappers may be using the ArcMap desktop application and its direct connection capability to access the landbase data stored in an enterprise geodatabase. In this example configuration the enterprise geodatabase will be using traditional versioning to track and manage changes to the landbase. Publishing the landbase feature service to support offline map area syncing requires checking feature service configuration option, Enable Sync. With sync enabled you now must decide what versioning strategy to deploy to support the mobile users. If your mobile users are not intended to create or modify the landbase features, then the correct Sync Version Creation option is: None. If you mobile users will be creating new landbase features or modifying existing landbase features then a version needs to be created to manage the flow of landbase edits. Two version management options are available for traditional versioning: “Create a version for each downloaded map” or “Create a version for each user”. A more detailed explanation of these sync version feature service options is available with the Esri online documentation. Scaling your Feature Services At the beginning of each workday there will likely be a spike in the number of mobile devices attempting to sync to the published feature services. This spike is caused by the mobile workers opening Field Maps and the offline map area enabled web map on their mobile device. This will initiate a synchronization. The capacity of a single feature service to scale to accommodate this morning spike in concurrent devices requesting to sync is defined within the pooling portion of a feature service’s properties. The pooling parameters which directly impact capacity are: Minimum number of instances per machine” and “Maximum number of instances per machine”. These parameters are adjustable after the feature service is published. The Minimum number of instances per machine value represents the base level of concurrent users that a single server can support.  As the number of requests for data increases the server will automatically start creating new additional instances.  This will continue until either the concurrent demand for data is met or the Maximum number of instances per machine is reached. If the number of concurrent user requests exceeds the maximum number of instances value, additional users will be put into a queue and must wait for an instance to become available. After the morning spike in synchronization, the number of concurrent users requesting data from the feature service will decline.  This will cause the number of instantiated instances to decline until the minimum number of instances per machine value is reached. Publishing Your Field Collected Data Hosted Feature layers provide another method of publishing data for synchronization to mobile devices. The publication process is integrated into the Hosted Feature layer initial creation process. Once a Hosted Feature layer is created it is automatically published as a feature service. Similar to an Enterprise published feature service, a hosted feature service is not sync enabled when it is initially created. Enabling sync is a task for the hosted feature layer owner, modifying the settings. Checking the box to Enable Sync is all that is required to enable syncing. Registering the Basemap to your Org The final data source to prepare for offline map areas is the basemap. For our example we will use an Esri curated basemap. Most of the Esri provided basemaps are not compatible with offline map area syncing. All ArcGIS Enterprise referenced default basemaps are NOT valid for offline map areas. Only a subset of the ArcGIS Online basemaps is valid for use within offline map areas. A listing of the 19 Esri vector basemaps for offline map areas (for export) can be viewed here. Using these ArcGIS Online basemaps in ArcGIS Enterprise requires that the basemap be registered as a layer in Enterprise, and that a valid ArcGIS Online user account is embedded within the registered vector tile layer. It is recommended that this ArcGIS Online user account is not an employee’s active account, nor should it be an account with administrative rights. This embedded user account should be a non-user account created expressly for access to the basemaps. A Persistent Listener Consistently and reliably transmitting new and updated information to a utilities entire mobile workforce requires web services. These published feature services have specific parameters which need to be configured to support syncing. Once a service is configured for syncing, it becomes a persistent listener, that is ready to respond to a mobile devices’ request for changes in data. This persistent listener is available on stormy days as well as sunny days to ensure that the utilities mobile workers have the current information they require to safely maintain a reliable utility system. About this Blog Series This is the third blog article in our series on offline map areas. In future blog articles we will continue to explain the details of how offline map areas work and the specific decisions an administrator will need to make during deployment. The first blog provided an overview of offline map areas. The second blog provided details on how to prepare the data for offline usage and synchronization. The fourth blog will provide details on the creation of offline map areas, how they are stored and managed in the portal environment. The fifth and final blog will provide details on the deployment and management of offline map areas for a large mobile workforce. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

Unified Pipeline Tools ribbon: New BFF of Pipeline Referencing and Utility Network Users By Jeff Allen, Tom Coolidge, and Tom DeWitte   Users of ArcGIS Pipeline Referencing and ArcGIS Utility Network have a new best friend in the Unified Pipeline Tools ribbon. That is because the new ribbon for the first time enables users of both extensions to access tools from both on one tab instead of having to move back and forth among multiple individual tabs. So now, not only does ArcGIS enable you to build pipe network models using both linear referencing and connectivity modeling methods on the same geodatabase, but it also greatly simplifies the performance of workflows requiring tools from both extensions.   Here's one example of how this advance can minimize clickiness and increase productivity. Let’s say that I am a mapping technician responsible for maintaining the transmission pipe network. On my desk is the construction packet for a newly replaced section of one of the transmission pipelines. Using the standard ArcGIS Pro configuration of tools and ribbons, I am going to use the following ribbons while updating my transmission pipe network to reflect this newly completed change to the pipe system. First, I am going to use the Map ribbon to explore and pan into my area of interest. Then I will switch to the edit ribbon to create a set of new pipeline features along the replaced section. Next I will use the location referencing ribbon and its route realignment tools to realign the pipeline to the new section of pipe. Lastly, I will use the Utility Network ribbon to update terminal settings and validate my edits. That’s four tabs to complete the task. Now if we have the Unified Pipeline Tools ribbon, we will complete all explore, create, save, route realign, and validation in one ribbon. Benefits Of Accessing Tools from Both Extensions on One Ribbon That is a significant increase in efficiency. The Unified Pipeline Tools ribbon is a step forward for users of ArcGIS Pipeline Referencing and ArcGIS Utility Network. It places at their fingertips a new capability to achieve greater efficiency when performing tasks and workflows that necessarily require accessing tools on multiple tabs. Personalize the New Ribbon The Unified Pipeline Tools ribbon comes pre-configured with what our research has revealed to date to be the most used tools for managing transmission pipe networks. Since this is based in ArcGIS Pro, users can add additional tools to the ribbon to make it even more aligned and complete for their most common workflows and tasks. Get the Unified Pipeline Tools ribbon now Esri users interested in getting the Unified Pipeline Tools ribbon should email Tom Coolidge at tcoolidge@esri.com.   ... View more

By Tom DeWitte, Kevin Ruggiero, Mike Hirschheimer Part 2 of 5 How do I get critical utility pipe, conductor, and cable data into the hands of my utility field workers? Before the advent of mobile computers that were not the size of briefcases, the only practical answer was paper. During a disaster event, the mapping department would have to print hundreds of copies of the same map pages to be handed out to utility field workers and foreign crews. With first-generation mobile computers, a digital copy of the pipe, conductor, and cable data was loaded onto the mobile computers by office staff for use by utility field workers. This snapshot of data was static, and likely did not include updates still in backlog and unable to reflect the transactional updates to the utility’s network as utility field workers were responding to the disaster event. Neither of these options are ideal for providing utility field workers with the up-to-date information they require to safely perform their jobs. What utilities desire is a mechanism that can automatically and frequently pass changes to the data from the office to the field and from the field to the office. This method of only passing changes is called bi-directional synchronization, though many will simply call it delta syncing. Bi-directional synchronization keeps the entire organization viewing and using the same data, whether they are in the office, or in the field. Syncing with ArcGIS Deploying bi-directional synchronization with ArcGIS requires proper configuration of the data, and the server environment. It also needs a mobile computing device with ArcGIS Field Maps. Offline map areas is the capability that provides bi-directional synchronization support to enterprise organizations with hundreds, thousands, or tens of thousands of utility field workers.   If this capability is of interest to you, you may be wondering, can I take my data offline? What modifications do I need to make to my data to prepare for bi-directional synchronization with my organization’s mobile devices? -In this blog we will answer those questions. Preparing the Data for Offline Map Areas Preparing the data for offline map areas is the first step in our four-step process. When preparing data for bi-directional synchronization with offline map areas we need to determine which formats of data are supported. Valid data sources are:                 -Enterprise Geodatabases                 -Hosted Feature Layers File based spatial data formats such as shapefiles, file geodatabases, and CAD files do not support this advanced form of synchronization.   Being a feature class or table within an Enterprise Geodatabase or a hosted feature layer does not guarantee the ability to support bi-directional synchronization. There are additional requirements of the data structure. Prepare to be Published as a Feature Service For enterprise geodatabases, the first set of requirements is the requirement to publish the data as a feature service. This requirement is the following: feature class or table is registered with the geodatabase, each feature class or table must have a Global ID field, and all relationship classes must have Global ID as the primary key. A detailed explanation on preparing data for use in offline feature services can be found in the ArcGIS Pro online help. Hosted feature layers when created are already prepared for offline sync. All you must do is update the Setting properties and check the box to Enable Sync. A detailed explanation for enabling a hosted feature layer for offline can be found in the ArcGIS Enterprise online help. Providing the complete map view that mobile workers require to perform the tasks assigned to them requires a wide range of data. Multiple data repositories can be referenced within a single web map for offline map areas. These different data repositories will often use different methods for tracking changes. Tracking Changes with Enterprise Geodatabases For the software to be able to identify the changes which have occurred since the last successful synchronization requires an ability to track changes within the enterprise geodatabase feature class or table. There are three options within the enterprise geodatabase for tracking changes to a feature class or table. These three options are called: Non-Versioned Archiving, Traditional Versioning, and Branch Versioning. Non-Versioned Archiving Non-Versioned Archiving is the database method for tracking a change to an individual record. This is perfect for workflows such as inspections, damage assessments, and field observation reports. These short transaction activities preserve every modification made to a record. For each iteration of change ArcGIS automatically assigns a “from date” and a “to date” which defines when that iteration of change was the current state of the record. A more detailed explanation of non-versioned archiving can be found in the ArcGIS Pro online help. Traditional Versioning Traditional Versioning was Esri’s original method for tracking a group of changes as a single transaction. This group of changes includes additions, updates, and deletes to many records across many feature classes and tables. This is critical to preserving the integrity of the full utility system for common changes such as documenting the extension of the utility system to a new subdivision.  This method uses a set of database tables to track the sequence in which the changes were made and submitted. A more detailed explanation of traditional versioning can be found in the ArcGIS Pro online help. Branch versioning Branch versioning is Esri’s current and recommended method for tracking a group of changes as a single transaction. This updated long-transaction methodology uses a simpler database structure to track changes. Modifications and additions to records will be marked with a date and time. A more detailed explanation of branch versioning  can be found in the ArcGIS Pro online help. You are not required to select one of these change tracking methods for your offline map areas deployment. Offline map areas have the capability to bi-directionally synchronize against multiple feature services and hosted feature layers; each using a different method of tracking changes. Preparing Utility Network Data The Utility Network is the feature service-based capability to manage networks of pipes, conductors, and cables. The Utility Network has many of the same data preparation requirements as offline map areas. They both require registration with the geodatabase, global IDs, and relationship classes based on the primary key field being global ID. Utility Networks require editor tracking and branch versioning. The branch versioning tracking of date and time will be used by the offline map area bi-directional synchronization to identify what changes have been made across the records in the data that need to be transmitted to the mobile application running ArcGIS Field Maps. For most utilities using ArcGIS, the utility mapping department is the only department allowed to edit the utility network configured data. This data repository, when published as a feature service will therefore be read-only to the utility field workers.   Preparing Landbase Data In larger utility organizations, it is not unusual to have a separate department for landbase mapping and utility mapping. The editing of the landbase is also a long-transaction method. The changes to curb lines, building footprints, and lot lines are commonly tracked as a group of changes which need to be posted together. In this landbase mapping department scenario, traditional versioning has been configured as the method for tracking these changes. The landbase mapping department is the only group allowed to edit the landbase data. This published feature service will be read-only to the utility field workers. Preparing Field Collected Data Field collected data can be any inspection or field observation report that will be documented by the utility field workers. In this configuration the field collected data is being stored as Hosted Feature Layers. Hosted Feature Layers use the Non-Versioned Archiving method of tracking changes. Preparing the Basemap Utilities have specific needs of their basemaps.  First, the basemap must cover a very large geographic area. Second, the basemap needs to provide highly detailed display scales of the basemap. This dual need for both a large geographic extent and highly detailed display scales does not work well with traditional image tile layers.  But vector tile layers are very well suited to these requirements. Esri provides a large set of vector tile basemaps with a beautiful range of cartography styles. But not all vector tile layer basemaps provided by Esri support exporting for offline map areas. There is a separate list of these “For Export” vector tile basemaps. The list of vector tile basemaps configured for exporting can be found here. Putting it all Together The ability of ArcGIS Field Maps and its offline map areas capability to pull data from multiple repositories provides the flexibility utility enterprise GIS administrators need to pull data from across the organization. In our utility scenario a single web map has been configured to use Utility Network data from a branch versioned Enterprise Geodatabase, landbase data from a traditional versioned Enterprise Geodatabase, inspection data from non-versioned archived hosted feature layers, and a vector tiles layer (for export) from Esri managed services. The ability to pull data from multiple repositories provides the flexibility utility enterprise GIS administrators need to pull data from across the organization. This avoids complex back-office duplication or replication processes to prepare the data. With a single web map an organization can put all the pipe, conductor, cable, landbase, inspection, and basemap data a utility field worker needs to safely perform their daily tasks. About this Blog Series This is the second blog article in our series on offline map areas. In future blog articles we will continue to explain the details of how offline map areas work and the specific decisions an administrator will need to make during deployment. The first blog provided an overview of offline map areas. The third blog will provide details and options for publishing the selected data repositories. The fourth blog will provide details on the creation of offline map areas, how they are stored and managed in the portal environment. The fifth and final blog will provide details on the deployment and management of offline map areas for a large mobile workforce. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions.                     ... View more

By Tom DeWitte, Kevin Ruggiero, Mike Hirschheimer Part 1 of 5 Where am I and where are the buried pipes, conductors, and cables? These are the questions every utility field worker asks every hour of every day they are in the field doing their jobs. These questions are asked whether it is a blue-sky working day or storm clouds are overhead and Mother Nature has decided to rearrange everything on the ground. When Mother Nature decides to rearrange things, what was a safe, quiet, and orderly community can be turned into something unrecognizable. Cellular networks can be knocked offline. All landmarks of location can be moved or wiped out. During these disaster events, utility field workers are part of the first responders to arrive onsite. These folks need a reliable source of information that is available to them when external communication such as cellular networks are not. Each utility maintains a mapping system which enables them to keep track of where their water, gas, electric, cable, and communication assets are located. At most utilities, the mapping system is a geographic information system (GIS) by the name of ArcGIS. So, how do these utilities keep their field workers informed with this asset location information when they are unable to communicate back to the utility data center?                -The answer is offline map areas. Offline Map Areas Offline map areas provide the ability to maintain a copy of your mapping data on your mobile device (phone, tablet, laptop). This capability of ArcGIS allows Esri-based mobile applications such as ArcGIS Field Maps to access this locally stored data and present it to the utility field worker even when all communication networks are down. This capability of ArcGIS can be extended to foreign crews and the devices they bring with them. A simple email can not only thank them for assisting with the event, but also provide instructions to download the application, log into the local utility’ ArcGIS Portal, and download the required offline map area(s) for the region they have been assigned to assist.  All of this can be completed in a few minutes before they enter the disaster zone. Data to Take Offline The data to be taken offline can be grouped into four categories: Network, Landbase, Field Collected Data, and Basemap. Each of these categories is comprised of many data layers and tables. Network: This is the utility’ inventory of pipes, conductors, cables and all connecting assets and structural assets which comprise the network. For ArcGIS implementations at a utility this will increasingly be organized with the Utility Network Management capabilities. Landbase: This is the utility-maintained set of referential data such as utility boundaries, governmental boundaries, property boundaries, and street curb edges. Field Collected Data: This is the set of layers that utility field workers will add to and update with the information they collect. Examples of this type of data are damage assessments, meter outage status, and inspections.  Basemap: This data layer provides the transportation network (streets, rail, and pedestrian) as well as parks, buildings, and in some locations even sidewalks and trees. To get these datasets into the hands of the utility field worker requires four primary steps. These steps are preparing the data, publishing the data, creating offline map areas, and deploying it to the mobile device.   Preparing the Data When preparing the data, it is important to understand that these four categories do not have to reside in the same data repository. Each can reference a separate data repository.  For example, your network data will most likely be stored in an Enterprise Geodatabase running inside of a relational database such as Oracle or SQL Server. But your field collected data could be a set of hosted feature layers and tables, which are stored in the Portal’s PostgreSQL database. Your landbase could be a separate relational database, or the same database as your network.  Most data are not fully configured to support the offline map area workflow. The data will likely need to be modified. The purpose of this modification is to support the tracking of changes to the data. Changes are tracked to enable the ArcGIS system to know what records have changed since the last time a user’s mobile device was synchronized to send and receive updates.   Publishing the Data To get the data from the office data repositories to the hundreds or thousands of mobile devices used by a utility’s field workers requires the communication capabilities of web services. There are multiple types of web services available with ArcGIS. Not all web service types available in ArcGIS support the exporting of data to an offline map area. The four types of ArcGIS web services which do support exporting data to an offline map area are feature service, hosted feature service, vector tile service, and image tile service. Of these four types, only the feature service and hosted feature service support delta synchronization data exchange with the offline map areas. The other two service types, vector tile service and image tile service, only support exporting data to the offline map area. They do not support synchronization.   When publishing multiple data repositories, each repository must be published as a separate service. For our four categories of data, each will be published as a separate service. The publishing of the basemap is a special situation. These Esri-curated layers provide beautiful cartography and impressive spatial accuracy. But only a special subset of basemaps allow inclusion in offline map areas. These are a special subset of the basemaps published by ArcGIS Online. Of that subset, only the vector tile services meet the geographic extent and scale resolution needs of utilities. A listing of Esri provided vector basemaps which support usage with offline map areas is available with this link.  Creating Offline Map Areas With the data now published it is time to create a web map which defines how the data is to be presented to the utility field worker. Once the web map is defined with its layer scale constraints, symbology, searches, bookmarks, and smart form configurations, it is ready to create the offline map areas. Each offline map area is an extraction of the web map defined data, for a specified geographic extent. A utility’s region or operating area is a natural subdivision of the utility’s full territory to define the geographic extent of each offline map area. This extraction of the data is stored with Portal. Each service in the web map will create a separate SQLite database, and the basemap will be written to a vector tile package. When the offline map area is initially defined a scheduled task will regularly update the portal’s stored copy of the data with changes made to the core data repositories. NOTE: A single web map can support up to 16 offline map areas. If more offline map areas are required, simply copy the original web map then define additional offline map areas. Deploying to the Mobile Device With the offline map areas created, it is now time to deploy to the utility field workers. This is where supporting foreign crews with the utility information they need to understand where the utility infrastructure was before the disaster event becomes much simpler. A free download of the ArcGIS Field Maps application from the Google Play, Amazon Store, or Apple Store gets the mobile software onto their phone or tablet. An email to the foreign utility worker with a login and password to the ArcGIS Portal hosting the offline map areas will give the utility field worker secure access to the asset data they need. Once logged into the ArcGIS Field Maps application, the utility field workers and the foreign utility workers will see the groups and content the GIS/IT department has approved them to access. Within this group will be the web map and its listing of defined offline map areas. A simple tap on the download icon will initiate the download of the offline map areas the worker needs.   Once downloaded the offline map areas will display the total storage space required on the device, as well as the date when the offline map area was last synchronized.   Asset Location and Information Always Available With the offline map areas downloaded to the mobile device, your utility field workers and foreign utility workers will always have the information they need. Whether it is a blue-sky day or a disaster event your field workers will never be without the information they need to perform their tasks safely, accurately, and efficiently. About this Blog Series This initial blog article provides an overview of the value and key steps to deploying offline map areas.  In future blog articles we will dive deeper into the details of how offline map areas work, and the specific decisions an administrator will need to make during deployment. The second blog article will provide details on how to prepare the data for offline usage and synchronization. The third blog article will provide details and options for publishing the selected data repositories. The fourth blog article will provide details on the creation of offline map areas, how they are stored and managed in the portal environment. The fifth and final blog article will provide details on the deployment and management of offline map areas for a large mobile workforce. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions.       ... View more

  By Tom DeWitte and Tom Coolidge The hands-on work of constructing, maintaining, and operating utility networks happens away from the office in the field. That’s why so much GIS development effort is focused on assuring workers have reliable access to the GIS capabilities they need wherever they are in their utility’s service territory. Technology is key to ever-improving mobile GIS capabilities, and fortunately technology is a never tiring march into the future. Consistently this march has helped to make our lives better. Some of us have seen the idea of the TV remote evolve from our dad’s telling us “hey Tom change the TV channel to 4”. Then we got off the couch, walked to the TV and turned a dial to change the channel to 4. Over time this evolved to TV remote control devices with lots of buttons that most people did not fully understand how to fully use but allowed us to stay on the couch and change the channel to 4.  Today, the remote has few buttons, and we can talk to it and say, “I want to watch channel 4.” And the TV “magically” changes to channel 4. There is a pattern to how technology evolves to solve a problem. The initial solution often is a little complicated. Then successive evolutions of that solution become increasingly intuitive and easy to use. That pattern is evident at utilities across the world. Paper forms were initially replaced with digital forms that looked like the paper form. Those forms are now being replaced with smart forms that are more intuitive, easier to use and quicker to complete. Viewing a map in the field has similarly evolved from a static paper document to an interactive map in an application with lots of buttons, to today’s smart device mobile maps. Available on Smart Devices This new generation of mobile map viewers runs on Apple, Android, and Microsoft smart devices (phones, tablets, and laptops). These mobile map viewers, such as ArcGIS Field Maps, follow the design pattern best practices of the device operating system.  This means they have very few buttons and use the iconography of the operating system. If you already know how to use the email and text messaging applications on the device, you have completed the first portion of your training. Location Aware For well over 100 years, utility workers would look at a paper map and the first question they would ask is: where am I located on this map? Later, they would look at a mobile map on a first-generation mobile application which was likely running on a non-location aware device and ask the same question. None of the many buttons on that legacy application could help them.  Now, with location aware smart devices, the answer is shown automatically. A blue dot displays your location on the screen and the map is automatically centered to your location. You do not even have time to ask the question.  Ever been caught in the middle of a field or subdivision wondering which direction is the utility asset you are looking for? When using a location aware device with a mobile map application, the application will point you in the correct direction. Real-Time Collaboration Today’s mobile devices enable real-time collaboration with other mobile devices. This comes in handy when you are using text messaging to arrange dinner plans with the entire family while finishing up a day at work. In the utility space similar real-time collaboration scenarios exist, such as an outage event. In an outage event the status of the customer’s connection is changing in real time as other utility workers are repairing parts of the pipe or wire network. The benefits of this are obvious for worker safety and productivity.  What may not be as obvious is that most legacy mobile map applications only worked offline and were unable to communicate in real-time. ArcGIS Field Maps recently enhanced its capabilities so that when the device is connected to the network a simple tap on the screen will open the connected version of the map enabling real-time communication. Workers in the field now have the same map, whether offline or online. High Quality Basemaps In the paper map world, you got one basemap. It was a simple basemap that if you were lucky included curb lines, parcel lot lines and some street names. In the first generation of mobile maps this single basemap was duplicated. Workers in the field had the same content, but with the ability to pan and zoom across multiple map sheets.   In the smart device generation of mobile applications, these basemaps offer a seamless map for the entire planet. Instead of one basemap, you can now have over a dozen basemaps at your disposal when connected to the network. This includes imagery and cartographic maps with color schemes optimized for a variety of situations.  These basemaps now include building footprints, and in some locations even sidewalks and trees. Easy to Use If the marketing department were asked to come up with a slogan for a typical first-generation mobile map, it might be: We have a button for that. The header of the application was full of dozens of buttons. Today’s smart device mobile map viewers, such as ArcGIS Field Maps use context driven logic to simplify the application to a few buttons. In context driven logic applications a specific capability is not exposed until the user selects a feature which can use it. An example of this is the compass tool. It does not display until the user selects a feature such as a valve or regulator station to which that user wishes to be directed. Easy to Find First-generation mobile maps would provide a dropdown menu of predefined searches for the field user to select from. This worked if the field user never needed to ask a question that the administrator did not anticipate when initially setting up the mobile application. Today’s smart device mobile map viewers use single line searches, like the ones we use every day in our personal lives when searching and shopping on the internet. In these modern applications the field user only needs to enter the address, assetID, or station name.   Simply type in the name, address, or AssetID and the search engine returns all items which match. Select the desired search return item from the list and the map zooms to the location of the result. Aware of Surroundings The ability to leverage location also includes the ability to proactively warn utility field workers when they are approaching a known safety hazard. Common hazards that utilities track include dangerous animals and violent customers. When this capability is used on a smart device, ArcGIS Field Maps will connect to the devices notification system and issue notification alerts of the approaching hazard.   The March of Technology Continues This pattern of technology evolution continues to simplify and improve the safety and efficiency of utility field workers. Smart device mobile map viewers are the latest generation. But they will eventually be replaced by the next generation which will likely be 3D enabled and use augmented reality. And in time that generation of mobile map viewers will be replaced by an even more advanced and easy to use generation of mobile map viewers. The march of technology continues. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

Improving Field Worker Safety with Geofences By Tom Coolidge and Tom DeWitte If we were betting men, we suspect most everyone reading this blog would readily recall the centuries-old saying that begins, “An ounce of prevention…” That saying, of course, was uttered in 1736 by Benjamin Franklin. In full, the saying is “an ounce of prevention is worth a pound of cure.” The meaning is simple. It is usually far easier to stop something bad from happening in the first place than it is to repair the damage afterwards. ArcGIS today provides gas utilities and pipelines a powerful, yet easy to easy to use, capability to put that saying into practice in a way that can significantly benefit field workers. Sadly, there are too many real cases to emphasize the point. Take dog bites for example. Multiple news reports through the years talk about utility field workers being bitten by dogs as they go about their work. While most do not result in serious injury, some do. We recall one just last year when a utility field worker needed to be airlifted to a nearby hospital after being mauled. Looking at the news articles publicly available and doing some quick back of the envelope estimations, it is possible the number of times utility field workers are attacked by an animal of one kind, or another tops a thousand annually! But it’s just not just animals. There also are occasions when a utility field worker is assaulted by a human. These instances can be incredibly dangerous, too. Instances of verbal threats and physical assaults against utility field workers are not as uncommon as we would wish them to be. Its Dangerous Out There The fact is, maintaining our utility infrastructure can be a dangerous job. To reduce these dangers, utilities go to great effort to keep track of locations of previous dog attacks, property owner threats, assaults, and other known dangers. This information is typically stored in a company’s Customer Information System (CIS), or in a simple spreadsheet. Unfortunately, that information is not always relayed to the field workers before they arrive at a location with a history of danger.   How many of these incidents could be mitigated or even avoided if the utility field worker was consistently notified whenever they came within proximity of a known hazard? Danger Approaching Notifying a utility field worker that they are approaching a known danger is a geospatial problem. This makes it a problem that can be solved with a location aware mobile device such as a smartphone or tablet, and a mobile mapping application such as ArcGIS Field Maps.  ArcGIS Field Maps can use the new geofencing capability to notify the utility worker as they approach the known hazard. When the mobile device carried by the utility worker comes within the defined geofence distance a message is sent to the mobile device’s notification system. The mobile device’s notification system can vibrate, make noise, and display a message on the display screen to inform the user that they are approaching a known danger. Regardless of whether the utility field worker has been dispatched to a location with a known hazard, or the danger is at the neighboring address, the utility field worker will always be notified when they come in close proximity to the known hazard. Geofencing Known Hazards Geofencing is a recent enhancement to ArcGIS Field Maps. This new capability can be applied to any point, line, or polygon layer in the web map. The features within that layer are configured with a buffer distance. The buffer distance defines the geographical fence around each feature in the layer. This Field Maps capability is unique in that it is a completely client-side geofencing and notification system. That means it will work when the device is connected to the network, and it will also work when the device is not connected to the network. With the deployment of this capability utility workers can always be made aware of known hazards. Nearing a Known Hazard What happens when the utility field worker approaches a known hazard? For most utility field workers, they will feel their mobile device vibrating. Depending on their configuration of notifications, the vibration will be followed by a noise.  When they pull the mobile device out of their pocket or pouch, they will see the notification informing them of the known hazard they are approaching. These notifications can be tailored to different levels of threat and can be configured to include information from the known hazard record, such as customer name and address. Communicating Knowledge Maintaining our infrastructure is a job with many dangers. A lack of communication of known information should never be the root cause of an incident. In this day of everyone having a mobile device, isn’t it time to deploy an ounce of prevention to improve your organization’s safety record? PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

Staying Safe When it Really Matters By Tom Coolidge and Tom DeWitte On rare occasions natural gas from a leaking pipe accumulates inside a building. Gas-filled buildings are known as “Gas-Filled Occupancies” or GFOs for short. GFOs pose an obviously high level of risk due to the potential of an explosion if the accumulation ignites. The question needing answered by first responders and all in proximity to the building is just how far away from a GFO do they need to be to be safe? And the natural follow-on question is how can first responders best be aware of that answer during their emergency activity? In terms of existing industry practice, there is no one industry-standard answer to those questions. Each gas utility determines its own policy and practice based on its own unique circumstances and environment. Callers reporting an odor of gas sometimes are told to move outside at least 330 feet away, while others are told to move some other given distance based upon the gas utility’s own criteria or to an unspecified “safe location.” Gas utilities now can better answer those questions for themselves thanks to an initiative of the American Gas Association (AGA). The AGA’s Gas-Filled Occupancy Task Force that was initiated by the AGA Safety and Occupational Health Committee evaluated a decade of GFO explosions to examine the risk relationship between the GFO explosion location and the location of resulting fatalities or injuries. The criticality of GIS as a tool for first responders and all involved in GFO incidents jumped off the pages to me as I read the Task Force’s recent “Gas-Filled Occupancies – Emergency Response” technical paper. Reading the AGA paper also reminded me of one of my favorite IMGIS 2021 Conference presentations that focused on the use of GIS in GFO response situations. The research done by the Task Force found that there were 78 fatalities and approximately 350 reported injuries as a result of a GFO explosion between 2010 and 2020. Not surprisingly, in general, the further a person is away from a GFO explosion, the safer they are. It was concluded that utility worker and fire department fatalities typically occurred within 50 to 100 ft of structure. Most injuries related to a GFO explosion typically were within 100 to 200 ft of the structure and could be characterized as typically minor. If you haven’t already, I strongly encourage you to read the Task Force technical paper. It is really good work on a topic of obvious importance! The AGA Task Force work is specific to GFOs. Among previous guidance referenced by gas utilities is another that is more general. That reference is found in the U. S. Department of Transportation Emergency Response Guidebook. The guidebook specifies there should be a 100-meter (330 feet) evacuation distance from a natural gas leak in an open area. Given the Task Force’s findings, one approach we expect to see more of is the designation of different zones based on specified distances from a building. The zone closest to the GFO being the highest risk zone, the one furthest away the lowest. This is where a modern GIS can play a significant role. GIS can present all responders and those supporting them with a common operating picture indicating both levels of risk within a certain distance of a building and the location of pipe network components that can isolate the area to make the building – and them - safe. An early example of this approach was highlighted at Esri’s 2021 IMGIS Conference in a presentation titled “How close is too Close?” by Lindsay Dreckman of the Metropolitan Utilities District of Omaha, Nebraska (M.U.D.). Based on the work done by the Task Force, M.U.D. knew the distances from GFO incidents where most fatalities and injuries occurred. M.U.D.’s goal number one was to create a tool to generate three safety zones. Looking at the M.U.D. GFO emergency response policy, those zones include Exclusion, Hazardous, and Risk-Reduction. The Exclusion Zone is from the building’s exterior wall to 100 ft and where zero of our personnel are permitted to access. From 100-200ft is the Hazardous Zone, where work can occur but only after coordination with the public safety incident command. And from 200-330ft is the Risk-Reduction Zone, where we can establish a command post but with protection from a potential explosion. It is important to remember that this is an emergency event. Field crews and first responders need both an immediate visual aid and an automated warning to inform them of these safety zones. A mobile application with an interactive map, such as ArcGIS Field Maps is ideal for this need.  Here is an example highlighting that capability. When this mobile application is loaded onto the field technicians’ phone, or tablet, they will have a mobile device that informs them of these risks. This is possible because today’s mobile devices are location aware. The map can display their location with imagery of the area and the safety zones overlayed. Recent enhancements to ArcGIS Field Maps now enable client side notifications based on that mobile device location awareness.  This is called geofencing.  With geofencing the same field technician can have their mobile device vibrate, make sounds and have a banner notification appear on the mobile device when they approach the safety zones. This client side notification is unique because it works regardless of the phone connection to the network. All in all, it turns out that the AGA Task Force now gives gas utilities an extremely valuable reference to inform their decision-making on GFO response policies and practices. And, not surprisingly, location is key. That puts GIS at the heart of improving safety in a fast-evolving and understandably hectic time of emergency response. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

Automation with Lookup Tables Part 3 0f 5 By Tom DeWitte and Tom Coolidge Our first blog of this series provided an overview of the steps a utility can take to improve the productivity of the utility field worker. If you missed it, you can access it here. The second blog of this series explained how barcodes can help to automate field data collection. If you missed it, you can access it here. In this blog article, we will dig into the second method in automating field data collection. The second method for making life easier for the utility field worker is to deploy a configuration of ArcGIS Field Maps that uses lookup tables to auto-populate what the organization already knows about the asset or data collection activity. As noted in the first blog article, this is the second of four methods to automate field data collection. Minimize manual data entry Auto-populate what is already known Leverage sensors on your mobile device Use geography Solving the Steel Issue Within pipe utility organizations, automating the documentation of steel pipe construction has been a challenge. Unlike plastic pipe, valves, and fittings, there is no industry adopted barcoding standard for steel.  The polyethylene pipe and component manufacturers have adopted the ASTM F2897 barcode standard. But steel has not done so. For many pipe utility organizations, documenting steel pipe construction is a very manual process. It requires the utility field worker to: Obtain the manufacturer paperwork or a copy of manufacturer paperwork to acquire the information about the steel components. Read through the many pages of documentation to find the desired information. Manually enter the retrieved information into the mobile application. Now leave the truck, walk to the construction trench, or direct bore and capture GNSS coordinates of steel components. Return to the truck to retrieve the information for the next unique steel component. Repeat process for each unique steel pipe, steel valve, and steel fitting. Automating the documentation of steel construction requires a different method to streamline data entry. Lookup tables can be that method. What are Lookup Tables Lookup tables are Esri geodatabase tables. The purpose of these tables is to store information that the organization already knows about the specifications of the pipe segment or pipe component prior to construction. In the design and cost estimation phase of a project, this information is often referred to as compatible units. To avoid complications with using these geodatabase tables to auto-populate an asset’s record, the lookup table should be a schema duplicate of the featureclass it is supporting. For example, there should be a separate asset catalog table for just the pipe data.  This geodatabase table would be a schema duplicate of the PipelineLine featureclass in the gas and pipeline industry data model, UPDM.                 Pipe Asset Catalog – PipelineLine schema                 Device Asset Catalog – PipelineDevice schema                 Fitting Asset Catalog – PipelineJunction schema   Auto-Populating what is known When addressing the steel construction documentation issue, these lookup tables contain the information the pipe utility organization knows about the steel pipe or component prior to construction. This is typically more information than is provided by the plastic barcodes. For steel this includes knowing the additional pipe characteristics for Barlow’s equation, such as Specified Minimum Yield Strength (SMYS), outside diameter and design factors. These tables can also contain other steel pipe or component characteristics, such as coating type, and pipe specification. These asset catalog tables are pre-populated with the known information. This pre-population occurs prior to the project entering the construction phase. At many gas and pipeline organizations, this list does not change dramatically between projects. Gas and pipeline organizations have codes and standards that they follow, and a select number of vendors from whom they purchase pipe and pipe components. Deploying the Automation Deploying a technology which is continually, and aggressively enhancing its capabilities, such as ArcGIS, means that sometimes there are different deployment patterns for ArcGIS Field Maps with a certain version of ArcGIS Enterprise versus ArcGIS Field Maps with ArcGIS Online. For ArcGIS Enterprise 10.9.1, the ability to use form calculations in Field Maps did not exist when Enterprise was released in December of 2021. For this environment you will need to use geodatabase attribute rules for the automation.  For this article we will focus on the use of ArcGIS Field Map form calculations, as that provides a consistent capability for an ArcGIS Online deployment and an ArcGIS Enterprise version 11.0 deployment. A key benefit of using ArcGIS Field Map form calculations is that they work in both a connected and disconnected network environment. Step 1: Create asset catalog tables. When duplicating the schema of your system of record layers, it is useful to use the Featureclass to XML Workspace tool available in ArcGIS Pro. This preserves the coded value domains assigned to the data fields. Step 2: Add the data field AssetCatalogID to both the asset catalog table and the asset layer The AssetCatalogID field should be a short integer field to eliminate the issue of trailing and leading spaces when querying the table for the record. Step 3: Populate the asset catalog tables with asset information. Step 4: For each asset layer to query the asset catalog tables, create a coded value domain with a listing of all asset catalog entries. The code of the coded value domain will be used to query the asset catalog record containing the utility known information about the asset. Asset Layer assetcatalogid = asset catalog table assetcatalogid This coded value domain needs to be assigned to the “assetcatalogid” field which was added to the asset layer in step 2. The description of this coded value domain will be what field users see when selecting an asset catalog item to describe the newly constructed asset When working with a subtype heavy asset layer such as PipelineJunction (enterprise) has a separate subtype for each type of fitting. A separate coded value domain should be used for each subtype (ie. type of fitting). This shortens the picklist presented to the field user. These different coded value domains can still query the same asset catalog table. Step 5: Assign the arcade script to query asset catalog table The last piece of this configuration of Field Maps is to use the ArcGIS Field Maps web application to assign a form expression to the asset layer field to be automated. In this example that will be the manufacturer field. Here is an example of the form calculation to assign to manufacturer: //Form Calculation:  Pipe_Barcode_Manufacturer //Description: Read the BARCODE, then decode and populate the MANUFACTURER value from the BARCODE value //Description: If an ASSETCATALOG value is entered, query the AssetCatalog Pipes table to retrieve MANUFACTURER value. //Field:  MANUFACTURER //Edit scenario 1: A barcode has been entered if ($feature.barcode != null)     return mid($feature.barcode,0,2) //Edit scenario 2: No barcode, no Asset Catalog entry   else if ($feature.assetcatalog == null)    return ($feature.manufacturer) //Edit scenario 3: No barcode, AssetCatalog value has been entered var assetcatalogId = $feature.assetcatalog; if (assetcatalogId == null) {     return; } //Query assetcatalog table to retrieve MANUFACTURER values var cuTable = FeatureSetByName($map, "AssetCatalog Pipes", ['manufacturer'], false);   //Filter the selected Assetcatalog table records based on assetcatalogid value //Use the first record returned from the query var cuAttribute = First(Filter(cuTable, 'assetcatalog  = @assetcatalogId')); if (cuAttribute == null) {     return; } else {     return cuAttribute.manufacturer     } With the asset catalog tables deployed and the form expression in place to query the table, this automation is ready to put in the hands of your field users. Going to the Field The user experience is very intuitive for the field user. Simply tap on the Asset Catalog data field.  This will open the coded value domain list.  The field user taps on the desired type of asset to select from the list. Once selected, the form expressions will immediately initiate. This will populate the edit form with the asset catalog table retrieved data. With the steel asset data retrieved and populated, the field user can focus on completing the rest of the edit form. Automating Data Entry The gas and pipeline industry has long struggled with how to automate the documentation of new steel pipe construction. Lookup tables in ArcGIS Field Maps is one approach to solving this inefficiency and additional cost. Using lookup tables is a very straight forward method for automating data entry by field users. This method of automation is applicable to many instances where field users are being asked to enter information that the organization already knows. About This Blog Series This blog article is the third in a series of five blog articles. Upcoming blogs will continue explaining in greater detail how to configure the Esri ArcGIS Field Maps mobile application to deploy these examples. PLEASE NOTE: The postings on this site are our own and don’t necessarily represent Esri’s position, strategies, or opinions. ... View more

The attached style file contains the latest version of the symbols designed for deployment with the Utility and Pipeline Data Model (UPDM). These symbols are synchronized to UPDM 2021 and it's asset group and asset type unique value combinations. ... View more