Latest Contributions by RobertKrisher

Creating maps has always been an important aspect of providing value with a GIS. Even with the proliferation of web and mobile GIS apps, the foundation of most of these experiences is configuring a map that presents users with the information they need. Did you know that the ArcGIS Solutions team has created several productivity tools that help make creating, configuring, and deploying maps easier? This will highlight several important tools included in the Utility Data Management Support tools (UDMS) from the ArcGIS Solutions team that will make your life easier. If you’re not familiar with the Utility Data Management Support tool you can access their documentation and downloads for each release of ArcGIS through the above link. The tools we will be discussing in this article are: Update Data Sources Map Layers to CSV Set Field Properties to CSV Configure UN Layer Before we begin looking at these tools, let’s look at a feature included in ArcGIS Pro that helps make it easier to configure different maps. Copy and Paste Properties Use Paste Properties to move configuration between different layers. When creating a map for a project you are typically taking configuration from one or more existing maps, merging them, then adding new configuration on top of the resulting product. Most users are familiar with saving/loading layer files or copying layers between maps to achieve this, but did you know that you can also copy and paste the properties between two layers? This feature lets you take the configuration of one layer and paste all the properties onto another layer, even if they are pointed at different feature classes. If you attempt to copy and paste the properties between two layers that are too different, like a point and line feature class, you will receive an error: You will not be able to paste symbology between layers with different geometry types You cannot paste properties between two layers that don't support the same types of properties. However, in these cases it is still possible to copy some of the properties between the layers. The Paste Properties command has a flyout menu that lets you select which property you want to paste onto the layer. Using this menu will allow you to paste supported properties between different types of layers, if the property is available on both layer types. You can paste individual properties on a layer to overcome the limitation of pasting between different types of layers. This trick is especially useful for pasting definition queries between layers, since having consistent definition queries between layers allows you to multiselect layers and change all their definition queries at the same time. Next up, we’ll look at a tool that makes it easier to take already configure maps and deploy them to multiple environments. Update Data Sources Use Update Data Source to quickly update the data sources for all the maps in your project. Including changing between local and remote geodatabases or feature services. As GIS and IT professionals we often share our work across multiple environments. Maybe you maintain internal and external-facing copies of the same map, or more commonly you may maintain a staging environment that mirrors your production deployment. When you make a change in one of these environments and need to promote it to the other the most efficient way of doing this is to take the map you want to copy, point it to the appropriate environment, then replace/overwrite the map in the target environment. A key part of this process is to update all the data sources in the map to point to the correct environment. For maps that contain many layers, or subtype group layers, this can be a tedious and slow process. This is where the Update Data Sources tool comes into play. It lets you choose one or more maps in your project and point them to a specific workspace. The tool even allows you to repoint maps between local (file or mobile) geodatabases, enterprise geodatabases, and even feature services. Field Properties When we think about configuring maps it’s easy to think that its all about symbology and labeling but properly configuring the fields in a map is arguably just as, if not more important. Because each map is designed to meet the needs of a specific set of users it’s important that we only present attribute information to users that is relevant to their workflow. An important part of creating a curated experience is to: Hide fields that aren’t relevant to the workflow Order visible fields in a way that is meaningful to users Highlight any fields that require user attention Ensure all the fields required for contingent values are visible Going through this process with a handful of layers, with a handful of fields, for a few maps is straightforward; what do you do when your GIS contains hundreds of layers with hundreds of fields and hundreds of maps? You need something to work more effectively. This is where there are several tools that can help make the process easier, and more efficient. The first tool to look at is the Map Layers to CSV tool. Use the Map Layers to CSV tool to export your map's field properties to a CSV file. This tool allows you to export the field property configuration of a given map to a CSV file.  However, the spreadsheet isn’t just used for informational purposes. Once you’ve exported this file you can easily review the field configuration for a map and make the necessary adjustments to the various field properties (visibility, display order, highlighting, alias, etc). Make adjustments to the field properties CSV using an editor of your choice. After populating the spreadsheet, you can then use the Set Field Properties to CSV tool to apply the configuration in the CSV back to the original map. Use the Set Field Properties from CSV tool to update the field properties for the layers in your map using a CSV file. This approach has two main benefits. First, it allows you to quickly make a significant number of changes to the configuration of a map. Secondly, it allows you to save a copy of your desired configuration locally so you can easily review it with customers, apply it to other projects, or track how it has changed over time. Popup configurations are typically configured to match the field properties of a layer, so make sure you configure your field properties before you adjust your popups to save yourself time! Configure UN Layers All the tools up until this point have been applicable to any map, but this next tool is unique to the utility network. However, the next tool is designed specifically to add information about the utility network to a map that contains layers and tables that belong to a utility network. The Configure Utility Network Layers tool allows you to add information to the popups, labels, and definition queries to the utility network layers in a given map. This additional information is very useful for GIS administrators and super-users who are first learning the utility network. This configuration typically isn’t added to maps used by most users because the extra information it adds is not meaningful to their needs or in some cases there may be a performance cost associated with loading the popup. When a configuration has a possible performance impact, it will be noted below. Use the Configure UN Layers tool to add informative information to the popups, labels, and other properties of your utility network layers. The tool has options to add five different pieces of information to a layer, let’s examine what each of these looks like. Rule Popup The Rules Popup allows you to quickly see what types of features are allowed to be connected, attached, or contained to each other. This option adds a section to the bottom of a popup that will show all the network rules associated with the feature. This information is useful when new users are trying to learn what types of features are allowed to be connected or related to each other and is especially useful when trying to work through invalid connectivity errors. Category Popup Use the category popup to quickly see what categories a feature has assigned. What features are allowed to act as controllers in the network? What kind of equipment serves a protective or isolation role in the network? Each utility network contains a mix of system-defined and user-defined network categories that allow you to answer these questions. By adding Network Categories to the popup for a feature you can better understand that feature’s role within the network. Subnetwork Popup Use the subnetwork popup to see information about the subnetworks on a given feature. When looking at the subnetwork information on a feature we can only see the subnetworks that it belongs to. However, when trying to understand the larger picture it is often useful to understand relationships between the current feature’s subnetwork(s) and other subnetworks in the system. By adding Subnetwork information to the popup, you will see information about any subnetworks adjacent to the current subnetwork. When a feature belongs to subnetworks in different tiers, you will see the information listed for each tier that it belongs to. This situation is more common for hierarchical domain networks but also happens for subnetwork controllers in partitioned domain networks that exist at the boundaries between two tiers (i.e. Distribution and Transmission). Note: This configuration can impact the performance of loading the popup because the expression queries the feature layers to identify adjacent subnetworks. Features in a hierarchical domain typically belong to subnetworks in multiple tiers. Category Display Filter Use the category display filter to add display filters to your layer to help you quickly identify types of features. Seeing the Network Category assigned to a feature is useful, but when familiarizing yourself with your data it is often useful to visualize the network categories as well. The Category Display Filter will add display filters to the map that allow you to toggle between different network categories you are interested in. Use the display filters pane to toggle between manual display filters. This is a quick way to understand the roles of different features in your map (Subnetwork Controller, Isolation, etc.) without needing to review thousands of popups. Note: This configuration can impact the performance of drawing the map at small scales. This is because a display filter is a client-side filter, which means that all the features within the extent of the map must be loaded from the database but only the ones that match the expression will be displayed. Create a display filter for isolating or protective devices to help with reliability studies. Terminal Popup The Terminal Popup lets you review the terminal configuration assigned to the feature along with information about the terminals and paths for that terminal configuration. Asset Types that have a Terminal Configuration assigned to them allow a device or junction object to have explicit, predefined Terminals that other features can use to connect to them. This is useful when you need to distinguish the inlet and outlet ports of a device that sets or regulates pressure, a device that is stepping up/down the voltage of electricity, or when you need to distinguish between the lines connected to either side of a device. Adding the Terminal Configuration information to the popup for a feature makes it quick to see which features have terminals, whether those terminals are directional (upstream/downstream), and whether the device itself has internal paths that can affect tracing through the device. Conclusion If you’re interested in discovering more tools to help with utility network projects, check out the Utility Data Management Support Tools GitHub page for the full list. You can also learn about more tools to help with data modeling/migration as well as reporting. If you have specific questions about how to solve different workflows, please visit the ArcGIS Utility Network channel on the Esri Community site. This is a large community with thousands of active members. ... View more

Are you looking to do a utility network implementation, and want to know some of the tricks of the trade to make the process easier? Or maybe you have previously implemented a utility network, and are looking for ways to streamline your processes? Either way this article will show you the five most important tools provided by the ArcGIS Solutions team to help with all the configuration and quality assurance that goes into each data migration, most of which you’ve probably never heard of! We've previously written articles describing how to use special tools to rename or withhold configuration in an asset packagewhile keeping the original schema intact. All the tools in this article are either part of the Utility Network Package toolbox or the Utility Data Management Support tools. The UN Package tools are included with the utility network data models provided by the ArcGIS Solutions team. The Utility Data Management Support tools can be downloaded, along with their online help document, through their GitHub page. You can watch the records of the ArcGIS Utility Network: Utility Network Foundations presentation from the 2024 Esri User Conference for a discussion of how to use many of these tools. The tools we will be discussing in this article are: Create Utility Network Copy Workbook Apply Utility Network Copy Workbook Summarize Utility Network Errors Sync C Tables Asset Group/Types Assign Terminal Connections Creating Utility Network Copy Workbook Most data migration projects make use of a Utility Network Foundation. These are data models for different industries that come pre-configured for use with ArcGIS Utility Network. Most of these foundations have several variations. There is an Expanded model, that includes all the different configurations for many different workflows, but there may be other Essential models that include a subset of the model tailored for different companies in the industry (distribution, transmission, etc) or even a simpler that is designed to be expanded on. This raises the natural question, what do you do if you start with an Essentials model and want to borrow some of the elements from an Expanded model? The answer is you use the Create/Apply Utility Network Copy Workbook tools. The first tool, Create Utility Network Copy Workbook, lets you pick two utility network foundations. The source asset package should be the model that contains all the configuration, the Expanded model. The target asset package should be the model you want to import the configuration into, your Essentials model. Use the Create Utility Network Copy Workbook tool to create a workbook that allows you to import configurations from an expanded model into your current asset package. Once you run the tool it will create a workbook that shows all the asset types in the source model that are available or you to import into your target model. In this example we want to import the service meters from the Expanded model into our Essentials model. The screenshot below shows the asset types for service meters from the expanded model: Service Meter asset types from the expanded water utility network foundation.   The following screenshot shows the asset types for service meters in the essentials model:   Service Meter asset types from the essentials utility network foundation.   To do this we open the Utility Network Copy Workbook created by the tool and look for rows that define the Service Meters.   Rows from the utility network copy workbook from the service meter asset group.  Once you know which asset types you want to copy, you populate the To Source/AssetGroupName/AssetTypeName columns in the spreadsheet. This will define which asset group(s) the asset types will be added to, and what they will be named in your model after copying. You only need to populate the To AssetGroupCode/AssetTypeCode in the spreadsheet if there are specific codes you want to use in your target Asset Package. Otherwise, Asset Package.   Populate the To columns in the utility network copy workbook to identify the features you want to add to your asset package. When these asset types are copied into your target model, all the configuration and rules associated with them will also be copied. If you only want to copy some of the configuration, you can use the right most columns to control what you do and do not want to copy with each asset type. You can control which configurations are imported into your asset package using the workbook. By default, all related configurations will be added.  In this example we want to add the detailed Service Meter asset types to our model, so we would populate the spreadsheet as follows: The workbook in the above screenshot shows five additional asset types will be added to the model.   We will discuss what happens when you run the tool in the next section. Apply Utility Network Copy Workbook Once you’ve created and populated a Utility Network Copy Workbook, the next thing you need to do is apply it to your target Asset Package. To do this you run the Apply Utility Network Copy Workbook tool. In this section we will continue our example from the previous section, where we are copying the detailed Service Connection asset types from the Expanded Model into the Essentials model. After running the tool against the populated Utility Network Copy Workbook, we can verify the changes in our asset package. The Asset Type domain has been updated to include the new asset types. We can see that the existing asset type (originally Service Meter) is still in place, and new asset types have been added to the domain. We can also verify the other configuration was copied as well by looking at the various B_* tables in the asset package. In the example below we can see that new rules have been added to the model for the added asset types. If we look closely at the rules, we will see that no junction-edge rules have been imported for the new Service Meters. This is because the Expanded model has more detailed asset types for Service Lines that we didn’t import. There are two ways we can address this: First, if we want to keep the single asset type in the Essentials model, we will need to manually configure rules for the new Service Meters. Otherwise, if we want to have more granular asset types for Service lines in our target asset model, we can add those to our model as well. This will then allow the Service Meters to connect to the Service Lines. However, we will now have a problem that the imported Service Lines won’t have rules for all the Service Valves, Fittings, etc in the simpler model. This shows an important consideration when copying information from an Expanded model into an Essentials model. Because the tool will only import configuration for the asset types that exist in both models, you will always need to manually adjust some of the configuration to incorporate it into your model. Summarize Utility Network Errors The quality assurance provided by Network Rules and Feature Restrictions of the utility network allow you to quickly find data quality issues that are meaningful for your industry. While the utility network includes tools designed to help you identify and resolve tools on a row-by-row basis, using pop-ups or the error inspector, they often fall short when it comes to dealing with the thousands of errors that can occur during an initial data migration. The Migration toolset released with ArcGIS Pro 3.5 includes tools to help with the most common types of errors, and I recommend you read the Analyzing Topology Errors and Resolving Topology Errors articles if you haven't already. However, if you are on an older release of ArcGIS Pro or have errors not handled by those tools you should look at using the Summarize Utility Network Errors tool. Once you have deployed a utility network and enabled the network topology, run the Summarize Utility Network Errors tool to create a mobile geodatabase and report. By default, the tool summarizes all the topology and subnetwork errors in the Dirty Areas table of your utility network. The tool will add two layers to your map: Error Summary – This table summarizes all the types of errors discovered in the database. Errors By Type – This table breaks down the errors by table, asset group, and asset type. There are many ways to use this information, but a common way to develop an action plan is to look at the Errors By Type table and try to identify a corrective action for each row in the table. Some of these errors will require configuration changes, others will require manual or automated data cleanup. There are a few tricks that can help make this process easier. The first is to create a chart using the Errors By Type table to help visualize the different problems. This can help you prioritize which errors to fix first by identifying the error code with the most errors. The Summary geodatabase created by the tool also includes several additional classes that aren’t added to the map by default that are useful. One such class is the Errors feature class. This polygon feature class shows the location and details of every error the tool identified. It is best used when you want to look at examples of errors in your data, as it lets you quickly navigate to the locations where specific errors were discovered. Sync C_Tables Asset Group/Type Most data migration projects make use of Asset Packages to stage and apply migrated data between systems. Each asset package contains a set of tables that represent the schema, data, and configuration of a utility network. Tables in the asset package that are prefixed with a “C_” represent information that will be loaded into system tables using special tools. Keeping these tables in sync with the rest of the data in the asset package can be tedious, but this is where the Sync C_ Tables tool can help. Some of the fields in these tables, like feature Global IDs, are easy to populate during a data migration while others, like Asset Group/Asset Type names, can be difficult or inefficient to populate during a data migration. Because of this, the ArcGIS Solutions team created the Sync C_ Tables tool that can be run on an asset package that will synchronize the Asset Group and Asset Type information in the C_ tables of an asset package with the Asset Group/Asset Type information on the corresponding rows in the Asset Package. This means you can populate each C_table with a minimal amount of information shown below. After running the tool, the asset group and asset type fields will be populated. If this information is not synchronized before the Asset Package is deployed this will result in validation errors that must be corrected before you can proceed. Assign Terminal Connections Terminals are a new way of representing connectivity used by Utility Networks. When data is migrated into a utility network, features that connect to a device with terminals must know which terminal they are connected. Failure to populate this information can result in ambiguous connectivity errors if the two features are allowed to connect using different terminals, but the terminal being used isn’t specified. Fortunately, the ArcGIS Solutions team developed the Assign Terminal Connections tool that can analyze utility network data and specify which terminals on a device each line is connected to by making some reasonable assumptions. Running this tool will look at your devices with terminals, their terminal configurations, rules, and the lines connected to them and use this information to connect each line to the terminal the tool thinks is most appropriate. Once the tool has finished running you should validate the changes to ensure that all ambiguous connectivity errors are resolved. This process works well for bi-directional terminal configurations, because the terminals on these devices are typically interchangeable. However, you should carefully review any locations where the tool populated terminal connectivity on a device with a directional terminal configuration, like pumps or step transformers. In these situations, the terminal that each feature is connected to is important because it controls what is upstream and downstream, so if the tool assigns a feature to the wrong feature this will affect tracing. An easy way to perform this is to use the Layers tab of the Attributes pane to review all the features with directional terminal configuration, then use the Modify Terminal Connections pane to correct any features that are improperly connected. Conclusion If you’re interested to discovering more tools to help with utility network project, check out the Utility Data Management Support Tools GitHub page for the full list. You can also learn about more tools to help with configuring maps as well as reporting. If you want to watch a video discussing many of these tools, watch the ArcGIS Utility Network: Utility Network Foundations presentation from the 2024 Esri User Conference for a discussion of how to use many of these tools. If you have specific questions about how to solve different workflows, please visit the ArcGIS Utility Network channel on the Esri Community site. This is a large community with thousands of active members.   ... View more