Best Practices for working with Utility Network: Publishing Utility Network Service
Authors: @MichaelHirschheimer , @Saranya-Kesavan
Over the past few years, Mike and I have published multiple Utility Network services in an ArcGIS Enterprise environment. Over the course, we have made silly mistakes as we relied too much on muscle memory and felt the pain of having to redo work. We’ve also learned about some simple tweaks to the publishing map that resulted in overall performance improvements. In this blog we will share our wisdom gained from those experiences and provide you with some preventative strategies to publish a Utility Network service quickly.
Utility Network Foundational solutions
One of the key benefits of Utility Network is the industry standard data model that comes with it. Starting with Esri’s Utility Network Foundation solution, organizations do not have to spend their time and implementation budget on building custom data models and information products for their network management in GIS (but are welcome to configure the data model as needed, to fit their specific needs).
Industry specific Utility Network Foundations can be readily downloaded from the ArcGIS Solutions website.
These foundation solutions include a data model that configures ArcGIS Enterprise and the ArcGIS Utility Network to represent <Industry> networks. It deploys foundational services, simplifying the creation of additional web and mobile apps and provides a desktop data editing environment. A collection of tasks, with step-by-step directions, are also provided to help deploy the solution to an Enterprise environment.
Pertaining to those tasks, we will be sharing our wisdom in the following topics
- Importance of the publishing map
- Troubleshooting version management related issues
- Feature layers in the publishing map
- Maximum number of coded domains
- Troubleshooting register data source issues
- Optimal visible scale considerations for feature layers
- Backing up ArcGIS Pro project
Importance of the publishing map
It will take multiple maps to support your Utility Network implementation. There will be a map specific to publishing the Utility Network (called Electric Utility Network in the Foundation project. See #1 in screenshot below). Each layer’s data source in the publishing map will come from a database connection and there are specific requirements that must be met. Once published, you can create editing and viewing maps using subtype group layers (see #2 and #3 in the screenshot below). Their data source will be from a featureservice which will allow you to access the Utility Network in mobile and web applications. More information to come on editing and viewing maps in future blogs.
Troubleshooting version management related issues
To publish a Utility Network Service, all feature layers and tables that are added to the publishing map must be registered as a branched version. An efficient way to find out the publishing map’s conformance to this is by inspecting the Configuration tab: Share Tab > Web Layer > Publish Web Layer > Configuration tab
If you see an option to enable “Version Management” capability in the Share As Web Layer pane, you are all set. This confirms that all the feature layers are registered as branched versions in your publishing map.
CAUTION: Do not spend any time changing symbology, renaming layer names or other map changes, before checking that version management capability is present in the configuration tab. This could be all for nothing if you can’t enable Version Management.
Sometimes, things do not go as planned/expected. Without the Version Management capability, Utility Network functionality won’t be available upon publishing. Troubleshooting steps include:
- Make sure the layers added to the map come from a Geodatabase connection file whose versioning type is Branch. You can check it here: Catalog > Right click on Geodatabase connection file > Geodatabase Connection Properties
- Inspect each layer in the map and confirm its versioning type is Branch. (Right click on a feature layer > Properties > Source)
- If the layer is not flagged as branched in the properties dialog, remove it from the map and add it again using the branched geodatabase connection file.
Feature layers in the publishing map
At Esri, we recognized that the number of visible layers with visible content can have an impact on performance. Querying each layer in the map resulted in multiple requests being sent and responses being received from the database or service. In the case of Utility Network feature dataset, the dilemma of whether to add all the objects to a publishing map is understandable, considering the sheer number of it. But our experts vouch for better performance if all the objects in the Utility Network feature dataset are added to the publishing map.
Here’s why -- When a user is editing data from an ArcGIS Server published service, each update is a separate edit operation. This means when you change the attribute of a single feature, that update is a single edit transaction, which requires locking all the objects associated with the object being edited. So, if you update a single attribute on 10 different features, one at the time, that is 10 edit transactions. On top of it, if an object in the feature dataset is not in the map, then additional database queries need to be made. This will result in a performance hit in ArcGIS Pro.
Another tip here -- if a feature class participates in a relationship class, add the related object to the map. Because, when the feature class is queried/edited, ArcGIS Pro will try to open the related object. If it is not on the map, then additional database queries need to be made. To simplify this querying process and thereby to optimize performance it is advised to add the related object to the publishing map.
While adding multiple layers in the map helps to optimize overall performance with feature edits, one might experience slowness when the map is being rendered with all the layers. This slowness can be minimized by turning off the visibility for those layers that need not be seen on the map. This will come in handy when you pan and zoom in your publishing map. You can also choose to pause map rendering altogether if you do not want to visualize your data in the publishing map.
Finally, if you use landbase along with Utility Network, think about end-user workflows. If landbase edits need to be seen in the same version where facilities are being edited (for example, in a new Underground Residential Development, new roads are being added), then the landbase layers should be in the same service. So, add the landbase to your Utility Network feature service publishing map. On the other hand, if landbase and Utility Network are in separate feature services, landbase edits would be in its own version and the facility edits would be in a different version.
Maximum number of coded domains
Another setting that is sometimes forgotten is the maximum number of coded domains. The default value of 25,000 may not be large enough for a Utility Network. Change the value to 1 million: 1000000
But why? Let’s do the math. There are at least 12 database objects in your Utility Network. As an example, the Electric Utility Network model contains an average of 38 fields and 13 subtypes in each object. Suddenly the number creeps to a range closer to 25,000. If you choose to enhance the data model, the number of coded domains can exceed more than the default.
Remember, with the default setting, 25,000 is the maximum number of domain codes that can be returned from all fields, subtypes, layers & tables. When the number of domain codes returned by the MaxDomainCodeCount property exceeds the default value, the service may continue to run normally, but you will see the domain descriptions not displaying as expected for some attributes. In addition, errors will be captured in your ArcGIS Server logs.
So, remember to increase the maximum number of coded domains in the configuration tab while sharing the web layer. Share Tab > Web Layer > Publish Web Layer > Configuration tab > Click the Feature’s pencil icon > Configuration Parameters tab > tip open Advanced
You can also edit it with the ArcGIS Server manager, as shown below.
Troubleshooting register data source issues
When you analyze the map to publish your Utility Network Service for the first time, you are most likely to encounter an error indicating that the “Layer’s data source must be registered with the server.”
An effortless way to navigate this error is to right click on error message for one of those layers and choose to register data source with server. You will be prompted to add your data store to your server.
Upon successfully resolving the error for one layer, the other layers will be automatically updated. Click Analyze in the Share as Web Layer pane to make sure all the errors are cleared out.
Optimal visible scale range considerations for feature layers
It is a best practice to determine the scale range at which individual feature layers will be rendered. Failing to do so will result in performance issues when the map is being rendered.
For example, if the Line feature layer is set to be visible at the county level, there could be millions of objects that are rendered and that takes a considerable amount of processing and time. Despite that time and effort, end users cannot perceive any meaningful information. Similarly, a point layer that is visible at all scales will not mean anything when the map is zoomed in at the city level. So, pay attention to your workflows and determine the appropriate rendering scales for each layer. This will dramatically reduce the load when the map is rendering.
Setting a visible scale range at the feature layer level will ensure that these visibility scale ranges are passed to all the maps downstream that will be using the feature service. Depending on the feature service clients, you could fine tune the scale ranges as needed.
Backing up ArcGIS Pro project
Upon successful publishing of the Utility Network service, it is always a good idea to back up the ArcGIS Pro project so that you have a copy of all the configurations & the database connection files. This will come in handy when you need to make changes to the existing service, have multiple administrators managing the Utility Network environment, when upgrading the enterprise version or when gearing up for creating another service.
There are different ways you could back up your ArcGIS Pro project: saving the project from your local machine to a network drive or uploading it to your portal. If you choose to upload to portal, you can share it with others for collaboration. Also, you can use groups to ensure that only a certain member of staff can access it. There is also an option to mark that project as Authoritative which indicates it’s a trustworthy project to use.
Conclusion
The purpose of this blog is to share the knowledge we’ve gained from our firsthand experiences on publishing a performance optimized Utility Network service. We hope our insights save you time by avoiding common pitfalls. The items discussed can be considered as best practices to follow along with the tasks you will receive as part of Utility Network Foundation solution.
We look forward to sharing more of our findings in our upcoming blogs. Stay tuned!
About the Authors
Mike & Saranya are Solution Engineers on the Esri Infrastructure Team with a focus on the Utility Network. Between the two of us, we have 10+ years of applying ArcGIS to utilities, and we are continuously excited to work with customers to improve their experience.
While building a 10.8.1 environment and exercising Utility Network workflows, we’ve accumulated a wealth of knowledge that will aid in maintaining your System of Record. We look forward to sharing what we’ve learned with you in a series of blog posts. These blogs could be considered as best practices for Utility Network.
Mike Hirschheimer is a solution engineer on the Utilities Team with a focus on the Utility Network. He is a big sports fan who enjoys watching NFL & NBA games and playing disc golf.
Saranya Kesavan
Saranya Kesavan is a solution engineer on the Utilities Team with a focus on the Utility Network. Outside of work, she is an avid reader and a big-time motorcycle enthusiast.