Supported workflow to add new fields to existing Utility Network WebMaps with subtype group layers

[TóthRóbert]

Hi Utility Network community,

We are looking for the supported workflow to add a newly created service field to an existing Utility Network WebMap that uses subtype group layers. The new field is added to a feature class in the Utility Network dataset and appears correctly in the Feature Service REST schema after a service restart. The existing WebMap, however, does not pick up the new field in its stored layer / sublayer configuration. Removing and re-adding the subtype group layer is not a practical option, because the WebMap contains significant subtype group layer and subtype sublayer configuration. The goal is to define a repeatable operational procedure for adding the new field to the existing WebMap without rebuilding the subtype group layer from scratch.

Current development environment:

ArcGIS Enterprise 11.5, ArcGIS Pro 3.5, Branch versioned Utility Network ofc; 
The Feature Service does not lock the database schema
Utility Network feature layers are added to the map as subtype group layers
WebMap created from ArcGIS Pro using Share As Web Map
The WebMap used by custom thin client and can also be opened back in ArcGIS Pro as Portal content

workflow:

A new field is added to a feature class that is part of the Utility Network dataset in ArcSDE (MSSQL) and is included in the published Feature Service.
After restarting the service, the new field is visible in the Feature Service REST endpoint
The new field appears in the REST layer field list, so the Feature Service schema is updated correctly.
When the WebMap is opened back in ArcGIS Pro, the new field is not available in the relevant UI panels where field visibility, field order, popup, table, form, or editing-related configuration is managed.

The WebMap contains substantial configuration at subtype group layer and subtype sublayer level, including:

• definition queries
• subtype sublayer symbology
• label class definitions
• field visibility and field order per Asset Group / subtype
• detailed popup configuration
• attribute table configuration
• potentially form configuration
• potentially feature template-related configurations

Because of this, the old workaround of removing the layer from the WebMap and adding it again is not acceptable. Re-adding the subtype group layer would require rebuilding a large amount of WebMap-level configuration and would be expensive and error-prone
What is the supported workflow to add a newly available service field into an existing WebMap layer / subtype sublayer configuration?
Surely the recommended workflow is not to manually edit the WebMap JSON every time a new field is introduced?
The operational goal is to have a clear, repeatable procedure that an administrator can follow when a new field must be introduced into an existing Utility Network WebMap. The procedure does not need to be fully automated, but it should avoid rebuilding the complete subtype group layer configuration.

Any insights, experience, or recommendations would be greatly appreciated.

Regards,
Robert

[RobertKrisher · Accepted solution]

That does seem like a problem, have you logged a case with support on this yet? I found that when I restarted ArcGIS Pro I was able to see the new field.

[TóthRóbert]

Thanks Robert.

Your comment pointed me in the right direction.

After some additional testing, I believe I understand what confused me during the original investigation.

The first place I checked was the Portal WebMap item, not ArcGIS Pro. At that point, the new field was visible in the Feature Service REST schema, but it was not present in the WebMap JSON stored in Portal. Based on that, I assumed the existing WebMap configuration had not picked up the new service field.

That part still seems to be true: the Portal WebMap item does not update itself automatically. The field only becomes part of the WebMap configuration after ArcGIS Pro picks it up and the WebMap is saved/updated back to Portal.

I had also restarted ArcGIS Pro during my first round of testing. However, after reopening Pro, the first thing I did was immediately update/reload the Portal WebMap. I did not first check whether the new field had already become available in the Pro before performing that WebMap update operation in Catalog. 

After repeating the workflow more carefully, it appears that once ArcGIS Pro is restarted, the new field is picked up from the Feature Service and becomes available in the WebMap-related field configuration UI. From that point, saving/updating the WebMap propagates the field into the WebMap configuration stored in Portal.

One additional detail that made the behavior difficult to spot is the placement of newly discovered fields in the field visibility lists. During my tests, new fields did not always appear in the same position. First they were placed at the beginning of the hidden field list, while than they appeared at the end. In a layer with many fields, it is quite easy to miss them unless you explicitly search for the field name.  Because of that, I initially interpreted the behavior as the field not being present at all, while in reality it had already been picked up by Pro.

At this point, the workflow appears to be working, and the issue seems to have been caused by my misunderstanding of where and when the field refresh actually occurs.

Thanks again for pointing me in the right direction.

Regards,
Robert

[RobertKrisher]

My experience has been that if the layer already has field properties explicitly defined and saved in the map (which it won't until you change something and save), the field will appear at the end of the layer's field properties. If the layer doesn't have field properties explicitly defined, then fields will appear in the same order as the feature service.

One of the things I like to do for fun/paranoia is set up the Schema Report tool to run on a weekly basis in my major geodatabases, then I will periodically use the Compare Schema tool to check for any data model changes that may have been 'forgotten' to be communicated/documented. I will then go through and chase down who made the schema change (pro tip, check the GP metadata history) then make sure that these changes are accounted for in any of the relevant maps/services (do users in group X need to see this field? does it need to be highlighted? what order should it appear in?).

[TóthRóbert]

Thanks Robert,

The Schema Report / Compare Schema workflow is a great idea. I can see how useful that would be once multiple teams start making schema changes in parallel.

The field ordering observation is interesting as well. In my case, the behavior still seems a bit more nuanced, since the WebMap layers already had explicit field configuration and hidden fields before the new field was introduced. So I am not sure that alone explains everything I observed during testing, but it is definitely useful context and something I will pay closer attention to in future schema updates.

Thanks again for taking the time to share your experience and knowladge.

Regards,
Robert