ArcPy/CIM Symbology Automation Support for Utility Network Subtype Layers

[SavageTrimble]

Currently, when working with heavily managed Utility Network (UN) datasets, enterprise administrators cannot scale or automate map production using Python scripts. Subtype Group Layer sublayers isolate their rendering pipeline from the standard ArcPy/CIM environment.

When attempting to programmatic change symbols on a Subtype Feature Layer using standard workflows (e.g., layer.symbology or low-level layer.getDefinition('V3')), the application either silently drops the edits, throws an attribute error, or corrupts the internal XML schema validation, throwing a "Requested operation could not be completed" banner and rendering a blank white square in the UI.

Even using CIMREGISTRY pointers on dynamically generated CIMUniqueValueClass entries fails because the UI-cached values are hidden from the active layer definition matrix until manually committed via mouse clicks in the Symbology pane.

Proposed Solution / API Extension

Extend the ArcGIS Pro arcpy.mp or arcpy.cim module to natively support schema-aware unique value mapping on Utility Network sublayers. Specifically:

1. Provide a stable high-level method such as UniqueValueRenderer.addValueCombination([field1, field2], style_item_name) that forces serialization straight into the core map document.

2. Allow Symbol.applyStyleItem() or an equivalent property-setter to interact with read-only subtype symbol objects when a valid style sheet is actively imported into the project items framework.

[JeffBarrette]

@SavageTrimble We need more information to fully understand the issue.  One option would be to work with Esri Support where they can help set up a reproducible case we can look at.  Another option, yet less direct, is trying to describe your schema in more detail and provide a script that is attempting to modify the renderer?

Some questions:

How is the script running?  Is it stand-alone, or run from script tool in the application? 

Does the script modify the underlying data just prior to trying to change the renderer (in the same script)? Have you tried arcpy.RefreshLayer()?

Would it be helpful if you had an additional isSubtypeGroupLayer Layer property?

Thanks,

Jeff - arcpy.mp team

[SavageTrimble]

Answer To Questions:

1. How is the script running? The behavior has been reproduced both as a standalone script execution inside the built-in ArcGIS Pro Python Window and when wrapped natively inside an ArcGIS Python Toolbox (.pyt) script tool running within the application context.
2. Does the script modify the underlying data just prior to trying to change the renderer? Have you tried arcpy.RefreshLayer()? No, the script does not modify any underlying database records or schema tables prior to editing the symbology; it is purely attempting to alter the layer's visual properties in map memory.
3. Have you tried arcpy.RefreshLayer()? Yes, arcpy.RefreshLayer() and a hard canvas toggle (layer.visible = False then True) were both tested. Neither resolves the structural problem. The issue is not a canvas redraw failure; it is a serialization/validation lock inside the application's C++ data provider thread.
4. Would it be helpful if you had an additional isSubtypeGroupLayer Layer property? Yes, exposing isSubtypeGroupLayer and an explicit isSubtypeSublayer boolean property would be incredibly helpful for defensive scripting. However, the root requirement is allowing the ArcPy/CIM pipeline to successfully serialize and commit unique value combinations to these sublayers without the application sandboxing the edits or throwing a UI crash.

Description of Technical Issue:

When interacting with an out-of-the-box Utility Network subtype sublayer, attempting to programmatic convert or apply a multi-field UniqueValueRenderer (specifically targeting ASSETGROUP and ASSETTYPE) fails to serialize into the active map document.

• The Volatile UI Cache Lockout: If a user programmatically applies the multi-field renderer configuration via sym.renderer.fields = ['ASSETGROUP', 'ASSETTYPE'], the high-level Symbology pane updates visually. However, if the script immediately calls l.getDefinition('V3'), the underlying CIMUniqueValueRenderer.groups array returns an empty list (0 rows).
• The Glitch: The application evaluates the generated rows as a volatile visual cache isolated within the UI pane thread. Because they are not serialized down into the actual layer definition object, Python cannot access, manipulate, or map style items to those specific rows via code.
• The Crash Trigger: If a developer attempts to bypass this by instantiating a clean arcpy.cim.CIMUniqueValueRenderer() and injecting it onto the sublayer to force the rows to generate, ArcGIS Pro hard-crashes. This happens because replacing the root renderer object completely strips out the deep, hidden GlobalID pointers, subtype domains, and network association rules natively embedded in the UN sublayer's background XML schema.

Testing Script:

To demonstrate this behavior on any standard ArcGIS Solutions Utility Network layer (set to Unique Values in the UI), run the attached snippet. It illustrates how the UI and the CIM completely decouple, trapping the generated rows in a memory dead-zone:

import arcpy

# Target any standard ArcGIS Solutions UN sublayer currently using Unique Values
layer_name = "Water Device - Fire Hydrant" 

aprx = arcpy.mp.ArcGISProject("CURRENT")
active_map = aprx.activeMap
layer = active_map.listLayers(layer_name)[0]

# 1. Attempt standard high-level multi-field assignment
sym = layer.symbology
sym.renderer.fields = ['ASSETGROUP', 'ASSETTYPE']
layer.symbology = sym  # The UI pane now visually shows the fields assigned

# 2. Immediately inspect the low-level CIM definition to modify the newly generated rows
cim_def = layer.getDefinition('V3')
uv_renderer = cim_def.renderer

# BUG: This will print 'Found 0 active rows' even though rows are generated/visible in the UI pane!
if hasattr(uv_renderer, 'groups') and len(uv_renderer.groups) > 0:
    active_items = getattr(uv_renderer.groups[0], 'items', [])
    print(f"Success: Found {len(active_items)} rows available for style mapping.")
else:
    print("BUG DETECTED: Found 0 active rows in the CIM rendering list. The data provider has sandboxed the rows.")

[JeffBarrette]

@SavageTrimble is it possible that "items" in the arcpy API is called "classes" in the CIM.  We are not 100% consistent, but we tend to try and match what you see in the UI.  In this case, the CIM is different and I don't recall the CIM lineage.

If you change:

active_items = getattr(uv_renderer.groups[0], 'items', [])

to

active_items = getattr(uv_renderer.groups[0], 'classes', [])

Are you getting what you expect?

Jeff - arcpy.mp team

[SavageTrimble]

@JeffBarrette Thank you for the suggestion regarding the property names. To confirm, in the ArcGIS Pro CIM V3 Specification, a CIMUniqueValueGroup object explicitly uses the attribute items (which contains an array of CIMUniqueValueClass objects). There is no .classes attribute on the group container in the V3 schema.

To verify this definitively, we tested both properties in the ArcGIS Pro 3.x Python window. As expected, targeting .classes fails and returns an empty fallback list, while targeting .items is syntactically correct but returns 0 items due to the UI-to-CIM cache isolation bug.

Here is the debugging proof using standard Python reflection (dir()):

import arcpy

layer_name = "Water Device - Fire Hydrant" 
aprx = arcpy.mp.ArcGISProject("CURRENT")
active_map = aprx.activeMap
layer = active_map.listLayers(layer_name)[0]

cim_def = layer.getDefinition('V3')
uv_renderer = cim_def.renderer

if hasattr(uv_renderer, 'groups') and len(uv_renderer.groups) > 0:
    first_group = uv_renderer.groups[0]
    
    # Reflecting properties on the group container
    group_attributes = dir(first_group)
    
    print(f"Contains 'items'? {'items' in group_attributes}")     # Returns: True
    print(f"Contains 'classes'? {'classes' in group_attributes}") # Returns: False
    
    # Checking the actual length of the valid 'items' array
    print(f"Actual items count: {len(getattr(first_group, 'items', []))}") # Returns: 0

[JeffBarrette]

@SavageTrimble I really apologize for the disconnect here.  Our two worlds are not lining up.  I do little with UN data.  The Esri UN data I have and use to test does NOT have fire hydrants.  I'm assuming it is a point layer.  I'm using a unique value, point feature layer called 'Pole' in place of fire hydrant.  When I run your script using my layer, I get opposite results.

If possible, could you send me a very small sample of your data to [email protected].  This way I can hopefully see what you are seeing. 

Jeff - arcpy.mp team

[SavageTrimble]

Thank you so much for sending those screenshots! Seeing your Pole layer results clears up the entire disconnect instantly.

You are completely right—using the unified parent feature class layer directly is the standard way to load the data, which is why your script is working flawlessly. On a completely unrestricted layer, the high-level ArcPy setter serializes perfectly, and the CIM classes property evaluates the data and populates the rows exactly as your screenshot shows.

The reason our two worlds aren't lining up comes down to one critical layer configuration: Definition Queries.

To support our field crews' operational understanding, we split that single, unified parent layer out into multiple distinct map layers, assigning each one a specific Definition Query to isolate asset types (e.g., filtering the parent device layer to show only Fire Hydrants via an ASSETGROUP SQL statement).

The Core Bug Exposed:

When a feature layer has an active Definition Query applied to it, attempting to change or initialize a multi-field unique value renderer via the high-level API (sym.renderer.fields = ['ASSETGROUP', 'ASSETTYPE']) fails to serialize properly in memory.

ArcGIS Pro updates the text fields in the Symbology UI pane, but the background data provider sandboxes the change as a volatile local cache. Because of the active SQL filter query, the application fails to serialize those new classes down into the actual layer definition object in map memory. On a layer with an active definition query:

1. getattr(first_group, 'classes', []) returns a length of 0.

2. The Table of Contents remains empty, and the unique value grid stays blank.

3. The rows are trapped in a memory dead-zone until a user manually clicks "Add All Values" in the UI pane to force the database scan.

How to Easily Reproduce This on Your End:

You don't need our data or any water models to see this! You can replicate this exact behavior right now using your existing Pole layer:

1. Open your map properties and apply a standard Definition Query to your Pole layer (e.g., ASSETGROUP = <Some_Integer_Code>).

2. Ensure the primary symbology is set to Unique Values.

3. Run your field-switching script to assign ['ASSETGROUP', 'ASSETTYPE'].

You will instantly see the bug manifest: the UI pane text fields will change, but the underlying classes array will suddenly return a stubborn 0 rows because the definition query constraint blocks the automated script serialization.

Let me know if you require more information.

Luke