UN version 7, ArcGIS Pro 3.5.4 validation errors - a showstopper?

[DavidSDouglas]

We're planning to upgrade in Production, and have done it in our Test environment. We're having big problems with validation  and can't risk an upgrade in production without a fix. Has anyone one else experienced similar issues? I'm asking before contacting support with the local ESRI partner - it's easier if some else can confirm this issue.

We upgraded from ArcGIS Pro 3.3 and ArcGIS Server 11.1     to 3.5.4 and 11.5
Also Utility Networks upgraded from version 5 to 7 (that took 20 hours)

.NET 8.0 is installed on servers (previously 6.0)

Some functions are running faster than before, faster even than the production environment which has not yet been upgraded - it has most resources. The servers are Windows Server 2022 Datacenter Azure Edition 21H2. The SQL Server database is Microsoft SQL Azure (RTM) - 12.0.2000.8

We have divided Denmark into 27 large polygons and validate these each night using a Python script:

GPextent= ('{} {} {} {}'.format(x1, y1, x2, y2))
arcpy.un.ValidateNetworkTopology(serviceUrlProd,GPextent)

After upgrading, this takes about 30 minutes to run, about half the previous time - also compared to production.

However, interactive validation in ArcGIS Pro using the Validate function on the Utility Network tab times out after 5 minutes even with the smallest extent (a few metres - which took only a few seconds before, and in production). If I attempt to validate again within the next few minutes I get:

Edit operation failed. Message: Unable to complete operation.
Details: Serialization lock in progress [[Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Lock TimeoutX] [Serialization lock in progress - operation not allowed]

Eventually the lock is released - and validation will time out again on the next attempt. This has not been seen in the current production version,even if validation timed out after using too large an extent.

Using Validate Network Topology (Utility Network Tools), these small extents do validate - but it takes 30 minutes!

During editing, no dirty area appears after altering attributes. They do however appear if a feature is deleted.

[samshrma998]

@RobertKrisher . Based on your reply it means whenever the topology goes through enable/disable process new records gets added to the EID_Mapping. 

What is the idle process to maintain the growth of this table while enabling/disabling might be required. 

Is it good practice if the EID mapping table is cleared manually using SQL statement after disabling topology. Assuming mappings will be created again when topology is enabled

[RobertKrisher]

Do not manually run SQL statements to clear out history, that is a sure-fire way to corrupt the versions in your geodatabase. Enabling the network topology populates the table with the new mappings.

The safe way to manage the size of these tables is to

1. Be considerate about how often you are disabling and enabling your network topology or performing bulk updates on branch versioned data.
2. If you need to remove rows, use the Prune branch history tool (read more here). This will safely delete records older than a certain date. If you are not routinely performing other data maintenance tasks with your database, the tool will not be able to remove as much history. I know that sounds vague, read the link I provided above for a more precise description.

[PierreloupDucroix]

Enabling/disabling the topology is an integral part of normal network usage, as it's necessary for tasks such as adding asset types and their associated network rules, categories, or attributes.

Therefore, I understand that we must account for the growth in the EIDMAPPING table each time such an administrative action is performed. This means we must also manage the number of versions persisting in the database and regularly clean up (per scheduled task?) the versions using the Prune Branch History tool.

[RobertKrisher]

@PierreloupDucroix you'll want to ensure that versions are reconciled on a regular basis. You can automate this, but you need to make sure that you don't reconcile a version that already has conflicts! Otherwise, you will automatically accept the conflicts as-is without any record that they existed (the default behavior). If you use Python you can accomplish this with the Reconcile Versions tool by seting the parameter proceed_if_conflicts_not_reviewed to NOT_PROCEED (which isn't the default value).

If you're using REST, or some other means you will need to use the APIs to query for persisted conflicts on the version before you reconcile.

[VishApte_NGIS]

Hi @RobertKrisher 

I recently upgraded a customer to ArcGIS Enterprise 11.5, UNM V7 and ArcGIS Pro 3.5.2. And customer is getting "Edit Operation Failed: Serialization lock in progress" error intermittently when performing Validate Network Topology operation as described in this thread. 

Unversioning and reversioning the UNM dataset is not an option. We will lose the history of the network changes and ability to see the network at a point in time. 

We have done few rounds of Disable Topology and Enable Topology to tune topological rules and recently to upgrade UNM from V5 to V7. But nothing over the board. If EIDMAPPING table is the culprit, is there a official way to only prune that table?

Thanks,

Vish

[RobertKrisher]

There is an option to only prune system tables: Prune Branch History (Data Management)—ArcGIS Pro | Documentation. Make sure you read the documentation surrounding the tool to understand the preconditions for it to work effectively.

[VishApte_NGIS]

Thanks @RobertKrisher 

To understand a little more about prune version history with Only Prune Systems Tables option:

If we prune version history to say 1-Jan-2025  with "Only Prune Systems Tables" checked, we can still connect to a historic moment before 1-Jan-2025, say 1-July-2024 and visualize the network on that date but the trace will not run or will run incorrectly as the data for it doesn't exist in EIDMAPPING and other UN system tables, is it a correct assumption?

As per the documentation at https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/overview/prune-branch-history.htm 

• If the Only Prune System Tables parameter is checked, only the eligible internal system tables of the controller datasets will be pruned or reported to the log file.

Is is possible to know which internal system tables will be pruned?

Thanks,

Vish

[RobertKrisher]

@VishApte_NGIS in the case of the utility network, all the UN_* tables will be pruned. In terms of what will happen if you run a trace from a moment earlier than the earliest historical moment, the trace may fail or return inconsistent results. This is because the information stored in the system tables to perform tracing will no longer be in sync with the information in your user tables.