nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

UN version 7, ArcGIS Pro 3.5.4 validation errors - a showstopper?

DavidSDouglas

We're planning to upgrade in Production, and have done it in our Test environment. We're having big problems with validation and can't risk an upgrade in production without a fix. Has anyone one else experienced similar issues? I'm asking before contacting support with the local ESRI partner - it's easier if some else can confirm this issue.

We upgraded from ArcGIS Pro 3.3 and ArcGIS Server 11.1 to 3.5.4 and 11.5
Also Utility Networks upgraded from version 5 to 7 (that took 20 hours)

.NET 8.0 is installed on servers (previously 6.0)

Some functions are running faster than before, faster even than the production environment which has not yet been upgraded - it has most resources. The servers are Windows Server 2022 Datacenter Azure Edition 21H2. The SQL Server database is Microsoft SQL Azure (RTM) - 12.0.2000.8

We have divided Denmark into 27 large polygons and validate these each night using a Python script:

GPextent= ('{} {} {} {}'.format(x1, y1, x2, y2))
arcpy.un.ValidateNetworkTopology(serviceUrlProd,GPextent)

After upgrading, this takes about 30 minutes to run, about half the previous time - also compared to production.

However, interactive validation in ArcGIS Pro using the Validate function on the Utility Network tab times out after 5 minutes even with the smallest extent (a few metres - which took only a few seconds before, and in production). If I attempt to validate again within the next few minutes I get:

Edit operation failed. Message: Unable to complete operation.
Details: Serialization lock in progress [[Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Lock TimeoutX] [Serialization lock in progress - operation not allowed]

Eventually the lock is released - and validation will time out again on the next attempt. This has not been seen in the current production version,even if validation timed out after using too large an extent.

Using Validate Network Topology (Utility Network Tools), these small extents do validate - but it takes 30 minutes!

During editing, no dirty area appears after altering attributes. They do however appear if a feature is deleted.

Find more posts tagged with

Accepted answers

Thomas_Brown

David,

Please provide the incident number. I'll make sure the issue is escalated and you will receive the proper support to narrow down this issue.

There are a few issues at play here... first to understand why your validates are so slow (which are likely leading to the serialization lock issue). Assumption is the validate you are performing is in a version?

Thank you.

Tom

All comments

DavidSDouglas

Yes, we upgraded in Production in January, and it went smoothly. Again, we discarded all previous versioning and then re-established versioning, with has improved performance, especially, and massively, when disabling and enabling network topology, and with the Upgrade Dataset for UN (SQL Server). It is now possible to validate all of Denmark at once (1:2,500,000), whereas before, it timed out after 5 minutes if you zoomed out beyond about 1:200,000. Strangely, however, the first validation in an ArcGIS Pro session now takes several minutes, even when it's a single dirty area in a tiny area. Subsequent validations take around 40 seconds (it seems hardly to matter if you validate the whole country or a small area) - before, it took around 15 seconds.

PK1

Hi @DavidSDouglas have you upgraded your prod environment? Any further learnings? Thanks.

rkrisher

@VishApte_NGIS in the case of the utility network, all the UN_* tables will be pruned. In terms of what will happen if you run a trace from a moment earlier than the earliest historical moment, the trace may fail or return inconsistent results. This is because the information stored in the system tables to perform tracing will no longer be in sync with the information in your user tables.

VishApte_NGIS

Thanks @RobertKrisher

To understand a little more about prune version history with Only Prune Systems Tables option:

If we prune version history to say 1-Jan-2025 with "Only Prune Systems Tables" checked, we can still connect to a historic moment before 1-Jan-2025, say 1-July-2024 and visualize the network on that date but the trace will not run or will run incorrectly as the data for it doesn't exist in EIDMAPPING and other UN system tables, is it a correct assumption?

As per the documentation at https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/overview/prune-branch-history.htm

If the Only Prune System Tables parameter is checked, only the eligible internal system tables of the controller datasets will be pruned or reported to the log file.

Is is possible to know which internal system tables will be pruned?

Thanks,

Vish

rkrisher

There is an option to only prune system tables: Prune Branch History (Data Management)—ArcGIS Pro | Documentation. Make sure you read the documentation surrounding the tool to understand the preconditions for it to work effectively.

VishApte_NGIS

Hi @RobertKrisher

I recently upgraded a customer to ArcGIS Enterprise 11.5, UNM V7 and ArcGIS Pro 3.5.2. And customer is getting "Edit Operation Failed: Serialization lock in progress" error intermittently when performing Validate Network Topology operation as described in this thread.

Unversioning and reversioning the UNM dataset is not an option. We will lose the history of the network changes and ability to see the network at a point in time.

We have done few rounds of Disable Topology and Enable Topology to tune topological rules and recently to upgrade UNM from V5 to V7. But nothing over the board. If EIDMAPPING table is the culprit, is there a official way to only prune that table?

Thanks,

Vish

rkrisher

@PierreloupDucroix you'll want to ensure that versions are reconciled on a regular basis. You can automate this, but you need to make sure that you don't reconcile a version that already has conflicts! Otherwise, you will automatically accept the conflicts as-is without any record that they existed (the default behavior). If you use Python you can accomplish this with the Reconcile Versions tool by seting the parameter proceed_if_conflicts_not_reviewed to NOT_PROCEED (which isn't the default value).

If you're using REST, or some other means you will need to use the APIs to query for persisted conflicts on the version before you reconcile.

ducroix_magis

Enabling/disabling the topology is an integral part of normal network usage, as it's necessary for tasks such as adding asset types and their associated network rules, categories, or attributes.

Therefore, I understand that we must account for the growth in the EIDMAPPING table each time such an administrative action is performed. This means we must also manage the number of versions persisting in the database and regularly clean up (per scheduled task?) the versions using the Prune Branch History tool.

rkrisher

Do not manually run SQL statements to clear out history, that is a sure-fire way to corrupt the versions in your geodatabase. Enabling the network topology populates the table with the new mappings.

The safe way to manage the size of these tables is to

Be considerate about how often you are disabling and enabling your network topology or performing bulk updates on branch versioned data.
If you need to remove rows, use the Prune branch history tool (read more here). This will safely delete records older than a certain date. If you are not routinely performing other data maintenance tasks with your database, the tool will not be able to remove as much history. I know that sounds vague, read the link I provided above for a more precise description.

samshrma998

@RobertKrisher . Based on your reply it means whenever the topology goes through enable/disable process new records gets added to the EID_Mapping.

What is the idle process to maintain the growth of this table while enabling/disabling might be required.

Is it good practice if the EID mapping table is cleared manually using SQL statement after disabling topology. Assuming mappings will be created again when topology is enabled

rkrisher

@PierreloupDucroixthe most common cause of this is by frequent enabling/disabling of the network topology. the table maps every feature to its corresponding network element. every time you disable/enable the network topology we must delete and recreate all the mappings. The table is versioned, so if you are doing this frequently and not managing your conflicts and history than the table can become large.

ducroix_magis

I am curious to learn more about the EIDMAPPINGS table, as it is growing considerably in our UN database.

DavidSDouglas

We succeeded in upgrading the Test environment at last, without any validation problems afterwards. The difference is that we removed versioning first - this took around 10 hours to run - and then enabled versioning again. It was much faster than a previous attempt as we temporary upgraded the Azure SQL service for the Test database to have 16 CPUs instead of 4. Every feature had probably been altered several times since we migrated 2 years ago (from one older UN version, and from Intergraph G/Technology after a company merger) - the original migration left many errors that have been corrected either by script or manually in the last 2 years. One table in the database, UN_8_EIDMAPPINGS has 230 million rows - it now has 9 million. After each step we tested ArcGIS - still able to validate. After reversioning, we upgraded geodatabase, tested, disabled network topology (much faster than before - I suspect versioning has caused us performance problems previously). Then Upgrade Dataset - it took 10 minutes - last time (before removing versioning) it took 20 hours! Even though we had quadrupled the CPUs, it must be the de-versioning that made the difference. One odd thing however - it took a while to work out we had to have the ArcGIS Server services running (they have to be stopped when disabling network topology or removing/re-establishing versioning), but the feature services using UN data had to be stopped on the portal. If the ArcGIS Server services were not running we got "ERROR 003320: The connected portal's federated servers do not meet the minimum supported release for the Utility Network - 10.7" - which was bizarre, as all servers were up-to-date. Anyway, after a long frustrating process, we now feel confident we can update to 3.5.5 and 11.5 in Production in the new year.

DavidSDouglas

I installed the 3.5.5 update, which has some UN Validation bug fixes - but that didn't help. However, I noticed we hadn't installed the ArcGIS Server 11.5 Feature Services Security Patch (from October) on the servers, so I did that, and it has helped the interactive validation problem - we can now validate, but only pretty small extents, otherwise it times out. We also are unable to run prune branch history - it failed after 36 hours working on UN_8_EID_MAPPINGS (no idea what that is, but there's 230 million rows in the database table). Almost all the other tables were pruned. We also tried to remove versioning and re-establish it, but there's still the same massive numbers of rows in UN_8_EID_MAPPINGS. We're now planning to start again with a fresh copy of the production database in Test, and document every step of the way. Can it validate using 3.3.0 before upgrading? I'm sure it can, but we will document it. Then we will try to remove versioning - if that doesn't succeed it's not a 3.5.5 / 11.5 issue. If it does, try validation again. It may be faster to do the database upgrade again after removing versioning has massively lowered the number of rows in the tables, with added RAM and CPU just like in Production. That's so we can estimate how long it will take when we upgrade in Production (our control room is manned 24/7, so taking say 20 hours to upgrade the database is a major inconvenience).

Thomas_Brown

David,

Please provide the incident number. I'll make sure the issue is escalated and you will receive the proper support to narrow down this issue.

Thank you.

Tom

DavidSDouglas

Yes, updating to the latest geodatabase version was a requirement for upgrading, now that objectid is 64 bit

ducroix_magis

Hello,

Did you update the geodatabase version in addition to the UN version (using the Geodatabase Update tool)?

This resolved a significant performance issue we were experiencing in ArcGIS Pro (related to version posting, not network validation).

This could be a lead to investigate further.

DavidSDouglas

Not yet, but will do soon. In the last major support cases we have been asked to perform many time -consuming tests (creating new features, services, changing versioning, exporting to file geodatabase, recording video of screen etc, but there was no resolution of the the problem. That problem was that after running the Export to CAD tool, or Flip Line, ArcGIS Pro began to call a feature service repeatedly for no evident reason, taking 20 - or sometimes even 200 seconds per call! The only solution was to exit and reenter ArcGIS. Then, when we upgraded to 3.5.4 in Test, we saw that the bug was no longer present. We can't have been the only customer to experience this, and I was hoping that by posting our current problem here, someone might recognise it (even though it's only a few weeks since 3.5.4 was released).

rkrisher

Have you opened a case with support yet?

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.