topic Re: Failed to validate the server managed database in ArcGIS Enterprise Questions

Failed to validate the server managed database

LavanyaVasudevanNES — Wed, 19 Nov 2025 02:04:24 GMT

Hello Everyone,

We are running into issues at 11.3, and 11.5 where clients performed the latest Windows Server 2019, 2022 patches, restarted the machines and datastore no longer validates hence "breaking" Federation. I have opened an ESRI ticket and we are seeing nothing unusual.

1. We have ensured all folders have service account access, no luck

2. We have restarted the the services and machines ( portal, datastore, server) , no luck

3. Describedatastore shows healthy status, still no luck

4. We even tried to restart a broken hosted service via portal admin end point to see if it will fix the issue, no luck

5. Ensured all ports were open, disabled firewall, still no luck

This is happening to machines both on the AWS environment and On -Premise Hyper-V boxes. Since this is a production environment, we are hoping to use the Repair Datastore as the last resort.

This is broken at 11.5 even on a machine that hasnt been updated since January so makes me thing something else is happening. Three different IT departments, three different Clients are involved. All of the machines were functional before Wednesday the 12th of November 2025.

The machines worked perfectly last Wednesday. No other changes were made except for the update patches to Windows. I monitor a few more machines that are going to get the patches applied this next weekend and am hoping to find a root cause/solution before then. I am attaching a screenshot of the updates that were applied, machines restarts and AGE is broken.

I am really running out of resources and time at this point. Any help is appreciated.

thankyou

lavanya

Re: Failed to validate the server managed database

JakeSkinner — Wed, 19 Nov 2025 09:18:31 GMT

Hi @LavanyaVasudevanNES,

You should have some datastore backups in the following location \arcgisserver\backup\relational\dbbackup. You can try performing a fresh install of ArcGIS Data Store and restore one of the backups located here for the relational store. Take a look at the below video for steps on how to do this:

https://community.esri.com/t5/arcgis-enterprise-documents/restore-and-upgrade-relational-data-store-after/ta-p/1664356

Re: Failed to validate the server managed database

LavanyaVasudevanNES — Wed, 19 Nov 2025 16:50:21 GMT

Thankyou Jake.

This is a great solution but it still doesnt get to the root cause of why all of a sudden the clients are losing federation/validation.

Re: Failed to validate the server managed database

JakeSkinner — Wed, 19 Nov 2025 16:55:17 GMT

@LavanyaVasudevanNES are the datastore logs (i.e. C:\arcgisdatastore\logs) reporting anything?

Re: Failed to validate the server managed database

LavanyaVasudevanNES — Wed, 19 Nov 2025 17:08:06 GMT

We set the log more to Debug on the server and here is what is returning:

<Msg time="2025-11-19T11:05:06,179" type="DEBUG" code="9999" source="Data Store" process="8844" thread="1" methodName="" machine="MachineName" user="" elapsed="" requestID="">Health check replication....</Msg>
<Msg time="2025-11-19T11:05:06,180" type="DEBUG" code="9999" source="Data Store" process="8844" thread="1" methodName="" machine="MachineName" user="" elapsed="" requestID="">Replication health check is not enabled.</Msg>
<Msg time="2025-11-19T11:05:06,180" type="DEBUG" code="9999" source="Data Store" process="8844" thread="1" methodName="" machine="MachineName" user="" elapsed="" requestID="">Replication is not enabled for this relational data store.</Msg>

Re: Failed to validate the server managed database

JakeSkinner — Wed, 19 Nov 2025 18:35:41 GMT

There will be additional logs specifically for the Data Store located at \arcgisdatastore\logs. I would recommend checking there as well.

Re: Failed to validate the server managed database

LavanyaVasudevanNES — Wed, 19 Nov 2025 21:08:36 GMT

Hey Jake,

This is from the datastore. The logs were empty on 11, 12,13 and then 14th it start filling up with this. I do have an ESRI case where I have sent the complete log file to: Esri Case #04048032

Thankyou again for providing such quick responses.

Re: Failed to validate the server managed database

A_Wyn_Jones — Thu, 20 Nov 2025 10:14:54 GMT

Can you check the Windows Services start up type, I have found Windows update to changes these sometimes.

Try restarting ArcGIS server whilst everything else is running - If this works, you may need to change ArcGIS Server and Datastore to start up type to Automatic (Delayed)

Re: Failed to validate the server managed database

LavanyaVasudevanNES — Thu, 20 Nov 2025 14:52:02 GMT

Thankyou Wyn, they were set to Automatic. I had to reset them back to Automatic(Delayed) and that still didnt fix the federation/validation. I still dont understand why Windows Updates reset this and service account permissions to folders.

I am now looking into: \arcgisportal\logs\database\pg_log and seeing a whole lot of

2025-11-13 08:00:20.878 PST [4800] FATAL: terminating connection due to administrator command
2025-11-13 08:00:20.878 PST [4060] FATAL: terminating connection due to administrator command
2025-11-13 08:00:20.878 PST [3096] FATAL: terminating connection due to administrator command
2025-11-13 08:00:20.878 PST [3428] FATAL: terminating connection due to administrator command
2025-11-13 08:00:20.888 PST [2112] LOG: background worker "logical replication launcher" (PID 1252) exited with exit code 1
2025-11-13 08:00:20.908 PST [4088] LOG: shutting down
2025-11-13 08:00:22.271 PST [2112] LOG: database system is shut down

and then at about 4:30 CST, the log file shows:

2025-11-13 14:37:27.785 PST [10908] LOG: starting PostgreSQL 15.5, compiled by Visual C++ build 1931, 64-bit
2025-11-13 14:37:27.786 PST [10908] LOG: listening on IPv6 address "::", port 7654
2025-11-13 14:37:27.786 PST [10908] LOG: listening on IPv4 address "0.0.0.0", port 7654
2025-11-13 14:37:27.843 PST [10188] LOG: database system was shut down at 2025-11-13 14:20:23 PST
2025-11-13 14:37:27.870 PST [10908] LOG: database system is ready to accept connections

and all other log files after this are EMPTY.

Re: Failed to validate the server managed database

LavanyaVasudevanNES — Tue, 25 Nov 2025 16:44:13 GMT

Okay we have tried to unregister the datastore, restore from backup and it has failed.

The server itself was restored to Pre-windows update phase and everything validated and worked perfectly fine. Unfortunately, due to IP conflicting issues, we have had back to the post Windows Update phase. Now Datastore is broken/not validating and federation gives an error. Luckily this is sufficient to get the GIS team working on stuff, but we are back to the original problem. This is surely something from the windows update that is messing with the GIS elements. We suspect: https://support.microsoft.com/en-us/topic/november-11-2025-kb5068791-os-build-17763-8027-485c3175-0dc0-45c6-8328-553863ea2cbc is the problem child.