topic Re: Issue with ArcGIS Server security-configuration.json after failed upgrade in ArcGIS Enterprise Questions

Issue with ArcGIS Server security-configuration.json after failed upgrade

PhilippeVDV — Wed, 17 Sep 2025 06:56:07 GMT

Hi folks

We upgraded from AGE 11.2 to AGE 11.5

Portal, ArcGIS Server, relational ArcGIS Datastore & tile cache ArcGIS Datastore on separate VMs (VMWare)

Portal content folder, arcgis server directories & config-store on a fileshare.

Upgrade of Portal went fine

Upgrade of ArcGIS Server was not succesful

I found another post (Solved: Upgrade step 2 of 2 : Failed. Refresh auto-deploye... - Page 2 - Esri Community) that suggested to try another browser. However as ArcGIS Server was completely broken already, we needed to restore the snapshot of the VM. As the Portal upgade was succesful, I decided to only let restore the ArcGIS Server VM. After the restore we had an 11.5 Portal federated with a 11.2 ArcGIS Server. ArcGIS Server directories and the config-store on the fileshare were reverted to the 11.2 files.

Next, I did a second attempt to upgrade the 11.2 ArcGIS Server. But I saw an error in the ArcGIS Sever logs:

<Msg time="2025-09-15T16:36:34,941" type="SEVERE" code="6618" source="Admin" process="11988" thread="1" methodName="" machine="xyzxyzxyz" user="" elapsed="0.0" requestID="">Failed to return security configuration. Cannot invoke "com.esri.arcgis.discovery.admin.store.ConfigStore.securityConfigLastModified()" because "this.d" is null</Msg>

The timestamp of the error matches this step in the upgrade process:

When arrived at the Continue Server Upgade step, it failed again but other behavior now. After clicking Continue Server Upgrade, the process immediately terminated (1 second) and an authentication-prompt appeared. Seems that the federation / communication between Portal and ArcGIS Server was not OK anymore and after digging in the logs I found the error above. Most services are missing.

I try to understand what went wrong the 2nd time: error seems to point to the security-config.json, but that file was just present on the fileshare

No idea what the error (Cannot invoke "com.esri.arcgis.discovery.admin.store.ConfigStore.securityConfigLastModified()" because "this.d" is null</Msg>) tries to say

Any idea what might be the issue? 11.5 Portal federation with 11.2 ArcGIS Server validated well (only complains about the version not matching beween Portal and ArcGIS Server but this makes sense during an upgrade).

What do you usually do when an ArcGIS Enterprise upgrade fails in a later stage and the different components are on several VMs? As Portal upgrade went fine I decided to only restore the ArcGIS Server VM and to retry only the upgrade of ArcGIS Server (Portal upgrade took already 1.5 hours). Restore all VMs and start all over again with the Portal upgrade?

Best regards

Phil

@JonathanQuinn @Scott_Tansley

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

Scott_Tansley — Wed, 17 Sep 2025 07:45:33 GMT

Hi. I’ve had a similar severity predicament recently. I ended up uninstalling the software component. Reinstalling at the current version. Restoring the agssite file, testing/proving it and the. Upgrading on a fresh build. A lot of work but it worked well.

I haven’t seen your error before so can’t give specific advise on it. Sorry.

Scott

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

A_Wyn_Jones — Wed, 17 Sep 2025 08:16:39 GMT

Is ArcGIS Server fully patched at 11.2?

It seems like a permissions error related to your file share, for a sanity check - i would run this tool against the ArcGIS Server, using the same service account:

https://enterprise.arcgis.com/en/server/11.2/get-started/windows/what-s-included-with-arcgis-server.htm#ESRI_SECTION1_A257E9A10181416BA314EAFF5BE85D98

I would also check you don't have any active replication on your fileshare which may be affecting the file system....namely DFS: https://enterprise.arcgis.com/en/server/latest/deploy/windows/choosing-a-nas-device.htm#:~:text=and%20geodatabases.-,Immediate%20consistency,-Choose%20a%20storage

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

PhilippeVDV — Wed, 17 Sep 2025 10:57:19 GMT

Thanks for your reply

Yes, 11.2 was fully patched

I'll check whether the permissions on the folders changed after restoring the 11.2 arcgis server directories + config-store) + will check about the active replication

What is your opinion on restoring snapshots when the upgrade fails in a later stage of the upgrade (here: upgrade of ArcGIS Server failed). Restore all snapshots of all machines and beginning all over again with the upgrade of Portal? Or is it OK to keep our upgraded Portal and only restore the snapshot of the ArcGIS Server?

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

PhilippeVDV — Wed, 17 Sep 2025 11:01:36 GMT

Hmm, that is a lot of work. I hope I'll be able to find another solution (i.e. restore ArcGIS Server, complete the upgrade of ArcGIS Server and finally, upgrade the 2 Datastores)

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

Scott_Tansley — Wed, 17 Sep 2025 23:50:43 GMT

I hear you it was a lot of work. Unfortunately, because I'm under contract and have to deliver within set timescales it's sometimes better to invest the time, clean up the server and approach it in that way rather than spend hours looking for a needle in a stack of needles. I have to put tried and tested methods into play that will deliver, rather than trying to find the actual fix. As an example, my most recent incident saw me unable to upgrade, repair or uninstall an Enterprise Portal. I had to brute-force it's removal and implement the approach above to get the project over the line and not impact my forward programme of work. Hard yards and drastic action but the client have now completed their UAT, and ultimately it was the most expeditious way of moving forward. Once reimplemented as 11.3 (clean install and restore of content), the upgrade to 11.5 was a breeze, also the server and datastore upgraded in the way that I would have wanted them to.

I hope you solve this.

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

PhilippeVDV — Thu, 18 Sep 2025 13:02:19 GMT

Hello Scott, I'm afraid you workaround (clean install of 11.2, import through webgisdr & upgrade to 11.5) will not work for us due to another bug:

ArcGIS Enterprise 11.1-11.2: Restoring tile cache breaks scene layers

Customer has hosted scene layers and unfortunately we already faced the issues described in this post: restore of tile cache datastore fails in 11.2 + none of the workarounds provided by Esri worked for other customers. That bug would be fixed in 11.3

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

Scott_Tansley — Thu, 18 Sep 2025 22:26:55 GMT

My recommendation was to clean install only the ArcGIS Server and restore from the agssite file. There is no need to restore all components or to use webgisdr.

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

DavidColey — Fri, 19 Sep 2025 14:36:34 GMT

The only time I have had a failed ArcGIS Server upgrade was here:

https://community.esri.com/t5/arcgis-enterprise-questions/arcserver-hosting-site-11-2-won-t-create-the/m-p/1361752

When we were upgrading from 11.1 to 11.2. For whatever reason the ArcGIS Server.exe windows service could not be created.

We rolled back the Portal server and the Hosting Site ArcGIS Server servers from their respective snapshots. I then cleaned/upgraded/deleted/re-created any corrupted or otherwise mal-behaving items and hosted feature layers.

I recommend you do the same if you have not already. Of I made sure Server requirements were in place (i.e dot net) and then most importantly, I convinced my IT security team to allow me to disable active virus scanning during the re-upgrade.

This alone reduced what was about a 3-4 upgrade and post-upgrade time down to 30 minutes per machine for my 2-machine cluster Host site. Disabling AV during also meant that I no longer had any issue with the domain arcgiserver account needed to run the windows services.

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

PhilippeVDV — Fri, 19 Sep 2025 15:32:52 GMT

Hi Scott

Just to be sure that we understand each other.

Do you mean the following procedure?

1) Uninstall the 11.2 ArcGIS Server (that is federated with a 11.5 Portal, after a successful Portal upgrade)

2) Empty all arcgis server directories (except arcgiscache) + config-store folders

3) Clean install of ArcGIS Server 11.2

4) Import of 11.2 agssite.file

5) Validation step

6) Upgrade of ArcGIS Server to 11.5

Best regards

Phil

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

PhilippeVDV — Fri, 19 Sep 2025 15:40:47 GMT

Hi David

Yes, I always ask IT to disable AV during upgrades. We restored all snapshots (not only ArcGIS Server machine). But after a new upgrade I see now again the first error: ArcGIS Server upgrade step 2 of 2 : Failed. Refresh auto-deployed services during upgrade’ . Recently, other people faced the same with 11.5: https://community.esri.com/t5/arcgis-enterprise-questions/upgrade-step-2-of-2-failed-refresh-auto-deployed/td-p/411501/page/2 I'll discuss that issue in the respective post

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

Scott_Tansley — Sat, 20 Sep 2025 09:12:17 GMT

Yes. The federation is contained in the agssite not the raw software. I can't promise that this will work, but it puts you in a clean place and, hopefully, you still have VM backups at this point?

Re: Issue with ArcGIS Server security-configuration.json after failed upgrade

PhilippeVDV — Mon, 29 Sep 2025 15:27:40 GMT

Hi @Scott_Tansley . I couldn't reply earlier. Your workflow described above worked for me. That is : after the successful upgrade of Portal to 11.5 and the failed upgrade of ArcGIS Server to 11.5 the ArcGIS Server was completely broken; after following the procedure you suggested (uninstall ArcGIS Server 11.5, clean install of ArcGIS Server 11.2 and import of the 11.2 agssite file, I had an 11.5 Portal federated with an 11.2 ArcGIS Server. That is the good news. But when upgrading the 11.2 ArcGIS Server directly to 11.5, we see again the same error (Failed to return security configuration. Cannot invoke "com.esri.arcgis.discovery.admin.store.ConfigStore.securityConfigLastModified()" because "this.d" is null</Msg>). The only way to workaround it is to upgrade to ArcGIS Server 11.3. Then no error at all. Unfortunately, when upgrading beyond 11.3, then we still have the other error (Upgrade step 2 of 2 : Failed. Refresh auto-deployed services during upgrade). Esri inc support has no clue about both issues yet . (The step with upgrading to 11.3 was our own idea). So now we have an 11.5 Portal federated with an 11.3 ArcGIS Server with a registered 11.2 relational Datastore and an 11.2 tile cache Datastore. But as we can get rid of the first error, fine for me to upgrade ArcGIS Server first to 11.3 before proceeding to 11.5. Hopefully Esri inc finds a solution for the 2nd issue