Portal Service can't be started when Upgrade from 10.9.1 to 11.1

[anbinh]

Hi community,

I am doing an in-place ArcGIS Upgrade from 10.9.1 to 11.1.

After successfully running the portal upgrade installation, clicking "Finish" button to close the Installation Wizard. As I am expecting the browser takes me to "Import License Files" page, just found out that the Portal for ArcGIS Service has stopped, and the service account was somehow switched to "Local System Account". 

When trying to start the portal service manually, it throws the error message saying "The Portal for ArcGIS service on Local Computer started and then stopped. Some services stop automatically if they are not in use by other services or programs.". 

Tried both GMSA and local service account to run the upgrade, still get the same issue, hence cannot start the Portal for ArcGIS service and cannot reach the https://**:7443/arcgis page.

For your context, the ArcGIS Enterprise is hosted on Azure, and the portal content directory is Azure Storage. We have another DEV environment with same deployment pattern, but no such an issue with the Upgrade.

Can anyone share some thoughts or have experienced such issue before?

Regards,

AB

[ArchitSrivastava]

Hello @anbinh ,

A good starting point would be check the follow:

• Check the account running the portal service privilege's on the desired folder
  • C:\arcgisportal (appropriate location of different)
  • C:\Program Files\ArcGIS\Portal (appropriate location of different)
• Check logs generated by Portal for ArcGIS
  • To access the logs, browse to the logs directory and open the most recent log file (for example, C:\arcgisportal\logs\<machine name>\portal\portal-20141201.095803-8596-0.0.log).
• Check Windows Event viewer logs
• Check if any remnant processes are running prior to starting the service.

If possible, can you share the logs with me, we can have a look at it to figure where something is going wrong.

Hope it helps!

-Archit

[A_Wyn_Jones]

Hi @anbinh,

Did you have issues applying patches to Portal at version 10.9.1? The message you're seeing indicates a corrupted Portal installation.

Please roll back your deployment to the latest snapshot at 10.9.1 - when you've got back to 10.9.1, please take a WebGISDr export of your deployment to ensure your data is not lost.

I'd suggest you open a support ticket with Esri as soon as possible.

Personally, I have seen this happen a few times lately, with conditions unknown as to what causes it.

The fix for me was to uninstall Enterprise at version 10.9.1, clean windows registry relating to ArcGIS Enterprise, reinstall ArcGIS Enterprise 10.9.1, fully patch it and then import WebGISDr.

In my instance, a patch would refuse to install. Once the system was rebuilt the patch could be installed successfully and the following upgrade to 11.1 was successful.  

[anbinh]

Thanks guys for the help!

Here is an update:

I have partially taken @A_Wyn_Jones 's advice, rolled back to the portal at 10.9.1. Taking a step back, we have tried to apply patches to the 10.9.1 portal. However, just similar as Wyn's experience, the patch was seemingly successfully installed (installation wizard completed its run), but the patch was actually not installed. The patch cannot be found from "Installed Patches" listed in the patch notification wizard, neither can be found from "Installed Updates" in the Control Panel. 

The failure of patching may be very related with the failure of Upgrade. Both involves installation and certain permission, and both installation wizard finished its progress in my case. Just that patch was not applied, or upgrade messed up the portal. 

I have reached out to both system team to investigate the system such as group policy setting, and Esri Support to further investigate the application. 

I will keep this post updated. 

Cheers,

AB

[A_Wyn_Jones]

@anbinh That's good news.

Indeed this is difficult to ascertain the root cause as the conditions are unknown to replicate. 

Just as a demonstration of how a corrupt Windows registry can mess up the installation of Portal for ArcGIS, here's a screenshot from one of these I recently ran into:

If you do elect to stay on the same machine, ensure the Windows registry is cleaned and the control panel list of installed programs is reporting correctly before installing ArcGIS Enterprise 10.9.1 again.

In my case, I also ran a sfc /scannow to check for any missing windows resources (which I had one missing.corrupt .dll). https://support.microsoft.com/en-gb/topic/use-the-system-file-checker-tool-to-repair-missing-or-corr...

An alternative method would be to move to another machine, this blog should help you move to another machine whilst maintaining a friendly DNS alias:https://www.esri.com/arcgis-blog/products/arcgis-enterprise/administration/migrate-to-a-new-machine-...

Hope it all goes well!

[KaitlynStevens]

Hello @anbinh @A_Wyn_Jones and others - 

Esri is investigating an upgrade failure with the characteristics of what has been mentioned here. Please contact Esri Tech Support to open an incident if you are experiencing a similar upgrade failure.

Thank you,

Kaitlyn

[anbinh · Accepted solution]

Update:

Closing this thread as it has been confirmed by Esri that it is due to a problematic Security Patch. Esri has released a repair tool at version 11.1 by the time when this post is updated. Still waiting for the repair tool to be released at 10.9.1.

Thanks all for the input. 

Regards,

Anbin

[ThomasColson]

"For your context, the ArcGIS Enterprise is hosted on Azure, and the portal content directory is Azure Storage. We have another DEV environment with same deployment pattern, but no such an issue with the Upgrade"

@anbinh Would you care to share how you've gotten your Azure storage account to work as the Portal Content Directory? We have a case open with ESRI going into its second month, and no one can figure out why we can't get it to work. 

[anbinh]

Hi Thomas,

I guess it is quite common to use Azure Storage for portal content directory nowadays. When the portal site is initiated, we created the new site via the portaladmin directory by providing the content store in such JSON format:

{

               "type": "cloudStore",

               "provider": "Azure",

               "connectionString": {

                              "accountName": "storageaccount Name",

                              "accountEndpoint": "blob.core.windows.net",

                              "accountKey": "paste here account key secret",

                              "credentialType": "accessKey"

                              },

               "objectStore": "https://storageaccount name.blob.core.windows.net/portal-content-store"

}

Hope it helps.