ryan.martin_APHA

ArcGIS Server 10.4.1 to 10.5.1 Failed Upgrade and Solution

Discussion created by ryan.martin_APHA on Feb 21, 2018
Latest reply on Feb 28, 2018 by csgeosol1

This will not be relevant to everyone but I am sharing in case it helps someone else overcome the difficulties I experienced.

 

We recently attempted an upgrade of ArcGIS Server from 10.4.1 to 10.5.1 on one of our test/R&D servers running Windows Server 2008 Standard SP2 but we encountered difficulties. Specifically this error message at stage 2 of 2 of the upgrade process:

Upgrade step 2 of 2 : Failed. Refresh auto-deployed services during upgrade

 

If you run the task Asynchronous you can use the ArcGIS Server back end for the duration the upgrade task is processing and when it fails it locks you out again. In the server logs for the ArcGIS Server back end, which I accessed while the Asynchronous task was running, I discovered this error repeated numerous times with type "SEVERE" and code "6563":

 

"Failed to create the service System/PublishingToolsEx.GPServer"

If you look at the Asynchronous task list and select the upgrade job it looked like this (effectively a list of services starting and restarting continuously due to errors with the publishing tools service, then it eventually fails):

Upgrade step 2 of 2 : Started.

Upgrade step 2 of 2 : License check started.

Checking and validating server licenses.

Upgrade step 2 of 2 : License check completed.

Upgrade step 2 of 2 : Configuring application server.

Starting server on machine '{SERVERDOMAIN}'.

Upgrade step 2 of 2 : Configured application server.

Upgrade step 2 of 2 : Configured platform services.

Upgrade step 2 of 2 : Updating auto-deployed services.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Starting the service 'CachingControllers' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsManagement' on machine '{SERVERDOMAIN}'.

Stopping the service 'GeoAnalyticsTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'ReportingTools' on machine '{SERVERDOMAIN}'.

Stopping the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Starting the service 'PublishingTools' on machine '{SERVERDOMAIN}'.

Upgrade step 2 of 2 : Failed. Refresh auto-deployed services during upgrade

I tried a number of obvious things including restarting the services and restarting the server to no avail. After some online research I got to the root cause of the error with troubleshooting advice from foreign language forums. The upgrade only works with IE9 or newer installed on the host machine.

 

We were running Windows Server 2008 Standard SP2 (so not Server 2008 R2) and its default browser was IE7. Apparently the upgrade process for ArcGIS Server 10.5.1 uses the browser on the host machine to run upgrade scripts even if it doesn't tell you as much in the error messages. 

 

To be fair the system requirements for 10.5.1 do say IE9 and upwards, but there's little guidance that says what error messages and failure types you will get with incompatible browsers and why this happens, plus any FAQs to solve these problems. I'd imagine the problem isn't that common but an ESRI knowledge base article on it would be nice.

 

We tested the fact the IE9 and up browser has to be installed on the host machine by the fact we tried to launch the upgrade through IE11 on a remote machine and the task still failed. However, after installing IE9 on Windows Server 2008 Standard SP2 the upgrade completed successfully.

 

I might add for anyone using Windows Server 2008 Standard SP2 that upgrading to a supported browser is no easy job either. Firefox and Chrome no longer support the OS so these are no goes as its not worth the risk using a non-vendor supplied legacy version. IE9 is effectively your only choice and this has three pre-requisite patches that need to be installed first before the IE9 upgrade file will run: KB948465 (Service Pack 2 if you've not yet installed this) then KB971512 then KB2117917. 

 

You may also need to use an Internet Explorer 9 offline installer from the Microsoft website since the other Microsoft website downloads found from Google didn't run for us saying that the installer did not support our operating system. Note the IE9 installers are labelled "Windows Vista" but that's fine as Windows Server 2008 SP2 is based on Vista, the newer Server 2008 R2 was based on Windows 7.

 

Happy to tackle any questions for someone with a similar issue.

 

Thanks

 

Ryan

Outcomes