Upgrade of Hosting GIS Server from 10.7.1 to 10.8.1 Fails During "Continue Server Upgrade"

638
2
09-04-2020 01:41 AM
GavinCollins1
Esri Contributor

We have tried to perform an upgrade of a ArcGIS Enterprise (10.7.1) portal which comprises of a base deployment and an additional federated GIS server. During the upgrade of the Hosting GIS Server, the setup.exe completed successfully however when we attempted to compelte step 6 described in the upgrade documentation, an error is repeatedly returned when we click Continue Server Upgrade.

The error message is "ArcGIS Server upgrade step 2 of 2 : Failed. Refresh auto-deployed services during upgrade." 

In the ArcGIS Server logs we found an anomaly entry for the OfflinePackaging.GPServer being corrupt in the config-store. We repaired this file after noticing it contained malformed text compared to the expected parameters such as the description of the service however the Continue Server Upgrade continues to present the same error message. Historic arcgis server logs indicate this issue existed prior to the upgrade being ran.

ArcGIS Server Logs:
<Msg time="2020-09-03T10:24:38,841" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Started.</Msg>
<Msg time="2020-09-03T10:24:38,849" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Removing local configurations.</Msg>
<Msg time="2020-09-03T10:24:38,877" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Removed local configurations.</Msg>
<Msg time="2020-09-03T10:24:38,879" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Machine certificates and SSL logic started.</Msg>
<Msg time="2020-09-03T10:24:38,885" type="INFO" code="7295" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Validating key store</Msg>
<Msg time="2020-09-03T10:24:49,952" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Machine certificates and SSL logic completed.</Msg>
<Msg time="2020-09-03T10:24:49,953" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Web server starting.</Msg>
<Msg time="2020-09-03T10:24:57,424" type="SEVERE" code="6652" source="Admin" process="31964" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Unable to read the service 'OfflinePackaging.GPServer' from the configuration store. A JSONObject text must begin with '{' at character 1 of </Msg>
<Msg time="2020-09-03T10:24:57,663" type="INFO" code="7565" source="Server" process="31964" thread="27" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Recycling is configured for service Utilities.Geometry.GeometryServer. Recycling will start at 04/09/2020, 00:00 and repeat every 24 hours.</Msg>
<Msg time="2020-09-03T10:24:57,894" type="INFO" code="7565" source="Server" process="31964" thread="27" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Recycling is configured for service Utilities.GeocodingTools.GPServer. Recycling will start at 04/09/2020, 00:00 and repeat every 24 hours.</Msg>
<Msg time="2020-09-03T10:24:57,895" type="INFO" code="7565" source="Server" process="31964" thread="27" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Recycling is configured for service System.DynamicMappingHost.MapServer. Recycling will start at 04/09/2020, 00:00 and repeat every 24 hours.</Msg>
<Msg time="2020-09-03T10:24:57,913" type="INFO" code="7565" source="Server" process="31964" thread="27" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Recycling is configured for service System.PublishingTools.GPServer. Recycling will start at 04/09/2020, 00:00 and repeat every 24 hours.</Msg>
<Msg time="2020-09-03T10:24:57,949" type="INFO" code="7565" source="Server" process="31964" thread="27" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Recycling is configured for service System.PublishingToolsEx.GPServer. Recycling will start at 04/09/2020, 00:00 and repeat every 24 hours.</Msg>
<Msg time="2020-09-03T10:24:57,953" type="INFO" code="7565" source="Server" process="31964" thread="27" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Recycling is configured for service Utilities.PrintingTools.GPServer. Recycling will start at 04/09/2020, 00:00 and repeat every 24 hours.</Msg>
<Msg time="2020-09-03T10:24:57,961" type="INFO" code="6549" source="Admin" process="31964" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Server machine 'MachineName.Domain' started successfully.</Msg>
<Msg time="2020-09-03T10:24:57,976" type="INFO" code="7565" source="Server" process="31964" thread="27" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Recycling is configured for service System.SpatialAnalysisTools.GPServer. Recycling will start at 04/09/2020, 00:00 and repeat every 24 hours.</Msg>
<Msg time="2020-09-03T10:24:58,49" type="INFO" code="9011" source="Rest" process="31964" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Application initialized.</Msg>
<Msg time="2020-09-03T10:24:58,49" type="INFO" code="9011" source="Rest" process="31964" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Application initialized.</Msg>
<Msg time="2020-09-03T10:24:59,295" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Web server started.</Msg>
<Msg time="2020-09-03T10:24:59,295" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Completed.</Msg>
<Msg time="2020-09-03T10:24:59,298" type="INFO" code="7537" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Upgrade step 1 of 2 : Removed the config-store-connection.xml, and created config-store-connection-upgrade.xml for upgrade.</Msg>
<Msg time="2020-09-03T10:25:00,422" type="INFO" code="7720" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">The cloud regions configuration file was deleted from this server machine. If you have other server machines in the site, please make sure that the file has been deleted from them as well.</Msg>
<Msg time="2020-09-03T10:25:00,588" type="INFO" code="7702" source="Server" process="42044" thread="1" methodName="" machine="MachineName.Domain" user="" elapsed="" requestID="">Web server started successfully.</Msg>


 
Afterwards we researched the error message and found various fixes on the GeoNet which we have attempted with no success;

  • Technical Article https://support.esri.com/en/technical-article/000018642 however this applies an upgrade to 10.5.1 therefore non-applicable to us.
  • Re-ran 10.8.1 setep.exe as administrator and in repair mode.
  • Clearing browser cache.
  • Verify ArcGIS Server domain account permissions over its installation and content directory.
  • Use a different web browsers - IE11, Google Chrome.
  • Run web browser as an administrator.
  • Add the fully qualifed domain name of the machine to the trusted sites list in IE.
  • Restart the ArcGIS Server service.
  • Reboot the machine hosting ArcGIS Server.
  • Open Server Manager using the FQDN.

Keen to hear from anybody else who has encountered this error message during the Contine Server Upgrade.

2 Replies
DanielMurillo
New Contributor III

Hello @GavinCollins1 

I am right now encountering the same issue, were you able to find a solution for that failure?

0 Kudos
DanielMurillo
New Contributor III

A heads up just in case anybody sees this having the issue on version 10.8.1. We resolved the issue by running all patches available on ArcGIS Patch Notification.

Then proceeded to click retry and the process continued and succesfully ended.

0 Kudos