Failed to restore ArcGIS Portal

[MichaelSchoelen]

We are attempting to recover from a corrupted Portal machine (10.5.1 on Windows Server 2008 R2) . This is not high-availability.

• We successfully rebuilt the Enterprise to the same configuration from before the crash.
• This included a fresh install of Portal for ArcGIS-- we cleared all existing directories
• We ran WEBGISDR to import the backup.
• All of the servers restored successfully.
• When getting to the Portal, the WEBGISDR failed with the error below.

The restore of ArcGIS Server has completed in 00hr:10min:26sec.

Starting the restore of Portal for ArcGIS:

Admin Url: https://staff.gis.website.com/portal.

Failed to restore the Portal for ArcGIS.

Admin Url: https://staff.gis.website.com/portal.
{"error":{"code":500,"details":null,"message":"Failed to import site. com.esri.arcgis.portal.admin.core.PortalException: java.lang.Exception: Cannot connect to
database: jdbc:postgresql://localhost:7654/gwdb FATAL: the database system is starting up"}}

The restore of Web GIS components has completed in 00hr:25min:07sec.

Stopping the webgisdr utility.

C:\Program Files\ArcGIS\Portal\tools\webgisdr>

This error caused the portal to go back into initial configuration mode, and its possible the only option is to uninstall and reinstall again. 

Looking at the initial portal logs, I'm seeing an error related to HA (again, this portal was never HA):

HA configuration files created.</Msg>
<Msg time="2018-10-30T08:10:29,595" type="INFO" code="209160" source="Portal Admin" process="6920" thread="14" methodName="" machine="MACHINE01.DOMAIN.WEBSITE.COM" user="" elapsed="">Configuring database for HA. </Msg>
<Msg time="2018-10-30T08:10:52,966" type="INFO" code="209161" source="Portal Admin" process="6920" thread="14" methodName="" machine="MACHINE01.DOMAIN.WEBSITE.COM" user="" elapsed="">Configured database for HA.</Msg>
<Msg time="2018-10-30T08:10:52,966" type="INFO" code="209062" source="Portal Admin" process="6920" thread="14" methodName="" machine="MACHINE01.DOMAIN.WEBSITE.COM" user="" elapsed="">Storing the config info of machine for portal HA.</Msg>
<Msg time="2018-10-30T08:10:52,971" type="INFO" code="209063" source="Portal Admin" process="6920" thread="14" methodName="" machine="MACHINE01.DOMAIN.WEBSITE.COM" user="" elapsed="">Stored the config info of machine for portal HA.</Msg>
<Msg time="2018-10-30T08:10:52,971" type="INFO" code="209064" source="Portal Admin" process="6920" thread="14" methodName="" machine="MACHINE01.DOMAIN.WEBSITE.COM" user="" elapsed="">Updating the JDBC Url for portal HA.</Msg>
<Msg time="2018-10-30T08:10:52,972" type="INFO" code="209065" source="Portal Admin" process="6920" thread="14" methodName="" machine="MACHINE01.DOMAIN.WEBSITE.COM" user="" elapsed="">Updated the JDBC Url for portal HA.</Msg>

[JonathanQuinn]

Those aren't errors, they're simply messages indicating that the deployment is getting itself ready to be HA regardless if you intend it to be HA or not. They can be ignored as they aren't related to the failure.

What version are you using? If it's pre-10.6, then you can rename the newly created dbXXXXX folder to db and then move the .config-store-connection.json file out of the db folder and into the framework\etc folder. That'll get Portal back up and running quickly.

At 10.6 and on, Portal will roll back to it's previous state if the restore fails.

About the error, does this happen every time? Rebuilding the site per the instructions above will make rolling back manually easier.

[MichaelSchoelen]

Oh, noted. We're doing a reinstall right now, so we can try that next time.

We are running 10.5.1, and this has happened both times when restoring the portal.

[MichaelSchoelen]

Ran it again, and received the same error message 

[JonathanQuinn]

Hm, so perhaps the database is taking longer than expected to start up. Do you see any errors in the pg logs under the lgos directory\database? If you were to turn on DEBUG messages before the restore, you should be able to watch for when the database is told to start, and then you can monitor when it does by watching for the postgresql.exe processes. You may want to reach out to Tech Support as this may require some digging.