AnsweredAssumed Answered

Web GIS DR tool Stopped Working

Question asked by SJehle on Sep 4, 2018
Latest reply on Oct 30, 2018 by mpschoelen0

Hello,

 

I've been managing a highly available production and DR portal 10.5.1 environments for several months now, and as of 2 monthes ago the webgisdr tool stopped importing portal content into the DR portal environment. I've been unable to troubleshoot why. The resulting error from the import:

 

Failed to restore the Portal for ArcGIS.

Admin Url: https://alias.domain.com/webdadaptor.

{"error":{"code":500,"details":null,"message":"Failed to import site. com.esri.arcgis.portal.admin.core.PortalException: java.lang.Exception: Cannot connect to database: jdbc:postgresql://localhost:7654/gwdb FATAL: the database system is starting up"}}

 

After doing some more digging, the portal's postgres sql log shows the following:

 

FATAL:  terminating connection due to administrator command

LOG:  disconnection: session time: 15:51:16.671 user={adminaccount} database=gwdb host={serverIP} port=63277

LOG:  autovacuum launcher shutting down

LOG:  shutting down

LOG:  database system is shut down

--4minute break--

LOG:  database system was interrupted; last known up at 2018-07-08 06:00:52 PDT

LOG:  connection received: host=127.0.0.1 port=55163

FATAL:  the database system is starting up

LOG:  received fast shutdown request

LOG:  redo starts at F/97000028

LOG:  shutting down

LOG:  database system is shut down

 

After that point the portal admin page reverts back to Create new site or Join new site page. The data store, host server and another federated server environment all import correctly. The strange thing is, once I re-enter the PSA account the disaster recovery site loads up with the same portal site before the import started. Has anyone come across a similar issue? 

 

To give a little bit of a background, the portal environments are identical except for server names and IP addresses. All servers are running on the same OS (Windows Server 2016). The DR environment machines have had their host files modified to always redirect the web context URL to the VIP in their data center as well. They both use the same context URL allowing for the webgisdr tool to work in the past. I've tried a wide variety of things: re-establishing the PSA and retrying the import, reinstalling portal in the DR environment, reconfiguring the DR environment from scratch, importing a backup that previously worked and no luck.

 

I noticed the release of a portal security patch recently and that is one of my next steps. However I'm a little hesitant to move forward with applying the patch before identifying the current issue. I'm also trying to identify if any windows patches or ports are being blocked or are interfering with the backup import, but I wanted to put the question here before going down that rabbit hole.

 

Thanks!

Outcomes