AnsweredAssumed Answered

Join Portal sites issue (Portal for ArcGIS 10.6.1)

Question asked by sylvanvij on Dec 3, 2018
Latest reply on Dec 18, 2018 by sylvanvij

Jonathan Quinn

Hi. We are trying to create a High Availability Portal. We followed the ESRI documentation and created Portal for ArcGIS 10.6.1  site 1 on Machine 1. Then we installed portal of same version on Machine 2(Identical to Machine 1).

 

After that, we tried to Join the machine 2 to Site 1. But after few minutes of processing, we get back an error saying "Can not configure the local machine in standby mode for high availability." We have shared our portal's content directory and log directory to a shared drive and portal installation / service account has full rights on that folder.

 

Event viewer Log (Secondary Machine):

When we looked at the Event viewer of Machine 2, we noticed that, after few moments of giving the "Join Site" request from Machine 2 , we could see an error from PostgreSQL saying "pg_ctl: PID file "C:/arcgisportal/db/postmaster.pid" does not exist".

After this, few minutes later, the C:/arcgisportal/db folder which got created few moments agon on Machine 2, disappeared.

 

These are few excerpts from Postgresql's log file from Machine 1 (Primary):

 

2018-12-03 05:31:14 PST: [6656]: LOG: connection received: host=Machine2 IP port=65238
2018-12-03 05:31:14 PST: [6656]: LOG: connection authorized: user=portaladmin database=gwdb
2018-12-03 05:31:14 PST: [6656]: LOG: disconnection: session time: 0:00:00.179 user=portaladmin database=gwdb host=Machine2 IP port=65238
2018-12-03 05:31:15 PST: [9028]: LOG: connection received: host=Machine2 IP port=65239
2018-12-03 05:31:15 PST: [9028]: LOG: replication connection authorized: user=someuser
2018-12-03 05:31:21 PST: [7048]: LOG: connection received: host=127.0.0.1 port=56544
2018-12-03 05:31:21 PST: [7048]: FATAL: no pg_hba.conf entry for host "127.0.0.1", user "DOMAIN USER NAME USED FOR PORTAL INSTALLATION", database "DOMAIN USER NAME USED FOR PORTAL INSTALLATION", SSL off
2018-12-03 05:31:21 PST: [7048]: DETAIL: Client IP address resolved to "Machine 1 Host name", forward lookup does not match.
2018-12-03 05:31:23 PST: [9028]: LOG: disconnection: session time: 0:00:07.945 user=repuser1543832114496 database= host=Machine2 IP port=65239
2018-12-03 05:31:23 PST: [5032]: LOG: connection received: host=Machine1 IP port=56549
2018-12-03 05:31:23 PST: [5032]: LOG: connection authorized: user=portaladmin database=gwdb
2018-12-03 05:31:23 PST: [5032]: LOG: disconnection: session time: 0:00:00.104 user=portaladmin database=gwdb host=Machine1 IP port=56549
2018-12-03 05:31:25 PST: [10228]: LOG: connection received: host=Machine1 IP port=56551

 

Portal log looks like following when we try to join machine 2 (Primary machine log) to Site 1:

INFO Starting the web server. 2018-12-03T14: 38: 07,704 Portal "Machine1 Host Name" 217053 8472
WARNING The web server was stopped. Restarts it.  2018-12-03T14: 38: 07,704 Portal "Machine1 Host Name" 217064 8472
INFO Scheduler is closed down. 2018-12-03T14: 37: 04,450 Sharing "Machine1 Host Name" 219999 4180
INFO Stops watchdog thread.  2018-12-03T14: 36: 51,259 Portal "Machine1 Host Name" 217066 8472
WARNING The database server was stopped. Restarts it. 2018-12-03T14: 36: 48,596 Portal "Machine1 Host Name" 217060 8472
INFO Starting driving the watchdog thread. 2018-12-03T14: 36: 43,259 Portal "Machine1 Host Name" 217055 8472
DEBUG Have successfully resumed running the process monitor on the local machine.  2018-12-03T14: 36: 43,259 Portal Admin "Machine1 Host Name" 9999 4180
INFO Starter process monitor. 2018-12-03T14: 36: 43,258 Portal "Machine1 Host Name" 217046 8472
DEBUG Resumes running process-monitor plugin. 2018-12-03T14: 36: 43,258 Portal "Machine1 Host Name" 216057 8472
DEBUG Resumes running process monitor on the local machine.  2018-12-03T14: 36: 43,255 Portal Admin "Machine1 Host Name" 9999 4180
INFO Have successfully rolled HA configuration back. 2018-12-03T14: 36: 43,255 Portal Admin "Machine1 Host Name" 209022 4180
DEBUG Node Agent Plugin is paused. 2018-12-03T14: 36: 43,254 Portal "Machine1 Host Name" 216055 8472
DEBUG Have successfully put the process monitor on hold on the local machine.  2018-12-03T14: 36: 34,601 Portal Admin "Machine1 Host Name" 9999 4180
DEBUG Sets the process monitor plug-in.  2018-12-03T14: 36: 34,601 Portal "Machine1 Host Name" 216056 8472
DEBUG Puts process monitor on the local machine. 2018-12-03T14: 36: 34,560 Portal Admin "Machine1 Host Name" 9999 4180
INFO Rolls the HA configuration back.
DEBUG Connecting to the configuration store using connection string 'jdbc: postgresql: //"Machine1 Host Name": 7654, "Machine2 Host Name": 7654 / gwdb? TargetServerType = master'.  2018-12-03T14: 36: 33,63 Portal Admin "Machine1 Host Name" 219999 4180
DEBUG Connection string: jdbc: postgresql: //"Machine1 Host Name": 7654, "Machine2 Host Name": 7654 / gwdb? TargetServerType = master 2018-12-03T14: 36: 33,63 Portal Admin "Machine1 Host Name" 219999 4180
INFO HA: Monitoring standby codes.  2018-12-03T14: 36: 14,68 Portal "Machine1 Host Name" 216004 8472
INFO HA: Noden "Machine1 Host Name" is configured to be a master. 2018-12-03T14: 36: 14,68 Portal "Machine1 Host Name" 216003 8472
INFO HA: Monitoring standby codes. 2018-12-03T14: 35: 13,235 Portal "Machine1 Host Name" 216004 8472
INFO HA: Noden "Machine1 Host Name" is configured to be a master. 2018-12-03T14: 35: 13,235 Portal "Machine1 Host Name" 216003 8472
INFO HA: Monitoring standby codes. 2018-12-03T14: 34: 12,463 Portal "Machine1 Host Name" 216004 8472
INFO HA: Noden "Machine1 Host Name" is configured to be a master. 2018-12-03T14: 34: 12,462 Portal "Machine1 Host Name" 216003 8472
INFO HA: Monitoring standby codes.  2018-12-03T14: 33: 11,664 Portal "Machine1 Host Name" 216004 8472
INFO HA: Noden "Machine1 Host Name" is configured to be a master. 2018-12-03T14: 33: 11,664 Portal "Machine1 Host Name" 216003 8472
INFO HA: Monitoring standby codes. 2018-12-03T14: 32: 10,878 Portal "Machine1 Host Name" 216004 8472
INFO HA: Noden "Machine1 Host Name" is configured to be a master.  2018-12-03T14: 32: 10,878 Portal "Machine1 Host Name" 216003 8472
INFO Stops watchdog thread.  2018-12-03T14: 31: 28,778 Portal "Machine1 Host Name" 217066 8472
INFO Starting driving the watchdog thread.  2018-12-03T14: 31: 13,542 Portal "Machine1 Host Name" 217055 8472
INFO Starter process monitor.  2018-12-03T14: 31: 13,542 Portal "Machine1 Host Name" 217046 8472
DEBUG Have successfully resumed running the process monitor on the local machine.  2018-12-03T14: 31: 13,542 Portal Admin "Machine1 Host Name" 9999 4180
DEBUG Resumes running process-monitor plugin.  2018-12-03T14: 31: 13,542 Portal "Machine1 Host Name" 216057 8472
DEBUG Resumes running process monitor on the local machine.  2018-12-03T14: 31: 13,535 Portal Admin "Machine1 Host Name" 9999 4180
INFO HA: Monitoring standby codes.  2018-12-03T14: 31: 09,788 Portal "Machine1 Host Name" 216004 8472
INFO HA: Noden "Machine1 Host Name" is configured to be a master.  2018-12-03T14: 31: 09,788 Portal "Machine1 Host Name" 216003 8472
INFO HA: Starter HA monitor.  2018-12-03T14: 31: 09,107 Portal "Machine1 Host Name" 216002 8472
DEBUG Node Agent Plugin is set to not be paused.  2018-12-03T14: 31: 09,106 Portal "Machine1 Host Name" 216056 8472
DEBUG Can not retrieve information about current license users from the database. 2018-12-03T14: 30: 55,901 Portal Admin "Machine1 Host Name" 218049 4180

java.lang.RuntimeException: java.lang.RuntimeException: org.postgresql.util.PSQLException: Connection to localhost:7654 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at com.esri.arcgis.portal.admin.core.client.SharingAPIClient.a(SharingAPIClient.java:925) at com.esri.arcgis.portal.admin.core.client.SharingAPIClient.getUsersCountInLevel(SharingAPIClient.java:946) at com.esri.arcgis.portal.admin.core.system.LicenseUtil.a(LicenseUtil.java:487) at com.esri.arcgis.portal.admin.core.system.LicenseUtil.b(LicenseUtil.java:451) at com.esri.arcgis.portal.admin.core.system.LicenseUtil.c(LicenseUtil.java:312) at com.esri.arcgis.portal.admin.core.system.LicenseUtil.access$000(LicenseUtil.java:47) at com.esri.arcgis.portal.admin.core.system.LicenseUtil$a.run(LicenseUtil$a.java:73) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.postgresql.util.PSQLException: Connection to localhost:7654 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at com.esri.arcgis.portal.admin.core.client.SharingAPIClient.a(SharingAPIClient.java:2707) at com.esri.arcgis.portal.admin.core.client.SharingAPIClient.a(SharingAPIClient.java:910) ... 13 more Caused by: org.postgresql.util.PSQLException: Connection to localhost:7654 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:247) at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195) at org.postgresql.Driver.makeConnection(Driver.java:452) at org.postgresql.Driver.connect(Driver.java:254) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at com.esri.arcgis.portal.admin.core.client.SharingAPIClient.a(SharingAPIClient.java:2704) ... 14 more Caused by: java.net.ConnectException: Connection refused: connect at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.postgresql.core.PGStream.<init>(PGStream.java:69) at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:158) ... 21 more

 

 

We dont have any firewall between the machines, and both are virtual servers. We have set Hostname and HostIdentifier on both the machines after installation, before creating the site on Machine1.

 

Any help on solving this issue is highly appreciated.

Outcomes