AnsweredAssumed Answered

Upgrade 10.8.1 issues

Question asked by Nick.xu@vancouver.ca on Sep 25, 2020

we have one load balancer, 2 portals, 2 servers, 2 data stores and all portal content and server directories are on a shared file server. Recently, we updated our sandbox from 10.6.1 to 10.8.1, before and after upgradation, we verified federation and data store, server status  and all works.

 

however, we noticed that when I open a rest service folder, sometimes, it directs to the login page. the server log file shows severe errors:

Failed to return all services configurations in the folder 'Locator'. Could not connect to the ArcGIS component at URL 'https://sandbox.ourdomain.ca:7443/arcgis/sharing/rest/search'. The ArcGIS component on that machine may not be running or the machine may not be reachable at this time.Error:

 

I checked the server covgis33 log file and the error:

<Msg time="2020-09-25T21:21:36,62" type="SEVERE" code="6605" source="Admin" process="17392" thread="1" methodName="" machine="COVGIS33.DS.CITY.ourCity.BC.CA" user="" elapsed="" requestID="0fce8b36-e385-472f-942a-de447ab2eb09">Failed to return all services configurations in the folder 'Locator'. Could not connect to the ArcGIS component at URL 'https://sandbox.ourDomain.ca:7443/arcgis/sharing/rest/search'. The ArcGIS component on that machine may not be running or the machine may not be reachable at this time.Error: </Msg>
<Msg time="2020-09-25T21:21:36,63" type="DEBUG" code="9999" source="Rest" process="17392" thread="1" methodName="" machine="COVGIS33.DS.CITY.ourCity.BC.CA" user="" elapsed="" requestID="0fce8b36-e385-472f-942a-de447ab2eb09">Time (ms): 28 (/arcgis/rest/services/Locator)</Msg>
<Msg time="2020-09-25T21:21:36,62" type="DEBUG" code="9999" source="Server" process="17392" thread="26" methodName="" machine="COVGIS33.DS.CITY.ourCity.BC.CA" user="" elapsed="" requestID="0fce8b36-e385-472f-942a-de447ab2eb09">Failed to return all services configurations in the folder 'Locator'. Could not connect to the ArcGIS component at URL 'https://sandbox.ourdomain.ca:7443/arcgis/sharing/rest/search'. The ArcGIS component on that machine may not be running or the machine may not be reachable at this time.Error: </Msg>
<Msg time="2020-09-25T21:21:36,62" type="DEBUG" code="9999" source="Admin" process="17392" thread="1" methodName="" machine="COVGIS33.DS.CITY.ourcity.BC.CA" user="" elapsed="" requestID="0fce8b36-e385-472f-942a-de447ab2eb09">java.io.IOException: Could not connect to the ArcGIS component at URL 'https://sandbox.ourdomain.ca:7443/arcgis/sharing/rest/search'. The ArcGIS component on that machine may not be running or the machine may not be reachable at this time.Error:
at com.esri.arcgis.discovery.admin.util.HttpUtil.handleConnectionIOException(HttpUtil.java:721)
at com.esri.arcgis.discovery.admin.util.PooledHttpClientUtil.postToURL(PooledHttpClientUtil.java:253)
at com.esri.arcgis.discovery.admin.util.PooledHttpClientUtil.postPropsToURL(PooledHttpClientUtil.java:289)
at com.esri.arcgis.discovery.admin.security.portal.ArcGISPortal.search(ArcGISPortal.java:2075)
at com.esri.arcgis.discovery.admin.security.portal.ArcGISPortal.checkAccessOnItems(ArcGISPortal.java:2009)
at com.esri.arcgis.discovery.admin.services.ServiceManager.getAllServiceCfgsInFederatedMode(ServiceManager.java:3071)
at com.esri.arcgis.discovery.security.manager.WebSecurityManager.showFolder(WebSecurityManager.java:572)
at com.esri.discovery.app.security.SecurityPreHandler.authorizeServices(SecurityPreHandler.java:432)
at com.esri.discovery.app.security.SecurityPreHandler.servicesSecurity(SecurityPreHandler.java:154)
at com.esri.discovery.app.security.SecurityPreHandler.preHandle(SecurityPreHandler.java:85)
at com.esri.rf.RServlet.service(RServlet.java:113)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:666)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRun(Nio2Endpoint.java:1684)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at org.apache.tomcat.util.net.AbstractEndpoint.processSocket(AbstractEndpoint.java:1104)
at org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper$2.completed(Nio2Endpoint.java:599)
at org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper$2.completed(Nio2Endpoint.java:577)
at org.apache.tomcat.util.net.SecureNio2Channel$1.completed(SecureNio2Channel.java:969)
at org.apache.tomcat.util.net.SecureNio2Channel$1.completed(SecureNio2Channel.java:898)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Unknown Source)
at java.base/sun.nio.ch.Invoker$2.run(Unknown Source)
at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Unknown Source)
</Msg>

 

I have created a esri ticket: Esri Case #02645437

 

Thanks!

 

Nick

 

Outcomes