I try to use Hadoop big data with ArcGIS Enterprise and GeoAnalytics (10.5.1). The environment is set to use Kerberos authentication to access the database. Is there anybody who can provide me any information how to connect to Hadoop with GeoAnalytics Server using Kerberos authentication?
Thanks for your help
Zakaria
Hi Zakaria,
GeoAnalytics Server does not currently support registering a secured HDFS as a Big Data File Share, including HDFS configured with Kerberos authentication.
Noah Slocum
GeoAnalytics Product Engineer
Hi Zakaria Douiri, I'm happy to say that we've added support for connecting to an HDFS file configured with Kerberos authentication with ArcGIS Enterprise 10.6.1. The documentation here explains how to register a Big Data File Share of that type.
Noah Slocum
GeoAnalytics Product Engineer
Hello all.
We are trying to connect a geoanalytics server to a hadoop server secured with Kerberos.
We went through the documentation steps to register the big data file share, we are able to register the data item, but it fails to create the manifest. We think there is a problem with the key tab location string. Or some other problem reading the keytab file. The arcgis service account has proper permisions in the keytab location.
When creating the data item we see in the ArcGIS Server logs some errors "Unexpected end of string near .... " the string of the keytab location. Why is that? The location string is right. We use \ as scape character.
Another question we have, since this is the first time we are trying to connect using kerberos. Should we install a kerberos client in the server? Or the client is embeded in the software?
Any help is appreciated
Thanks in advance
Hi Sarah.
Sorry for the delay.
We finnally ended up with a no out of the box solution for connecting to hive with kerberos authentication.
Hi Javier Distefano,
To clarify:
- Sarah
Hi all,
I installed a new environment, this time with ArcGIS Enterprise and GeoAnalytics 10.6.1.
I registered the big data file share as mentioned in the documentation, but it fails to create the manifest.
I received the following error in the log of ArcGIS Server (Geoanalytics box) :
Error executing tool. GenerateManifest Job ID: j600903669f994003bae83f7bc1c895fc : {"messageCode":"BD_101030","message":"Failed to execute GenerateManifest. Please contact your administrator with job ID 'j600903669f994003bae83f7bc1c895fc'.","params":{"toolName":"GenerateManifest","jobID":"j600903669f994003bae83f7bc1c895fc"}} Failed to execute (GenerateManifest).
I used the following json to register the big data file share :
{
"path": "/bigDataFileShares/test",
"type": "bigDataFileShare",
"info":
{
"connectionString": "{\"path\":\"hdfs://<server>:8020/user/hive/test\",\"accessMode\":\"Kerberos\",\"principal\":\"<user>\",\"keytab\":\"/var/toto.keytab\"}",
"connectionType": "hdfs"
}
}
Any help is well appreciated.
Thanks,
Zakaria
Hi Zakaria Douiri,
Can you please turn on debug on your GeoAnalytics Server and rerun the tool. The debug messages will be more informative about the issue.
Please double check that your keytab is up to date, and available to all machines in the GeoAnalytics Server site.
Thanks,
Sarah
Product Engineer, GeoAnalytics Server
Hi Sarah Ambrose,
The keytab file is copied in geoanalytics server. Also arcgis user has read/write access to that file (see snapshot).
Below the log file after turning on Debug level.
Thanks for your help,
Zakaria
<Msg time="2019-05-10T12:14:24,115" type="INFO" code="7426" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Generating manifest for data store /bigDataFileShares/fidapparcgis.</Msg>
<Msg time="2019-05-10T12:14:24,148" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Creating Request Parameters : {datasources=*, f=json}</Msg>
<Msg time="2019-05-10T12:14:24,148" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Sending Rest Handler Services Request : https://SERVER.COMPANY.COM:6443/arcgis/admin/local/clearRestCache</Msg>
<Msg time="2019-05-10T12:14:24,271" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Creating Request Parameters : {datasources=*, f=json}</Msg>
<Msg time="2019-05-10T12:14:24,271" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Sending Rest Handler Services Request : https://SERVER.COMPANY.COM:6443/arcgis/rest/admin/cache/clear</Msg>
<Msg time="2019-05-10T12:14:24,334" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">ARCGIS_PORTAL_TOKEN Authentication, Unable to validate token com.esri.arcgis.discovery.admin.security.AGSSecurityException: Server machine 'https://nbd.privateportal.COMPANY.COM:7443/arcgis/sharing/rest/community/self' returned an error. 'Invalid token.'</Msg>
<Msg time="2019-05-10T12:14:24,334" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request host: SERVER.COMPANY.COM</Msg>
<Msg time="2019-05-10T12:14:24,334" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request address: 100.64.22.6</Msg>
<Msg time="2019-05-10T12:14:24,336" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Referer: 4b8e8824-1659-4989-a5d8-e13f0bf424de</Msg>
<Msg time="2019-05-10T12:14:24,338" type="DEBUG" code="8521" source="Server" process="28230" thread="23" methodName="WebSecurityManager.authenticate" machine="SERVER.COMPANY.COM" user="ef773fd4-6171-4321-8134-2ffe232e76a9" elapsed="">HTTP Referer: 4b8e8824-1659-4989-a5d8-e13f0bf424de</Msg>
<Msg time="2019-05-10T12:14:24,338" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for Global:IsOnline</Msg>
<Msg time="2019-05-10T12:14:24,338" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for Global:IsOnline</Msg>
<Msg time="2019-05-10T12:14:24,363" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for folder:System</Msg>
<Msg time="2019-05-10T12:14:24,363" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for folder:System</Msg>
<Msg time="2019-05-10T12:14:24,398" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: folder:System</Msg>
<Msg time="2019-05-10T12:14:24,398" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: folder:System</Msg>
<Msg time="2019-05-10T12:14:24,411" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">ARCGIS_PORTAL_TOKEN Authentication, Unable to validate token com.esri.arcgis.discovery.admin.security.AGSSecurityException: Server machine 'https://nbd.privateportal.COMPANY.COM:7443/arcgis/sharing/rest/community/self' returned an error. 'Invalid token.'</Msg>
<Msg time="2019-05-10T12:14:24,411" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request host: SERVER.COMPANY.COM</Msg>
<Msg time="2019-05-10T12:14:24,411" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request address: 100.64.22.6</Msg>
<Msg time="2019-05-10T12:14:24,414" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Referer: 4b8e8824-1659-4989-a5d8-e13f0bf424de</Msg>
<Msg time="2019-05-10T12:14:24,415" type="DEBUG" code="8521" source="Server" process="28230" thread="23" methodName="WebSecurityManager.authenticate" machine="SERVER.COMPANY.COM" user="ef773fd4-6171-4321-8134-2ffe232e76a9" elapsed="">HTTP Referer: 4b8e8824-1659-4989-a5d8-e13f0bf424de</Msg>
<Msg time="2019-05-10T12:14:24,491" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: Global:IsOnline</Msg>
<Msg time="2019-05-10T12:14:24,491" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: Global:IsOnline</Msg>
<Msg time="2019-05-10T12:14:24,491" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for IncludeOrgIdWIthUsername</Msg>
<Msg time="2019-05-10T12:14:24,491" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for IncludeOrgIdWIthUsername</Msg>
<Msg time="2019-05-10T12:14:24,627" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: IncludeOrgIdWIthUsername</Msg>
<Msg time="2019-05-10T12:14:24,627" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: IncludeOrgIdWIthUsername</Msg>
<Msg time="2019-05-10T12:14:24,627" type="INFO" code="9029" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="ef773fd4-6171-4321-8134-2ffe232e76a9" elapsed="">Request user: ef773fd4-6171-4321-8134-2ffe232e76a9, Service: null/null</Msg>
<Msg time="2019-05-10T12:14:24,628" type="INFO" code="9029" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="ef773fd4-6171-4321-8134-2ffe232e76a9" elapsed="">Request user: ef773fd4-6171-4321-8134-2ffe232e76a9, Service: System/GeoAnalyticsManagement/GPServer</Msg>
<Msg time="2019-05-10T12:14:24,631" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In clearDatasources</Msg>
<Msg time="2019-05-10T12:14:24,631" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">SDSDatasourceManager clear datasource cache</Msg>
<Msg time="2019-05-10T12:14:24,631" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In clearDatasources</Msg>
<Msg time="2019-05-10T12:14:24,631" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Time (ms): 313 (/arcgis/rest/admin/cache/clear)</Msg>
<Msg time="2019-05-10T12:14:24,631" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Time (ms): 313 (/arcgis/rest/admin/cache/clear)</Msg>
<Msg time="2019-05-10T12:14:24,636" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for gpserver:System/GeoAnalyticsManagement</Msg>
<Msg time="2019-05-10T12:14:24,636" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for gpserver:System/GeoAnalyticsManagement</Msg>
<Msg time="2019-05-10T12:14:24,643" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for gpserver:System/GeoAnalyticsManagement/Service</Msg>
<Msg time="2019-05-10T12:14:24,643" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Populating cache for gpserver:System/GeoAnalyticsManagement/Service</Msg>
<Msg time="2019-05-10T12:14:24,647" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: gpserver:System/GeoAnalyticsManagement/Service</Msg>
<Msg time="2019-05-10T12:14:24,647" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: gpserver:System/GeoAnalyticsManagement/Service</Msg>
<Msg time="2019-05-10T12:14:24,647" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">RediscoveryAppServer handleRequest called. FQSN: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:24,647" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectProxy: 1</Msg>
<Msg time="2019-05-10T12:14:24,647" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectSync: 1</Msg>
<Msg time="2019-05-10T12:14:24,649" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request type = - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:24,649" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Delegating request to GPServerObjectProxy - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:24,649" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">handleRequest() in com.esri.arcgis.discovery.admin.rediscovery.service.ArcSOCGISService called for service: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:24,649" type="FINE" code="7615" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Wait time for 'System.GeoAnalyticsManagement.GPServer' service request is 2 milliseconds.</Msg>
<Msg time="2019-05-10T12:14:24,655" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="6981" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Set UserInfo to "" for Anonymous request.</Msg>
<Msg time="2019-05-10T12:14:24,658" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="6981" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Updating log level from WARNING to DEBUG</Msg>
<Msg time="2019-05-10T12:14:24,742" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: gpserver:System/GeoAnalyticsManagement</Msg>
<Msg time="2019-05-10T12:14:24,742" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Cache PUT: gpserver:System/GeoAnalyticsManagement</Msg>
<Msg time="2019-05-10T12:14:24,741" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Server object processing time for service request: System.GeoAnalyticsManagement.GPServer is 92 ms.</Msg>
<Msg time="2019-05-10T12:14:24,742" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">RediscoveryAppServer handleRequest called. FQSN: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:24,742" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectProxy: 1</Msg>
<Msg time="2019-05-10T12:14:24,742" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectSync: 1</Msg>
<Msg time="2019-05-10T12:14:24,742" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request type = submitJob - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:24,742" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">This is a SUBMIT_JOB request. Request is pre-processed by GPServer - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:24,742" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">handleRequest() in com.esri.arcgis.discovery.admin.rediscovery.service.ArcSOCGISService called for service: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:24,743" type="FINE" code="7615" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Wait time for 'System.GeoAnalyticsManagement.GPServer' service request is 1 milliseconds.</Msg>
<Msg time="2019-05-10T12:14:25,55" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Created new job ID 'jc3e728469c8a44b883459d3bdf220035' for request - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:25,58" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request properties before renewal: {"token":"ZW5dQQpb_cpqubcpPTAEbdNMB799ZfCGwjaKxBQCYEf0oYSLIyMMA0OgAs06qxrL9QvGadI9pRSseOYm2uTI_Kz0Xcgx0xyJR4Ff1IBK7VNGTJNEqV8XkkrlFtAgg4Ea","referer":"4b8e8824-1659-4989-a5d8-e13f0bf424de","privilege":"ADMINISTER","securityProvider":"portal"}</Msg>
<Msg time="2019-05-10T12:14:25,58" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request properties after renewal: {"token":"ZW5dQQpb_cpqubcpPTAEbdNMB799ZfCGwjaKxBQCYEf0oYSLIyMMA0OgAs06qxrL9QvGadI9pRSseOYm2uTI_Kz0Xcgx0xyJR4Ff1IBK7VNGTJNEqV8XkkrlFtAgg4Ea","referer":"4b8e8824-1659-4989-a5d8-e13f0bf424de","privilege":"ADMINISTER","securityProvider":"portal"}</Msg>
<Msg time="2019-05-10T12:14:25,74" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Submitting job with ID 'jc3e728469c8a44b883459d3bdf220035' - System/GeoAnalyticsManagement.GPServer. File: /mnt/config/geoanalyticsserver/directories/arcgisjobs/system/geoanalyticsmanagement_gpserver/.submitted/jc3e728469c8a44b883459d3bdf220035. Request properties: {"RequestTarget":"SERVER_OBJECT","RequestContextURL":"https://SERVER.COMPANY.COM:6443/arcgis","requestProperties":{"token":"ZW5dQQpb_cpqubcpPTAEbdNMB799ZfCGwjaKxBQCYEf0oYSLIyMMA0OgAs06qxrL9QvGadI9pRSseOYm2uTI_Kz0Xcgx0xyJR4Ff1IBK7VNGTJNEqV8XkkrlFtAgg4Ea","referer":"4b8e8824-1659-4989-a5d8-e13f0bf424de","privilege":"ADMINISTER","securityProvider":"portal"},"RequestUser":"ef773fd4-6171-4321-8134-2ffe232e76a9","RequestUserRoles":""}</Msg>
<Msg time="2019-05-10T12:14:25,75" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Time (ms): 712 (/arcgis/rest/services/System/GeoAnalyticsManagement/GPServer/GenerateManifest/submitJob)</Msg>
<Msg time="2019-05-10T12:14:25,75" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Time (ms): 712 (/arcgis/rest/services/System/GeoAnalyticsManagement/GPServer/GenerateManifest/submitJob)</Msg>
<Msg time="2019-05-10T12:14:25,74" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Server object processing time for service request: System.GeoAnalyticsManagement.GPServer is 332 ms.</Msg>
<Msg time="2019-05-10T12:14:25,278" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">ARCGIS_PORTAL_TOKEN Authentication, Unable to validate token com.esri.arcgis.discovery.admin.security.AGSSecurityException: Server machine 'https://nbd.privateportal.COMPANY.COM:7443/arcgis/sharing/rest/community/self' returned an error. 'Invalid token.'</Msg>
<Msg time="2019-05-10T12:14:25,278" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request host: SERVER.COMPANY.COM</Msg>
<Msg time="2019-05-10T12:14:25,278" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request address: 100.64.22.6</Msg>
<Msg time="2019-05-10T12:14:25,279" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Referer: 4b8e8824-1659-4989-a5d8-e13f0bf424de</Msg>
<Msg time="2019-05-10T12:14:25,286" type="DEBUG" code="8521" source="Server" process="28230" thread="23" methodName="WebSecurityManager.authenticate" machine="SERVER.COMPANY.COM" user="ef773fd4-6171-4321-8134-2ffe232e76a9" elapsed="">HTTP Referer: 4b8e8824-1659-4989-a5d8-e13f0bf424de</Msg>
<Msg time="2019-05-10T12:14:25,286" type="INFO" code="9029" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="ef773fd4-6171-4321-8134-2ffe232e76a9" elapsed="">Request user: ef773fd4-6171-4321-8134-2ffe232e76a9, Service: System/GeoAnalyticsManagement/GPServer</Msg>
<Msg time="2019-05-10T12:14:25,291" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">RediscoveryAppServer handleRequest called. FQSN: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:25,291" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectProxy: 1</Msg>
<Msg time="2019-05-10T12:14:25,291" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectSync: 1</Msg>
<Msg time="2019-05-10T12:14:25,291" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request type = - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:25,291" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Delegating request to GPServerObjectProxy - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:25,291" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">handleRequest() in com.esri.arcgis.discovery.admin.rediscovery.service.ArcSOCGISService called for service: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:25,291" type="FINE" code="7615" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Wait time for 'System.GeoAnalyticsManagement.GPServer' service request is 0 milliseconds.</Msg>
<Msg time="2019-05-10T12:14:25,325" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Time (ms): 60 (/arcgis/rest/services/System/GeoAnalyticsManagement/GPServer/GenerateManifest/jobs/jc3e728469c8a44b883459d3bdf220035)</Msg>
<Msg time="2019-05-10T12:14:25,325" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Time (ms): 60 (/arcgis/rest/services/System/GeoAnalyticsManagement/GPServer/GenerateManifest/jobs/jc3e728469c8a44b883459d3bdf220035)</Msg>
<Msg time="2019-05-10T12:14:25,325" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Server object processing time for service request: System.GeoAnalyticsManagement.GPServer is 34 ms.</Msg>
<Msg time="2019-05-10T12:14:25,691" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Checking the state of other ZooKeeper servers in the cluster.</Msg>
<Msg time="2019-05-10T12:14:25,691" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Checking ZooKeeper connections for the cluster....</Msg>
<Msg time="2019-05-10T12:14:25,693" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Adding new client for SERVER.COMPANY.COM:2181</Msg>
<Msg time="2019-05-10T12:14:25,701" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">My current config:
server.1=SERVER.COMPANY.COM:2182:2190:participant;0.0.0.0:2181
version=100000000</Msg>
<Msg time="2019-05-10T12:14:25,720" type="VERBOSE" code="7597" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Executing geoprocessing job 'jc3e728469c8a44b883459d3bdf220035'.</Msg>
<Msg time="2019-05-10T12:14:25,720" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">RediscoveryAppServer handleRequest called. FQSN: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:25,720" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectProxy: 1</Msg>
<Msg time="2019-05-10T12:14:25,720" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectSync: 1</Msg>
<Msg time="2019-05-10T12:14:25,720" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">handleRequest() in com.esri.arcgis.discovery.admin.rediscovery.service.ArcSOCGISService called for service: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:25,733" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Updating log level from WARNING to DEBUG</Msg>
<Msg time="2019-05-10T12:14:25,734" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Setting job information in server environment.</Msg>
<Msg time="2019-05-10T12:14:25,823" type="DEBUG" code="9999" source="Admin" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Releasing client 100.64.22.6:2181</Msg>
<Msg time="2019-05-10T12:14:30,601" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">ARCGIS_PORTAL_TOKEN Authentication, Unable to validate token com.esri.arcgis.discovery.admin.security.AGSSecurityException: Server machine 'https://nbd.privateportal.COMPANY.COM:7443/arcgis/sharing/rest/community/self' returned an error. 'Invalid token.'</Msg>
<Msg time="2019-05-10T12:14:30,601" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request host: SERVER.COMPANY.COM</Msg>
<Msg time="2019-05-10T12:14:30,601" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request address: 100.64.22.6</Msg>
<Msg time="2019-05-10T12:14:30,603" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Referer: 4b8e8824-1659-4989-a5d8-e13f0bf424de</Msg>
<Msg time="2019-05-10T12:14:30,605" type="INFO" code="9029" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="ef773fd4-6171-4321-8134-2ffe232e76a9" elapsed="">Request user: ef773fd4-6171-4321-8134-2ffe232e76a9, Service: System/GeoAnalyticsManagement/GPServer</Msg>
<Msg time="2019-05-10T12:14:30,605" type="DEBUG" code="8521" source="Server" process="28230" thread="23" methodName="WebSecurityManager.authenticate" machine="SERVER.COMPANY.COM" user="ef773fd4-6171-4321-8134-2ffe232e76a9" elapsed="">HTTP Referer: 4b8e8824-1659-4989-a5d8-e13f0bf424de</Msg>
<Msg time="2019-05-10T12:14:30,610" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">RediscoveryAppServer handleRequest called. FQSN: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:30,610" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectProxy: 1</Msg>
<Msg time="2019-05-10T12:14:30,610" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">In processServiceRequest in GPServiceActor. Current router size for GPServerObjectSync: 1</Msg>
<Msg time="2019-05-10T12:14:30,610" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Request type = - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:30,610" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Delegating request to GPServerObjectProxy - System/GeoAnalyticsManagement.GPServer.</Msg>
<Msg time="2019-05-10T12:14:30,610" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">handleRequest() in com.esri.arcgis.discovery.admin.rediscovery.service.ArcSOCGISService called for service: System.GeoAnalyticsManagement.GPServer</Msg>
<Msg time="2019-05-10T12:14:30,610" type="FINE" code="7615" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Wait time for 'System.GeoAnalyticsManagement.GPServer' service request is 0 milliseconds.</Msg>
<Msg time="2019-05-10T12:14:30,683" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Time (ms): 107 (/arcgis/rest/services/System/GeoAnalyticsManagement/GPServer/GenerateManifest/jobs/jc3e728469c8a44b883459d3bdf220035)</Msg>
<Msg time="2019-05-10T12:14:30,683" type="DEBUG" code="9999" source="Server" process="28230" thread="23" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Server object processing time for service request: System.GeoAnalyticsManagement.GPServer is 73 ms.</Msg>
<Msg time="2019-05-10T12:14:30,683" type="DEBUG" code="9999" source="Rest" process="28230" thread="15" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Time (ms): 107 (/arcgis/rest/services/System/GeoAnalyticsManagement/GPServer/GenerateManifest/jobs/jc3e728469c8a44b883459d3bdf220035)</Msg>
<Msg time="2019-05-10T12:14:34,672" type="VERBOSE" code="7423" source="Admin" process="7365" thread="26" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Server license version does not match effective current version.</Msg>
<Msg time="2019-05-10T12:14:35,202" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Current Service Environment: GPServiceEnvironment(jc3e728469c8a44b883459d3bdf220035,/mnt/config/geoanalyticsserver/directories/arcgisjobs/system/geoanalyticsmanagement_gpserver/jc3e728469c8a44b883459d3bdf220035,Some(GPServiceRequest(Some(ZW5dQQpb_cpqubcpPTAEbdNMB799ZfCGwjaKxBQCYEf0oYSLIyMMA0OgAs06qxrL9QvGadI9pRSseOYm2uTI_Kz0Xcgx0xyJR4Ff1IBK7VNGTJNEqV8XkkrlFtAgg4Ea),None,Some(4b8e8824-1659-4989-a5d8-e13f0bf424de))),System/GeoAnalyticsManagement,GPServer,Map(jobsVirtualDirectory -> /rest/directories/arcgisjobs, jobsDirectory -> /mnt/config/geoanalyticsserver/directories/arcgisjobs, virtualOutputDir -> /rest/directories/arcgisoutput, showMessages -> Info, outputDir -> /mnt/config/geoanalyticsserver/directories/arcgisoutput, javaHeapSize -> 512, maximumRecords -> 1000, toolbox -> ${AGSSERVER}/ArcToolBox/Services/GeoAnalytics Management.tbx, executionType -> Asynchronous))</Msg>
<Msg time="2019-05-10T12:14:35,265" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Running on SERVER.COMPANY.COM</Msg>
<Msg time="2019-05-10T12:14:35,454" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Creating local GAContext for process</Msg>
<Msg time="2019-05-10T12:14:36,518" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Initialized Spark 2.2.0</Msg>
<Msg time="2019-05-10T12:14:36,519" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Attaching progress listener to SparkContext</Msg>
<Msg time="2019-05-10T12:14:36,546" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Using status file '/mnt/config/geoanalyticsserver/directories/arcgisjobs/system/geoanalyticsmanagement_gpserver/jc3e728469c8a44b883459d3bdf220035/status.dat' to track cancel</Msg>
<Msg time="2019-05-10T12:14:36,566" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Executing function 'GenerateManifest'</Msg>
<Msg time="2019-05-10T12:14:36,612" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Validating function inputs</Msg>
<Msg time="2019-05-10T12:14:36,613" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Generating manifest (overwrite=false)</Msg>
<Msg time="2019-05-10T12:14:36,855" type="SEVERE" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">REST call to 'https://nbd.privateportal.COMPANY.COM:7443/arcgis/sharing/portals/self/servers' failed (reason=Server returned error: [Invalid token.])</Msg>
<Msg time="2019-05-10T12:14:36,856" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">REST call to 'https://nbd.privateportal.COMPANY.COM:7443/arcgis/sharing/portals/self/servers' failed (reason=Server returned error: [Invalid token.]) Server returned error: [Invalid token.]</Msg>
<Msg time="2019-05-10T12:14:36,856" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">com.esri.arcgis.gae.ags.RestUtil$ServerException: Server returned error: [Invalid token.]
at com.esri.arcgis.gae.ags.RestUtil$.assertResponseSuccess(RestUtil.scala:227)
at com.esri.arcgis.gae.ags.RestUtil$ResponseHandler$jsonResponseHandler$.handleResponse(RestUtil.scala:33)
at com.esri.arcgis.gae.ags.RestUtil$ResponseHandler$jsonResponseHandler$.handleResponse(RestUtil.scala:30)
at com.esri.arcgis.gae.ags.RestUtil$.call(RestUtil.scala:142)
at com.esri.arcgis.gae.ags.RestUtil$.post(RestUtil.scala:56)
at com.esri.arcgis.gae.ags.PortalClient.post(PortalClient.scala:245)
at com.esri.arcgis.gae.ags.PortalClient.getServerInfos(PortalClient.scala:239)
at com.esri.arcgis.gae.ags.GenericWebGISContext.x$9$lzycompute(WebGISContext.scala:141)
at com.esri.arcgis.gae.ags.GenericWebGISContext.x$9(WebGISContext.scala:140)
at com.esri.arcgis.gae.ags.GenericWebGISContext.self$lzycompute(WebGISContext.scala:140)
at com.esri.arcgis.gae.ags.GenericWebGISContext.self(WebGISContext.scala:140)
at com.esri.arcgis.gae.gp.fn.FnGenerateManifest$.generateManifest(FnGenerateManifest.scala:66)
at com.esri.arcgis.gae.gp.fn.FnGenerateManifest$.execute(FnGenerateManifest.scala:60)
at com.esri.arcgis.gae.gp.fn.GAFunction$class.executeFunction(GAFunction.scala:27)
at com.esri.arcgis.gae.gp.fn.FnGenerateManifest$.executeFunction(FnGenerateManifest.scala:20)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(GPToGAFunctionAdapter.scala:128)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1$$anonfun$apply$mcV$sp$1.apply(GPToGAFunctionAdapter.scala:128)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1$$anonfun$apply$mcV$sp$1.apply(GPToGAFunctionAdapter.scala:128)
at com.esri.arcgis.st.util.HasScope$Companion$class.withScope(HasScope.scala:93)
at com.esri.arcgis.gae.gp.JobEnvironment$.withScope(JobEnvironment.scala:40)
at com.esri.arcgis.st.util.HasScope.withScope(HasScope.scala:32)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1.apply$mcV$sp(GPToGAFunctionAdapter.scala:127)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1.apply(GPToGAFunctionAdapter.scala:59)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1.apply(GPToGAFunctionAdapter.scala:59)
at com.esri.arcgis.st.util.HasScope$.withScope(HasScope.scala:48)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter.execute(GPToGAFunctionAdapter.scala:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.esri.arcgis.interop.NativeObjRef.nativeVtblInvokeNative(Native Method)
at com.esri.arcgis.interop.NativeObjRef.nativeVtblInvoke(Unknown Source)
at com.esri.arcgis.interop.NativeObjRef.invoke(Unknown Source)
at com.esri.arcgis.interop.Dispatch.vtblInvoke(Unknown Source)
at com.esri.arcgis.system.IRequestHandlerProxy.handleStringRequest(Unknown Source)
at com.esri.arcgis.discovery.servicelib.impl.SOThread.a(SOThread.java:955)
at com.esri.arcgis.discovery.servicelib.impl.SOThread.a(SOThread.java:719)
at com.esri.arcgis.discovery.servicelib.impl.SOThread.run(SOThread.java:472)
</Msg>
<Msg time="2019-05-10T12:14:36,937" type="SEVERE" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Job 'jc3e728469c8a44b883459d3bdf220035' for tool 'GenerateManifest' failed: Server returned error: [Invalid token.]. </Msg>
<Msg time="2019-05-10T12:14:36,937" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">Job 'jc3e728469c8a44b883459d3bdf220035' for tool 'GenerateManifest' failed: Server returned error: [Invalid token.]. Server returned error: [Invalid token.]</Msg>
<Msg time="2019-05-10T12:14:36,937" type="DEBUG" code="9999" source="System/GeoAnalyticsManagement.GPServer" process="7365" thread="20" methodName="" machine="SERVER.COMPANY.COM" user="" elapsed="">com.esri.arcgis.gae.ags.RestUtil$ServerException: Server returned error: [Invalid token.]
at com.esri.arcgis.gae.ags.RestUtil$.assertResponseSuccess(RestUtil.scala:227)
at com.esri.arcgis.gae.ags.RestUtil$ResponseHandler$jsonResponseHandler$.handleResponse(RestUtil.scala:33)
at com.esri.arcgis.gae.ags.RestUtil$ResponseHandler$jsonResponseHandler$.handleResponse(RestUtil.scala:30)
at com.esri.arcgis.gae.ags.RestUtil$.call(RestUtil.scala:142)
at com.esri.arcgis.gae.ags.RestUtil$.post(RestUtil.scala:56)
at com.esri.arcgis.gae.ags.PortalClient.post(PortalClient.scala:245)
at com.esri.arcgis.gae.ags.PortalClient.getServerInfos(PortalClient.scala:239)
at com.esri.arcgis.gae.ags.GenericWebGISContext.x$9$lzycompute(WebGISContext.scala:141)
at com.esri.arcgis.gae.ags.GenericWebGISContext.x$9(WebGISContext.scala:140)
at com.esri.arcgis.gae.ags.GenericWebGISContext.self$lzycompute(WebGISContext.scala:140)
at com.esri.arcgis.gae.ags.GenericWebGISContext.self(WebGISContext.scala:140)
at com.esri.arcgis.gae.gp.fn.FnGenerateManifest$.generateManifest(FnGenerateManifest.scala:66)
at com.esri.arcgis.gae.gp.fn.FnGenerateManifest$.execute(FnGenerateManifest.scala:60)
at com.esri.arcgis.gae.gp.fn.GAFunction$class.executeFunction(GAFunction.scala:27)
at com.esri.arcgis.gae.gp.fn.FnGenerateManifest$.executeFunction(FnGenerateManifest.scala:20)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(GPToGAFunctionAdapter.scala:128)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1$$anonfun$apply$mcV$sp$1.apply(GPToGAFunctionAdapter.scala:128)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1$$anonfun$apply$mcV$sp$1.apply(GPToGAFunctionAdapter.scala:128)
at com.esri.arcgis.st.util.HasScope$Companion$class.withScope(HasScope.scala:93)
at com.esri.arcgis.gae.gp.JobEnvironment$.withScope(JobEnvironment.scala:40)
at com.esri.arcgis.st.util.HasScope.withScope(HasScope.scala:32)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1.apply$mcV$sp(GPToGAFunctionAdapter.scala:127)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1.apply(GPToGAFunctionAdapter.scala:59)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter$$anonfun$execute$1.apply(GPToGAFunctionAdapter.scala:59)
at com.esri.arcgis.st.util.HasScope$.withScope(HasScope.scala:48)
at com.esri.arcgis.gae.gp.GPToGAFunctionAdapter.execute(GPToGAFunctionAdapter.scala:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.esri.arcgis.interop.NativeObjRef.nativeVtblInvokeNative(Native Method)
at com.esri.arcgis.interop.NativeObjRef.nativeVtblInvoke(Unknown Source)
at com.esri.arcgis.interop.NativeObjRef.invoke(Unknown Source)
at com.esri.arcgis.interop.Dispatch.vtblInvoke(Unknown Source)
at com.esri.arcgis.system.IRequestHandlerProxy.handleStringRequest(Unknown Source)
at com.esri.arcgis.discovery.servicelib.impl.SOThread.a(SOThread.java:955)
at com.esri.arcgis.discovery.servicelib.impl.SOThread.a(SOThread.java:719)
at com.esri.arcgis.discovery.servicelib.impl.SOThread.run(SOThread.java:472)
</Msg>