I have federated Portal 10.5 with a single-machine ArcGIS Server 10.5 site. Portal is configured to use Active Directory for its Identity Store. Portal and ArcGIS Server are on different virtual servers and their respective web adaptors are on a separate Web Server and configured under different IIS sites. I have done ArcGIS Enterprise Basic deployment in the past successfully.
This table summarized the configuration
URL | Description |
---|---|
https://ags-web-dev.mydomain.org/server | Web Adaptor pointing to the ArcGIS Server |
https://ags-web.mydomain.org/portal | Web Adaptor pointing to the Portal for ArcGIS Site |
https://ags-dev.mydomain.org:6443/arcgis | Direct URL to the ArcGIS Server machine |
https://ags-portal.mydomain.org:7443/arcgis | Direct URL to the Portal for ArcGIS |
Almost everything appears to work correctly. For example
The only issue if that attempting to open ArcGIS Server Manager fail. The interface hangs on the "Please wait..." progress bar and the network traffic shows repeated failures to POST to the portal generateToken page via the ArcGIS Server proxy
The network inspector shows that ArcGIS Server returned a 500 server error caused by a "Connection closed" (see screenshot).
Are there any good ways to go about diagnosing the root cause of an issue like this?
Update
Installing Fiddler and enabling HTTPS traffic snooping shows that request is being sent from the ArcGIS Server machine to the Portal Web Adaptor.
This may be a side-effect of enabling Fiddler as a MITM proxy, but the Portal logs show this WARNING:
ArcGIS Server services URL 'https://ags-web-dev.mydomain.org/server' cannot be validated against 'https://ags-web-dev.mydomain.org/server/rest/info'. If the service URL is a proxy URL verify it is accessible to clients.
The JSON at the /info endpoint is
{ "currentVersion": 10.5, "fullVersion": "10.5.0", "soapUrl": "https://ags-web-dev.mydomain.org/server/services", "secureSoapUrl": null, "owningSystemUrl": "https://ags-web.mydomain.org/portal", "authInfo": { "isTokenBasedSecurity": true, "tokenServicesUrl": "https://ags-web.mydomain.org/portal/sharing/generateToken" } }
What information is being used to attempt this 'validation'?
Solved! Go to Solution.
A workaround!
The actual issue appears to be a lack of support for SNI in the ArcGIS Server proxy. Our web server was set up with multiple sites, each bound to a FQDN. On a hunch, I added a default HTTPS binding to the site that contains the Portal Web Adaptor and things began working.
At one point I did have a default binding on the site in order to configure the Web Adaptor (so I could use https://localhost/ to get at the configuration page), which is why it worked for a while and then "stopped". At the time, I didn't associate removing the default binding with the loading failures.
There is an Esri BUG-000093827 that discusses an issue with the Portal proxy and SNI support that was fixed in 10.4.1, but nothing I could find that discussed the state of SNI support in the ArcGIS Server proxy.
I'm hoping to get a resolution from Esri support that either
For now, moving on from this issue....
TLS Extension List sent from ArcGIS Server
extensions [extension_type: elliptic_curves,extension_type: ec_point_formats,extension_type: signature_algorithms]
TLS Extension List sent from Chrome Browser
extensions [extension_type: 31354,extension_type: renegotiation_info,extension_type: server_name,extension_type: extended_master_secret,extension_type: SessionTicket_TLS,extension_type: signature_algorithms,extension_type: status_request,extension_type: 18,extension_type: application_layer_protocol_negotiation,extension_type: 30032,extension_type: ec_point_formats,extension_type: elliptic_curves,extension_type: 10794]
Hm, so I think that when either Server or Portal use their respective proxies, the request is actually made from the software itself as the user running the software's service. I'm more certain this is the case for the Portal going through the sharing proxy, so I'm guessing that it's the same for Server. So, if we assume that's correct, what you can try to do is reach the Sharing API through the Portal web adaptor on the Server machine when you're logged in as the account running the ArcGIS Server Windows service. That will tell you if the URL it's proxying to can be reached by the Server as the account running the service. Another thing you can try is to sign into the Portal website and then bring up Server Manager, as that'll automatically sign you in with the cookies already present in the browser.
I'll try logging in as the ArcGIS Service account and see if that makes a difference.
Regarding logging into Portal first, that is actually how I discovered the issue. I had just Federated the server and clicked on the server link in the Portal Server setting page, which pointed to the server manager page.
I may not have verified that I was logged into Portal on subsequent tests, so I'll double-check that being logged into Portal does not change the behavior.
Thanks for the suggestions.
New information:
More information.
I’ve attached a network trace of the traffic captured between our ArcGIS Server and the Web Server that hosts the Web Adaptor for Portal. It appears that the Web Server (192.168.103.50) is sending a TCP Reset packet in response to the CLIENT HELLO send by the ArcGIS Server. This process repeats a few times and then, apparently, the ArcGIS Server side gives up.
We actually have three ArcGIS Servers set up right now (1 dev, 2 production) and the same error is happening in all cases, so it appears to be something systemic and specific to how the proxy page and/or tomcat/geronimo/Java is handling it's networking.
Might be time to write a test client in Java and see if that behaves the same as ArcGIS Sever....
Getting closer to a resolution. It appears that this is likely an IIS/SSL issue related to certificate negotiation. The IIS Web Server we are using has an ECC certificate installed (versus the more common RSA signed certificate) and that appears to be problematic.
We had an issue when installing ECC certificates on ArcGIS Server, as well. Chrome and other broswers reported that they could not negotiate a common cipher and the connection failed. We ended up using RSA certificates instead. The same type of issue appears to be in play here.
Some relevant stackexchange posts on this issue
https://serverfault.com/questions/774826/tls-1-2-client-hello-triggers-tcp-reset-from-2012-r2
Ah, ok, I have a basic understanding of certificates, not to the degree to know the difference between ECC and RSA. You may want to reach out to Technical Support so they can investigate whether there's a problem with a certain certificate type in the software.
I have had an open support case concurrent with this thread and have cross-referenced customer support to this topic as well. Hopefully the specific failure mechanism will be found soon...
Portal and server support ECDHE (Elliptic-curve Diffie Hellman) key exchanges. Is your listed here:
The original certificates that were installed on IIS were ECDHE_ECDSE which were not on the list. After discovering that discrepancy, IT installed new certificates that are ECDHE_RSA with P384.
My understanding is that the block cipher and message authentication are negotiated and, according to the Chrome Security toolbar, it can connect with AES_256_CBC_SHA1 (AES_256_CBC with HMAC_SHA1, specifically). That would appear to match the TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA entry in the list of supported ArcGIS Server ciphers, so our expectation was that ArcGIS Server would (at least) be able to use that cipher as well.
Unfortunately, the new certificate did not resolved the issue. So we are looking more closely at the details of the certificates and network traces to try and figure out where the TLS handshake breaks down.
I am certainly not a certificate/TLS/Networking expert by any means, so I may be misunderstanding the specifics of how this is supposed to work and any quirks of the Java networking stack when talking to Windows/IIS web servers.