Latest Contributions by DannyKrouk

Introduction This article addresses a basic use case like you might have in a pre-production environment.  All the ArcGIS Enterprise components (Portal, Server, and Data Store) exist on a single machine on an internal network.  The BIG-IP reverse proxy allows the system to be presented to another network (could be an internal client network or a public network) with all client traffic routed through the BIG-IP appliance.    This article is divided into three main sections.  The first section is intended for the ArcGIS Enterprise administrators and orienting them to their tasks in the language and terms they understand.  The second section is intended for the BIG-IP administrator, hopefully speaking to them in the language and terms that they understand.  The final section addresses additional options and details appropriate to both sets of administrators.  The goal of this article is to help ArcGIS and F5 administrators to work together by describing a proven set of procedures around which to collaborate.  This article does not seek to enable ArcGIS administrators to configure F5 themselves (or vice versa).  It is assumed that ArcGIS administrators are experts in ArcGIS and rely upon Esri documentation for details.  Similarly, it is assumed that F5 administrators are experts in BIG-IP and rely upon F5 documentation for details. While the use case is “basic”, the undertaking is not “simple”.  Success requires integrating technology and knowledge from several specialized domains (Esri technology, F5 technology, networking, certificates, etc.).  If the procedures in this article are not clear enough for action in your organizational context, it may be an indication that outside consulting support would be beneficial. This article is built on the following use case specifics: Use of F5’s BIG-IP as a “OSI Layer 7” or “full proxy” Use of ArcGIS Enterprise where the Web Adaptor is deployed with IIS on the Windows operating system ArcGIS Administrator Instructions The focus of the ArcGIS administrator is to deploy the ArcGIS Enterprise such that it is ready to be proxied by the F5 BIG-IP reverse proxy.  The deployment objective is depicted in the diagram below.   Depending on the specific requirements, it may not be necessary to deploy Web Adaptors when using a reverse proxy.  However, it is recommended that you do so for the following reasons: Using Web Adaptors reduces the configuration complexity in the BIG-IP reverse proxy. Web Adaptors allow you to validate the correctness of the ArcGIS Enterprise system independent of the reverse proxy; this can be very valuable in the case of troubleshooting. While Web Adaptors can be deployed on Linux-based systems using a Java Web Server, the frequency of deployment on Windows with IIS is why this initial guide focuses on that pattern. Step One: Design Decisions and Communication with F5 Administrators In this configuration, clients will address ArcGIS Enterprise through the reverse proxy.  The reverse proxy presents a CNAME (“DNS alias”, shown in green in the diagram below) which terminates the HTTPS sessions from the clients.  It then re-initiates new HTTPS sessions from itself to the web server that hosts the web adaptors.  The web server usually presents a certificate with the subject name of the A record (the host name, shown in red in the diagram) which terminates the incoming HTTPS sessions from the BIG-IP reverse proxy.   Before deploying your system, you need to make the following decisions: What is the CNAME (“DNS Alias”) under which the reverse proxy will represent the ArcGIS Enterprise system? What are the “contexts” (“Web Adaptor names”, shown in blue in the diagram) for the Portal for ArcGIS and ArcGIS Server Sites? You likely need to cooperate with your F5 administrators to agree upon the CNAME and ensure that they have a TLS certificate appropriate to terminate HTTPS communications on that CNAME.  At the same time, you can share with the F5 administrators the name (the A RECORD) of the machine on which your web server will operate and to which requests should be forwarded. Step Two: Deploy a Web Server with TLS Certificate It is useful to deploy your web server and configure it with a TLS certificate for HTTPS traffic before you do anything with ArcGIS Enterprise directly.  This allows you to make sure HTTPS is working with your web server, and it allows F5 administrators to configure the reverse proxy for the web server early in the process.  This allows early validation of the TLS certificate and HTTPS pathways. Configure HTTPS on Your Web Server Configuration steps to enable HTTPS with web servers vary by web server brand.  Since IIS is a very common web server, Esri has provided instructions for how to configure it for HTTPS as part of its Web Adaptor installation guide: https://enterprise.arcgis.com/en/web-adaptor/latest/install/iis/enable-https-on-your-web-server-server-.htm. When you enable HTTPS on your web server, you will need to provide a TLS certificate.  Among other attributes, certificates have Subjects and Subject Alternative Names (SAN’s).  The Subject of the certificate should match the name that the BIG-IP reverse proxy will use to address the web server.  This is often the A RECORD (machine1.domain.local in the diagram).  The SAN is a list of alternative names.  A good practice for SAN’s is to include: The Subject name (e.g., machine1.domain.local) The short version of the subject name (e.g., machine1) The CNAME that the reverse proxy will present (e.g., gis.domain.com), if possible Depending on your organization’s certificate authority and policies, you may or may not be able to put the CNAME in the SAN.  The benefit of doing so is that it allows you greater ability to validate the ArcGIS Enterprise independent of the reverse proxy. Validate Once your web server is configured for HTTPS, you should validate your configuration.  First, you should validate by navigating to your web server directly in a browser using the HTTPS protocol.  Second, if your F5 administrators have configured the BIG-IP reverse proxy to forward traffic to your web server, you can then navigate to the reverse proxy’s virtual server endpoint in a browser, also using HTTPS.   In each case, you want to confirm that you get no certificate warnings, and the target page draws correctly. Typically, web servers will have a default page which is returned when you go to the web server root.  That is a good option.  A better option is to deploy a web page that allows you to see more about what is going on.  The showHeaders pages (one for ASPX/IIS and one for JSP: https://github.com/dannykrouk/showHeaders) will echo back many useful details as shown below:   Consider deploying and using the showHeaders page to your web server and using it as the target for both your web server and reverse proxy tests.  If you deploy the page to the root of your web server, you can test with these requests: https://machine1.domain.local/showHeaders.aspx https://gis.domain.com/showHeaders.aspx Step Three: Deploy and Configure ArcGIS Enterprise  The ArcGIS Enterprise deployment here is a “single-machine base deployment” (https://enterprise.arcgis.com/en/get-started/latest/windows/base-arcgis-enterprise-deployment.htm#ESRI_SECTION1_690F8D4A3ABE4FB8AE926C118E9F8299).  Install and Configure the Basics Documentation to take you through the installation steps is available on Esri’s web site: https://enterprise.arcgis.com/en/documentation/install/. In this article, the Portal for ArcGIS’s Web Adaptor name (“context”) is “/portal” and the Hosting Server’s Web Adaptor name is “/server”.  When you federate your Hosting Server Site with your Portal for ArcGIS, you may use the Web Adaptor for both the Services URL and the Administration URL (https://enterprise.arcgis.com/en/portal/latest/administer/windows/configure-servers.htm), so long as you are not using Web Tier authentication.  If you are using Web Tier authentication, your Administration URL should follow this pattern:  https://machine1.domain.local:6443/arcgis. Validate the Basics; "System Confidence Check" Once you have your Web Adaptors configured and your Hosting Server federated, you should perform a brief “system confidence check” of the system to make sure the core functions work correctly.  A typical system confidence check would involve: Login to /portaladmin Validate Federation Publish a service Share the service The “confidence” that one establishes with these steps is that the deployed system does not have any fundamental configuration flaw that would prevent basic operation. Whatever you create (e.g. a published service) in the system confidence check should be deleted before you continue. Final Configuration for the Reverse Proxy The final configuration step is to configure the WebContextURL for the Portal and the Hosting Server.  This property is how each Esri server knows the name by which clients will address it through the BIG-IP reverse proxy. Portal for ArcGIS: https://enterprise.arcgis.com/en/portal/latest/administer/windows/using-a-reverse-proxy-server-with-portal-for-arcgis.htm#ESRI_SECTION1_7C753FB1F19349A398E5FFCC6079A821 {    "WebContextURL": "https://gis.domain.com/portal" } ArcGIS for Server: https://enterprise.arcgis.com/en/server/latest/deploy/linux/using-a-reverse-proxy-server-with-arcgis-server.htm#ESRI_SECTION1_13680C9069E14B1F8AE5793BE1ED25A6 {    "WebContextURL": "https://gis.domain.com/server" } Step Four: System Validation The fourth and final step for the ArcGIS administrator is “system validation”, independent of the BIG-IP reverse proxy.  If you validate this way, and something doesn’t work when requests flow through the reverse proxy, then the reverse proxy configuration is what needs attention.  If you do not validate this way, it can be much more complicated to establish the source of the problem. Temporarily Change Name Resolution on the Web Server Machine The trick to this validation is to temporarily convince the system that the CNAME (gis.domain.com) is at the ArcGIS Enterprise machine (machine1.domain.local).  If you have local administrator permissions on the machine1.domain.local machine, you can edit the hosts file.  If machine1.domain.local has the IP address 10.0.0.10, then the host file entry would look like this: 10.0.0.10        gis.domain.com This says, “gis.domain.com can be found at 10.0.0.10”. When you complete your edit to this file (which is usually found here: C:\Windows\System32\drivers\etc\hosts), open a command prompt and confirm that your setting is effective with ping: C:\>ping gis.domain.com Pinging gis.domain.com [10.0.0.10] with 32 bytes of data: Reply from <IP Address of machine1>: bytes=32 time=22ms TTL=124 … The result from the ping command should be the IP address in your hosts file, the placeholder value highlighted above for clarity. Test in a Browser on the Web Server Machine Now, open a web browser on machine1.domain.local and run your “system confidence check” again, this time addressing the system by its CNAME (https://gis.domain.com/portal/home/).   If your system functions correctly this way, back-out the hosts file change and ask the F5 administrators to complete their configuration work. F5 Administrator Instructions From a BIG-IP perspective, this is a simple configuration.  The ArcGIS Enterprise system has several components.  But, from the standpoint of the reverse proxy, there is a single back-end web server node listening on HTTPS/443.  The one configuration element that may be different to other systems is that ArcGIS Enterprise requires the reverse proxy to include an X-Forwarded-Host header. At a high level, your proxy configuration will be terminating all HTTPS traffic to the gis.domain.com CNAME and re-initiating HTTPS to the single back-end node, machine1.domain.local.  The web server at this address supports two “contexts”, one for each major component of the ArcGIS Enterprise system (Portal for ArcGIS and ArcGIS Server): /portal and /server.   As described in the introduction of this article, we assume that you have three networks associated with your BIG-IP proxy, a client network, a server network, and your administration network. This article assumes you are an expert in BIG-IP administration and only need cues to the steps to configure this virtual server and backend pool in the optimal order. Step One: Proxy the Web Server The CNAME for this system should have been established with you or communicated to you.  A matching TLS certificate for that CNAME should also be provided.  The certificate authority must be one that is trusted by the clients of this system. Install the TLS Certificate Install the TLS certificate for the Virtual Server in BIG-IP (e.g., Subject gis.domain.com) System > Certificate Management > Traffic Certificate Management > SSL Certificate List > Import SSL Certificate Create Client SSL Profile Create a client profile to terminate client TLS connections on the certificate for the gis.domain.com name. Local Traffic > Profiles > SSL > Client > Create   Create a Pool with a Simple Monitor Create a backend Pool for the web server (e.g. https://machine1.domain.local/ ) with a Simple Monitor to determine whether the node resource is “up” or “down”. Local Traffic > Pools > Pool List > Create   Create an HTTP Services Profile (Add the X-Forwarded-Host Header) The X-Forwarded-Host header allows ArcGIS Enterprise to know the value of the Host header that arrived at BIG-IP.  The ArcGIS Enterprise system will check this value against its configuration to ensure that clients have addressed it in an appropriate way.  If the X-Forwarded-Host header is not present, or contains a wrong value, ArcGIS Enterprise will issue an HTTP redirect to signal how it believes it should be addressed.  This can result in redirect loops.  In the case that ArcGIS Enterprise detects a redirect loop, it will break it and return an error. An X-Forwarded-Host header can be included with an iRule: when HTTP_REQUEST {         HTTP::header insert X-Forwarded-Host [HTTP::host] }   This directive takes the value of the arriving request’s Host header and sets it as the X-Forwarded-Host header value to the default pool. Create the Virtual Server Local Traffic > Virtual Servers > Create Select “Standard” for Type of Virtual Server.  Specify your SSL Profile (Client) you created earlier with your SSL certificate.  In specifying a Server profile, the configuration objective is to achieve a TLS tunnel to the backend.   This can be accomplished with the default “serverssl” setting within BIG-IP.  Set “Source Address Translation” to Auto Map.  On the Resources tab, select your pool that you created before.   Step Two: Validate Proxying Through to ArcGIS Enterprise There are two steps to validate the effectiveness of this configuration.  Validate Trust and Headers Assuming the showHeaders.aspx page was deployed on the backend web server, use a web browser to navigate to https://gis.domain.com/showHeaders.aspx.  The browser should be free of certificate trust warnings.  The page response body should prove-out the effectiveness of the X-Forwarded-Host header aspect of your configuration.     At this point, with the header flow through to the backend confirmed, there is no longer a need for the showHeaders.aspx tool.  If your system is intended for PRODUCTION, or any environment where the information exposes should not be exposed, the tool should be removed. Validate Effectiveness with ArcGIS Enterprise In a web browser, navigate to https://gis.domain.com/portal/home/ and click the “Sign in” link.  If you see a page in which credentials can be provided, this validates the effectiveness of the X-Forwarded-Host header relative to the ArcGIS Enterprise configuration.  If this test fails, the most likely cause is a disagreement between the value of the header and the corresponding WebContextURL configuration in ArcGIS Enterprise. Optional validation steps include: Confirm that the second ArcGIS Enterprise server role (ArcGIS Server) is accessible: https://gis.domain.com/server/rest/services. Determine the behavior when an unsupported context is used such as https://gis.domain.com/not-a-real-context/.  In this case, we expect an HTTP 404 code from the web server in the pool. At this point, the configuration of the system is complete, and it may be passed on for acceptance testing. Final Thoughts Having completed the instructions thus far, you should have a fully functional ArcGIS Enterprise system accessible via F5’s BIG-IP reverse proxy.  Congratulations.  This article offers the following closing thoughts. Testing These instructions stipulated “system confidence checks” and validations throughout the deployment/configuration process.  These measures were meant to determine whether it was reasonable to continue to the next step in the instructions.  These measures do not prove that the system is acceptable.  By the end, we assume that the entire system functions.  In other words, these tests establish that the system has functional coherence. Passing the system on for acceptance testing is the appropriate next step.  The goal of acceptance testing is to measure the deployed system against the business goals it must support. Health Checking The Simple Monitoring included in the instructions to the F5 administrators allows the BIG-IP to stop forwarding requests to the ArcGIS Enterprise system if the node (i.e., machine1.domain.local) is down or inaccessible.  For a system like this, with only one back-end Pool member, this (a Simple Monitor) is the full extent of the recommended health checking.  In principle, BIG-IP can be configured with out-of-band health checks that ask the ArcGIS Enterprise software to confirm its health or validate that specific HTTPS requests succeed with particular response payloads.  For example, the Portal for ArcGIS component of ArcGIS Enterprise exposes this endpoint: https://developers.arcgis.com/rest/enterprise-administration/portal/health-check-portal.htm.  In a single node system, the response to a health check problem essentially eliminates all access to the system.  The upside of this is that the reverse proxy can provide a generic error to clients that the system is down.  The downside of this is that a temporary problem, partial problem, or misinterpretation through the health check eliminates all access to the system when it might be still serviceable for many use cases.  The probability of false negatives (unhealthy findings) associated with formal/complex health checks may result in less system reliability to end users relative to the false positives (healthy findings) associated with the simple node monitoring.  In other words, for a single-node system complexity is the enemy of reliability; stick to a Simple Monitor. Credits This article was produced based on the work of Roger Schlogel, a consultant with Esri’s Professional Services.     ... View more