Using F5’s BIG-IP to provide access to a “Base Deployment” of ArcGIS Enterprise

1728
11
11-17-2023 05:08 PM
DannyKrouk
Esri Contributor
10 11 1,728

Introduction

This article addresses a basic use case like you might have in a pre-production environment.  All the ArcGIS Enterprise components (Portal, Server, and Data Store) exist on a single machine on an internal network.  The BIG-IP reverse proxy allows the system to be presented to another network (could be an internal client network or a public network) with all client traffic routed through the BIG-IP appliance. 

config_objective.png

 

This article is divided into three main sections.  The first section is intended for the ArcGIS Enterprise administrators and orienting them to their tasks in the language and terms they understand.  The second section is intended for the BIG-IP administrator, hopefully speaking to them in the language and terms that they understand.  The final section addresses additional options and details appropriate to both sets of administrators. 

The goal of this article is to help ArcGIS and F5 administrators to work together by describing a proven set of procedures around which to collaborate.  This article does not seek to enable ArcGIS administrators to configure F5 themselves (or vice versa).  It is assumed that ArcGIS administrators are experts in ArcGIS and rely upon Esri documentation for details.  Similarly, it is assumed that F5 administrators are experts in BIG-IP and rely upon F5 documentation for details.

While the use case is “basic”, the undertaking is not “simple”.  Success requires integrating technology and knowledge from several specialized domains (Esri technology, F5 technology, networking, certificates, etc.).  If the procedures in this article are not clear enough for action in your organizational context, it may be an indication that outside consulting support would be beneficial.

This article is built on the following use case specifics:

  • Use of F5’s BIG-IP as a “OSI Layer 7” or “full proxy”
  • Use of ArcGIS Enterprise where the Web Adaptor is deployed with IIS on the Windows operating system

ArcGIS Administrator Instructions

The focus of the ArcGIS administrator is to deploy the ArcGIS Enterprise such that it is ready to be proxied by the F5 BIG-IP reverse proxy.  The deployment objective is depicted in the diagram below.

deployment_pattern.png

 

Depending on the specific requirements, it may not be necessary to deploy Web Adaptors when using a reverse proxy.  However, it is recommended that you do so for the following reasons:

  1. Using Web Adaptors reduces the configuration complexity in the BIG-IP reverse proxy.
  2. Web Adaptors allow you to validate the correctness of the ArcGIS Enterprise system independent of the reverse proxy; this can be very valuable in the case of troubleshooting.
  3. While Web Adaptors can be deployed on Linux-based systems using a Java Web Server, the frequency of deployment on Windows with IIS is why this initial guide focuses on that pattern.

Step One: Design Decisions and Communication with F5 Administrators

In this configuration, clients will address ArcGIS Enterprise through the reverse proxy.  The reverse proxy presents a CNAME (“DNS alias”, shown in green in the diagram below) which terminates the HTTPS sessions from the clients.  It then re-initiates new HTTPS sessions from itself to the web server that hosts the web adaptors.  The web server usually presents a certificate with the subject name of the A record (the host name, shown in red in the diagram) which terminates the incoming HTTPS sessions from the BIG-IP reverse proxy.

design.png

 

Before deploying your system, you need to make the following decisions:

  1. What is the CNAME (“DNS Alias”) under which the reverse proxy will represent the ArcGIS Enterprise system?
  2. What are the “contexts” (“Web Adaptor names”, shown in blue in the diagram) for the Portal for ArcGIS and ArcGIS Server Sites?

You likely need to cooperate with your F5 administrators to agree upon the CNAME and ensure that they have a TLS certificate appropriate to terminate HTTPS communications on that CNAME.  At the same time, you can share with the F5 administrators the name (the A RECORD) of the machine on which your web server will operate and to which requests should be forwarded.

Step Two: Deploy a Web Server with TLS Certificate

It is useful to deploy your web server and configure it with a TLS certificate for HTTPS traffic before you do anything with ArcGIS Enterprise directly.  This allows you to make sure HTTPS is working with your web server, and it allows F5 administrators to configure the reverse proxy for the web server early in the process.  This allows early validation of the TLS certificate and HTTPS pathways.

Configure HTTPS on Your Web Server

Configuration steps to enable HTTPS with web servers vary by web server brand.  Since IIS is a very common web server, Esri has provided instructions for how to configure it for HTTPS as part of its Web Adaptor installation guide: https://enterprise.arcgis.com/en/web-adaptor/latest/install/iis/enable-https-on-your-web-server-serv....

When you enable HTTPS on your web server, you will need to provide a TLS certificate.  Among other attributes, certificates have Subjects and Subject Alternative Names (SAN’s).  The Subject of the certificate should match the name that the BIG-IP reverse proxy will use to address the web server.  This is often the A RECORD (machine1.domain.local in the diagram).  The SAN is a list of alternative names.  A good practice for SAN’s is to include:

  1. The Subject name (e.g., machine1.domain.local)
  2. The short version of the subject name (e.g., machine1)
  3. The CNAME that the reverse proxy will present (e.g., gis.domain.com), if possible

Depending on your organization’s certificate authority and policies, you may or may not be able to put the CNAME in the SAN.  The benefit of doing so is that it allows you greater ability to validate the ArcGIS Enterprise independent of the reverse proxy.

Validate

Once your web server is configured for HTTPS, you should validate your configuration.  First, you should validate by navigating to your web server directly in a browser using the HTTPS protocol.  Second, if your F5 administrators have configured the BIG-IP reverse proxy to forward traffic to your web server, you can then navigate to the reverse proxy’s virtual server endpoint in a browser, also using HTTPS.   In each case, you want to confirm that you get no certificate warnings, and the target page draws correctly.

Typically, web servers will have a default page which is returned when you go to the web server root.  That is a good option.  A better option is to deploy a web page that allows you to see more about what is going on.  The showHeaders pages (one for ASPX/IIS and one for JSP: https://github.com/dannykrouk/showHeaders) will echo back many useful details as shown below:

showHeaders.png

 

Consider deploying and using the showHeaders page to your web server and using it as the target for both your web server and reverse proxy tests.  If you deploy the page to the root of your web server, you can test with these requests:

Step Three: Deploy and Configure ArcGIS Enterprise 

The ArcGIS Enterprise deployment here is a “single-machine base deployment” (https://enterprise.arcgis.com/en/get-started/latest/windows/base-arcgis-enterprise-deployment.htm#ES...). 

Install and Configure the Basics

Documentation to take you through the installation steps is available on Esri’s web site: https://enterprise.arcgis.com/en/documentation/install/.

In this article, the Portal for ArcGIS’s Web Adaptor name (“context”) is “/portal” and the Hosting Server’s Web Adaptor name is “/server”.  When you federate your Hosting Server Site with your Portal for ArcGIS, you may use the Web Adaptor for both the Services URL and the Administration URL (https://enterprise.arcgis.com/en/portal/latest/administer/windows/configure-servers.htm), so long as you are not using Web Tier authentication.  If you are using Web Tier authentication, your Administration URL should follow this pattern:  https://machine1.domain.local:6443/arcgis.

Validate the Basics; "System Confidence Check"

Once you have your Web Adaptors configured and your Hosting Server federated, you should perform a brief “system confidence check” of the system to make sure the core functions work correctly. 

A typical system confidence check would involve:

  1. Login to /portaladmin
  2. Validate Federation
  3. Publish a service
  4. Share the service

The “confidence” that one establishes with these steps is that the deployed system does not have any fundamental configuration flaw that would prevent basic operation.

Whatever you create (e.g. a published service) in the system confidence check should be deleted before you continue.

Final Configuration for the Reverse Proxy

The final configuration step is to configure the WebContextURL for the Portal and the Hosting Server.  This property is how each Esri server knows the name by which clients will address it through the BIG-IP reverse proxy.

Portal for ArcGIS: https://enterprise.arcgis.com/en/portal/latest/administer/windows/using-a-reverse-proxy-server-with-...

{

   "WebContextURL": "https://gis.domain.com/portal"

}

ArcGIS for Server: https://enterprise.arcgis.com/en/server/latest/deploy/linux/using-a-reverse-proxy-server-with-arcgis...

{

   "WebContextURL": "https://gis.domain.com/server"

}

Step Four: System Validation

The fourth and final step for the ArcGIS administrator is “system validation”, independent of the BIG-IP reverse proxy.  If you validate this way, and something doesn’t work when requests flow through the reverse proxy, then the reverse proxy configuration is what needs attention.  If you do not validate this way, it can be much more complicated to establish the source of the problem.

Temporarily Change Name Resolution on the Web Server Machine

The trick to this validation is to temporarily convince the system that the CNAME (gis.domain.com) is at the ArcGIS Enterprise machine (machine1.domain.local).  If you have local administrator permissions on the machine1.domain.local machine, you can edit the hosts file.  If machine1.domain.local has the IP address 10.0.0.10, then the host file entry would look like this:

10.0.0.10        gis.domain.com

This says, “gis.domain.com can be found at 10.0.0.10”.

When you complete your edit to this file (which is usually found here: C:\Windows\System32\drivers\etc\hosts), open a command prompt and confirm that your setting is effective with ping:

C:\>ping gis.domain.com

Pinging gis.domain.com [10.0.0.10] with 32 bytes of data:

Reply from <IP Address of machine1>: bytes=32 time=22ms TTL=124


The result from the ping command should be the IP address in your hosts file, the placeholder value highlighted above for clarity.

Test in a Browser on the Web Server Machine

Now, open a web browser on machine1.domain.local and run your “system confidence check” again, this time addressing the system by its CNAME (https://gis.domain.com/portal/home/).  

If your system functions correctly this way, back-out the hosts file change and ask the F5 administrators to complete their configuration work.

F5 Administrator Instructions

From a BIG-IP perspective, this is a simple configuration.  The ArcGIS Enterprise system has several components.  But, from the standpoint of the reverse proxy, there is a single back-end web server node listening on HTTPS/443.  The one configuration element that may be different to other systems is that ArcGIS Enterprise requires the reverse proxy to include an X-Forwarded-Host header.

At a high level, your proxy configuration will be terminating all HTTPS traffic to the gis.domain.com CNAME and re-initiating HTTPS to the single back-end node, machine1.domain.local.  The web server at this address supports two “contexts”, one for each major component of the ArcGIS Enterprise system (Portal for ArcGIS and ArcGIS Server): /portal and /server.

design.png

 

As described in the introduction of this article, we assume that you have three networks associated with your BIG-IP proxy, a client network, a server network, and your administration network.

This article assumes you are an expert in BIG-IP administration and only need cues to the steps to configure this virtual server and backend pool in the optimal order.

Step One: Proxy the Web Server

The CNAME for this system should have been established with you or communicated to you.  A matching TLS certificate for that CNAME should also be provided.  The certificate authority must be one that is trusted by the clients of this system.

Install the TLS Certificate

Install the TLS certificate for the Virtual Server in BIG-IP (e.g., Subject gis.domain.com)

System > Certificate Management > Traffic Certificate Management > SSL Certificate List > Import SSL Certificate

f5_tls.png

Create Client SSL Profile

Create a client profile to terminate client TLS connections on the certificate for the gis.domain.com name.

Local Traffic > Profiles > SSL > Client > Create

f5_client_profile.png

 

Create a Pool with a Simple Monitor

Create a backend Pool for the web server (e.g. https://machine1.domain.local/ ) with a Simple Monitor to determine whether the node resource is “up” or “down”.

Local Traffic > Pools > Pool List > Create

f5_pool.png

 

Create an HTTP Services Profile (Add the X-Forwarded-Host Header)

The X-Forwarded-Host header allows ArcGIS Enterprise to know the value of the Host header that arrived at BIG-IP.  The ArcGIS Enterprise system will check this value against its configuration to ensure that clients have addressed it in an appropriate way.  If the X-Forwarded-Host header is not present, or contains a wrong value, ArcGIS Enterprise will issue an HTTP redirect to signal how it believes it should be addressed.  This can result in redirect loops.  In the case that ArcGIS Enterprise detects a redirect loop, it will break it and return an error.

An X-Forwarded-Host header can be included with an iRule:

when HTTP_REQUEST {

        HTTP::header insert X-Forwarded-Host [HTTP::host]

}

 

This directive takes the value of the arriving request’s Host header and sets it as the X-Forwarded-Host header value to the default pool.

Create the Virtual Server

Local Traffic > Virtual Servers > Create

Select “Standard” for Type of Virtual Server.  Specify your SSL Profile (Client) you created earlier with your SSL certificate.  In specifying a Server profile, the configuration objective is to achieve a TLS tunnel to the backend.   This can be accomplished with the default “serverssl” setting within BIG-IP.  Set “Source Address Translation” to Auto Map.

f5_virtual_server.png

 On the Resources tab, select your pool that you created before.

f5_virtual_server_to_pool_mapping.png

 

Step Two: Validate Proxying Through to ArcGIS Enterprise

There are two steps to validate the effectiveness of this configuration. 

Validate Trust and Headers

Assuming the showHeaders.aspx page was deployed on the backend web server, use a web browser to navigate to https://gis.domain.com/showHeaders.aspx.  The browser should be free of certificate trust warnings.  The page response body should prove-out the effectiveness of the X-Forwarded-Host header aspect of your configuration.

 validate_headers.png

 

At this point, with the header flow through to the backend confirmed, there is no longer a need for the showHeaders.aspx tool.  If your system is intended for PRODUCTION, or any environment where the information exposes should not be exposed, the tool should be removed.

Validate Effectiveness with ArcGIS Enterprise

In a web browser, navigate to https://gis.domain.com/portal/home/ and click the “Sign in” link.  If you see a page in which credentials can be provided, this validates the effectiveness of the X-Forwarded-Host header relative to the ArcGIS Enterprise configuration.  If this test fails, the most likely cause is a disagreement between the value of the header and the corresponding WebContextURL configuration in ArcGIS Enterprise.

Optional validation steps include:

  1. Confirm that the second ArcGIS Enterprise server role (ArcGIS Server) is accessible: https://gis.domain.com/server/rest/services.
  2. Determine the behavior when an unsupported context is used such as https://gis.domain.com/not-a-real-context/.  In this case, we expect an HTTP 404 code from the web server in the pool.

At this point, the configuration of the system is complete, and it may be passed on for acceptance testing.

Final Thoughts

Having completed the instructions thus far, you should have a fully functional ArcGIS Enterprise system accessible via F5’s BIG-IP reverse proxy.  Congratulations. 

This article offers the following closing thoughts.

Testing

These instructions stipulated “system confidence checks” and validations throughout the deployment/configuration process.  These measures were meant to determine whether it was reasonable to continue to the next step in the instructions.  These measures do not prove that the system is acceptable.  By the end, we assume that the entire system functions.  In other words, these tests establish that the system has functional coherence.

Passing the system on for acceptance testing is the appropriate next step.  The goal of acceptance testing is to measure the deployed system against the business goals it must support.

Health Checking

The Simple Monitoring included in the instructions to the F5 administrators allows the BIG-IP to stop forwarding requests to the ArcGIS Enterprise system if the node (i.e., machine1.domain.local) is down or inaccessible.  For a system like this, with only one back-end Pool member, this (a Simple Monitor) is the full extent of the recommended health checking. 

In principle, BIG-IP can be configured with out-of-band health checks that ask the ArcGIS Enterprise software to confirm its health or validate that specific HTTPS requests succeed with particular response payloads.  For example, the Portal for ArcGIS component of ArcGIS Enterprise exposes this endpoint: https://developers.arcgis.com/rest/enterprise-administration/portal/health-check-portal.htm.  In a single node system, the response to a health check problem essentially eliminates all access to the system.  The upside of this is that the reverse proxy can provide a generic error to clients that the system is down.  The downside of this is that a temporary problem, partial problem, or misinterpretation through the health check eliminates all access to the system when it might be still serviceable for many use cases.  The probability of false negatives (unhealthy findings) associated with formal/complex health checks may result in less system reliability to end users relative to the false positives (healthy findings) associated with the simple node monitoring.  In other words, for a single-node system complexity is the enemy of reliability; stick to a Simple Monitor.

Credits

This article was produced based on the work of Roger Schlogel, a consultant with Esri’s Professional Services.

 

 

11 Comments
AndreaB_
Occasional Contributor

Fantastic, thank you!!

MatejVrtich
Esri Contributor

@DannyKrouk thanks for the great explanation.

Would it be possible to describe the configuration of F5 BIG-IP without the use of ArcGIS Web Adaptors?

Thanks,

Matej

DannyKrouk
Esri Contributor

@MatejVrtich Thank you for the appreciation.

I don't have plans to write such an article.  Much of the work would be the same.  The main difference is that you have to re-write the 3xx series response location headers whenever they contain the back-end machine name/native port combination.  This is only needed for the front-side channel, not the admin pathways.  This document notes the need to do so (but leaves the implementation up to the reverse proxy admin ... because reverse proxies all have different ways of doing such things): https://enterprise.arcgis.com/en/portal/latest/administer/windows/using-a-reverse-proxy-server-with-...

AndreaB_
Occasional Contributor

Hi again @DannyKrouk ,

I just read this https://enterprise.arcgis.com/en/portal/11.1/administer/windows/using-a-reverse-proxy-server-with-po... 

AndreaB__0-1703103808084.png

Does this contradict your blog post? I'm confused on this one. 

Thanks!

DannyKrouk
Esri Contributor

Hello @AndreaB_  - Thanks for the question. 

I am not sure how this statement in the doc, "Portal for ArcGIS does not support SSL offloading through a reverse proxy server/load balancer. Therefore, if your configuration uses a reverse proxy server, it must forward traffic to either the ArcGIS Web Adaptor or directly to Portal for ArcGIS over HTTPS" is in conflict with this blog post.

The blog post describes how to forward traffic "to the ArcGIS Web Adaptor" (really to the web server hosting it).  So, that seems in alignment.

The bit about "does not support SSL offloading" is, perhaps, confusing because "SSL offloading" can mean different things.  But, both F5 and Wikipedia agree with the sense that I think Esri means in its documentation.   "Offloading" does not mean "terminate and re-initiate TLS sessions".  Rather, "offloading" means "terminate TLS sessions and leave the back-end traffic unencrypted".  This post does not describe an "SSL offloading" configuration; it describes a "terminate and re-initiate" configuration ... which is what Esri supports.

Am I missing something?

AndreaB_
Occasional Contributor

Hi @DannyKrouk

Thank you for the extra information. I'm sure it's not you that's missing something...I'm sure it's me! 🙂 This isn't my strong area so all of the extra information and explanation I can gather the better.  Yes, you answered the parts that were confusing me. I'm finding some of these IT terms are used pretty loosely, interchangeably, or generally, and that's not helping. If you know of any other videos or training in this area I would greatly take that advice also. I come from the Esri side so I'm missing some of the parts of the IT side that would help with this. I'm starting to wrap my head around it though and it's been fun to learn a new side of the Enterprise set-up.

Thanks!

DannyKrouk
Esri Contributor

@AndreaB_  I hope the additional explanation is helpful to you and others.  I suspect you are not the only person with a question like this.  Regarding other resources, I confess that I've not reviewed the content myself, but this is a big new offering from Esri that seems like it would have information in this space: https://architecture.arcgis.com/en/introducing-the-arcgis-architecture-center.html?rsource=https%3A%... (The Well Architected Framework).

AndreaB_
Occasional Contributor

Thank you so much for that link, @DannyKrouk. The intended audience for the ArcGIS Architecture Center website,  https://architecture.arcgis.com/en/overview/well-architected-framework/mission-and-intended-audience... , states as below:

"While the primary audience of this site is intended to be IT professionals, architects, and support teams, GIS professionals will also benefit from the topics covered here, as they increasingly are asked to learn and understand IT principles and concepts as part of their work in modern enterprise GIS systems."

I feel seen, thank you! Happy Holidays 🙂

TimWillson
New Contributor II

Not sure why my post was marked as spam. I'll try again.

@DannyKrouk, this is well detailed article that will be very helpful to our deployment. One question, how does the directions differ if Portal and Server are on separate servers? Thanks for your insight!

DannyKrouk
Esri Contributor

@TimWillson "It depends"? 😉

If the Portal and Server Web Adaptors are not on separate servers, then nothing about the load balancing approach would be different ... because BIG-IP is balancing the web servers, not the Esri software servers themselves.

If the Web Adaptors are on separate servers, there are many design alternatives.  Maybe you are presenting them on a single VIP and using path-based routing to land on separate backend pools.  Maybe you are exposing two VIPs, each routed to its separate backend pool.

I think, for there to be a specific answer, there needs to be a more detailed specific design.  And, that level of detail (in question and in answer) is a bit outside the intent of this article.  That is, this article is focused on foundational education such that you can work out the details with your team of F5 and GIS admin experts.

TimWillson
New Contributor II

@DannyKrouk  No worries. I wasn't sure if it was simple tweak. 👍 I will be giving this a shot. Thanks for your reply!