In my previous article, Architecture is a Funny Word, I introduced Esri’s Global Architecture Team. I talked a bit about the work we do (and don’t do) and how we can support you in your efforts to design, build, and implement a GIS that makes a real difference at your organization. You might want to check that blog out first, since it sets up some of the context for this one. However, if you think context is dull and overrated, then by all means carry on, dear reader.
While everyone uses both sides of their brain, it is said that ‘right-brained’ people tend to be creative, imaginative and innovative thinkers. They like to employ creative concepts to understand complex ideas, are often focused more on the ‘big picture’ rather than specific details. ‘Left-brained’ people are said to be detail-oriented, analytical, and logical thinkers. They like to approach problems methodically and with concrete evidence and facts, and know that people don’t trip over mountains, but over molehills.
It seems to me that GIS practitioners are a diverse mix of right and left-brainers. Which makes sense, since GIS is as much about analyzing data and identifying patterns as it is about communicating concepts and ideas with beautiful, compelling visuals. So, I figured it could be valuable (and fun) to try and explain architecture in a way that would appeal to a larger group of thinkers.
The Right Brain For my right-brained readers, I offer you an architecture analogy: Architecture is often compared to the architecture of the physical world of buildings, bridges, or even city planning. This analogy illustrates some key facets of the discipline- describing ‘what should be’ in terms of design, and how it should be adjusted going forward. However, this analogy only describes some of what architects do, and sometimes the ‘how’ and ‘why’ are more interesting, especially to you right-brainers, than the ‘what’. So why don’t we take a new analogy for a spin?
The fundamental idea of “yin-yang” is that seemingly opposite or contrary forces are actually complementary and interrelated. If you’ve ever sat in a room with technologists and business people trying to solve a problem together, you might see how those 'seemingly opposite forces' could represent business and technology. Technologists often feel separated from the business objectives of their organization, and business people sometimes see technology as a frustrating, separate field from their work. Of course, these two practices are completely interrelated and dependent on each other: the business dictates the use of technology, and technology guides and enables what’s possible for the business to achieve.
One of the key tenets of an architect’s duty is to make this relationship strong and clear by aligning the capabilities of technology to the needs of the business. In other words, we strive to highlight and strengthen the interconnectedness of these sometimes seemingly opposing forces. We do this by starting with a vision everyone can agree on and collectively work toward. Almost as importantly, we also need to make that alignment easier to see and understand, especially for leadership and other key stakeholders. This means communicating creatively, often with visuals, to explain complex ideas. Anyone who has had to act as the bridge between business and technology knows that bridge’s foundation relies more heavily on the art of building relationships than the science of technicalities.
The Left Brain
Alright, left-brained readers, it’s your time to shine. Let’s get serious about the methodical, concrete work architects on the Global Architecture team can do with you. Our work revolves around three key questions:
Where is your organization today?
Where do you need to be tomorrow?
What’s the best path to get there?
Notice how I didn’t say:
What does your GIS do today?
What do you want to do with GIS tomorrow?
Here’s what you need to buy or install to do it
Rather, we take a business-first approach to ensure that your investment in GIS is directly impacting outcomes for your organization and its stakeholders. We start by understanding the business challenges impeding your organization’s goals, and design new or augmented business workflows that resolve those challenges. Then, we identify what technology components, information, data, and skills are needed to enable those workflows. This process draws clear connections between technology solutions and business outcomes for your executive sponsors (or potential executive sponsors).
This work can’t be done in isolation. An architect documents perspectives from both the business and technology side before defining how to meet their needs in an efficient, sustainable and adaptable way. This means that architects speak the languages of both technologists and business managers. One way we do this is with diagrams. Kind of how GIS practitioners communicate with maps.
While technical architects communicate with system diagrams, my team and I communicate with conceptual and logical architecture diagrams. These might take the shape of a matrix illustrating where technology is not being effectively leveraged in your organization, a flow diagram outlining a new or augmented business workflow, or a roadmap that details how to get from where you are now to a place where you can overcome the challenges impeding your goals.
As Esri’s User Conference is just around the corner, I suspect many of you are getting excited to talk about the future for GIS at your organization. The Global Architecture Team is excited to work with you. Come chat with us. If you have any questions or would like to talk about any of these ideas further, email me at email@example.com. You can also reach my team at firstname.lastname@example.org.
“Nothing is as dangerous in architecture as dealing with separated problems” – Alvar Aalto
Like many of my colleagues on the Global Architecture Team at Esri, I spend a lot of time explaining who we are and what we do, and I think I know why. architecture is a funny word. It means a lot of different things to a lot of different people, which makes sense because depending on your domain, your experience with architects will differ greatly.
You might think of data architects, who design how data is collected, stored and arranged. Perhaps you’re more familiar with application architects, who are concerned with how applications are used and how they work together. Many people think of system architecture, and quickly find themselves in a cold sweat after picturing a horrifyingly complex system diagram. All this preamble to say: The Global Architecture Team doesn’t really do any of that.
What We Do
Our work is more closely aligned to the discipline of enterprise architecture. The role of the enterprise architect is to align the perspectives of business, applications, data, and technology to the organization’s overall strategy, while making sure the design follows organizational governance policies and standards. Now, we obviously we work at Esri, so we aren’t going to architect your entire business-technology landscape (sorry). Rather, we use enterprise architecture principles and frameworks and apply them specifically to the domain of GIS. One of those principles you’ll hear us talk about a lot is to take a ‘top-down, business first approach’.
This means starting with a vision for how GIS needs to support your business. This isn’t a list of maps or apps, and it definitely isn’t a data catalog. Rather, it’s a statement of what your organization will ‘look’ like when it has implemented and is operationalizing GIS effectively. Then we need to understand what business processes need to be designed or augmented with GIS. More simply, we figure out how the work your staff does needs to change to achieve the vision. Only once we have this level of understanding do we start aligning technology components, information products and data needed to enable those new or enhanced processes. Ultimately, we work with you to design the ‘why’ and then the ‘who’, ‘what’, ‘where’, and ‘how’ of GIS in your organization. My colleagues on the Global Architecture team and I are here to work with you to design the most impactful GIS that’s feasible in terms of sustainability, cost, and potential risk.
As Esri’s (first-ever virtual) Users Conference(esriuc2020) rapidly approaches, we know many of our customers are eager to connect and learn how they can do more. The Global Architecture team has supported organizations large and small, and in nearly every industry in their efforts to transform the way their organization works with geography and GIS. If you found this blog interesting and are looking for more on how our team can help amplify your success, you can reach me at email@example.com or my team at firstname.lastname@example.org.
“An idealist believes the short run doesn’t count. A cynic believes the long run doesn’t matter. A realist believes that what is done or left undone in the short run determines the long run.” - Sydney Harris
Organizations are serving a broader and more geographically disperse audience than ever before. Organizations that have a global reach may experience latency when trying to load content across the globe. Content Delivery Networks (CDNs) use edge servers to deliver content closer to the client application that requested it. Instead of a client application in Europe accessing a server in Northern California, content can be delivered from edge servers in Europe. CDN Settings allow us to control how often the edge servers return to the origin in Northern California to fetch updated content. The benefit here is twofold, less performance hit on the origin server and less latency for the end users.
ArcGIS Enterprise can take advantage of CDNs to push static files to edge locations. In this example, we use Amazon's CDN Service (CloudFront) to distribute static ArcGIS Enterprise files across the globe, allowing for quicker load times and less load on the back-end server. This blog details: content and its corresponding context path that are candidates for edge caching, a walk-through of how to use AWS CloudFront in front of ArcGIS Enterprise, and some additional considerations when using a CDN.
The first user in a particular edge server region always gets a request delivered directly from the origin. Subsequent users in that same region are then served content from the edge cache. Requests continue to be delivered from the edge servers until the configured Time To Live (TTL) is met.
ArcGIS Enterprise inherently supports secure content which requires a token to access. This content is not a good candidate for caching, as it changes frequently and each user obtains a different token. This will make all requests forward to the origin instead of being served from the edge cache.
ArcGIS Enterprise servers a mix of dynamic and static content. When adding a CDN in front of ArcGIS Enterprise it is important to understand the different type of content that should be cached. If we cache content that is meant to be dynamic, errors may occur in your applications. Below you can find a breakdown of the context paths and the associated content for both Portal for ArcGIS and ArcGIS Server.
Portal for ArcGIS
Static Content (context path)
Dynamic Content (context path)
Portal JS Files (portal/home/10.7.1/*)
Sharing API (portal/sharing/*)
Portal HTML Files (portal/*.html)
Portal Admin (portal/portaladmin/*)
Portal Image Files (portal/home/images/*)
* The above examples all use a Web Adaptor name "portal". If you named your web adaptor something different, replace the "portal" with you web adaptor name
* The above examples all use a Web Adaptor name "server". If you named your web adaptor something different, replace the "server" with you web adaptor name
Using CloudFront as your CDN with ArcGIS Enterprise:
There are three steps to deploying your ArcGIS Enterprise with CloudFront. The first step is creating the distribution and specifying what the default behavior for the cache is. This step is critical, as it allows dynamic content to pass thru the edge servers and onto the origin. The second step is adding additional cache behaviors. This step provides the ability to cache specific path patterns from the distribution. The final step is to map the existing DNS Record Set to the CloudFront distribution CNAME.
Add Cache behaviors to the above created distribution
Create a CloudFront Distribution:
From your AWS Console navigate to the CloudFront service under the "Networking & Content Delivery" section.
Select "Create Distribution" to enter the CloudFront deployment wizard.
Select "Get Started" under the Web at the "Select a Delivery Method" page.
Note: The Origin Domain Name will be the Load Balancer or Server that your ArcGIS Enterprise is accessible from. In this example we use the an Application Load Balancer but other properties are supported here as well. CloudFront will forward requests to this location when populating it's edge cache.
Ensure the default behavior of Cache Based on Selected Request Headers is set to None to support portal's Dynamic Content.
Note:Ensure Object Caching is set to Use Origin Cache Headers. Portal will send a response header with Cache-Control: no-cache by default. CloudFront has the ability to override this. Additional cache behaviors will be added in the next steps to overwrite this and allow static content to be cached.Note: Ensure you add your DNS to the "Alternate Domain Names". This comes into play when you map your DNS Records Set to the CloudFront Domain Name.
Find your distribution ID from the CloudFront homepage
Navigate to the "Behaviors" tab
Select "Create Behavior"
Populate the behavior with the following
Note: The path pattern determines what paths to apply this behavior. This option allows us to cache content. Note: Unlike the initial deployment ensure you chose the "Customize" option for "Object Caching". This allows you to specify a default, minimum, and maximum amount of time (in seconds) your object will live in its edge location. You can see mine are set to 86400 (1 day).
Repeat this step for the following "Path Patterns"Note: The path patterns are related to the table in the "ArcGIS Enterprise Content" Section above. You will notice the path pattern for a tiled map service there as well. Note: After an upgrade the path pattern for "portal/home/10.6.1" will have to be updated for the specific version of ArcGIS Enterprise you are using.
3. Point your DNS records to the CloudFront CNAME
These steps come after you have completed the installation and configuration of your ArcGIS Enterprise deployment
Ensure that the CloudFront cache is invalidated after an upgrade
It is time to bring this all together...in Part 1 of The Path from GIS Manager to GIS Leader, I mentioned how rebranding can help you change your image from mapmaker to enterprise IT solution provider. This rebranding can be applied to two areas, job titles and department names. Here are some real example job titles from GIS practitioners that have rebranded:
Content Delivery Manager - Charlotte, NC Water Department
Decision Analytics Manager - Charlotte, NC Department of Innovation & Technology
Data, Analytics & Visualization Services Director, York Region, Ontario, Canada
Geospatial Intelligence Manager - US National Geospatial Intelligence Agency
GeoAnalytics Information Officer - Pennsylvania Turnpike Commission
Geographic Information Officer - there are many examples of this one, especially at the state government level
Director of Enterprise Location Intelligence - Walgreens
Business Analytics, Intelligence & Reporting Manager – Edmonton Police Service
Business & Location Intelligence Manager - DCSI South Australia
Business & Location Innovative Services Supervisor - Cabarrus Co., NC
The ultimate accomplishment of all of this is to create a new strategic executive position that is above the GIS Manager and reports to a top executive. This is key, as guiding the strategic direction of enterprise GIS to enable location intelligence across the organization is a full-time job. The GIS Manager position should focus on the operations, managing the GIS staff and projects. While this goal of creating this new executive position may seem impossible, there is evidence to the contrary. If executives see value and potential for more value in technology, they can, and will, create new full-time executive positions to lead these critical initiatives.
When enterprise technology was first implemented at most organizations, the first step was to create something like a Chief Technology Officer (CTO). As enterprise technology expanded, and with it, the value it provided to the organization, additional positions are being created, like Chief Innovation Officer (CIO), Chief Data Officer (CDO), and Chief Analytics Officer (CAO). If the lead executive can understand the full potential and value of Location Intelligence and GIS, they will create a Chief Geospatial Officer (CGO). If this is not possible in your organization, it is possible to move up above the GIS Manager position to gain increased access and ability to affect change and expand the use of GIS. Here are more than thirty real examples of GIS practitioners that have done this from across the US, Canada, and Australia:
Here's an interesting point for those that are GIOs, some GIOs report to the CIO/CTO, but some report directly to another executive. I think this is an important point to make that who you report to can often offer insight to the importance executives put in your position. For instance, Joseph Sloop, the GIO at Forsyth Co., NC, reports to the County Manager. This shows that the top executive in that organization values that position so much, that they want to personally direct their work. Additionally, there are some examples of GIS leaders that may not have the leadership titles, but still are valued enough to report directly to top executives. This is the case for the following positions:
GIS Coordinator, Oak Hill, WV reports to City Manager
GIS Analyst, Morgantown, WV reports to the Assistant City Manager
GIS Manager, Montgomery Co., PA reports to the Chief Operating Officer
GIS Manager, Skagit Co., WA reports to the Central Services Division Manager
What I'm trying to do here is show examples of how your peers are changing their role in the organization to improve their personal professional development as well as increase the value of GIS. Use them as inspiration to do the same, change from GIS Manager to GIS Leader.
There is a tremendous opportunity for you as a GIS practitioner to move up in your organization and transition from GIS Manager to GIS Leader. This is a valuable endeavor as it improves you personally and professionally as well as your organization. Remember that your colleagues want your help and need your help, they just don't know it. Reach out to them, be proactive, sell them on the value of spatial analytics and location intelligence. Enable them to use the technology themselves with easy-to-use apps that work on any device, anywhere, at any time. Even though your job description may not include working on the five pillars of location intelligence (Strategy, Organization, Technology & Data, Culture and Literacy), make sure you are dedicating time to all of them, the people funding the enterprise GIS (taxpayers, shareholders, donors) deserve the ultimate level of return on investment and this means maximizing the capabilities of the ArcGIS platform and expanding the number of users.
Don't try to do this alone, reach out to Esri, our distributors, and our partners, as well as your peers and professional network. Learn from the examples of your peers that are doing this. Get some non-GIS training on subjects like management, leadership, IT, and business. Remember that GIS is not about maps, it's about digital transformation. Align your work to helping the business and to what the leaders at the top deem important, that's how to create value.
You can find additional resources related to this topic here. Please reach out to me if you'd like to connect, I enjoy helping others in this important transition. You can find me here on GeoNet as well as Twitter and LinkedIn.
The second of my important numbers (64, 76, 78, 84, 84) based on my research of 800+ GIS practitioners across the United State over the last two years, refers to the second pillar of location intelligence, Organization:
76% of organizations that use GIS do not have formal governance in place
While Organization covers many aspects of enterprise GIS, let's focus on Governance. In this context, governance refers to the organizational structure for GIS within the organization. GIS governance is a critical ingredient for any successful enterprise GIS. If your organization does not have formal GIS governance, you should consider creating it. Here are some resources to assist:
Matt Lewin, from Esri Canada, has several articles on the subject as well as a presentation video:
You should also reach out to your Esri Account Team for assistance and other examples of successful enterprise GIS governance.
The third of my important numbers mentioned above, refers to the third pillar of location intelligence, Technology & Data:
78% of GIS practitioners have not read the Architecting the ArcGIS Platform: Best Practices document
While Technology & Data covers a wide area, let's focus on Best Practices. This Architecting the ArcGIS Platform: Best Practices document is one of the most important documents produced by Esri, and it is updated at least three times a year. Following and implementing best practices are another key ingredient to a successful enterprise GIS. They are called best practices for a reason. They are designed to help you learn from the missteps of others. Many of the best practices in this document follow those established by the IT industry. By implementing these best practices, your GIS will become stronger, more stable and effective. Please work with your Esri Account Team to implement those that are applicable to your organization.
The fourth of my important numbers refers to the fourth pillar of location intelligence, Culture:
84% of organizations that use GIS do not have, and maintain, a Change Management Plan
While Culture covers a wide area, let's focus on Change Management. According to Prosci, Change Management is the discipline that guides how we prepare, equip and support individuals to successfully adopt change in order to drive organizational success and outcomes. The biggest challenge for a successful implementation is not the technology, it's the people. The technology is the easy part. You can deploy the best GIS app ever, but if nobody uses it, it's of no value. Change management can really help technology adoption. According to Prosci, organizations that combine a people-focused change management plan with a project plan are up to six times more likely to achieve their project goals. Here are some resources to help you implement Change Management:
The fifth of my important numbers refers to the fifth pillar of location intelligence, Literacy:
84% of organizations that use GIS do not have, and maintain, a Workforce Development Plan
While Literacy covers a wide area, let's focus on Workforce Development. GIS is changing rapidly. Everyone that uses GIS in an organization should be receiving some GIS training annually. Getting financial support for training can be a challenge, as can figuring out what GIS training options to take in what order. As an Esri customer, you have access to an Esri Training Consultant. These Training Consultants can create a custom workforce development plan for you at no cost. This plan will create learning pathways for each GIS user in your organization based on their roles, responsibilities and use of ArcGIS technology. Once you have a workforce development plan, it can help you get budget to support this critical need. There are also a collection of Learning Plans available on the Esri Training website. Contact your Esri Account Team to get connected to your Training Consultant.
Over the last ten years as technology has evolved, especially cloud and mobile devices, ArcGIS has transformed from a collection of software that runs on different devices, to a fully integrated cloud platform that allows the GIS practitioner to easily share their work with anyone, anywhere, anytime, on any device. This transformation has changed the role of the GIS Manager, from managing a team of GIS practitioners that did all of the GIS work, to that of an enterprise IT system leader that focuses on enabling anyone inside, or outside the organization, to directly use GIS technology to make better decisions and do their job. The GIS community has failed to keep up with this transformation in a few ways.
software skills that include Esri software, programming skills, database skills
Those requirements do not represent the skills required by the modern GIS Manager of a cloud-enabled enterprise IT system. The requirements now are more focused on enterprise IT system management, business, strategy, governance, innovation, marketing, change management, and workforce development. Due to this disconnect, where most GIS manager positions are defined incorrectly, there is an atmosphere of unawareness as to what they should be doing. This has led to a chronic underutilization of enterprise GIS in most organizations that have the technology. This trend of underutilization can be supported by the following documents:
“Generally, people outside of GIS think of GIS just as ‘maps’ or a graphic product, or the younger brother of CAD.” Five suggestions to get more out of your GIS:
Low awareness the biggest hurdle - However, understanding the complexity and potential about this technology amongst those technology innovators is still limited. “The geospatial industry is still undervalued and under-appreciated by the world at large. The onus is on us to collectively demonstrate how location data and tools can be applied to make dramatic improvements to society from making every journey safer and our air cleaner to helping businesses operate more efficiently,” underlines Edzard Overbeek, CEO, HERE Technologies.
As expected, lack of awareness among users and policymakers remains the biggest challenge for the industry, with over 38% listing it as a primary hurdle... (Graph 14)
This trend of underutilization has led to an outdated image of most GIS practitioners as mapmakers. While mapmaking is a valuable and treasured skill, if an organization has an enterprise GIS implemented, the GIS practitioners should be doing a lot more than just making maps. If the non-GIS practitioners in the organization think the GIS practitioners only make maps, that is all they will ask for. The problem is, they want help, and need help from the GIS practitioners to apply spatial analytics to their mission, they just don't know that they can help them in this manner.
One of the first things an enterprise GIS manager needs to do is change this image. There are a few ways to do this. Internal marketing and education is one way. One easy way to start to work on this is to change your elevator pitch. Many GIS practitioners when asked what they do for a living say, "I make maps." or "I do GIS, it's like Google Maps." This prolongs the mapmaker image, therefore it is crucial that you change your elevator pitch to something like, "I help people use the power of location to make better decisions."
Rebranding is another way, there is a growing movement for GIS practitioners changing their job titles and the name of their department to remove the term "GIS" from them, to distance themselves from the mapmaker image. Examples of this include Decision Analytics, Spatial Analytics, Location Intelligence, Geospatial Intelligence, GeoAnalytics, and Location Innovative Services.
Another part of internal marketing and education is proactively changing your communication focus with non-GIS users. Most GIS users tend to focus on the technology in their communication with non-GIS users. This is not an effective mode of communication, you need to communicate with non-GIS users in ways that interest them in what you can do to help them. The best way to do this is to shift the communication focus to capabilities. A capability is "the power or ability to do something" according to Lexico powered by Oxford. Adding capabilities to an organization and its staff will interest executives and managers. The capability that GIS can enable is best described as location intelligence. Watch this short video for a definition of location intelligence.
Enabling a new capability, like location intelligence, for an organization and its workers, is very different than implementing technology. In 2019, Esri Canada released a report they developed with IDC called Winning with Location Intelligence: The Essential Practices. This report identifies best practices for how to be successful with location intelligence. They surveyed two hundred organizations across Canada in all industries. These organizations had five hundred or more employees. They analyzed their use of location intelligence and identified common best practices. They categorize these best practices into what they call the five pillars of location intelligence.
These five pillars of location intelligence help us understand in more detail the skills required of an enterprise GIS manager. You can see that you cannot spend all of your time on Technology & Data. You must dedicate time and effort to the other four. My research shows that the majority of organizations that have implemented GIS are not spending time in these other four areas. Here are five important numbers that will be revealed in this blog series:
64, 76, 78, 84, 84
Starting with the first of the five pillars of location intelligence, Strategy, why is a strategy so important? In this article from McKinsey & Company entitled, Catch them if you can: How leaders in data and analytics have pulled ahead, their survey of over five hundred C-level executives and senior managers representing a full range of regions, industries and sizes showed that:
The creation of a strategy now ranks as the number one challenge to, and reason for, companies' success at data and analytics...
Here's the first of my important numbers, based on my research of 800+ GIS practitioners across the United States over the last two years:
64% of organizations that use GIS do not have, and maintain, a GIS Strategic Plan
What is a Geospatial Strategy?
A Geospatial Strategy is a business-oriented plan that defines how an organization will use GIS to achieve its goals. An effective geospatial strategy connects your business needs with the right people, processes and technology to help you overcome challenges and improve results.
When creating a Geospatial Strategy, it is critical that you focus on the business, not the technology. Your objective is to focus your work on assisting the business. You do this by first identifying and understanding what the business goals are, these should be documented in strategic plans, initiatives, Key Performance Indicators, etc. Then you identify and understand the challenges that the organization is facing that are keeping it from achieving the goals. Then you deploy GIS solutions that overcome the challenges so the goals are achieved. This results in real business value of GIS.
For more information on the Esri Best Practices on how to create a Geospatial Strategy, please check out this document, The Approach to Maximize Impact and be sure to contact your Esri Account Team.
Organizations often require a certain level of system uptime for their ArcGIS Enterprise deployments, such as 99 percent of the time or higher. For these organizations, implementing a strategy to ensure high availability is crucial.
High Availability (HA), while related to Disaster Recovery (DR), is a separate concept. Generally, HA is focused on avoiding downtime for service delivery, whereas DR is focused on retaining the data and resources needed to restore a system to a previous acceptable state after a disaster.
This post will focus on best practices to configuring local load balancer (Local Traffic Manager, LTM) in a single location for high availability, and will not include considerations to configuring Global Traffic Manager (GTM) for automatic failover between different locations.
To achieve high availability, you must reduce single points of failure through duplication and load balancing.
Load balancers act as a reverse proxy and distribute traffic to back-end servers. Third-party load balancer is required in a highly available ArcGIS Enterprise deployment to improve the capacity and reliability of the software. They handle client traffic to your portal and server sites, as well as internal traffic between the software components.
ArcGIS Web Adaptor
Though ArcGIS Web Adaptor is considered a load balancer, it’s inadequate to serve as the lone load balancer in a highly available deployment, since ArcGIS Web Adaptor also requires redundancy to achieve high availability.
ArcGIS Web Adaptor is an optional component, as load balancers can forward requests directly to your portal and server sites, yet it is a recommended component. The advantages of using web adaptors are:
It provides an easy way to configure a single URL for the system
It allows you to choose context names for the different system components, e.g. portal, server, mapping, etc.
It is integrated natively with other ArcGIS Enterprise software components, the portal and server sites, and will automatically handle health checks and configuration tasks, e.g. adding new machine to a server site
Web Context URL
The web context URL is the public URL for the portal. Since any item in portal has a URL - file, layer, map, and app, the portal's WebContextURL property helps it construct the correct URLs on all resources it sends to the end user.
External Access and DNS
ArcGIS Enterprise portal supports only one DNS for public portal URL (the web context URL), and currently there is no supported way to change the web context URL without redoing administrative tasks, e.g. federating server sites with your portal. If your ArcGIS Enterprise requires external access, e.g. to allow access to mobile users, contractors, partners, or agencies, without VPN, or if you anticipate that you will need to allow external access in the future, you must use an externally resolvable DNS name for the portal’s web context URL, e.g. https://gis.company.com/portal.
To secure external access to ArcGIS Enterprise, it is common to host a load balancer in a DMZ, and implement a Split Domain Name System (Split DNS), i.e. internal access to ArcGIS Enterprise DNS (e.g. gis.company.com) will be resolved to an internal load balancer IP, so internal users will stay behind the firewall, and external access to ArcGIS Enterprise DNS will be resolved to an external (DMZ) load balancer IP.
URLs used in federation
Several different URLs are used in a highly available ArcGIS Enterprise deployment.
This is the URL used by users and client applications to access ArcGIS Server sites. It’s the URL for the load balancer that handles ArcGIS Server traffic and passes requests either to the server site’s Web Adaptor or directly to the server machines.
This URL is used by administrators, and internally by the portal, to access an ArcGIS Server site when performing administrative operations. This URL is also used for publishing GIS services with reference to a registered data store, e.g. SQL Server enterprise geodatabase, to a federated server site. This must direct to a load balancer; if the administrative URL points to a single machine in the server site and that machine is offline, federation will not work. This can be the same URL as the services URL or can be a second load balancer (VIP) for each federated server site admin URL via port 6443. Configuring a dedicated VIP for each federated server site admin URL via port 6443 will require to open this port for administrators and publishers, and you can disable administrative access through the web adaptor, thus providing additional security controls for the organization.
I recommend using the same URL as the services URL since it simplifies configuration. ArcGIS Server administrative access will be controlled by ArcGIS Enterprise authentication and user roles, similarly to administrative access to portal, e.g. ArcGIS Portal Directory (portaladmin) and Organization Settings. To use web adaptor URL for administrative URL, you must enable administrative access in the server web adaptor.
Private portal URL
This is an internal URL used by your server sites to communicate with the portal. This must also direct to a load balancer and should be defined prior to federating. If you federate your server sites prior to setting the privatePortalURL, follow step 8 and 9 in the topic Configure an existing deployment for high availability to update the URL within your deployment. Similar to the administrative URL, this can be the same as the public URL for the portal (portal’s web context URL), or it can be a second load balancer (VIP) via port 7443.
For configuration simplicity, I would recommend using the public URL for the portal for the private portal URL. If you choose to use a dedicated load balancer VIP via port 7443 for private portal URL, you should configure the load balancer to check the health of the portal machines.
“The health check reports if the responding Portal for ArcGIS machine is able to receive and process requests. For example, before creating the portal, the health check URL reports the site is unavailable because it can't take requests at that time.”
ArcGIS Enterprise portal and server have health checks.
When you use ArcGIS Web Adaptors, the web adaptors will take care of performing the health checks against the portal and the servers. In this case you can configure the load balancer with basic TCP/443 health check or a static page health check, with standard health check settings for timeout, failure trigger, polling interval, and healthy threshold.
If you configure the load balancer to access portal and/or the servers directly, e.g. if you don’t include web adaptors in your architecture, or if you use dedicated load balancer URL for private portal URL (port 7443) or federated server administrative URL (port 6443), you should configure the load balancer to check the health of the portal and server machines.
There are some important considerations about health check settings on the load balancer. Most organizations using a load balancer use a static page as their health check (e.g. index.html) to determine whether the web server is healthy. This is a static file that only requires a disk fetch. Also, most web servers tend to bottleneck with I/O rather than CPU.
However, Esri’s ArcGIS Server is different in that our health check requires a very small amount of CPU because our check is more than just a disk fetch, as the software needs to determine whether certain processes are functional.
With health checks there is a timeout value on the poll. Most load balancer administrators set the timeout value very low because disk fetches are usually very fast (though they often leave some buffer for poor network latency).
When using a low timeout value with ArcGIS Server and when using a multi-machine setup, there is a chance one of the machines may exceed the low timeout value and a healthy machine will be removed.
Esri recommends a higher timeout value, ideally at least 5 seconds. Depends on the system, you might need to increase this value even more. You should monitor your environment and adjust this value accordingly. This may seem like a high number to Network Administrators since a health check on a simple page usually takes less than 10ms normally (plus network latency).
However, it is critical for the load balancers to distinguish between when a machine is just slow vs. when a machine is truly dead and non-responsive. If portal or server is truly down and not listening on a port at all, most load balancers will detect this even sooner than 5 seconds, so this timeout does not impact "normal" failures where a machine goes away.
The second consideration is the failure trigger - how many times does the poll need to fail before removing the machine from the load balancer.
As a rule, Network Administrators set the failure trigger to be more than 1 failure because they do not want a single network glitch to take down the system. As noted above, a small spike in CPU usage (which is common on ArcGIS Server systems) can cause a timeout, and it is not desirable to bring down a machine because of a single CPU spike on a single machine.
Esri has done significant internal testing with load balancers and found that a setting of 5 failures dramatically reduced the number of false positives while still detecting true outages. We found that a value of 3 was still highly susceptible to false positives.
The third consideration is the polling interval. Esri has found that 30 second polling intervals, coupled with 5 failures was a sweet spot of detecting true failures and ignoring false positives.
With this combination, the expected value (using a statistics term) or mean time to detection is 1 minute and 15 seconds of downtime before detection, with a worse case of 2 minutes and 30 seconds. It is possible to aim for a lower mean time to detection, but the trade-off is receiving false positives.
If the lower mean time to detection is preferred, it may be needed to increase capacity so there are sufficient resources to suffer a true failure on a machine and a false positive without overloading the remaining machines.
The final setting regarding health checks is the healthy threshold for the load balancer to start sending requests again. Esri does not have a recommendation for this and we have not observed many differences in this number, but we typically see 3 healthy consecutive polls before rejoining.
Throttling settings are worth considering. ArcGIS Server is CPU-bound which means most of the request time is spent using the CPU rather than waiting for I/O.
This means if there are 8 cores, ArcGIS Server can handle a little more than 8 simultaneous requests from a practical standpoint. If ArcGIS Server is busy, it starts queueing requests until there are hundreds of requests in line, and after that threshold it refuses connections. When there is a long backlog of requests it will result in a long wait time, but eventually the request gets processed.
ArcGIS Server has settings to control this behavior, so there are less of these abandoned requests being worked on, but it is also a best practice to control this at the load balancer level via its ability to throttle.
Esri does not have a numerical recommendation because it depends a lot on the specific architecture and the types of requests coming in, but it is typical to throttle at a significant lower value than Network Admins would typically do for a Web Server.
This is beyond the control of the Network Administrator, but the client applications should be written to handle a throttling event and do re-tries in increasing time intervals (e.g. first time you can immediately re-try, second time you wait a second, third re-try wait 5 seconds, etc.).
Esri does not recommend sticky sessions except in very rare circumstances. Sticky sessions can theoretically overload a machine. Esri has conducted load tests using sticky sessions to see if we found results that would overload our GIS Server, and this did not occur. We have also not received any customer complaints. That said, since our software is stateless we do not see the value in using sticky sessions.
Layer 4 vs. Layer 7
The final setting to mention is whether the load balancer does a "level 7" approach or a "layer 4" approach. This can be a debate among Network Administrators, but below is a concise summary of the situation describing the differences and advantages of each.
A layer 7 load balancer understands http and https, it therefore decrypts https content and then re-encrypts it. Because it understands http and https it can cache content and save requests going to the backend server.
A layer 4 load balancer views all the traffic as TCP packets and has no idea what the packets mean; they could be for ftp, https, smtp, but this does not matter to a layer 4 load balancer. As a result, it doesn't need to understand the http payload and can be faster.
ArcGIS Server can work with either approach and there is no recommendation on this setting for Network Admins, but there are some pieces of information that an Admin will want to be aware of.
ArcGIS Server payloads can be much larger than HTML pages, CSS, and JS content (the actual amount would be useful but is often dependent on the data the customer uses). This means there is more CPU load on a layer 7 load balancer decrypting and encrypting on a per-request basis.
Also, since much of ArcGIS Server data is dynamic and frequently changing, by default cache headers prohibit caching on the client and load balancer. If the data doesn't change much and the customer wants to use a layer 7 load balancer, they can change and control these cache settings.
Summary of Recommendations
If you configure your load balancers with ArcGIS Web Adaptors, you can configure the load balancer with basic TCP/443 health check or a static page health check against the web servers
If you configure the load balancer to access portal and/or the servers directly (via ports 6443 and 7443), use HTTPS health check endpoint:
Use a higher health check timeout value, at least 5 seconds
Use a value of 5 for the failure trigger
Use 30 second polling intervals
Use 3 healthy consecutive polls before rejoining
Use a throttle setting at a significant lower value than typical web servers
Do not use sticky sessions
ArcGIS Enterprise components come pre-configured with a self-signed server certificates, which allows the software to be initially tested and to help you quickly verify that your installation was successful. However, in almost all cases, an organization should request a certificate from a trusted certificate authority (CA) and configure the software to use it. The certificate can be signed by a corporate (internal) or commercial CA. Commercial CA (known-CA) must be used for externally resolvable DNS, e.g. load balancer VIP DNS; internal domain certificates can be used for internal servers. For ArcGIS Enterprise system with externally resolvable DNS, if the load balancer’s SSL method is SSL-passthrough (the load balancer does not decrypt and re-encrypt https content), it does not require certificate, and a commercial CA certificate must be installed on the mapped servers (e.g. web servers where the web adaptors are installed). If the load balancer’s SSL method is SSL re-encryption, a commercial CA certificate must be installed on the load balancer.