Using Amazon Cloud Development Kit and Systems Manager to Automate Deployment of ArcGIS Enterprise
Are you still manually clicking executables to deploy ArcGIS Enterprise? Wouldn’t it be nice to automate that process? By combining Amazon Cloud Development Kit (CDK), Amazon Web Services (AWS) Systems Manager, and Esri’s PowerShell Desired State Configuration (DSC) libraries, you can take ArcGIS Enterprise Deployments to the next level! By utilizing Infrastructure as Code to automate deployments, you will make ArcGIS Enterprise deployments efficient, consistent and simple.
Storing your infrastructure as code has become an essential task for IT departments. It allows System Administrators to quickly deploy consistent architectures, in a repeatable manner; and lowers the risk of user-based errors, during deployment and configuration. This blog will provide instruction on how to use Infrastructure as Code, to deploy AWS Infrastructure and then install & configure ArcGIS Enterprise with that infrastructure. This blog will use Amazon’s System Manager Run Commands to prep the instance and deploy ArcGIS Enterprise without having to log into a machine!
Currently, Esri offers CloudFormation templates and a CloudBuilder application that enables you to automate AWS deployments. The deployment patterns for those tools are rather ridged in terms of available architectures. The approach laid out in this blog allows for a more flexible architecture and also provides access to the code via the arcgis_cdk GitHub Repo.
The blog is broken into two sections:
- Deploying Infrastructure with the AWS CDK; and
- Deploying ArcGIS Enterprise with AWS Systems Manager & PowerShell DSC
It is recommended that you follow along with the links that point to detailed steps for deployment procedures within the repo.
Deploying Infrastructure with the CDK:
The CDK allows you to program in a language of your choice and deploy infrastructure to with that language. The arcgis_cdk uses the CDK to define infrastructure in python. It was designed with common architecture patterns that Esri see’s organizations use for ArcGIS Enterprise within AWS. The architectures are a tiered (components broken out into individual machines) highly available configuration and a tiered single instance stack. A full breakdown of the prerequisites, deployment steps, configuration parameters, and available architectures can be found within repo. A logical diagram of high-level configuration options can be found below.
Note: A configuration parameter allows the user to choose a highly available pattern. If this parameter is true, the architecture will include the lighter shade pieces of the diagram above (highly-available). If it is false, it will only include the full color components (not highly-available).
If you’re attempting this deployment, now would be a good time to break and follow initial steps to setup the CDK in your environment from the repo.
Once the arcgis_cdk has been deployed take a peek in your AWS account to notice the following infrastructure:
- A new VPC was created with multiple public/private subnets
- Instances were added to the private subnets (amount depends on your architecture configuration)
- Security Groups were created that only allow specific traffic to the instances
- An application load balancer was created with 2 target groups, a listener with path-based routing rules, and an ACM Certificate attached
- Route53 Public Zone was used to map a Record Set to the Application Load Balancer
- A Route53 private hosted zone was created and record sets were mapped to the internal IP address of each EC2 Instance
Our AWS infrastructure is now in place, we are ready to start installing ArcGIS Enterprise!
Deploying ArcGIS Enterprise Using Systems Manager & PowerShell DSC:
AWS System Manager is an AWS Service that allows visibility and control of your cloud infrastructure. This deployment uses AWS System Manager Run Commands. Run Commands can be used to execute code, without having to log into your EC2 Instances. There is a preexisting Run Command Document that will execute PowerShell on our instances, that were deployed from the previously referenced CDK exercises. Run Commands will be used to both prep the system and invoke our PowerShell DSC script. Alternatively, a run command document could be created from the PowerShell files in the arcgis_cdk repo.
Prep of Machines
Before we can install, license, and configurate ArcGIS Enterprise we must prep our instances for the PowerShell DSC scripts. This prep includes:
- Download Esri software and licenses from the S3 bucket created by CDK
- Configure a service account on each machine
- Setup the file share if your site is HA
- Prepare each server for Windows Remote Management.
A detailed walkthrough and sample script for preparing the instances can be found in repo or code attached to this post "EC2InstancePrep.ps1". I’ve also captured a screenshot of the PowerShell Script from the Run Command prior to executing the command.
The machines are now prepped for our ArcGIS Enterprise installation. The next step is using PowerShell DSC for the deployment of ArcGIS Enterprise!
ArcGIS Enterprise PowerShell DSC:
Esri's PowerShell DSC is used to Install, License, and Configure ArcGIS Enterprise. Once the infrastructure has been deployed by Amazon CDK, and the EC2 Instances have been prepped by Amazon Systems Manager, we are ready to deploy ArcGIS Enterprise using PowerShell DSC. This can be run from Systems Manager as well, or can be run from the orchestration EC2 Instance.
While PowerShell DSC is installing, licensing, and configuring ArcGIS Enterprise, it can take some time. It’s common for an highly-available deployment to take over 4 hours, non highly-available deployments are generally ready within 2 hours.
One configurable option for Amazon’s System Manager, is to deploy the output logs to S3. This allows you to check these logs to understand how your deployment is progressing. Once this piece is complete you will have a working base deployment of ArcGIS Enterprise!
Summary
This post covered Infrastructure as Code, by using Amazon CDK to deploy AWS Infrastructure and using Amazon Systems Manager to both, prep and deploy ArcGIS Enterprise. This example used Windows AMIs and PowerShell DSC but could extended to use Linux AMIs and Chef to deploy. I recommend choosing the language and operating system you’re most comfortable with and extending the solution. The GitHub repo is open if you would like to branch your own code or enhance what’s already there. The power of the tools leveraged in the blog, gives you ultimate freedom to deploy ArcGIS Enterprise into the architecture you need, with the, programming languages you’re already comfortable with.