Background
When our upstream product manager set out to prepare Acreage Grading data using an ArcGIS Pro 3D model by calculating the 3D nearest neighbor, we knew we were facing a challenging computational problem with exciting potential.
The Challenge
Our GIS Analyst initially developed a Python script that extracted each wellbore bottom point and calculated the 3D nearest neighbor with high precision. The script delivered excellent results, but the process took nearly 30 hours per county.
The leadership team wanted to scale this analysis to include the entire United States, creating Acreage Grading datasets for external customers and upcoming conference presentations. To achieve this, we needed a cloud-based architecture that could handle heavy geospatial processing while remaining cost-efficient.
The Solution
We designed a cloud automation workflow using ArcGIS Pro, AWS Step Functions, and FME to parallelize and optimize processing at scale.
With FME, we extracted and transformed data from multiple sources, including geodatabases, Denodo, and other formats. We then sliced the data by county, created File Geodatabases (GDBs), and uploaded them to an AWS S3 bucket.
Once the summary CSV file was uploaded, an S3 event triggered an AWS Step Functions workflow. Step Functions launched a Fargate container that orchestrated the workload across 10 EC2 instances with ArcGIS Pro pre-installed.
Each EC2 instance dynamically acquired a license from the shared pool, executed the Python script, and processed the files in parallel threads. With 8 threads per instance, the system processed 80 counties simultaneously, significantly accelerating throughput.
Logs and output data were written back to S3, and EC2 instances were automatically terminated after completion. Using Spot Instances reduced infrastructure costs, and by running during off-peak hours, we optimized license usage to just 90 minutes per batch.
Finally, the FME job merged the CSV outputs and loaded the results into a PostgreSQL table for visualization in Tableau dashboards.
Results
This automated pipeline reduced processing time from days to hours, improved scalability, and minimized infrastructure costs. The workflow now runs weekly to process updated well data efficiently and with full automation.
The combination of ArcGIS Pro, AWS Step Functions, and FME created a robust, repeatable solution that can be extended to other large-scale spatial analyses. Using Terraform for infrastructure provisioning and Ansible for command orchestration made deployment streamlined and maintainable.
Takeaways
This project highlights how ArcGIS technology can be optimized for cost and performance in cloud-native environments. Automation, scalability, and parallelization transformed a time-intensive GIS task into a highly efficient enterprise workflow.
We are deeply grateful to the ESRI Community for the collaboration, guidance, and inspiration that helped us achieve this success. This work demonstrates the power of integrating ArcGIS Pro with cloud services to deliver real-world, high-impact geospatial solutions.
