Idea: Accelerating 3D Nearest Neighbor Search Using Snowflake + ArcGIS Pro
We had a performance bottleneck running 3D nearest neighbor analysis (e.g., finding the closest wellbore) using ArcPy in ArcGIS Pro — it took 30 hours per US county on a single machine.
I solved this by parallelizing the workflow with AWS Step Functions, EC2, and Python multithreading. By slicing the data by county and running ArcPy on 10 EC2s (8 threads each), we reduced processing time to 1.5 hours. Outputs were written to File GDBs and stored in S3.
Now, I'm exploring the next step: offloading the processing to Snowflake. The idea is to store spatial data in Snowflake, use its multi-cluster compute to run 3D proximity logic in parallel (via SQL or Snowpark), and connect ArcGIS Pro for visualization.
This could eliminate the need for EC2 and scale analysis instantly using Snowflake’s native elasticity.
