Integrate ArcGIS Pro with Databricks Unity Catalog to Read Delta Lake Tables

[JamesMaskell]

Idea Proposal: Integrate ArcGIS Pro with Databricks Unity Catalog to Read Delta Lake Tables

We propose enabling ArcGIS Pro desktop users to connect directly to Databricks Unity Catalog and access Delta Lake tables containing GEOMETRY/GEOGRAPHY primitive types. These tables should be readable and visualised as vector feature class layers within ArcGIS Pro.

Benefits of Integration:

• Eliminate Data Migration Overhead
  Seamlessly access and process Delta Lake tables within ArcGIS Pro without the need for data duplication or transfer.
• Preserve User Workflows
  Empower the existing ArcGIS Pro user community to continue leveraging familiar mapping and spatial querying capabilities on modern data platforms.
• Scale to Big Data
  Efficiently handle large, partitioned datasets stored in Delta Lake, unlocking advanced spatial analytics on enterprise-scale data.
• Enhance Security and Governance
  Leverage Databricks Unity Catalog’s robust security model for fine-grained access control and auditing.

[JosephMcGehan]

Databricks is an order of magnitude faster than SQL for doing stuff.  Being able to connect to DB and visualize data would be extremely beneficial.  At the moment we have to export to text, and a very 1990's approach. 

[ALink]

We're starting to explore places where Databricks can help us scale jobs and workflows, so being able to explore spatial outputs from Databricks in ArcGIS Pro would be a huge win. Right now, the lack of good integration makes it feel like we have to choose between the speed / scalability of Databricks and the robust visualization capabilities of ArcGIS Pro. Better Databricks integration would make sure ArcGIS Pro remains a relevant part of our workflows.