Azure SQL as a Source in ArcGIS Data Pipelines

[Dion]

It would be highly beneficial to have the option to select an Azure SQL table as a source within ArcGIS Data Pipelines. Currently, the available options are somewhat limited, with support only for Google BigQuery and Snowflake.

In our current workflow, we use an Azure Function along with the ArcGIS API to transfer data from our Azure SQL table to an ArcGIS feature layer. This approach requires custom code, making us dependent on a consultant. We would prefer to handle this process internally using the low-code ArcGIS Data Pipelines solution.

[CarterHughes]

I second this. I have had to weigh the same myself. I use Azure Synapse but the data is persisted externally on Azure Storage as an external table so fortunately I can use that as a source for ArcGIS Data Pipelines myself. However, it would be much cleaner to query from a SQL Server database such as Azure SQL Database you use or Synapse SQL Serverless like I use because that opens the door to views etc. and doing transformations, filters, etc. further upstream easier.

To simplify some of your code, you could write the desired data to Azure Storage so ArcGIS Data Pipelines. I have Azure Functions that query via SQL and write to Azure Storage that that are very simple. Can also use copy activity from Fabric, Data Factory, or Synapse for low code solution. In my experience the ArcGIS API for Python is rather bloated for simple data extract and load data processing like this so I have done everything possible to avoid it.

Also this idea is equivalent to an existing idea SQL Server on Azure. Just stating so Esri can see we keep asking for this!