https://community.esri.com/t5/data-pipelines-ideas/arcgis-data-pipelines-connector-for-azure-data/idi-p/1591930 <P>The&nbsp;<A href="https://www.esri.com/arcgis-blog/products/arcgis-online/announcements/integrate-arcgis-data-pipelines-with-microsoft-power-automate-workflows/" target="_self">ArcGIS Data Pipelines Connector in Power Automate</A> is great! I am certainly going to implement it. However, it is a band-aid solution for users like myself who execute data engineering at the enterprise scale and use Azure Data Factory / Azure Synapse / Microsoft Fabric for orchestration (and more). Power Automate was a great first step but I cannot version control it nor have it as robustly tested nor publish changes across environments like I can for my Azure Synapse environment for example.</P><P>I believe this is also a great idea for implementation because it would help delegate some of the more complex aspects of data engineering like orchestration and even logging to other third parties like Microsoft. There are other open ideas requesting some of this functionality already like logging. Customers who need that can use an orchestration tool from another third party and allow Data Pipelines focus solely on geospatial data engineering within the ArcGIS ecosystem.</P> Tue, 04 Mar 2025 19:49:50 GMT CarterHughes 2025-03-04T19:49:50Z https://community.esri.com/t5/data-pipelines-ideas/arcgis-data-pipelines-connector-for-azure-data/idi-p/1591930 <P>The&nbsp;<A href="https://www.esri.com/arcgis-blog/products/arcgis-online/announcements/integrate-arcgis-data-pipelines-with-microsoft-power-automate-workflows/" target="_self">ArcGIS Data Pipelines Connector in Power Automate</A> is great! I am certainly going to implement it. However, it is a band-aid solution for users like myself who execute data engineering at the enterprise scale and use Azure Data Factory / Azure Synapse / Microsoft Fabric for orchestration (and more). Power Automate was a great first step but I cannot version control it nor have it as robustly tested nor publish changes across environments like I can for my Azure Synapse environment for example.</P><P>I believe this is also a great idea for implementation because it would help delegate some of the more complex aspects of data engineering like orchestration and even logging to other third parties like Microsoft. There are other open ideas requesting some of this functionality already like logging. Customers who need that can use an orchestration tool from another third party and allow Data Pipelines focus solely on geospatial data engineering within the ArcGIS ecosystem.</P> Tue, 04 Mar 2025 19:49:50 GMT https://community.esri.com/t5/data-pipelines-ideas/arcgis-data-pipelines-connector-for-azure-data/idi-p/1591930 CarterHughes 2025-03-04T19:49:50Z https://community.esri.com/t5/data-pipelines-ideas/arcgis-data-pipelines-connector-for-azure-data/idc-p/1650191#M109 <P>Well said — I agree with this completely. The <STRONG>Power Automate connector</STRONG> is a useful starting point, but for enterprise-scale data engineering it doesn’t offer the same robustness you get with orchestration platforms like <STRONG>Azure Data Factory, Synapse, or Microsoft Fabric</STRONG>.</P><P>Being able to integrate Data Pipelines with those tools would bring huge benefits: proper <STRONG>version control</STRONG>, <STRONG>promotion across environments</STRONG>, <STRONG>robust testing</STRONG>, and enterprise-grade <STRONG>logging/orchestration</STRONG>. That would let Esri focus on the geospatial data engineering strengths, while organizations leverage existing enterprise orchestration platforms for scheduling, monitoring, and governance.</P><P>+1 to this idea — tighter integration with enterprise orchestration tools would make Data Pipelines much more powerful in large-scale environments.</P><P>Regards,<BR />Venkat</P> Fri, 12 Sep 2025 20:16:59 GMT https://community.esri.com/t5/data-pipelines-ideas/arcgis-data-pipelines-connector-for-azure-data/idc-p/1650191#M109 VenkataKondepati 2025-09-12T20:16:59Z