Connect Data Pipelines to PostgreSQL

400
6
03-07-2024 07:13 AM
Labels (1)
pzitman
New Contributor

Hi!

Is it possible to connect Data Pipelines to an Azure Database for PostgreSQL (Flexible Server), and is there somewhere a list of databases that are supported by Data Pipelines?

6 Replies
rob_hewy
Occasional Contributor

Not sure how to up vote this but I gave it 1 kudo

Sarah_Hanson
Esri Contributor

Currently, ArcGIS Data Pipelines is not able to establish a connection to Azure Database for PostgreSQL. The list of supported inputs, including databases, can be found here: Work with input data.

For more context, Data Pipelines is only able to connect to databases that are accessible over the open internet, and is not able to establish connection to data stores that are behind an organization's firewall. If we were to add support for connecting to Azure Database for PostgreSQL, or any relational database management system, the organization would need to adjust their security settings to allow database connections originating from outside of their network. Would your organization be open to allowing this?

If yes, please create a Data Pipelines Idea to suggest this functionality for a future release. Once your idea is created, other community members who also would like to see the idea realized will be able to upvote it by giving it kudos.

Thank you!

Sarah

0 Kudos
rob_hewy
Occasional Contributor

What if the database is public but over at amazon?

0 Kudos
Sarah_Hanson
Esri Contributor

If the database can be configured to allow connections that originate outside of the network, but it is different than Azure Database for PostgreSQL, I would ask that you please create a separate idea detailing your needs as well. Similar to how ArcGIS Pro and Enterprise support a finite list of cloud-based database services (relevant documentation here), Data Pipelines would be expected to do the same. Thanks!

0 Kudos
rob_hewy
Occasional Contributor

The one I am thinking of is in RDS. I will test with Pro first.

 

0 Kudos
rob_hewy
Occasional Contributor

A workaround option might be koop as middleware but I not sure if the:

http://geoservices.github.io/query.html 

Is a supported format, could be added as an idea for future data inputs.

https://koopjs.github.io/docs/available-plugins/providers

There are 2 third party providers for PostgreSQL/PostGIS

It would mean a second VM/Container to run koop.

If middleware is an option then pg_featureserv might work too as it outputs GeoJSON but you would still need a second VM/Container to run that middleware.

https://access.crunchydata.com/documentation/pg_featureserv/1.2.0/introduction/

 

Would be nice not to need the middleware.

 

 

 

 

0 Kudos