Hi!
Is it possible to connect Data Pipelines to an Azure Database for PostgreSQL (Flexible Server), and is there somewhere a list of databases that are supported by Data Pipelines?
Not sure how to up vote this but I gave it 1 kudo
Currently, ArcGIS Data Pipelines is not able to establish a connection to Azure Database for PostgreSQL. The list of supported inputs, including databases, can be found here: Work with input data.
For more context, Data Pipelines is only able to connect to databases that are accessible over the open internet, and is not able to establish connection to data stores that are behind an organization's firewall. If we were to add support for connecting to Azure Database for PostgreSQL, or any relational database management system, the organization would need to adjust their security settings to allow database connections originating from outside of their network. Would your organization be open to allowing this?
If yes, please create a Data Pipelines Idea to suggest this functionality for a future release. Once your idea is created, other community members who also would like to see the idea realized will be able to upvote it by giving it kudos.
Thank you!
Sarah
What if the database is public but over at amazon?
If the database can be configured to allow connections that originate outside of the network, but it is different than Azure Database for PostgreSQL, I would ask that you please create a separate idea detailing your needs as well. Similar to how ArcGIS Pro and Enterprise support a finite list of cloud-based database services (relevant documentation here), Data Pipelines would be expected to do the same. Thanks!
The one I am thinking of is in RDS. I will test with Pro first.
A workaround option might be koop as middleware but I not sure if the:
http://geoservices.github.io/query.html
Is a supported format, could be added as an idea for future data inputs.
https://koopjs.github.io/docs/available-plugins/providers
There are 2 third party providers for PostgreSQL/PostGIS
It would mean a second VM/Container to run koop.
If middleware is an option then pg_featureserv might work too as it outputs GeoJSON but you would still need a second VM/Container to run that middleware.
https://access.crunchydata.com/documentation/pg_featureserv/1.2.0/introduction/
Would be nice not to need the middleware.