topic Re: Can I use ArcGIS python API in Azure DataBricks platforms?? in ArcGIS API for Python Questions

Can I use ArcGIS python API in Azure DataBricks platforms??

SuraviMahanta — Tue, 14 May 2019 06:19:58 GMT

I went through different ArcGIS tutorial and guide, however I didn't find any way to use ArcGIS with python API in DataBricks.

Re: Can I use ArcGIS python API in Azure DataBricks platforms??

DominicStubbins — Wed, 15 May 2019 12:22:37 GMT

I was able to install the arcgis package from PyPI into databricks (azure databricks). just follow the import library workflow and type "arcgis" into the PyPI library box.

However I then ran into this issue https://docs.databricks.com/user-guide/faq/python-command-cancelled.html and have not had a chance to go back and work through this, seems like there is a big overlap in libraries of different versions so there may be challenges..

I also had a bit more success installing the library into the notebook context using

dbutils.library.installPyPI("arcgis")
dbutils.library.restartPython()

in the first notebook cell.

Re: Can I use ArcGIS python API in Azure DataBricks platforms??

SimonGIS — Tue, 03 Oct 2023 06:56:39 GMT

Is this still the preferred approach for getting ArcGIS API for Python into Databricks? I have Geoanalytics Engine all setup in a Databricks Workspace on Azure, but just looking for steps on how to incorporate the ArcGIS API for Python as well.

I assumed I would be able to install at the cluster level so all notebooks would be able to import it when required:

arcgis==2.2.0

However this fails

Re: Can I use ArcGIS python API in Azure DataBricks platforms??

Roxana_ElenaUrdea — Thu, 28 Mar 2024 10:31:40 GMT

Hi Simon, I also have issues on using arcgis library on Databricks cluster. Even the library seem to be correctly installed, fails importing arcgis.

Does anyone has similar issues?

Thank you,

Roxana

Re: Can I use ArcGIS python API in Azure DataBricks platforms??

EsriLocalLogic — Fri, 12 Apr 2024 00:45:21 GMT

@SimonGIS @Roxana_ElenaUrdea I was able to get the arcgis library installed on Databricks by running the command sudo apt-get install libkrb5-dev in a terminal. You can open a terminal window by clicking the little icon in the bottom right of a Databricks Notebook page (see attached image). After that, I was able to run %pip install arcgis in a notebook cell, and it installed okay (albeit with some non-critical warnings). Hope that helps!

Re: Can I use ArcGIS python API in Azure DataBricks platforms??

Roxana_ElenaUrdea — Mon, 29 Jul 2024 16:04:40 GMT

Hello @SimonGIS and thank you for your reply!

Using the terminal it works but you must always run this command every time you load the Databriks cluster. Do you think it's possible to avoid it?

Thank you,

Roxana

Re: Can I use ArcGIS python API in Azure DataBricks platforms??

Koen — Thu, 22 Aug 2024 19:31:10 GMT

Hey @EsriLocalLogic ,

This seems like an acceptable resolution for the time being, however, you must always run this command every time you load the Databricks cluster. Do you think it's possible to avoid this? Have you been able to figure out a more permanent resolution?

Re: Can I use ArcGIS python API in Azure DataBricks platforms??

NicholasGiner1 — Wed, 21 May 2025 20:35:59 GMT

Hi - yes we have this capability now. Please see the following blog:

https://www.esri.com/arcgis-blog/products/api-python/announcements/new-use-the-arcgis-api-for-python-in-databricks-notebooks