Can I use ArcGIS python API in Azure DataBricks platforms??

[SuraviMahanta]

I went through different ArcGIS tutorial and guide, however I didn't find any way to use ArcGIS with python API in DataBricks.

[DominicStubbins]

I was able to install the arcgis package from PyPI into databricks (azure databricks). just follow the import library workflow and type "arcgis" into the PyPI library box.

However I then ran into this issue https://docs.databricks.com/user-guide/faq/python-command-cancelled.html and have not had a chance to go back and work through this, seems like there is a big overlap in libraries of different versions so there may be challenges..

I also had a bit more success installing the library into the notebook context using

dbutils.library.installPyPI("arcgis")
dbutils.library.restartPython()

in the first notebook cell.

[SimonGIS]

Is this still the preferred approach for getting ArcGIS API for Python into Databricks?  I have Geoanalytics Engine all setup in a Databricks Workspace on Azure, but just looking for steps on how to incorporate the ArcGIS API for Python as well.

I assumed I would be able to install at the cluster level so all notebooks would be able to import it when required:

arcgis==2.2.0

However this fails

[Roxana_ElenaUrdea]

Hi Simon, I also have issues on using arcgis library on Databricks cluster. Even the library seem to be correctly installed, fails importing arcgis.

Does anyone has similar issues?

Thank you,

Roxana

[EsriLocalLogic]

@SimonGIS @Roxana_ElenaUrdea I was able to get the arcgis library installed on Databricks by running the command sudo apt-get install libkrb5-dev in a terminal. You can open a terminal window by clicking the little icon in the bottom right of a Databricks Notebook page (see attached image). After that, I was able to run %pip install arcgis in a notebook cell, and it installed okay (albeit with some non-critical warnings). Hope that helps!

[Koen]

Hey @EsriLocalLogic ,

This seems like an acceptable resolution for the time being, however, you must always run this command every time you load the Databricks cluster. Do you think it's possible to avoid this? Have you been able to figure out a more permanent resolution? 

[Roxana_ElenaUrdea]

Hello @SimonGIS and thank you for your reply!

Using the terminal it works but you must always run this command every time you load the Databriks cluster. Do you think it's possible to avoid it?

Thank you,

Roxana