Can I use ArcGIS python API in Azure DataBricks platforms??

[SuraviMahanta]

I went through different ArcGIS tutorial and guide, however I didn't find any way to use ArcGIS with python API in DataBricks.

[DominicStubbins]

I was able to install the arcgis package from PyPI into databricks (azure databricks). just follow the import library workflow and type "arcgis" into the PyPI library box.

However I then ran into this issue https://docs.databricks.com/user-guide/faq/python-command-cancelled.html and have not had a chance to go back and work through this, seems like there is a big overlap in libraries of different versions so there may be challenges..

I also had a bit more success installing the library into the notebook context using

dbutils.library.installPyPI("arcgis")
dbutils.library.restartPython()

in the first notebook cell.

[SimonGIS]

Is this still the preferred approach for getting ArcGIS API for Python into Databricks?  I have Geoanalytics Engine all setup in a Databricks Workspace on Azure, but just looking for steps on how to incorporate the ArcGIS API for Python as well.

I assumed I would be able to install at the cluster level so all notebooks would be able to import it when required:

arcgis==2.2.0

However this fails