Errors with dependency installation and Deep Learning Tool model availability in Anaconda 3.8

[JackCharde1]

Hey everyone (and @KarthikDutt, who asked me to post this here!)

I ran into an issue while using Jupyter Notebook and arcgis.learn for the first time while following this Esri Youtube Video

I found the github location for the workflow and tried following the documentation but ran into dependency installation issues and deep learning tool model existence in certain versions of Anaconda.

I could not get this to work in a Jupyter Notebook. Maybe someone might have some insight!

I downloaded Anaconda, version 3.8, and installed ArcGIS API for Python (arcgis) per the blog "Three Ways to Get Jupyter Notebooks and the ArcGIS API for Python".

With my data in a notebook, I tried according to the video at timestamp 2:07:

ln[1]: from arcgis.learn import prepare_data, MultiTaskRoadExtractor

ln[2]: data_path=..... <<(I selected the path where my image chips were stored)

ln[3]: data = prepare_data(data_path, chip_size=256, batch_size=135)

I received a traceback error on ln[3] saying I was missing a number of dependencies required to use MultiTaskRoadExtractor. So, I opened the Anaconda command prompt and tried installing them, only to be told that there were a bunch of conflicts that prevented install.

So I researched how to create new Anaconda environments, and created one for Python 3.6, because some forums said some dependencies like earlier versions better. I installed arcgis so I could access Jupyter Notebooks in Python 3.6. Also, in my Python 3.6 environment, the dependencies now installed without issue. But then when I tried the above code again in a 3.6 Notebook, ln[1] returned a traceback error saying:

"ImportError: cannot import name 'MultiTaskRoadExtractor' from 'arcgis.learn' (C:\Users\JohnC\anaconda3\envs\snakes\lib\site-packages\arcgis\learn\__init__.py)"

To see if there was a model existence issue, I compared the 'lib\site-packages\arcgis\learn\__init__.py" files between environments, and the Python 3.6 environment did not have the MultiTaskRoadExtractor model. Version 3.8 does.

So I tried creating another anaconda environment in Version 3.7 to see if I could install these model dependencies and see if the MultiTaskRoadExtractor model was available for import into a Notebook. It also was not. It seems only Version 3.8 has this model, but the dependencies also won't download to the 3.8 environment.

So I'm unsure what to do, and I've exhausted my level of knowledge with this stuff (which is next to nothing).

Do you have any suggestions? Is there more information you need from me to answer this? Let me know and thank you in advance.

[JackCharde1]

And here I've attached snippets of each of the environments  arcgis.learn __init__.py files to show that only v3.8 inlcudes MultiTaskRoadExtractor. v3.6 and v3.7 don't. Maybe there's something I'm missing?

[Anonymous User]

I tried creating a new environment and it worked fine for me, I will suggest you to try this:

1. Clean existing conda packages by the following command

  `conda clean -a`

2. Create a new python 3.7 environment and try installing arcgis with all deep learning dependencies there.

Make sure the latest version of the Python API for ArcGIS gets installed, for me it was 1.8.4.

[JackCharde1]

@Anonymous User I just attempted this, per your instruction.

I created a fresh 3.7 environment in conda, installed arcgis (it was 1.8.4, like yours), and then installed the dependencies, according to the very bottom of this link. 

The attachment "Pre_DL_..." shows that the MultiTaskRoadExtractor model is present when installing arcgis 1.8.4.  

When I went to install the deep learning dependencies, conda said that arcgis would be downgraded, from 1.8.4 --> 1.8.2 (Attachment: "arcgis_will_down..."). I said yes, and it installed the dependencies. 

I went back to the __init__.py file after I installed the dependencies, and from my computer's File Explorer, it shows that MultiTaskRoadExtractor is still available (Attachment: "Post_DL_...").

However, when I use conda to open Jupyter Notebook in my browser, and run the code:

             ln[1]: from arcgis.learn import prepare_data, MultiTaskRoadExtractor

I get a traceback error saying it cannot import the model. Searching for the arcgis.learn __init__.py file through the jupyter notebook directory, I find that the model is not available (Attachment: "JupyterNotebook__...").

Now I should mention that I can open a notebook from within ArcGIS Pro and get all of this to work on the base environment. But it just doesn't want to work in an external 3.7 environment through Jupyter Notebook.

[Anonymous User · Accepted solution]

Hi @JackCharde1 

Can you replace the arcgis API version in the installation command and try again.

conda install -c esri -c fastai -c pytorch arcgis=1.8.4 scikit-image=0.15.0 pillow=6.2.2 libtiff=4.0.10 fastai=1.0.60 pytorch=1.4.0 torchvision=0.5.0 --no-pin

Thanks,

Sandeep

[JackCharde1]

That worked! Thank you so much @Anonymous User !

I just checked it in a browser-based notebook and encountered no errors running the successive lines of code.

Next step: get more CUDA!

Cheers 👍🍻