Installing conda prerequisites error

[MeshaSagram]

Hello,

I am exploring the deep learning functions in ArcGIS Pro and have started following this tutorial: arcgis-python-api/detecting_swimming_pools_using_satellite_image_and_deep_learning.ipynb at master ·... 

To get started with the arcgis.learn module in python, I need to install the prerequisites. The website says:

Data preparation, augmentation and model training workflows using arcgis.learn have a dependency on PyTorch and fast.ai deep learning libraries. They can be installed using conda as follows:

conda install -c fastai -c pytorch fastai=1.0.39 pytorch=1.0.0 torchvision

But when I enter that into my python window in Arc, I get a syntax error. I tried installing Anaconda and Git, and I entered this into the Anaconda Prompt and it seemed to install everything. But when I continue with the tutorial, I still get the message that it needs to be installed:

data = prepare_data(data_path, batch_size=32)
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\Mesha\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\learn\_data.py", line 132, in prepare_data
_raise_fastai_import_error()
File "C:\Users\Mesha\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\learn\_data.py", line 22, in _raise_fastai_import_error
raise Exception('This module requires fastai, PyTorch and torchvision as its dependencies. Install it using "conda install -c pytorch -c fastai fastai=1.0.39 pytorch=1.0.0 torchvision"')
Exception: This module requires fastai, PyTorch and torchvision as its dependencies. Install it using "conda install -c pytorch -c fastai fastai=1.0.39 pytorch=1.0.0 torchvision"

Is there anyone who can tell me what I need to change so that this works? I can't seem to find any resources online that address this. Thank you!

[TomThompson1]

Hey Sandeep Kumar, 

I'm also seeing this issue with arcgis.learn. I've following the guide in Geospatial deep learning with | ArcGIS for Developers and installed the modules within a new environment successfully. 

When running prepare_data() function from the learn module within Jupyter Notebooks, it throws the _raise_fastai_import_error(), claiming that it requires those modules listed in the original post. However, interestingly, when I run the _data.py file directly within VS Code, it succeeds and no exception is thrown. 

I've confirmed that the environment I'm running is the same between my Jupyter Notebook & VS Code, AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1. So I do not know why the exception occurs in Jupyter Notebooks, but does not in VS Code.

This problem I'm seeing may also be the root cause of why others experience this problem, so I'm posting here with these findings. I'll do some more debugging today and if I find the solution to my problem, I'll post a reply with the fix.

Tom Thompson, Esri Canada Technical Support.

[DanPatterson_Retired]

did you try --no-pin in conda?

[TomThompson1]

Hi Dan Patterson‌,

I did try the --no-pin when installing the modules in my old environment and it did not resolve the problems.

I'll create a third environment and test it again just to make sure there were no dependency miss-matches.

Thanks for the suggestion,

Tom

[TomThompson1]

Hey Dan Patterson,

Tested using the --no-pin and the documented solution to installing these modules:

conda install -c fastai -c pytorch fastai=1.0.54 pytorch=1.1.0 torchvision --no-pin

This did not resolve the problem.

Great idea though, I was hopeful. Thanks again for the suggestion.

Tom

[DanPatterson_Retired]

Sandeep is on to something... you can't to the installation from your command prompt, you have to be in the cloned conda environment

[TomThompson1]

I've been installing all modules through the Python Command Prompt. That screen capture is of VS Codes terminal, so it is a bit misleading.

[TomThompson1]

Some progress on this. I believe this issue is related to the _data.py file. Something, at least on my machine, is causing the import statements within the try/except to stop working.

Specifically, it is the import statement:

from fastai.vision.data import imagenet_stats, ImageItemList

Any statement defined after this line is skipped and fails, and so an exception is thrown and HAS_FASTAI will always equal false.

Interestingly, the interpreter throws no errors. It just exits and appears to have executed successfully. My debugger wasn't working, so I implemented some print statements to test.

try:
 print("---_data.py Modules & Versions---")
 import fastai
 print("fastai installed at version: ",fastai.__version__)
 import torchvision
 print("torchvision installed at version: ",torchvision.__version__)
 import torch
 print("torch installed at version: ",torch.__version__)
 print("Before Import from fastai.vision.data import imagenet_stats, ImageItemList")
 from fastai.vision.data import imagenet_stats, ImageItemList
 print("After Import from fastai.vision.data import imagenet_stats, ImageItemList")‍‍‍‍‍‍‍‍‍‍‍

This is quite bizzare.

What do you guys think?

Tom

[Anonymous User]

Hi Tom,

I see you are using ArcGIS Pro python command prompt, Create a new cloned environment and install the deep learning dependencies using this command  

conda install -c esri arcgis fastai pillow scikit-image --no-pin

[TomThompson1]

Hi Sandeep Kumar,

I did as you suggested. Built a new clone and used the conda command you provided.

Same problems continue. Something to do with the import statements in the _data.py file. Occurs on the import of:

from fastai.vision.data import imagenet_stats, ImageItemList

Nothing is executed after this line in the _data.py and so my Python scripts leveraging the prepare_data() function fail, claiming that I do not have the required modules.

My next test is to update ArcGIS Pro to v2.5.0, as I'm at V2.4.3. Once updated I'll create another fresh clone and use the command you provided.

Tom

[Anonymous User]

tom can you try this, after you import arcgis in the same notebook

import torch 

also just to make sure ipython is working properly try this command in the python prompt

python -m ipykernel install --user