topic Re: SSD model trained on colab got errors on ArcGIS Pro in ArcGIS API for Python Questions

SSD model trained on colab got errors on ArcGIS Pro

KyawNaingWin — Mon, 29 Mar 2021 06:17:25 GMT

I am following a use-deep-learning-to-assess-palm-tree-health tutorial. Since I do not have GPU on my computer, I used colab to train the SSD model with arcgis.learn and it was successfully completed.

When I use the trained model (either .emd or dlpk) in "Detect Objects Using Deep Learning" tool, i got error right after model definition selection.

ERROR 002667 Unable to initialize python raster function with scalar arguments. [D:\ESRI_DL\palm_tree_health\ImageChips\models\palmtree_detector_ssd25\ArcGISObjectDetector.py]
Traceback (most recent call last):
File "C:\Users\Kyaw\AppData\Local\Temp\ArcGISProTemp21936\palmtree_detector_ssd25.dlpk\ArcGISObjectDetector.py", line 166, in initialize
self.child_object_detector.initialize(model, model_as_file)
File "C:\Users\Kyaw\AppData\Local\ESRI\conda\envs\arcgispro-py3-dl\Lib\site-packages\arcgis\learn\models\_inferencing\_DynamicSSD.py", line 195, in initialize
self.ssd = SingleShotDetector.from_emd(data=None, emd_path=model)
File "C:\Users\Kyaw\AppData\Local\ESRI\conda\envs\arcgispro-py3-dl\Lib\site-packages\arcgis\learn\models\_ssd.py", line 346, in from_emd
ssd = cls(data, emd['Grids'], emd['Zooms'], emd['Ratios'], pretrained_path=str(model_file), backend=backend, backbone=backbone, ssd_version=ssd_version)
File "C:\Users\Kyaw\AppData\Local\ESRI\conda\envs\arcgispro-py3-dl\Lib\site-packages\arcgis\learn\models\_ssd.py", line 228, in __init__
self.load(pretrained_path)
File "C:\Users\Kyaw\AppData\Local\ESRI\conda\envs\arcgispro-py3-dl\Lib\site-packages\arcgis\learn\models\_arcgis_model.py", line 1300, in load
raise e
File "C:\Users\Kyaw\AppData\Local\ESRI\conda\envs\arcgispro-py3-dl\Lib\site-packages\arcgis\learn\models\_arcgis_model.py", line 1298, in load
self.learn.load(name, purge=False)
File "C:\Users\Kyaw\AppData\Local\ESRI\conda\envs\arcgispro-py3-dl\Lib\site-packages\fastai\basic_train.py", line 269, in load
state = torch.load(source, map_location=device)
File "C:\Users\Kyaw\AppData\Local\ESRI\conda\envs\arcgispro-py3-dl\Lib\site-packages\torch\serialization.py", line 527, in load
with _open_zipfile_reader(f) as opened_zipfile:
File "C:\Users\Kyaw\AppData\Local\ESRI\conda\envs\arcgispro-py3-dl\Lib\site-packages\torch\serialization.py", line 224, in __init__
super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
AttributeError: 'WindowsPath' object has no attribute 'tell'

Unable to initialize python raster function with scalar arguments.

I tested with the pre-trained model provided on the tutorial page, it works without any problem.

What could be the problem? Has anybody train SSD on colab for ArcGIS Pro?

Re: SSD model trained on colab got errors on ArcGIS Pro

mvandoornik — Wed, 31 Mar 2021 08:33:07 GMT

I have the same error (also trained my model on Colab). In the past I fixed the error by upgrading my PyTorch package from 1.4.0 to 1.6.0. You could try this, I'm curious if that will work for you. With the pre-trained models from ESRI this upgrade is not necessary, for some reason.. I have no idea why.

Re: SSD model trained on colab got errors on ArcGIS Pro

KyawNaingWin — Wed, 07 Apr 2021 05:12:54 GMT

Thanks for sharing your experience.

I tried as you suggested but getting same error.

I set epochs =100 and use early_stopping to get the best model in all trainings. The model can detect palm trees on sub_set of image (about 1200 x 600) on the colab but miss a lot. It either did not work on ArcGIS Pro.

Re: SSD model trained on colab got errors on ArcGIS Pro

mvandoornik — Tue, 06 Apr 2021 09:03:23 GMT

Last week I managed to fix the error again (I'm currently working on a new laptop with ArcGIS Pro 2.7).

The error was solved after upgrading both Torch and Torchvision. It might be the Torchvison upgrade that fixed the error. I used the following command to get a stable CUDA-enabled version of both packages:

python -m pip install torch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html

Hope this will help you.

Re: SSD model trained on colab got errors on ArcGIS Pro

KyawNaingWin — Thu, 08 Apr 2021 04:31:37 GMT

Not succeeded yet. Will try again.

Yesterday, I trained the model on AGOL notebook where I can get a pre installed run time with GPU costing 0.5 credit per min.

Used the same codes I had tried on the Colab. All the modules needed for ArcGIS deep learning training are already installed there. The model trained on AGOL notebook work on my ArcGIS Pro. Currently AGOL notebook runtime use Arcgis 1.8.1 module. I have retrieved all the module names of the runtime and will install them on the Colab and test again.

Best

Re: SSD model trained on colab got errors on ArcGIS Pro

DanPatterson — Fri, 16 Apr 2021 03:23:59 GMT