Error when training deep learning model

[SteveCacioppo]

I'm trying to train my model and am getting the following error:

Start Time: Friday, August 19, 2022 8:59:06 AM

Traceback (most recent call last):
  File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 308, in execute
    data_bunch = prepare_data(in_folders, working_dir=out_folder, **prepare_data_kwargs)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\arcgis\learn\_data.py", line 1822, in prepare_data
    ArcGISInstanceSegmentationItemList.from_folder(path / "images")
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\fastai\data_block.py", line 479, in _inner
    self.train = ft(*args, from_item_lists=True, **kwargs)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\fastai\data_block.py", line 303, in label_from_func
    return self._label_from_list([func(o) for o in self.items], label_cls=label_cls, **kwargs)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\fastai\data_block.py", line 279, in _label_from_list
    y = label_cls(labels, path=self.path, **kwargs)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\arcgis\learn\models\_maskrcnn_utils.py", line 97, in __init__
    if is_no_color(list(color_mapping.values())):
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\arcgis\learn\models\_maskrcnn_utils.py", line 70, in is_no_color
    return (np.array(color_mapping) == [-1.0, -1.0, -1.0]).any()
AttributeError: 'bool' object has no attribute 'any'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 390, in <module>
    execute()
  File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 384, in execute
    del data_bunch
UnboundLocalError: local variable 'data_bunch' referenced before assignment

Failed script (null)...

Failed to execute (TrainDeepLearningModel).

Failed at Friday, August 19, 2022 8:59:11 AM (Elapsed Time: 4.73 seconds)

I think this is what's causing the model to fail, but I'm not sure how to rectify this issue.

AttributeError: 'bool' object has no attribute 'any'

Here are the parameters I am using to train the model on a sample area:

I've been messing around with this for over a week and I'm stuck.  Any suggestions?

[SteveCacioppo · Accepted solution]

I finally figured this one out.  First of all, when using the MaskRCNN model for object detection, the class values for the input training data must start at 1 (I was using 0).  Secondly, you need to have at least two class values in order to train the deep learning model when using the MaskRCNN.  Here are the inputs I used that worked:

[AlexaVerónicaGarcíaBarrantes]

Hi Steve 

I'm using PASCAL instead of Mask, do you know if I still need to have two class values?

[byrondelgado]

Hi Steve, 

I read and followed your instructions but still no luck. Please, can you be more specific with the values recommendations for the values assigned to the samples?  Here attached are the messages I get when trying to train the model. 

Thank you and regards...

[PavanYadav]

@byrondelgado Hi, have you been able to get it work. From the attached error, it looks like there was an internet issue. thanks in advance for any details you may share. 

thanks

Pavan

[Omoige23]

Hello, please I am having a similar issue here, I am using CPU and not GPU for this processing. I am aware the CPU can be used. I have reduced batch sizes from 16-12-8-4-2 yet unsuccessful. I will appreciate solutions achieved by anyone who has encountered this before. Thank you

[PavanYadav]

@Omoige23 this just appears wrong to me. If CPU or batch size was a problem, the tool would have returned error/messages. I see it trained up to 20 epochs and at the time of saving the model it failed. Is it possible for your to share the training data? If yes, please email me at pyadav AT esri DOT com. 

The origional post was about

UnboundLocalError: local variable 'data_bunch' referenced before assignment 

 I assume you don't see this error. Right?

[Omoige23]

Thank you Pavan Yadav,

This is the data I tried using to familiarize with the process. https://learn.arcgis.com/en/projects/classify-mangroves-using-deep-learning/

I planned to apply this method to another AOI in the research my student is carrying out. I would be glad to receive any help to guide me in helping my student actualize her dream. 

I have also sent same message to you via LinkedIn so i can connect with you there.

Hope to hear from you soon