Train Deep Learning Model for Building Footprints

[BC_Admin509]

From a previous issue we were suggested to run the Deep Learning tool set from the beginning and create our own model based on our own building footprints and imagery instead of using a pre trained model as this ran into errors with larger buildings.

My current workflow is utilizing an area of 10.7 sq miles on a large portion of the city of Kennewick with 12,000 building footprints pre created. The imagery is captured by EagleEye  at 12in resolution with 4 bands (4th band is IR). When I reach the train Deep Learning Model tool, it will run for some time in both the Arc Pro 2.8 and 2.7 environments but will error after some time stating that imagery does not contain label files. This is also using the 64 gb virtual machine CPU and not a local GPU on windows 10 64bit and using a duplicated python environment with the deep learning environment packages installed on both for each different version from ESRI website.

Per the error pasted below, I can see the images that are missing labels in both the text file and when I compare the label file and image file contents however if I edit these the tool errors out again with a different error. I have tried checking and unchecking the box for "Output no feature files" when using the Export Training Data for Deep Learning tool and receive the same errors for both. I have provided 2 images of both the Export Training data and Train Deep learning tools.

Please check your dataset. 387 images dont have the corresponding label files.
Traceback (most recent call last):
File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 232, in <module>
execute()
File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 207, in execute
show_accuracy=show_accuracy)]
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\arcgis\learn\models\_arcgis_model.py", line 708, in fit
self.learn.fit_one_cycle(epochs, lr, callbacks=callbacks, **kwargs)
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastai\train.py", line 23, in fit_one_cycle
learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastai\basic_train.py", line 200, in fit
fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks)
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastai\basic_train.py", line 99, in fit
for xb,yb in progress_bar(learn.data.train_dl, parent=pbar):
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastprogress\fastprogress.py", line 47, in __iter__
raise e
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastprogress\fastprogress.py", line 41, in __iter__
for i,o in enumerate(self.gen):
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastai\basic_data.py", line 75, in __iter__
for b in self.dl: yield self.proc_batch(b)
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\torch\utils\data\dataloader.py", line 345, in __next__
data = self._next_data()
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\torch\utils\data\dataloader.py", line 385, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastai\data_block.py", line 651, in __getitem__
if self.item is None: x,y = self.x[idxs],self.y[idxs]
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastai\data_block.py", line 120, in __getitem__
if isinstance(idxs, Integral): return self.get(idxs)
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\fastai\vision\data.py", line 271, in get
res = self.open(fn)
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\arcgis\learn\models\_maskrcnn_utils.py", line 137, in open
return ArcGISMSImage.open(fn, div=self._div, imagery_type=self._imagery_type)
File "C:\Users\CK1\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\Lib\site-packages\arcgis\learn\_utils\common.py", line 193, in open
raise Exception(message)
Exception:
Tried opening image using gdal and encountered the following error

'NoneType' object has no attribute 'ReadAsArray'


===================================================================

Tried opening image using pillow and encountered the following error

[Errno 2] No such file or directory: 'G:\\GIS\\Caleb\\Bldg Footprint Test\\Image_Chips\\ImageChips\\images\\000004687.tif'
Failed to execute (TrainDeepLearningModel).