topic Re: Export training data for deep learning in ArcGIS API for Python Questions

Export training data for deep learning

MattiasTivlin — Tue, 05 Nov 2019 11:00:37 GMT

Hi all!

I'm trying to learn about python and deep learning. I have tried to follow the sample notebooks and watch the training courses but now i'm stucked.

I have done some training samples in Arcgis Pro with the training samples manager and saved it as a shapefile.

Then i export the samples with GP-tool "Export Training Data For Deep Learning" with KITTI Labels as Meta Data Format.

In jupyter notebook i write this to load the samples

from arcgis.learn import prepare_data
data = prepare_data('Path to my samples')
data.show_batch

The error i get

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone4\lib\site-packages\arcgis\learn\_data.py in prepare_data(path, class_mapping, chip_size, val_split_pct, batch_size, transforms, collate_fn, seed, dataset_type, resize_to, **kwargs)

572 return ner_prepare_data(dataset_type=dataset_type, path=path, class_mapping=class_mapping, val_split_pct=val_split_pct)

573 else:

--> 574 raise NotImplementedError('Unknown dataset_type="{}".'.format(dataset_type))

575

576 if dataset_type == 'RCNN_Masks':

NotImplementedError: Unknown dataset_type="KITTI_rectangles".

What am i doing wrong?

Regards Mattias

Re: Export training data for deep learning

DanPatterson_Retired — Tue, 05 Nov 2019 11:34:33 GMT

The error suggests that the details are to the path name you used. If it was a shapefile did you add the *.shp file extension? In any event more information on the file type and path name are needed

Re: Export training data for deep learning

MattiasTivlin — Tue, 05 Nov 2019 11:51:24 GMT

Here's the export tool and to the right is the folders with images that was created.

data = prepare_data('Output Folder/chips') so how should prepare_data look like?

Re: Export training data for deep learning

MattiasTivlin — Tue, 05 Nov 2019 11:54:29 GMT

I have tried to follow this seminar

https://www.esri.com/training/catalog/5d8a8d4155cf937306dba7d9/arcgis-deep-learning-tools-for-imagery/

Re: Export training data for deep learning

DanPatterson_Retired — Tue, 05 Nov 2019 13:56:51 GMT

you might want to try it through the python interface. The warning beside the output folder name suggests something is amiss

Export Training Data For Deep Learning—Help | ArcGIS Desktop

Re: Export training data for deep learning

MattiasTivlin — Tue, 19 Nov 2019 14:41:23 GMT

I exported the training data via python interface but i got the same error when preparing data

Re: Export training data for deep learning

Anonymous User — Mon, 02 Dec 2019 08:34:09 GMT

Mattias Tivlin‌ what kind of deep learning task do you want to do ? If it is an object detection task, you can try exporting data in the format 'pascal voc objects'.

Re: Export training data for deep learning

Javier_AntonioEscudero — Wed, 04 Dec 2019 16:37:27 GMT

Hi Mattias

Try to use metadata mode PASCAL VOC rectangles instead of KITTY rectangles.
Use PNG Format.
Replace all commas to points for each file in labels folder.

That worked for me.

Javier Escudero

Re: Export training data for deep learning

MattiasTivlin — Fri, 06 Dec 2019 07:32:55 GMT

Great! I will try that

Re: Export training data for deep learning

MattiasTivlin — Thu, 12 Dec 2019 08:18:03 GMT

Its working now

Changed the comma to Points. Thank you so much for the help

Re: Export training data for deep learning

Anonymous User — Wed, 24 Jun 2020 16:31:00 GMT

This is helpful, but I am still wondering why the KTTI Labels export is not working - could you elaborate on why the PASCAL method works, but KTTI labels does not? Thanks!

Re: Export training data for deep learning

Wimalavan_Iersel — Wed, 12 Aug 2020 14:24:47 GMT

I am also curious about this issue with the KITTI labels. I have the same error as Mattias, but I can't switch to PASCAL since I want to use the training data for pixel classification. Any ideas? My KITTI labels already have points, so that can't fix it either... Thanks!

Re: Export training data for deep learning

Anonymous User — Sat, 22 Aug 2020 17:48:17 GMT

Hi Wimala,

If you want to do pixel classification you can use the format 'Classified Tiles'.

Thanks,

Sandeep

Re: Export training data for deep learning

AlexaVerónicaGarcíaBarrantes — Mon, 06 Nov 2023 19:37:09 GMT

Hi Javier

specifically where do I replace all commas to points? I went fo the folder created after exporting the training data and all the files look like in the picture. I don't see any points or commas, should they be created differently?

Re: Export training data for deep learning

AlexaVerónicaGarcíaBarrantes — Mon, 06 Nov 2023 20:42:45 GMT

Hi Javier

I replied to your thread before but seems like it didn't upload

Specifically, where do I replace the commas to points, after creating the image chips I went to the Label folder and they all look like in the picture. Are they supposed to be in a different format?