topic Re: Deep learning - invalid argument 0: Sizes of tensors must match except in dimension 0 in ArcGIS API for Python Questions

Deep learning - invalid argument 0: Sizes of tensors must match except in dimension 0

VHolubec — Fri, 29 Nov 2019 16:16:19 GMT

Dear all,

I have issue with deep learning tools. I exported 12 000 training chips from Esri Imagery basemap to TIFF - see settings.png

The TIFFs looks fine as well as the labels file.

On calling data = prepare_data(data_path, class_mapping={1:'Pool'}, batch_size=64) I am getting error - UserWarning: It's not possible to collate samples of your dataset together in a batch.
warn(message)

For data.show_batch() - now I am getting result - RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 19 and 10 in dimension 3 at ..\aten\src\TH/generic/THTensor.cpp:711

I am not sure what does it mean? Does the Export training data tool corrupt some chips? Is there a way how to find out which chips are corrupted? Or how to fix it?

The chips were created in PRO by Export training data tool, so these could be fine.

Thank you for any idea.

Vladimir

Re: Deep learning - invalid argument 0: Sizes of tensors must match except in dimension 0

Anonymous User — Fri, 29 Nov 2019 18:56:56 GMT

Try to export the training data with chip size > 224 and choose jpg or png as your Image format. Also the cell size is very less, i do not know what kind of objects you are try to detect.

Re: Deep learning - invalid argument 0: Sizes of tensors must match except in dimension 0

VHolubec — Mon, 02 Dec 2019 14:52:53 GMT

Dear Sandeep Kumar,

I am learning a model for pools. When I set bigger chip size, I am not able to get to detail like the attached one.

- the size is 200 to 200 pixels and cell size 0.045 (because the map is in WebMercator, I believe it´s in meters - am I right? ) - and the real size of the area is around 10x meters.

If I set the chip to 256x256 and pixel size to 0.2, I don´t think the resul is usable - see

So I don´t know, if there is any other way, how to set the export of training chips more proper way?

The error I mentioned in the initial post - have you seen this? It seems for me some issue with the chips themselves. - RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. pls help - PyTorch Forums

but I have no idea, how to identified the bad chips.

Thank you for any ideas.

Regards,

Vladimir

Re: Deep learning - invalid argument 0: Sizes of tensors must match except in dimension 0

Anonymous User — Mon, 02 Dec 2019 15:09:50 GMT

You can try a bigger chip size like 256px and for pools i would suggest a cell size of around 50cm (.5 in webmercetor) so that the model can learn the surrounding context too.

Re: Deep learning - invalid argument 0: Sizes of tensors must match except in dimension 0

VHolubec — Mon, 02 Dec 2019 19:37:38 GMT

Dear Sandeep Kumar, with this settings it works. The problem is, that it seems to me so coarse (in comparision to previous settings) - see

and the batch looks like this - no so ideal - no on every chips the pools is highlighted.

Please could you explain, why the ptevious settings did not work? Is there any limit to chip size and number of chips, which Python API could handle?

Thank you,

Vladimir

Re: Deep learning - invalid argument 0: Sizes of tensors must match except in dimension 0

Anonymous User — Wed, 04 Dec 2019 09:36:09 GMT

There is no limit on the chip number, but with a greater chip size you would require more memory to handle it. the show batch looks reasonable to me, to get a little bit zoomed in chips you can try reducing the cell size to 0.3 or 0.4.