topic Re: Classify Pixels Deep Learning Package - Sentinel 2 in ArcGIS Image Analyst Questions

Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Sun, 07 Mar 2021 20:11:14 GMT

Hi all,

I've been trying to put to work the deep learning package corine_landcover.dlpk, as available in Esri Living Atlas. I'm having a unexpected result from the deep learning process, without no warnings or errors identified by ArcGIS Pro. I've tested 2 approaches:

- Scenario 1: raster mosaic, sentinel 2 type, with all bands template, using SRS WGS 1984 UTM Zone 29N.

- Scenario 2: raster dataset, Sentinel True Colour, exported R::Band 4; G::Band 3; B::Band 2, SRS Web Mercator.

Scenario 1, my first approach, I've experimented several batch_size, from 4, 8 to 16. I've selected Mosaicked Image and Process All Rasters single. batch_size parameter has not affected the results. Processor type (GPU or CPU), has affected the results. Both are not understandable.

All processing have took account the visible map area as input parameter for the inferencing.

Starting point:

Scenario 1 - GPU - Mosaicked Image / Batch size 4/GPU:

Scenario 1 - CPU - Mosaicked Image / Batch size 4/CPU:

For scenario 2 same results, GPU and CPU.

For the GPU case I don't have a clue what the issue is. I've ArcGIS Pro Advanced 2.7.1, Image Analyst ext, Deep Learning Framework (with all the requirements, including visual studio). My GPU is RTX 3070 8 Gb who I consider has enough power to do the math. No errors or warnings are presented in the inference processing.

For the CPU the results are clearly not adequate. The test area is about 145.6716km². Most of the classification is Inland Waters, and that doesn't make any sense has you see above in the sentinel 2 scene. I know the model is Unet and should be using GPU, but nevertheless I thought it could be SRS (WGS 1984 UTM Zone 29N) the issue.

I tested again, scenario 2, the below image is GPU processing

CPU mode was about the same.

I understand that the Pytorch version in use in ArcGIS Pro supports only CUDA 10.2 and no CUDA 11.x GPU's. I've seen around the threads issues alike, with different graphic cards, Touring and Pascal, and this type of issue are happening in pixel classification using deep learning.

The CPU results, I don't understand them. I have an i5-10600K CPU with 32 Gb RAM.

Any comments on this?

Re: Classify Pixels Deep Learning Package - Sentinel 2

Anonymous User — Mon, 08 Mar 2021 05:14:49 GMT

Hi @TiagoCarvalho1979 ,

I would recommend you to:

1. Use this tool here Manage Sentinel-2 imagery (arcgis.com) to create your mosaic layer, It expects sentinel 2 Level 1 C data.

2. Change the processing template of the newly created layer to None. (Right click on the layer > Properties > Processing template)

3. Try using the layer with the model.

Let me know if you face any issues.

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Mon, 08 Mar 2021 11:21:12 GMT

Hi @Anonymous User

well in GPU processing I've got the same result.

So for using the tool I had to install ArcMap. In my understanding the Create Mosaic in ArcGIS Pro has the capability to configure a Sentinel 2 template and import data. But I followed all the steps just as a precaution.

The first import option in the tool (import 10 m band) in the deep learning model didn't work, because the model tries to get the index bands for inference and fails in the import process if not all bands are present. So I had to select in the tool the multispectral option.

With the multispectral option the model started and did not presented errors.

But as you see above the results are still the same.

Any clues?

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Mon, 08 Mar 2021 11:25:32 GMT

@Anonymous User one thing, not all the processing was wrong. See this upper left corner in the image:

Could be the tiles? Any thoughts on this?

Thanks

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Mon, 08 Mar 2021 11:37:20 GMT

I also done a CPU based inference. The results are less granular, although I've delimited the inference based on a polygon of a pre-defined AoI. The results:

Now for the upper left limit in my previous post, compare btw GPU and CPU processing:

Re: Classify Pixels Deep Learning Package - Sentinel 2

Anonymous User — Mon, 08 Mar 2021 12:26:18 GMT

Hi @TiagoCarvalho1979 ,

This tool here https://www.arcgis.com/home/item.html?id=e6e1f20cb0374d28a6eed24f5c2ff51b works in ArcGIS Pro too.

Then you need to use the multispectral layer with processing template set to None.

If you think it works well on CPU the problem might be related to GPU, I recommend you to try setting batch size to one and run on GPU. Just to test it.

Thanks

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Mon, 08 Mar 2021 12:14:20 GMT

Hi @Anonymous User ,

the link above is not working:

I tested with batch size = 1. The processing is below:

Again upper left corner did work:

My issue related with CPU processing is that the granularity of the processing is lesser. See bellow the same are by CPU inference:

classification is not the same.

@Anonymous User do you consider that since pytorch version 1.4.0 does not support CUDA 11.x that this is a issue related to more recent Ampere architecture GPU cards?

Thank you for your time.

Re: Classify Pixels Deep Learning Package - Sentinel 2

Anonymous User — Mon, 08 Mar 2021 12:30:01 GMT

Hi @TiagoCarvalho1979 ,

I am not sure about your GPU related question because I have not tested it with that specific GPU. About your question related to granularity you can check the resolution of the raster. Also I have fixed the link, it is same tool and that works with ArcGIS Pro too You don't need arcmap for that if you are already using ArcGIS Pro.

Thanks

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Mon, 08 Mar 2021 13:10:19 GMT

Hi @Anonymous User

both rasters have the same pixel value 10 meters. So my conclusion is how the built-in libraries models that handle the data work for the differente cases. Since I don't have another GPU, Touring or Pascal, available I cannot test the model using other type of GPU.

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Mon, 08 Mar 2021 17:58:57 GMT

Hi all,

new feedback for the discussion. What happens is something like this:

so in the example above the GPU is working. I was identifying if the problem was regarding to the pre-defined extent of the processing. So looking at the CUDA chart the GPU is processing without any problems of memory neither GPU. Then a peak happens in CUDA and the processing breaks with the above results showing. This is a strange behaviour. I use my GPU for 3D rendering and analysis and nothing like this happens, and I have all drivers update.

Any comments on this?

Re: Classify Pixels Deep Learning Package - Sentinel 2

TimG — Tue, 09 Mar 2021 03:36:25 GMT

I got strange results but found the following helped:

-Change the processing template to None of the original Sentinel View (as mentioned)

-Right-Click on the Sentinel View layer -> Data -> Export Raster

-Set the Clipping Geometry to "Current Display Extent" (as a test)

-Create the TIFF with these settings, Pixel Type to 32 bit unsigned and Output Format to TIFF

The nvidia-smi.exe output

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Wed, 10 Mar 2021 12:02:09 GMT

Hi Tim,

I will try to do your approach but I can give the feedback that a colleague of mine with RTX3000 (laptop Touring architect) using my workflow has with success did inferencing on the data, without converting.

As I see from your screenshot you have and RTX 2080, so you also have a touring GPU. Since pytorch 1.4.0 supports only CUDA 10.2, and because CUDA 10.2 is not supported in new Ampere GPU although having CUDA 11 installed the cudnn/cudatoolkit in use will be 7.x/10.x.

I will try to do your workflow and give feedback.

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Wed, 10 Mar 2021 22:36:06 GMT

Hi @TimG

did create the new raster (export to 32 bits unsigned). Same result. Thanks for the help.

Re: Classify Pixels Deep Learning Package - Sentinel 2

TimG — Wed, 10 Mar 2021 23:15:05 GMT

Hmmm, this is mine (image below). The left side 32bit unsigned, right side 8bit unsigned. Other thoughts

- Your 'Map' set to WGS 1984 Web Mercator?

- Cell size set to 10 on the Classify Pixels for Deep Learning?

Re: Classify Pixels Deep Learning Package - Sentinel 2

TiagoCarvalho1979 — Thu, 11 Mar 2021 19:51:46 GMT

Hi @TimG

the CUDA issue just happens, in 16bit and 32bit unassigned. 8 bit even worse. Really I don't have a clue why this issue happens.

Any comments?

Thanks