Deep learning with multiband rasters containing morphologic data

[Anonymous User]

Hello there!

I am currently working with Arcgis Pro (I think Version 2.7.1) and i want to classify cemented sand areas on a dry beach via deep learning, pixel based. I took UAV-based images and have now constructed a range of Geodata from this data: Orthophoto, DEM, TerrainRuggednessIndex and others. My target is to not only use the 3-band-Orthophoto for training and classifying, but the morphologic data (DEM, Ruggedness etc.) too. I wanted to achieve this by generating a Multiband raster containing the 3 RGB-bands AND the morphologic data, and then use this several-layer-strong Multiband-Raster to train my model.

My Question is now, does the Deep Learning Toolset in Arcgis Pro use all rasterbands in the given Rasterdata to classify pixels? As far as I know, Deep learning is not dependent on the actual depiction of the raster in the data frame. But I didn't find answers yet, if it is only usable for 3-bands-strong rgb pictures.

It would be very kind if somebody could help me out!

Best regards

Lorenz Siebrecht, Germany

[Tim_McGinnes]

Hi Lorenz,

Yes, multiband\mutispectral imagery does work with deep learning - for example there are many examples of landcover classification using multiband landsat or sentinel.

There is a good page here with some useful info: Working with Multispectral Data 

[Anonymous User]

Hello Tim,

first: thank you very much for your answer!

So this works even with, lets say, a Digital Elevation Model as an additional raster band? Could I use only an single-band Elevation Model raster as an Input for Deep learning? I always thought "multispectral imagery" does only reference of different spectral ranges that are recorded; I just want to clarify that I don't want to use images of any kind, but morphologic raster data like DEMs, curvature rasters, Surface roughness etc.

I would be thankful if you could clarify your answer regarding this idea.

Best regards!

[Tim_McGinnes]

Yes, it should work for any numerical input data. The way I think of it is that the deep learning model is just learning the relationship between the numerical values in the bands, so it shouldn't really matter what those numbers represent. Now, how well it works is the difficult bit.

The most relevant deep learning ArcGIS example I could find is this one: Land cover classification using sparse training data, which while still having RGB does have a fourth band being used to train the model. It also is trying to detect individual classes against background nodata, which sounds similar to your situation.

There is also good info in the following multispectral examples:

Detecting settlements using supervised classification and deep learning 

Landcover mapping using hyperspectral imagery and deep learning 

I think the main issue you might have is that the commonly used backbone weights (ResNet etc.) were all originally trained on 3 band RGB data, so might not train very well if you are using data that is very different. Have a look at the Train the Model section from that second link above for info on this.