Multiple Datasets for Deep Learning - Dataset preparation best practices question

[GEOACEADMIN]

Hi all,

Please forgive my ignorance as I am new to using the Deep Learning toolkits, but I am currently trying to use 1x1m-resolution DEM data in conjunction with orthoimagery to train a model. The orthoimagery, as downloaded from the source, is in 6-inch resolution in a separate coordinate reference system. Does anyone have any recommendations on what should be done to the 6-inch resolution file to optimize it for a machine learning workflow (to be used with DEM data)?

My first thoughts are to make the CRS match, and at least convert from imperial to metric pixel sizes. Should I resample to match the 1x1m (effectively reducing resolution, which isn't desirable if it can be avoided)? Maybe I put it into a .25m x .25m resolution? I'm not sure if any of this matters and am just looking for a nod in the right direction if anyone has a couple minutes to aid.

Thank you!

[PavanYadav]

Hi @GEOACEADMIN Can you share what model architecture or deep learning workflow do you want to use? And, can you share what is the end goal of the project e.g. image translation, object detection, pixel classification or other? I am trying to understand how you will use a DEM and an Image as a training data to train a deep learning model. Will both DEM and image chips participate in the training? 

thanks
Pavan

[GEOACEADMIN]

Hi @PavanYadav thanks for your response. The end goal of the project is for object detection. We want to be able to use DEM-derived products (hillshade as an example) alongside aerial imagery to identify areas where a culvert is likely to be installed. Yes, we'd like DEM (or DEM-derived projects as mentioned; we have to play around with several options a bit...but we know all will be 1mx1m resolution) and image chips to participate in the training.

We're pretty early on in the game and in Machine Learning in general, so model architecture is yet to be determined. If you have any recommendations based off all that info then I'm all ears, or really if you know of any resources which would help us to dive into ML on geospatial data so we can figure out the answers ourselves then I'd certainly give it a look.

[PavanYadav]

Hi @GEOACEADMIN thanks for sharing the details. About the model architectures and initial exploration of the deep learning capabilities, you might find these helpful: 
Detect Objects Tutorial: https://learn.arcgis.com/en/projects/use-deep-learning-to-assess-palm-tree-health/

Deep Learning Models: https://pro.arcgis.com/en/pro-app/latest/tool-reference/image-analyst/overview-of-the-deep-learning-... (this lists supported models with examples and other details) 

About using DEM based product with a an aerial images as a training data is something I have not done. But, if both are important to detect the locations (please think about how a human will use both DEM and aerial image to detect the location). You can give it a try. Generally, you input one image (it can be multispectral e.g. more than 3-bands). At the moment I can only think of creating a composite raster using the DEM and the aerial image. But, I cannot say for sure if this will give good results. I would say try with just aerial image, see if the results are good. It might just work if you have enough samples. next  try with both DEM+aerial. It's an interesting project and I would love to hear what approach you end up using.  All the best!

Cheers

Pavan

Product Engineer | Raster

[GEOACEADMIN]

Many thanks @PavanYadav ! I'll report back when we're on the other side.