topic Re: Using pre-trained deep learning models with declassified satellite imagery in Imagery and Remote Sensing Questions

Using pre-trained deep learning models with declassified satellite imagery

EMK — Wed, 05 Feb 2025 09:40:51 GMT

Hello everybody.

I am a PhD student trying to apply ESRI's pretrained deep learning models (SAM and/or Agricultural Field Delineation) to declassified satellite imagery, specifically Corona or Hexagon low-resolution mapping camera images.

In a different forum I was told that as these are single-band images, I would need to add texture to them (e.g. integrating Landsat bands to them) to make them multiband images for this type of analysis. Which I did. I was also advised that utilising a DEM that corresponds to my study location and approximate time period (of the imagery) would improve my results. So, I found the appropriate DEM and clipped it to my study area. A different expert said that the deep learning is more effective when DEM-derived products are also included in this analysis, specifically slope and curvature.

I now have 4 different raster images: the textured declassified image, an SRTM DEM, slope and curvature. I uploaded them individually to ArcGIS Online to experiement with the aforementioned deep learning packages. I tried both packages on just the textured image and on a composite band image. And received error messages every single time. I'm not an experienced GIS-user so it is possible that I have made some dumb mistakes in preparing my images for this type of analysis. But at this point, I'm not sure where to even start troubleshooting.

Has anybody here ever successfully used one of these pre-trained models on this type of declassified imagery? Or heard of anybody who has been able to do so? If so, I would appreciate any advice, contacts, workflows, recommended reading, etc. in order to move forward. This data preparation is only a small (although potentially invaluable) part of my total research but I need to be extremely careful to not get bogged down in lengthy, time-consuming tasks (e.g., accumulating individual training samples)...or anything too complicated for somebody that doesn't know coding or in-depth GIS programming.

Thank you, in advance, for any help!

Re: Using pre-trained deep learning models with declassified satellite imagery

BobBooth1 — Wed, 05 Feb 2025 15:54:26 GMT

It may be helpful to you to work through this tutorial to get to understand some of the issues involved in deep learning.

https://learn.arcgis.com/en/projects/improve-a-deep-learning-model-with-transfer-learning/

These models work best on imagery that is like the imagery they were trained on.

SAM may be an option, but I think it also expects three band, RGB imagery.

See this tutorial:

https://learn.arcgis.com/en/projects/detect-objects-with-text-sam/

You might try using a DL tool to convert your black and white imagery to color before using it in one of the DL classification tools.

See this blog post:

https://www.esri.com/arcgis-blog/products/arcgis-pro/design-planning/colorizing-historic-black-and-white-aerial-imagery-using-deep-learning/

Re: Using pre-trained deep learning models with declassified satellite imagery

RavinderVirk — Wed, 05 Feb 2025 16:25:39 GMT

Did you read the pre-trained model imagery input requirements for SAM and Agricultural field delineation models? EX: Ag field delineation pre-trained model uses Input: Sentinel-2 L2A 12-bands multispectral imagery using Bottom of Atmosphere (BOA) reflectance product in the form of a raster, mosaic or image service. [Introduction to the model—ArcGIS pretrained models | Documentation] and SAM uses Input 8-bit, 3-band imagery. The pre-trained models work fine with correct input is used.

Why using AGOL where you have to use credits to run the pre-trained model. Why not use the ArcGIS Pro and use your dataset to train/create a model. This maybe of help: Use the Train Deep Learning Model wizard—ArcGIS Pro | Documentation. NOTE: To do deep learning in Pro, you need to install the deep learning libraries compatible with the Pro version: GitHub - Esri/deep-learning-frameworks: Installation support for Deep Learning Frameworks for the ArcGIS System.

Note: Creating and managing labels is now very easy in Pro [not time consuming :)]. Create and manage labels—ArcGIS Pro | Documentation

Finally, did you create an image composite which has 3 or 12 bands as per model requirements?.. Just check the properties of your dataset [Content > properties > Source > Raster info], and make sure you have 3 bands and they are 8 bit, if using SAM or textSAM. You have to use the composite bands gp tool to combine your raster dataset [R = declassified corona; B = textured image; C = ?] . Once it is 3 band 8 bit, try running the pre-trained model.

Hope this helps a bit.

Re: Using pre-trained deep learning models with declassified satellite imagery

RavinderVirk — Wed, 05 Feb 2025 16:29:53 GMT

"I now have 4 different raster images: the textured declassified image, an SRTM DEM, slope and curvature. I uploaded them individually to ArcGIS Online to experiement with the aforementioned deep learning packages. I tried both packages on just the textured image and on a composite band image."

It seems you still have a raster image which has a single band, minus the texture declassified image where you may have only 2 bands - texture + b/w. [Corona is a single band b/w imagery]. This still does not fit the pre-trained model requirements!

Re: Using pre-trained deep learning models with declassified satellite imagery

BobBooth1 — Wed, 05 Feb 2025 16:50:51 GMT

I also suggest using ArcGIS Pro, rather than ArcGIS Online. This tutorial should help you get set up:

https://learn.arcgis.com/en/projects/get-ready-for-deep-learning-in-arcgis-pro/

Re: Using pre-trained deep learning models with declassified satellite imagery

EMK — Wed, 05 Feb 2025 18:21:09 GMT

Thank you everybody for your answers! I have read most of these links before but I will look back through them to make sure that I haven't missed anything. My textured Corona image IS a three band (colour) image. Texture was added to the original b/w image in Google Earth Engine, following the advice of Ujaval Gandhi of Spatial Thoughts, by combining it with the appropriate Landsat imagery. I'm currently attempting the actual deep learning experimentation with ArcGIS Online as I've tried this with ArcGIS Pro and even though I have a decent laptop, I'm just a bit above the minimum limits of required processing power for anything deep learning related. It will work but ties up my laptap for hours, which isn't very practical. All DL libraries are already installed and compatible with my current version of Pro. My university is currently allowing me credits for ArcGIS Online (as I need to work remotely and don't have access to high-spec computers) as long as I'm reasonable in what I use.

I use Google Earth Engine or ArcGIS Pro for the other image prep/processing tasks. And I did make a composite band image using the textured image (1 band?), slope (2nd band?) and curvature (3rd band?) but I may need to doublecheck that everything was actually 8 bit. Thank you for reminding me about that. I'll also look into the Managing Labels documentation...but up until now, I've held off on training my own model due to time constraints.

Again, thank you for all the advice. I'll go back to troubleshooting, reading and experimenting. And if something DOES finally work, I'll let you know 🙂