Classify Pixels Using Deep Learning - Patchy Results

1239
7
10-25-2021 11:47 AM
Clairetlm
New Contributor II

Hi all!

I have been trying do a land cover classification using deep learning in ArcGIS Pro. I'm looking to use this deep learning method so that the deep learning package (dlpk) can be used to make land cover comparisons over the years and/or scale it up to a larger area. However, the results so far seem to be patchy/ incomplete and it wrongly identifies the land cover. So I guess there is something in the process that I missed out on and would be grateful if anyone can help. Here are the steps that I took: 

I downloaded a Sentinel 2 image. It has 3 bands (432, RGB) and a pixel depth of 8 bit. 

Clairetlm_0-1635185469450.png

Using the training samples manager, I created a classified raster using SVM. This is how the classified raster looks like: 

Clairetlm_2-1635185876873.png

Export Training Data for Deep Learning:

Clairetlm_5-1635187164969.png

Train Deep Learning Model

Clairetlm_1-1635185824419.png

Classify pixels using deep learning: 

Clairetlm_4-1635186005263.png

Results: 

Seems like only 1 class ('water') has been inaccurately highlighted. 

Clairetlm_3-1635185909495.png

The above are the default settings. I have tried various combinations from changing the tile size, stride, ignore_classes = 0, predict_background = FALSE, using the shapefile generated from the training samples manager as input feature class when exporting training data. And I'm at a loss as to what else needs to be done to rectify this. 

Any help is greatly appreciated! 

Thank you in advance! 

Claire

0 Kudos
7 Replies
by Anonymous User
Not applicable

Hi @Clairetlm ,

 

Can you post the contents of esri_model_definition.emd file from the exported training data folder ?

 

Specifically the "Classes" key in it, I have pasted an example below.

 

"Classes" : [
{
"Value" : 1,
"Name" : "possibly_a_human",
"Color" : [
109,
82,
172
]
}
],

 

Thanks,

Sandeep

0 Kudos
Clairetlm
New Contributor II

Hi @Anonymous User!

Thank you for your reply! Really appreciate it!

Here are the contents of the emd file:

{
"Framework" : "e.g. TensorFlow or PyTorch",
"ModelConfiguration" : "e.g. UNet or FasterRCNN",
"ModelType" : "e.g. ObjectDetection or ImageClassification",
"ModelFile" : "e.g. ./trained.model",
"Description" : "e.g. which data model was trained on, when, by whom, mAP, etc",
"ImageHeight" : 256,
"ImageWidth" : 256,
"ExtractBands" : "e.g. [0, 1, 2]",
"DataRange" : "e.g. [0.1, 1.0] (optional)",
"ModelPadding" : "e.g. 64 (optional)",
"BatchSize" : "e.g. 8 (optional)",
"PerProcessGPUMemoryFraction" : "e.g. 0.8 (optional)",
"MetaDataMode" : "Classified_Tiles",
"ImageSpaceUsed" : "MAP_SPACE",
"Classes" : [
{
"Value" : 1,
"Name" : "Water",
"Color" : [
0,
92,
230
]
},
{
"Value" : 7,
"Name" : "Regrowth",
"Color" : [
163,
255,
115
]
},
{
"Value" : 10,
"Name" : "Lowland Open",
"Color" : [
215,
194,
158
]
},
{
"Value" : 12,
"Name" : "Urban",
"Color" : [
255,
0,
197
]
}
],
"InputRastersProps" : {
"RasterCount" : 1,
"SensorName" : "Undefined",
"BandNames" : [
"",
"",
""
]
},
"AllTilesStats" : [
{
"BandName" : "",
"Min" : 0,
"Max" : 255,
"Mean" : 68.462112971714561,
"StdDev" : 50.586463101038021
},
{
"BandName" : "",
"Min" : 0,
"Max" : 255,
"Mean" : 73.301121303013204,
"StdDev" : 41.25798619653375
},
{
"BandName" : "",
"Min" : 0,
"Max" : 255,
"Mean" : 57.21778052193779,
"StdDev" : 40.796065454854656
}
],
"WellKnownBandNames (FYI, these band names can be used in ExtractBands)" : [
"Red",
"Green",
"Blue",
"Infrared",
"CoastalBlue",
"Yellow",
"RedEdge",
"ShortWaveInfrared",
"NearInfrared",
"NearInfrared_1",
"NearInfrared_2",
"MidInfrared",
"MidInfrared_1",
"MidInfrared_2",
"Thermal",
"Thermal_1",
"Thermal_2",
"Panchromatic",
"PseudoRed",
"PseudoGreen",
"PseudoBlue",
"QA"
]
}

Thank you so much for helping!

Claire

0 Kudos
TimG
by
New Contributor III

How many tiles were exported in the training data?  How many epochs did you train for?

0 Kudos
Clairetlm
New Contributor II

Hi @TimG!

In the above case, there were 27 tiles and it was trained for 10 epochs. I've also tried reducing the tile size (from 256 to 64) and a total of 671 tiles were exported. Trained on 30 epochs, the results had a similar issue as well:

Clairetlm_0-1636273279122.png

 

0 Kudos
TimG
by
New Contributor III

Im a bit stumped, but I bet you will find that the blue area in the classification is equivalent to the green in the original image - it just has been coloured differently.

Can you try one thing - grab several tiles from the labels directory and pull them into ArcGIS Pro.  Show a screen shot of the tiles.  I am wondering if all the classes are not getting exported.  Although the .emd suggests otherwise.  Also what does your stats.txt file show?

0 Kudos
Clairetlm
New Contributor II

Hi @TimG!

Thanks for your reply! Yeah...the blue area in the classification is identifying regrowth (green) areas instead. I'm really puzzled as well!

Screenshot of first 8 images:

Clairetlm_3-1636696902394.png

 

Screenshot of first 8 labels:

Clairetlm_5-1636697170792.png

 

 

Here is a screenshot of the stats.txt file: 

Clairetlm_0-1636696501209.png

 

0 Kudos
TimG
by
New Contributor III

Im still stumped so don't have any great ideas.  When you bring a label image in to ArcGIS, can you please change the symbology to match your classes. The Value field says 0-3, is there a Class field in the label that matches the stats.txt file classes of 1, 7,10,12?  

 

 

0 Kudos