Image Classification Segmenation vs Segment Mean Shift

[MicZatorsky_AEC]

Would the Image Classification Tool  Segmentation be expected to return a different result to the Spatial Analyst tool Segment Mean Shift given the same image input and parameters? 

Given input image for both:

With the same parameters for Spatial detail, Spectral Detail and Minimum segment size in pixels:

The process:

Classification > Segmentation

Results in: 

Segment Mean Shift, with the same parameters:

Results in:

[RonaldHaug]

Hi Mic!

I don't do this everyday, but I love spatial analysis, so I'm gonna pipe in here. What I found by doing a google search on "Image Classification Segmenation ArcGIS Pro" are a number of esri help pages which talk about the process. I illustrate the model builder workflow below. As you can see it contains  Segment Mean Shift as part of the process, but it also includes training as well, so this provides a greater degree of massaging the data to take into account objects inside the raster extent. See https://pro.arcgis.com/en/pro-app/latest/help/analysis/image-analyst/understanding-segmentation-and-... and all the links on the left-hand side of the page.

When I looked up Segment Mean Shift  https://pro.arcgis.com/en/pro-app/latest/tool-reference/spatial-analyst/segment-mean-shift.htm I read all the info on it, and it does everything it says it does, AND it contains the code for how it works. Pretty cool, because you gotta do code sometimes when you're doing spatial analysis.

I wanna see it like this, though the analogy is not the best. Segment Mean Shift is like the pick-ups on your electric guitar. Image Segmentation is like the reverb on your amp, and all those crazy pedals you got plugged into one another and the mixer.

I hope this is a start for you. I'm hoping my more experienced community members chime in now and make me blush in embarrassment for my creative personal license here.

[DanPatterson]

Segment Mean Shift (Spatial Analyst)—ArcGIS Pro | Documentation

did you use the pre-processing steps outlined under Useage?

[MicZatorsky_AEC]

Thanks, both - my question is why, given identical input, is the output different?

[MervynLotter]

Are you sure the input is identical in both cases?  I suspect that the input layer for the Classification wizard approach comes from your Contents pane. Once a raster is added to a map it becomes a layer in the Contents pane as certain stretching or display adjustments are added to it (like Minimum-Maximum, Percent Clip, etc). Any of these are the property of the layer in the Contents pane and not of the raw image file lying on disk somewhere. Once they are part of a layer file, these properties are honored and maintained when the layer is included in the Classification wizard (or GP tool).  Peter Becker has stressed that it is important to use stretching to bring out the most in your imagery before running it through the segmentation tools. So it is important to tweak the display properties for your raster before running the segmentation tool. Then add this layer to the tools.  

Which brings me to the part where I suspect that when you ran the Segment Mean Shift tool, you are using the raw file on disk as input and not the layer in the contents pane? Is that so? That may be part of the reason why there are differences. 

I also am not sure whether raster statistics may be playing a role in the above.

[MicZatorsky_AEC]

I'm using a layer - but thanks for suggesting it, as it is important.

Both tools have the same layer as input - i.e., the same data source, extent, and symbology. The input image is 1.2m resolution in a defined, meter base coordinate system, a 3-band RGB JPG with statistics. Pyramids are absent (by design). The image layer Symbology Stretch type was set to None, and then a raster Function Stretch was applied with the output set to 8-bit unsigned. 

The output layer from the Stretch function was the input into Image Classification Segmentation and Segment Mean Shift. All tools were run with the input layer at the full extent of the data.

I acknowledge the distinction between a layer and its source image. I confirmed Segment Mean Shift uses the layer, not the image, directly by activating a different map before running the tool; the tool failed when it didn't find the layer. It would have run regardless of the active map if the tool used a disk image.

Both Segmentation and Segment Mean Shift wrote results to a FGDB raster.

Segment Mean Shift has segment size parameters that Segmentation does not, but I left this as the default (no limit).  

There are differences between:

1) A preview of the Segmentation, 

2) the output of running Segmentation and 

3) the output of running the Segmented Mean Shift.   

All are close, but none are the same. All have Symbology stretch set to None. I've highlighted two areas of difference.  I'm not critical of the preview; it's probably running at screen resolution like raster functions, not full resolution so that the final output would vary. But I was hoping the interactive Segmentation could be used to determine parameters, then call Segment Mean Shift with Python with the same parameters and get the same result.

The Segment Mean Shift algorithm is deterministic; given the same input data and parameters, it should produce the same output every time. Is it sensitive to the kernel size?  That is system-defined, not by me. so I don't know what each is using.

[MervynLotter]

Thanks for confirming the input source used in these tools but I am afraid someone from Esri would now need chime in with any differences to algorithms or defaults used.