topic Re: Getting the Mean value out of the Max values of a raster data set in Python Questions

Getting the Mean value out of the Max values of a raster data set

anTonialcaraz — Tue, 27 Nov 2018 11:41:35 GMT

Hi,

As the title says, I'm trying to calculate in python the mean value out of all maximum values in a list of rasters (696) in a GDB.

First thing would be to calculate all max values but I'm not sure how to "hold" all those values to then calculate the mean.

Any help will be greatly appreciate it.

Running ArcGis 10.5.1

Re: Getting the Mean value out of the Max values of a raster data set

XanderBakker — Tue, 27 Nov 2018 12:11:49 GMT

Hi anToni alcaraz

I'm not entirely sure what you want to achieve. When you mention you want to calculate the maximum, do you refer to a single value for the raster (like the metadata property of the raster) or the maximum for each pixel over the series of rasters or the maximum in a certain neighborhood (moving window) for each raster. And the mean would be using the values of each pixel for the series of maximum rasters?

You can create temp rasters using the in_memory workspace if they are not to big in case you do not want to create a physical result for the intermediate max rasters.

Re: Getting the Mean value out of the Max values of a raster data set

MartinSirkovsky1 — Tue, 27 Nov 2018 12:25:33 GMT

Basically you would iterate over the rasters, find the maximum, add the result to array, and at the end you would calculate mean of the values in array.

You could use Get Raster Properties function to get the maximum of the raster.

Re: Getting the Mean value out of the Max values of a raster data set

JakeSkinner — Fri, 10 Dec 2021 20:36:36 GMT

You could try the following:

import arcpy
arcpy.env.workspace = r"C:\TEMP\PYTHON\Data.gdb"

# Create list
maxVals = []

# Iterate through each raster and append max val to list
for raster in arcpy.ListRasters("*"):
    val = arcpy.GetRasterProperties_management(raster, "MAXIMUM")
    maxVal = int(val.getOutput(0))
    maxVals.append(maxVal)

# Sum list
sum = 0
for val in maxVals:
    sum += val

# Find Mean
meanVal = sum / len(maxVals)
print(meanVal)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Getting the Mean value out of the Max values of a raster data set

DanPatterson_Retired — Fri, 10 Dec 2021 20:36:39 GMT

Can't let a numpy opportunity go by...

import arcpy
arcpy.env.workspace = r"C:\Data\rasters"
import numpy as np
r_max = np.finfo(np.float).min
for raster in arcpy.ListRasters("*"):
    ras = arcpy.RasterToNumPyArray(raster)
    r_max = max(r_max, np.nanmax(ras))
print(r_max)


255‍‍‍‍‍‍‍‍‍‍‍

Re: Getting the Mean value out of the Max values of a raster data set

anTonialcaraz — Fri, 10 Dec 2021 20:36:42 GMT

Thanks Xander and Martin for your suggestions.

Sorry Xander I may have not explained the issue correctly.

My ultimate goal here is to standardise a number of raster layers (696 of them). My first objective was to get the maximum value (a unique single value) for all those rasters. I've done that; but after examining the maximum value I realise that doing the mean of all the maximum values of all rasters would be a more accurate value (to later standardise them).

What Martin says is what I need to do but I'm not familiar with numpy and arrays and I'm not quite sure how to proceed. I also wonder if there is a function in scipy that would do that directly? Below is just the start of the script but as I said I don't know how to proceed.

import arcpy
import math
import os
import numpy
import scipy
from arcpy import env
arcpy.env.overwriteOutput=True
from arcpy.sa import*
arcpy.CheckOutExtension("Spatial")



env.workspace = r"D:\000_TEST_MAGNITUDE_summary\MAX_VAL_calculation\Vel_rasters.gdb



# List rasters and get MAX values

rasterlist1 = arcpy.ListRasters()

Vel_max = arcpy.GetRasterProperties_management(rasterlist1, "MAXIMUM")


# Create array and add result from above

Vel_array = arcpy.RasterToNumPyArray(rasterlist1)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Getting the Mean value out of the Max values of a raster data set

anTonialcaraz — Tue, 27 Nov 2018 16:46:43 GMT

Thank you Jake. That looks promising. Just trying now but I got an error in line 16. Is it '+' or '='?

Re: Getting the Mean value out of the Max values of a raster data set

JakeSkinner — Tue, 27 Nov 2018 16:50:46 GMT

It's actually +=. It's essentially a quicker way of writing:

sum = sum + val‍

Re: Getting the Mean value out of the Max values of a raster data set

XanderBakker — Fri, 10 Dec 2021 20:36:44 GMT

Could you try this and share what it prints (it's basically the same code as Jake shared earlier)?

import arcpy
import os

# settings
ws = r"D:\000_TEST_MAGNITUDE_summary\MAX_VAL_calculation\Vel_rasters.gdb"
arcpy.env.workspace = ws

# List rasters
raster_names = arcpy.ListRasters()

# loop through rasters
max_values = []
for raster_name in raster_names:
    raster_path = os.path.join(ws, raster_name)
    vel_max = arcpy.GetRasterProperties_management(raster_path, "MAXIMUM")
    print(" - {}: {}".format(raster_name, vel_max))
    max_values.append(float(vel_max.getOutput(0)))

mean_val = float(sum([max_values])) / len(max_values)
print("Mean value: {}".format(mean_val))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Getting the Mean value out of the Max values of a raster data set

DanPatterson_Retired — Fri, 10 Dec 2021 20:36:47 GMT

Mean of max's? no problem

import arcpy
arcpy.env.workspace = r"C:\Data\rasters"
import numpy as np
r_max_mean = []
for raster in arcpy.ListRasters("*"):
    ras = arcpy.RasterToNumPyArray(raster)
    r_max_mean.append( np.nanmax(ras) )
print("max_list\n{}\nmean_max {}".format(r_max_mean, np.nanmean(r_max_mean)))


max_list
[255, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99]

mean_max 103.875‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Getting the Mean value out of the Max values of a raster data set

anTonialcaraz — Fri, 10 Dec 2021 20:36:50 GMT

I see. At first run I got an error in line 13 below. As the rasters are floating point I removed 'int' from the line (as below).

env.workspace = r"D:\000_TEST_MAGNITUDE_summary\MAX_VAL_calculation\Vel_rasters.gdb"



# Create list

maxVals = []

# Iterate through each raster and append max val to list

for raster in arcpy.ListRasters("*"):
    val = arcpy.GetRasterProperties_management(raster, "MAXIMUM")
    maxVal = val.getOutput(0)
    maxVals.append(maxVal)

# Sum list

sum = 0
for val in maxVals:
    sum = sum + val

# Find Mean

meanVal = sum / len(maxVals)


print(meanVal)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

But now I'm getting this error which I guess it may be related.

Re: Getting the Mean value out of the Max values of a raster data set

JakeSkinner — Fri, 10 Dec 2021 20:36:53 GMT

Change int to float:

# Iterate through each raster and append max val to list
for raster in arcpy.ListRasters("*"):
    val = arcpy.GetRasterProperties_management(raster, "MAXIMUM")
    maxVal = float(val.getOutput(0))
    maxVals.append(maxVal)‍‍‍‍‍

Re: Getting the Mean value out of the Max values of a raster data set

DanPatterson_Retired — Tue, 27 Nov 2018 17:17:44 GMT

scipy relies on numpy, see my second example, it returns a list of the maximums, then simply takes the mean of that

Re: Getting the Mean value out of the Max values of a raster data set

anTonialcaraz — Wed, 28 Nov 2018 09:16:34 GMT

Thank you Xander. It seems there is a minor issue in line 19.

I also tried with float(len(max_values) but getting the same error. Other than that, I'm sure is perfect. Very nice the way it prints the max values for each raster. Both Jake's and Dan's are working and giving the same result. Many thanks.

Re: Getting the Mean value out of the Max values of a raster data set

anTonialcaraz — Wed, 28 Nov 2018 09:17:26 GMT

That works beautifully. Really appreciate it.

Re: Getting the Mean value out of the Max values of a raster data set

anTonialcaraz — Wed, 28 Nov 2018 09:20:34 GMT

That works perfect Dan. All the slightly different approaches above give same result. Really appreciate your help once again.