topic Re: Loop through multiple folder and Calculate mean of raster of Same folder in Python Questions

Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Sat, 11 Dec 2021 22:48:51 GMT

Hi All,

I am writing a script to loop through multiple folders and calculate mean of the raster of each folder's data sets and save it in same folder. Below script i am working, but its not producing the mean rasters for each year folder, after running the script its generating only name of "mean" raster in main folder, and overlapping it .

Below images showing the name of the raster in one folder, another folder the raster name is same, only year value is changed in different year.
here how to take the year value in name of mean raster (E.g. for 2001 folder, output mean raster name would be Mean_Temp_2001.tif, E.g. for 2002 folder, output mean raster name would be Mean_Temp_2002.tif)

Below i have attached data sets also (Test_data folder)v which i am working on.

Thanks.

import arcpy, os
from arcpy import env
arcpy.CheckOutExtension("Spatial") 
arcpy.env.overwriteOutput = True
env.workspace = r"D:\Test"
outraster = env.workspace


walk = arcpy.da.Walk(env.workspace, topdown=True, datatype="RasterDataset")
for dirpath, dirnames, filenames in walk:
    print dirpath
    rasterList = []
    for file in filenames:
        raster = os.path.join(dirpath, file)
        rasterList.append(raster)
        rasMean = arcpy.sa.CellStatistics(rasterList,"MEAN")
        rasMean.save(os.path.join(outraster,"mean.tif"))
        print rasMean‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Loop through multiple folder and Calculate mean of raster of Same folder

IanMurray — Fri, 24 Mar 2017 19:41:12 GMT

If you are running this as a standalone script, you will need to have arcpy check out a Spatial Analyst Extension.

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Fri, 24 Mar 2017 19:48:01 GMT

i added the line,but its behave the same as earlier .

arcpy.CheckOutExtension("Spatial") ‍

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Sat, 25 Mar 2017 10:30:58 GMT

@lan , i heve updated above code, but its producing only single output name "mean" and overlapping the output for multiple time in main folder, output not save in source of input raster folder

Re: Loop through multiple folder and Calculate mean of raster of Same folder

NeilAyres — Sat, 11 Dec 2021 22:48:53 GMT

Take this out of the loop.

rasMean = arcpy.sa.CellStatistics(rasterList,"MEAN")
rasMean.save(os.path.join(outraster,"mean.tif"))
print rasMean‍‍‍

Assemble the rasterList first, then calculate the mean

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Sat, 25 Mar 2017 11:46:43 GMT

Thank you, May i get one example of this

Re: Loop through multiple folder and Calculate mean of raster of Same folder

DanPatterson_Retired — Sat, 25 Mar 2017 12:49:31 GMT

Neil is suggesting dedenting the last 3 lines so that cell statistics is performed on the whole list rather incrementally as your script is doing. Note, all the folders must be within one folder or a subfolder within it

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Sat, 11 Dec 2021 22:48:56 GMT

Hi, I have updated my script, Below script printing only folder name, its not calculating the mean of raster from different folder

Script:

import arcpy, os
from arcpy import env
arcpy.CheckOutExtension("Spatial")
arcpy.env.overwriteOutput = True
env.workspace = r"D:\Test"
outraster = env.workspace


walk = arcpy.da.Walk(env.workspace, topdown=True, datatype="RasterDataset")
for dirpath, dirnames, filenames in walk:
    print dirpath
    rasterList = []
    for file in filenames:
        raster = os.path.join(dirpath, file)
        rasterList = arcpy.ListRasters("TIF")
    for Rasters in rasterList:
        rasMean = arcpy.sa.CellStatistics(Rasters,"MEAN", "DATA")
        file_name_only = os.path.splitext(Rasters)[0]
        tifname = file_name_only[-4:]
        rasMean.save(os.path.join(outraster,'Mean_{0}.tif'.format(tifname)))
        print rasMean‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

After running the script, its printing folder only

D:\Test
D:\Test\2001
D:\Test\2002
D:\Test\2003
D:\Test\2004
D:\Test\2005
D:\Test\2006
D:\Test\2007
D:\Test\2008
D:\Test\2009‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Loop through multiple folder and Calculate mean of raster of Same folder

DanPatterson_Retired — Wed, 29 Mar 2017 07:02:12 GMT

You do realize that rasmean is a raster and not a single number don't you? And don't you want lines 16-21 indented one more level?

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Sat, 11 Dec 2021 22:48:59 GMT

Patterson i have updated the line 16-19, still script behave the same

for Rasters in rasterList:
        rasMean = arcpy.sa.CellStatistics(Rasters,"MEAN", "DATA")
        file_name_only = os.path.splitext(Rasters)[0]
        tifname = file_name_only[-4:]‍‍‍‍

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Wed, 29 Mar 2017 08:40:33 GMT

xander_bakker‌, please cooperate with us to solve the issue.

Re: Loop through multiple folder and Calculate mean of raster of Same folder

DanPatterson_Retired — Wed, 29 Mar 2017 11:29:56 GMT

perhaps you could show the edited code instead of snippets which are then out of context and incorrect as they stand

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Wed, 29 Mar 2017 11:37:48 GMT

please tell me which way to solve the issue.

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Wed, 29 Mar 2017 11:58:30 GMT

192 views , still received input from 3 -4 developer only.

Re: Loop through multiple folder and Calculate mean of raster of Same folder

XanderBakker — Wed, 29 Mar 2017 13:18:02 GMT

Hi shouvik jha , It is always a good thing to add data to the thread (if possible) since sometimes the issue is data related. I haven't looked at the specific situation of you case yet, but I notice that you are using names for your rasters that start with a number. It is beter to avoid that.

I have been traveling a lot lately, but if the thread is not solved by tomorow evening I will see if I can do something if you attach the data to the thread.

Kind regards, Xander

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Wed, 29 Mar 2017 13:26:56 GMT

Xander Bakker‌, Thanks for your respond. I have attached the datasets to thread.

Re: Loop through multiple folder and Calculate mean of raster of Same folder

RustyRex — Wed, 29 Mar 2017 17:25:37 GMT

I think you missed Dan's comment. rasmean is a raster object, although I would expect something to be printed. Did you check the workspace directory to see if any .tif have been saved there? Also i would print your rasterList to help debug. I suspect one of your problems is there. CellStatistics is looking for a list of rasters and your are looping through the list of rasters and giving it one item from the list which implies that rasterList is a list of list.

Re: Loop through multiple folder and Calculate mean of raster of Same folder

XanderBakker — Sat, 11 Dec 2021 22:49:02 GMT

Hi shouvik jha , try the script below (change the input path at line 7):

def main():
    import arcpy
    import os

    arcpy.CheckOutExtension("Spatial")
    arcpy.env.overwriteOutput = True
    ws = r'C:\GeoNet\LMFCMSF\Test_Data'

    walk = arcpy.da.Walk(ws, topdown=True, datatype="RasterDataset")
    for dirpath, dirnames, filenames in walk:
        print "Processing folder:", dirpath

        rasters = []
        for filename in filenames:
            raster = os.path.join(dirpath, filename)
            if filename.upper().endswith('.TIF'):
                if filename.upper().startswith('MEAN') == False:
                    rasters.append(raster)
        print " - rasters found:", len(rasters)

        if len(rasters) != 0:
            print " - calculate mean..."
            file_name_only = os.path.splitext(rasters[0])[0]
            tifname = file_name_only[-4:]
            ras_mean = arcpy.sa.CellStatistics(rasters, "MEAN", "DATA")

            ras_mean_name = os.path.join(dirpath,'Mean_{0}.tif'.format(tifname))
            ras_mean.save(ras_mean_name)
            print " - save mean raster as:", ras_mean_name
        else:
            print " - skipping folder..."

    print "Finished..."


if __name__ == '__main__':
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This will print the following text:

Processing folder: C:\GeoNet\LMFCMSF\Test_Data
 - rasters found: 0
 - skipping folder...
Processing folder: C:\GeoNet\LMFCMSF\Test_Data\2001A
 - rasters found: 12
 - calculate mean...
 - save mean raster as: C:\GeoNet\LMFCMSF\Test_Data\2001A\Mean_2001.tif
Processing folder: C:\GeoNet\LMFCMSF\Test_Data\2002A
 - rasters found: 12
 - calculate mean...
 - save mean raster as: C:\GeoNet\LMFCMSF\Test_Data\2002A\Mean_2002.tif
Processing folder: C:\GeoNet\LMFCMSF\Test_Data\2003A
 - rasters found: 12
 - calculate mean...
 - save mean raster as: C:\GeoNet\LMFCMSF\Test_Data\2003A\Mean_2003.tif
Processing folder: C:\GeoNet\LMFCMSF\Test_Data\2004A
 - rasters found: 12
 - calculate mean...
 - save mean raster as: C:\GeoNet\LMFCMSF\Test_Data\2004A\Mean_2004.tif
Finished...‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The resulting rasters seem to be OK...

Kind regards, Xander

Re: Loop through multiple folder and Calculate mean of raster of Same folder

ShouvikJha — Fri, 31 Mar 2017 05:39:36 GMT

xander_bakker‌, Thank you very much. Now its working fine.

Re: Loop through multiple folder and Calculate mean of raster of Same folder

XanderBakker — Fri, 31 Mar 2017 13:15:21 GMT

I'm glad it works. If you need additional explanation of the steps, please let me know.