nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Calculate daily average of raster files based on their names in python

AmenehTavakol

I have hundreds of SMAP soil moisture data and need to calculate the daily average of them. For every day there are 8 files (every 3 hours) and I need a code to calculate the average based on their names.

I have a code but i am interested to know how can I define a code that average data based on their names?

# Import arcpy module
import arcpy


# Local variables:
SMAP_L4_SM_gph_20150331T043000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c15_tif = "SMAP_L4_SM_gph_20150331T043000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c15.tif"
SMAP_L4_SM_gph_20150331T073000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c0d_tif = "SMAP_L4_SM_gph_20150331T073000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c0d.tif"
SMAP_L4_SM_gph_20150331T103000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c14_tif = "SMAP_L4_SM_gph_20150331T103000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c14.tif"
SMAP_L4_SM_gph_20150331T133000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf5_tif = "SMAP_L4_SM_gph_20150331T133000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf5.tif"
SMAP_L4_SM_gph_20150331T163000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf3_tif = "SMAP_L4_SM_gph_20150331T163000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf3.tif"
SMAP_L4_SM_gph_20150331T193000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf4_tif = "SMAP_L4_SM_gph_20150331T193000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf4.tif"
SMAP_L4_SM_gph_20150331T223000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c0f_tif = "SMAP_L4_SM_gph_20150331T223000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c0f.tif"
SMAP_L4_SM_gph_20150331T013000_Vv2030_001_Geophysical_Data_sm_surface_bdea9eb8_tif = "SMAP_L4_SM_gph_20150331T013000_Vv2030_001_Geophysical_Data_sm_surface_bdea9eb8.tif"
v_name_ = "D:\\NASA\\Data\\SMAP\\2014\\Processed\\%name%.tif"

# Process: Raster Calculator
arcpy.gp.RasterCalculator_sa("( \"%SMAP_L4_SM_gph_20150331T043000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c15.tif%\" + \"%SMAP_L4_SM_gph_20150331T073000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c0d.tif%\" + \"%SMAP_L4_SM_gph_20150331T103000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c14.tif%\" +\"%SMAP_L4_SM_gph_20150331T133000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf5.tif%\" + \"%SMAP_L4_SM_gph_20150331T163000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf3.tif%\" + \"%SMAP_L4_SM_gph_20150331T193000_Vv2030_001_Geophysical_Data_sm_surface_bdea9bf4.tif%\" + \"%SMAP_L4_SM_gph_20150331T223000_Vv2030_001_Geophysical_Data_sm_surface_bdea9c0f.tif%\" + \"%SMAP_L4_SM_gph_20150331T013000_Vv2030_001_Geophysical_Data_sm_surface_bdea9eb8.tif%\") / 8", v_name_)


Your help is really appreciated.

Encuentra más publicaciones etiquetadas con

Accepted answers

blake.terhune

Assuming the characters before the date in the filename are always the same length and all of your rasters are together in the same folder, see if this prints out the raster file names for each day that exists in the filename dates.

<SPAN class="keyword token">import</SPAN> arcpy
<SPAN class="keyword token">from</SPAN> collections <SPAN class="keyword token">import</SPAN> defaultdict

<SPAN class="keyword token">def</SPAN> <SPAN class="token function">main</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; raster_dir <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"D:\NASA\Data\SMAP\2014\Processed"</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>workspace <SPAN class="operator token">=</SPAN> raster_dir
&nbsp;&nbsp;&nbsp; raster_days <SPAN class="operator token">=</SPAN> defaultdict<SPAN class="punctuation token">(</SPAN>list<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> r <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>ListRasters<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; raster_days<SPAN class="punctuation token">[</SPAN>r<SPAN class="punctuation token">[</SPAN><SPAN class="number token">15</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">23</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">)</SPAN>

&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> date<SPAN class="punctuation token">,</SPAN> rasters <SPAN class="keyword token">in</SPAN> raster_days<SPAN class="punctuation token">.</SPAN>items<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>date<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> r <SPAN class="keyword token">in</SPAN> rasters<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">"\t{}"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>

<SPAN class="keyword token">if</SPAN> __name__ <SPAN class="operator token">==</SPAN> <SPAN class="string token">'__main__'</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; main<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

This creates a dictionary where the key is a date string (YYYYMMDD) and the value is a list of raster filenames that all have that date. If that works, you should be able to use this dictionary to calculate your statistics however you want.

EDIT:

Fixed typo on raster_days variable name in for loop.

All comments

blake.terhune

Would it be easier to use the MEAN raster property for each raster?

AmenehTavakol

Actually I need a raster file for every day (instead of 8 raster files for every day) which every pixel contains the average of soil moisture of the same pixel in 8 different time spells. By now, I can not define how arcpy identify files based on their name which contains the year month and year.

blake.terhune

Can you use raster properties to average the average of every raster for the day? In other words, loop over each raster for the day to grab its average and put it in a list, then average the list; like numpy.mean()

Or are you also asking how to look at all these raster files and identify which ones are from the same day with the file name?

curtvprice

I think you want to look into the Cell Statistics tool. The order doesn't matter, you could use Model Builder‌ or Python (arcpy.ListRasters() or arcpy.da.Walk) to create your list of rasters and pass it to the tool.

AmenehTavakol

Exactly my question is how to look at all these raster files and identify which ones are from the same day with the file name? and write a code that can define data by days and average the according to every day.

AmenehTavakol

Do you have an example of the code. I am very new in python so your help is really appreciated.

AmenehTavakol

I put the final codes to help others with the same question:

<SPAN class="keyword token">import</SPAN> arcpy
<SPAN class="keyword token">from</SPAN> shutil <SPAN class="keyword token">import</SPAN> copy2
<SPAN class="keyword token">from</SPAN> collections <SPAN class="keyword token">import</SPAN> defaultdict
arcpy<SPAN class="punctuation token">.</SPAN>CheckOutExtension<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"Spatial"</SPAN><SPAN class="punctuation token">)</SPAN>

<SPAN class="keyword token">def</SPAN> <SPAN class="token function">main</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; raster_dir <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"D:\NASA\Data\SMAP\OutPut"</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>workspace <SPAN class="operator token">=</SPAN> raster_dir
&nbsp;&nbsp;&nbsp; raster_days <SPAN class="operator token">=</SPAN> defaultdict<SPAN class="punctuation token">(</SPAN>list<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> r <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>ListRasters<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; raster_days<SPAN class="punctuation token">[</SPAN>r<SPAN class="punctuation token">[</SPAN><SPAN class="number token">15</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">23</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> date<SPAN class="punctuation token">,</SPAN> rasters <SPAN class="keyword token">in</SPAN> raster_days<SPAN class="punctuation token">.</SPAN>items<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>date<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; outCellStatistics <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa<SPAN class="punctuation token">.</SPAN>CellStatistics<SPAN class="punctuation token">(</SPAN>rasters<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"MEAN"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; copy2<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">"D:\NASA\Data\SMAP\OutPut\meanc_ras.tif"</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"D:\NASA\Data\SMAP\OutPut\Processed\meanc_ras_{}.tif"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>date<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>

<SPAN class="keyword token">if</SPAN> __name__ <SPAN class="operator token">==</SPAN> <SPAN class="string token">'__main__'</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; main<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

blake.terhune

In the future, try Posting code with Syntax Highlighting on GeoNet. Makes code easier to read and copy.

curtvprice

I went ahead and fixed it.

AmenehTavakol

Dear all,

This is my last code: I need to copy my output file in a folder:

<SPAN class="keyword token">import</SPAN> arcpy
<SPAN class="keyword token">from</SPAN> shutil <SPAN class="keyword token">import</SPAN> copy2

<SPAN class="comment token"># Check out the ArcGIS Spatial Analyst extension license</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>CheckOutExtension<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"Spatial"</SPAN><SPAN class="punctuation token">)</SPAN>

<SPAN class="keyword token">from</SPAN> collections <SPAN class="keyword token">import</SPAN> defaultdict

<SPAN class="keyword token">def</SPAN> <SPAN class="token function">main</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; raster_dir <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"D:\NASA\Data\SMAP\OutPut"</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>workspace <SPAN class="operator token">=</SPAN> raster_dir
&nbsp;&nbsp;&nbsp; raster_days <SPAN class="operator token">=</SPAN> defaultdict<SPAN class="punctuation token">(</SPAN>list<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> r <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>ListRasters<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; raster_days<SPAN class="punctuation token">[</SPAN>r<SPAN class="punctuation token">[</SPAN><SPAN class="number token">15</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">23</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">)</SPAN>

&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> date<SPAN class="punctuation token">,</SPAN> rasters <SPAN class="keyword token">in</SPAN> raster_days<SPAN class="punctuation token">.</SPAN>items<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>date<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#for r in rasters:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#print("\t{}".format(r))</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; outCellStatistics <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa<SPAN class="punctuation token">.</SPAN>CellStatistics<SPAN class="punctuation token">(</SPAN>rasters<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"MEAN"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#outname = r"D:\NASA\Data\SMAP\OutPut\Processed"</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#outCellStatistics.save(outname)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; copy2<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"D:\NASA\Data\SMAP\OutPut\meanc_ras.tif"</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">"D:\NASA\Data\SMAP\OutPut\Processed\meanc_ras_{}.tif"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>date<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

But now I receive an error :

IOError: [Errno 2] No such file or directory: 'D:\\NASA\\Data\\SMAP\\OutPut\\meanc_ras.tif'

I don't know how to save my results!

blake.terhune

Looking at the code samples, try

outCellStatistics<SPAN class="punctuation token">.</SPAN>save<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">"D:\NASA\Data\SMAP\OutPut\Processed\outCellStatsImage.tif"</SPAN><SPAN class="punctuation token">)</SPAN>‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

LarryBiodun

Hello everyone. I am trying to do a similar thing to the original post however, I have thousands of daily files that I want to use the cell statistics to calculate the 8 day averages. Simply put instead of the final code above doing a mean of the daily values, I want the code to do a mean after every eight days and save each file with the beginning date of each 8 day average. Your help is greatly appreciated!

XanderBakker

Hi Larry Biodun , it is better to create a new question and provide more information on the input data (names for instance and storage type) and if possible attach some data. Depending on the names we can see if a similar approach can be applied to your problem.

When you create a new question do include a link to this thread.

LarryBiodun

Thanks Xander. I'll do that now

Únete a la Conversación

Los miembros registrados pueden publicar, seguir actualizaciones y más. ¿Nuevo aquí? Regístrate gratis.

Enlaces Rápidos

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.