topic Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder. in Geoprocessing Questions

I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

LochlanJones — Thu, 21 Mar 2019 07:12:43 GMT

I have one folder (folder 1) with 200 or more rasters and another folder (folder 2) with the same number. The rasters in folder 1 are all named genus_species_01 (e.g. acan_rufo_01, lito_bico_01, and so on). The rasters in folder 3 are the same but end with _03 instead. Each raster needs to be multiplied with it's corresponding raster from the other folder and the output must retain the species name and be saved in another folder. I am completely inexperienced when it comes to Python but I believe it is the only way to achieve this task.

The link below describes a possible solution to my problem only I am not proficient enough at formatting code (or understanding it) to be able to adapt it to my problem.

python - How to use the math tool ("times") in ArcGIS modelbuilder to multiply two raster sets while using an iterator? …

I understand what most steps do but in some cases I'm unsure of what I need to change. In particular some of it would need to be changed based on the fact my rasters are located in separate folders rather than all within the same geodatabase as it is in the linked solution. Another part I am unsure about is prefix = r.split("_")[0] + "_" . To my understanding, this will split the raster name at the '_' character in order to retrieve the identifying first part of the name. But as far as I know the [0] refers to the first file in the specified workspace. So if my folder has over 200 rasters would I need to include r.split("_")[0] through to [200]?

Another potential issue is that I need both the abbreviate genus and species so it can only split the name at the second '_' and not the first.

Is anyone able to offer some guidance?

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

DanPatterson_Retired — Sat, 11 Dec 2021 13:59:54 GMT

logistically, I would suggest collecting the raster names from each of the two folders first, make the pairing and do the math

flder0 = ['a_x01', 'a_x02', 'a_x03', 'a_x04']
flder1 = ['b_c_y01', 'b_c_y02', 'b_yc_03', 'b_yc_04']
xs = [i.rsplit('_')[1] for i in flder0]
ys = [i.rsplit('_')[-1] for i in flder1]
xsys = list(zip(xs, ys))
names = ["{}{}".format(*i) for i in xsys]

for i, nme in enumerate(names):
    eq = "{} = {} * {}".format(*[nme, flder0[i], flder1[i]])
    print(eq)

x01y01 = a_x01 * b_c_y01
x02y02 = a_x02 * b_c_y02
x0303 = a_x03 * b_yc_03
x0404 = a_x04 * b_yc_04‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Lines 1 and 2, would be a arcpy.ListRasters on each folder

Lines 3 and 4 would simply be splitting the names into their bits to

line 5 is just part of the process of formatting the output names

When assembled, lines 8 and on are a 'fake' run of the process.

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

JohannesBierer — Sat, 11 Dec 2021 13:59:56 GMT

If I do get it right, the name difference between the two raster to chose is name_01 and name_03 ?

Maybe this script can help you?

import arcpy, os
from arcpy.sa import *
arcpy.CheckOutExtension ("Spatial")

arcpy.env.overwriteOutput = True

# Paths to your rasterdata ?
arcpy.env.workspace = r'C:\pathToRaster1'
inws  = arcpy.env.workspace
inws1 = r'C:\pathToRaster2'
outws = r'C:\outRaster'

# List all rasters
rasters = arcpy.ListRasters()

# 2) Loop through your raster list
for r in rasters:

    prefix = os.path.splitext(r)[0] 
    suffix = os.path.splitext(r)[1] 

    prefix1 = prefix.rstrip("1") 

    raster2 = os.path.join(inws1, prefix1 + "3" + suffix)  

    print raster2

    outraster = os.path.join(outws, prefix1 + "4" + ".tif") # Change the number

    print outraster

    # perform the raster algebra
    mycalc = arcpy.sa.Raster(r) * arcpy.sa.Raster(raster2)

    # Save to disk (write to .tif)
    mycalc.save(outraster)

arcpy.CheckInExtension ("Spatial")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Sun, 12 Dec 2021 16:17:53 GMT

I have the same challenge as Lochlan Jones, but I am attempting to iterate through and multiply two sets of corresponding rasters, named 'date_ndvi' (e.g. '20050122_ndvi') and 'date_combined' (e.g.'20050122_combined'). Johannes Bierer's script works for me up until the raster algebra step, where I get Runtime error message:

Traceback (most recent call last):
 File "<string>", line 1, in <module>
RuntimeError: ERROR 000732: Input Raster: Dataset D:\1_USGS\6_2005\NDVI_Clips_scale\clouds_null\20050318_ndvi3.tif does not exist or is not supported

It is the correct path, and the raster exist. Would anyone know where I might have gone wrong?

Very grateful for any advice, as I have not found a solution for this anywhere else.

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

DanPatterson_Retired — Sat, 11 Dec 2021 13:59:59 GMT

Anna...

The errors can vary depending on your version of python and how programs trap errors.... but your file paths aren't the greatest.

You need to 'raw' encode them, use double backslashes or use forward slashes.

For future reference, in the world of python 3 and Unicode, this will become a more important issue.

p = "D:\1_USGS\6_2005\NDVI_Clips_scale\clouds_null\20050318_ndvi3.tif"
  File "<ipython-input-3-6b3231a93a98>", line 1
    p = "D:\1_USGS\6_2005\NDVI_Clips_scale\clouds_null\20050318_ndvi3.tif"
       ^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 16-17: malformed \N character escape

# ---- use 'raw' encoding for your paths

p = r"D:\1_USGS\6_2005\NDVI_Clips_scale\clouds_null\20050318_ndvi3.tif"

print(p)
D:\1_USGS\6_2005\NDVI_Clips_scale\clouds_null\20050318_ndvi3.tif‍‍‍‍‍‍‍‍‍‍‍‍

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

LochlanJones — Wed, 11 Sep 2019 00:37:31 GMT

Hi Anna, could you provide your code that you used? I can’t remember right now and I don’t want to make promises but I think I encountered a similar issue and I can look at it when I get home later today (approx 9 hours from now).

Also is the error file the first one in the list?

Get Outlook for iOS<https://aka.ms/o0ukef>

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Wed, 11 Sep 2019 00:46:15 GMT

Hi Dan Patterson‌, thank you for this! I am a complete novice to python, so I really appreciate being pulled up on these things early on. I will raw encode and try again!

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Wed, 11 Sep 2019 00:50:20 GMT

Thank you Lochlan Jones‌! Dan (see above) highlighted that my file paths were not great and needed recoding, so I will do that first to see if it changes anything. It was the first error in the list, but I would be very interested to see if the script worked for you in the end. It has been very hard to get online support on this, so I really appreciate your efforts! I'll post my code with updated file paths and result in the next hour.

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Sun, 12 Dec 2021 16:17:56 GMT

I have not had a chance to raw encode my file paths, so this may still be the issue, but if anyone can see other errors, please let me know. I had some indent errors, but when fixed the code ran until error 00732. The files ndvi_3 and ndvi_4 has also not been printed - i.e I cannot see the output in explorer/ArcCatalog... Again, I'm very new to any programming, so happy to take any feedback.

>>> import arcpy, os
>>> from arcpy.sa import*
>>> arcpy.CheckExtension("Spatial")
u'Available'
>>> arcpy.env.overwriteOutput = True
>>> arcpy.env.workspace = r 'D:\1_USGS\6_2005\NDVI_Clips_scale'
Parsing error SyntaxError: invalid syntax (line 1)
>>> arcpy.env.workspace = r'D:\1_USGS\6_2005\NDVI_Clips_scale\ndvi'
>>> inws = arcpy.env.workspace
>>> inws1 = r'D:\1_USGS\6_2005\NDVI_Clips_scale\clouds_null'
>>> outws = r'D:\1_USGS\6_2005\NDVI_outputs'
>>> 
>>> rasters = arcpy.ListRasters()
>>> for r in rasters:
... prefix = os.path.splitext(r)[0]
... suffix = os.path.splitext(r)[1]
... 
>>> prefix1 = prefix.rstrip("1")
... 
Parsing error IndentationError: unexpected indent (line 1)
>>> prefix1 = prefix.rstrip("1")
>>> raster2 = os.path.join(inws1, prefix1 + "3" + suffix)
>>> print raster2
D:\1_USGS\6_2005\NDVI_Clips_scale\clouds_null\20050318_ndvi3.tif
>>> outraster = os.path.join(outws, prefix1 + "4" + ".tif")
>>> print outraster
D:\1_USGS\6_2005\NDVI_outputs\20050318_ndvi4.tif
>>> mycalc = arcpy.sa.Raster(r) * arcpy.sa.Raster(raster2)
Runtime error 
Traceback (most recent call last):
 File "<string>", line 1, in <module>
RuntimeError: ERROR 000732: Input Raster: Dataset D:\1_USGS\6_2005\NDVI_Clips_scale\clouds_null\20050318_ndvi3.tif does not exist or is not supported

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

LochlanJones — Thu, 12 Sep 2019 13:27:14 GMT

Hi Anna, please see my (hopefully helpful) explanation of the code I successfully used below with my data. Something of note is that how your files are named is extremely important to this code working which is why it is not working for you.
For my answer I'll assume your first raster set is named date_ndvi.tif and your second raster set is named date_combined.tif. Because I don't know what you want your output rasters to be called I will assume it will be date_multiply.tif.
Please read my hastily put together explanation in the attached image below before continuing.

Okay, now make sure you have the correct filepaths specified in your code and then replace the following lines:

Replace prefix1 = prefix.rstrip("1") with prefix1 = prefix.rstrip("ndvi")
Replace raster2 = os.path.join(inws1, prefix1 + "3" + suffix) with raster2 = os.path.join(inws1, prefix1 + "combined" + suffix)
Replace outraster = os.path.join(outws, prefix1 + "4") with outraster = os.path.join(outws, prefix1 + "multiply" + suffix)

Now it should hopefully work.
The code wasn't working originally because it couldn't find your second set of rasters. In the raster2 line you added a "3" to your file name and then it couldn't find any rasters with that name. This is why it says 20050318_ndvi3.tif doesn't exist because you accidentally added a 3 to it.

I hope this helps. Let me know how it goes. I'm a beginner as well and I only know what this code means because I worked backwards and looked at it piece by piece after the code worked after having it given to me by the good samaritan Johannes Bierer.

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Thu, 12 Sep 2019 23:31:25 GMT

Lochlan Jones‌ and Johannes Bierer‌ - I am entirely grateful to you both! Lochlan, I cannot thank you enough for taking the time to spell out each step as you have above - it's incredibly helpful. You have saved my code and advanced my knowledge substantially! Thank you!

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Sun, 12 Dec 2021 16:17:58 GMT

Lochlan Jones‌, the code appears to work well for me up until the raster calc stage (and thanks to you I now follow what it is doing). Can you see why I get error 000732 on my calculation? The files exist and have the same filename, as far as I can see (see below), in the original folders (workspace and inws1) but the output folder (outws) is still empty since the error appears before I can execute mycalc.save(outraster)...

In terms of my poor file paths - I tried both changing my filepaths to forwarded slashes as suggested by Dan Patterson and dragging and dropping the file paths , both providing the error below. Is there any other way I can improve the file paths to make this work?

>>> import arcpy, os
... from arcpy.sa import*
... arcpy.CheckOutExtension ("Spatial")
... 
... arcpy.env.overwriteOutput = True
... arcpy.env.workspace = r'D:/1_USGS/6_2005/NDVI_Clips_scale/ndvi'
... inws = arcpy.env.workspace
... inws1 = r'D:/1_USGS/6_2005/NDVI_Clips_scale/clouds_null'
... outws = r'D:/1_USGS/6_2005/NDVI_outputs'
... 
... rasters = arcpy.ListRasters()
... 
... for r in rasters:
... prefix = os.path.splitext(r)[0]
... suffix = os.path.splitext(r)[1]
... prefix1 = prefix.rstrip("ndvi")
... raster2 = os.path.join(outws, prefix1 + "combine" + suffix)
... print raster2
... outraster = os.path.join(outws, prefix1 + "multiply" + suffix)
... print outraster
... 
D:/1_USGS/6_2005/NDVI_outputs\20050122_combine.tif
D:/1_USGS/6_2005/NDVI_outputs\20050122_multiply.tif
D:/1_USGS/6_2005/NDVI_outputs\20050207_combine.tif
D:/1_USGS/6_2005/NDVI_outputs\20050207_multiply.tif
D:/1_USGS/6_2005/NDVI_outputs\20050318_combine.tif
D:/1_USGS/6_2005/NDVI_outputs\20050318_multiply.tif
>>> mycalc = arcpy.sa.Raster(r) * arcpy.sa.Raster(raster2)
Runtime error 
Traceback (most recent call last):
 File "<string>", line 1, in <module>
RuntimeError: ERROR 000732: Input Raster: Dataset D:/1_USGS/6_2005/NDVI_outputs\20050318_combine.tif does not exist or is not supported
>>>

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

DanPatterson_Retired — Fri, 13 Sep 2019 01:59:38 GMT

D:/1_USGS/6_2005/NDVI_outputs\20050318_combine.tif

that is the output folder? but the input tif is in another folder. It is failing before you even get chance to complete the multiplication

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

LochlanJones — Fri, 13 Sep 2019 02:06:03 GMT

Hi Anna, your new problem is that the ‘raster2’ line has ‘outws’ in it instead of ‘inws’. You are telling it to look in your output folder for your second raster set (combined). Change it to inws and try it again.

Get Outlook for iOS<https://aka.ms/o0ukef>

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Fri, 13 Sep 2019 02:25:43 GMT

That worked!! Thank you both. I cannot even begin to say how much I appreciate your prompt help on this.

Lochlan Jones you should be a teacher! Really - thank you for all your help and willingness to share. The world of programming beginners needs more people like you!

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Fri, 13 Sep 2019 04:05:41 GMT

Now my only (and hopefully final!) issue is that I only recieve one output, which is the output of the last file (20050318_multiply), rather than three as I was expecting from this test.

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

LochlanJones — Fri, 13 Sep 2019 06:25:52 GMT

I believe your last issue may be you’re missing the last line of code ‘mycalc.save(outraster)’ and because of this it is overwriting each output with each run leaving you with only the output of the final multiplication. If you include this last line and run it it should save them all.

Get Outlook for iOS<https://aka.ms/o0ukef>

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

AnnaKreij — Sun, 15 Sep 2019 23:45:28 GMT

Thanks Lochlan Jones‌. I tried that and it brought me one step closer, but I receive the following error:

>>> for r in rasters:
... prefix = os.path.splitext(r)[0]
... suffix = os.path.splitext(r)[1]
... prefix1 = prefix.rstrip("ndvi")
... raster2 = os.path.join(inws1, prefix1 + "combine" + suffix)
... outraster = os.path.join(outws, prefix1 + "multiply" + suffix)
... print outraster
... mycalc = arcpy.sa.Raster(r) * arcpy.sa.Raster(raster2)
... mycalc.save(outraster)

Runtime error
Traceback (most recent call last):
File "<string>", line 1, in <module>
TypeError: 'function' object is not iterable

Would the be the raster calculation function that is not iterable?

If you don't have any more time to spend on this I understand - I've really appreciated all your help so far.

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

LochlanJones — Tue, 17 Sep 2019 01:36:01 GMT

Are you running everything before the ‘for r in rasters:’ at the same time as everything after? Or are you running the first part and then the second part? I think some of it won’t work if it isn’t all run at the same time.

Get Outlook for iOS<https://aka.ms/o0ukef>

Re: I need to multiply multiple rasters in a folder with their corresponding rasters in another folder.

LochlanJones — Tue, 17 Sep 2019 01:42:37 GMT

Can you also post the full code that’s giving you this error? It can be easy to miss one tiny thing. A common cause of this error message is missing the parentheses after a function. In this case the problem could be the line ‘rasters = arcpy.ListRasters()’. If you don’t have the parentheses at the end of it or you don’t run it at the same time as the ‘for r in rasters’ line it won’t work.

Get Outlook for iOS<https://aka.ms/o0ukef>