How to create a mosaic of rasters based on name

2443
11
Jump to solution
07-19-2021 08:43 AM
MarkBalman
Occasional Contributor

Hi all

Is it possible to create a mosaic of rasters according to file name using model builder? E.g. I have a lot of raster data using species binomial names i.e. genus_species with a suffix denoting range state (breeding, non-breeding etc). How would I loop through a folder of these species and create a new mosaic where a species might have two or more rasters? Is this even possible in model builder or would python be a better option (not much experience with python though).

Any tips/advice welcome!

Many thanks,

Mark

0 Kudos
2 Solutions

Accepted Solutions
curtvprice
MVP Esteemed Contributor

The Iterate Rasters iterator tool can loop through a wildcard match (Lep_jav_*.tif, for example) of rasters in a folder (and subfolders).

You would do the iteration in a submodel and pass the multi output result to a driver model that creates the mosaic. David is correct that Python is more flexible with things like this, but ModelBuilder has some very powerful iteration capabilities, and if you know a little python you can make it even more powerful by using Python snippets in the Calculate Value tool to do things like manipulate file and path names.

A quick tour of advanced techniques in ModelBuilder (ArcMap)

Examples of using iterators in ModelBuilder (Pro)

UPDATE: I could not resist figuring this out. I managed to do this with four models, a couple including a little bit of python (Calculate Value tool) to for example create the list if unique Genus_species pairs from the list of rasters and write it to to an in_memory table for later use (by the main Run All model). This model runs in Pro 2.8 (should work with all 2.x).

UPDATE: I got some advice from my favorite ModelBuilder expert at Esri that in general iterators should be the first thing in the model, with nothing in front of it. This model did work, but probably this setup is not best practice.

model.jpg

View solution in original post

DavidPike
MVP Frequent Contributor

Should be pretty simple.  My script should already have a dictionary for each species with values being a list of paths of corresponding rasters.

Then something simple such as 

#final step will be to iterate of the keys and use the values(raster paths in the list)
#as input into the mosaic raster tool
out_loc = r'output folder location path'
for species in species_path_dict:
    arcpy.MosaicToNewRaster_management (species_path_dict[species], out_loc, species + '.tif', "","", "", "1", "FIRST", "")

View solution in original post

11 Replies
DavidPike
MVP Frequent Contributor

I would say python is the way with something that requires name matching/grouping etc.  It seems fairly straightforward so perhaps share your data/file structure and someone might be able to put something together or point you in the right direction.

Also this is mosaic to new raster rather than crating a mosaic dataset?

0 Kudos
MarkBalman
Occasional Contributor

Hi David

Thanks for the reply. Indeed this is mosaic to new raster. Am currently trawling through some threads on python ways of doing this as don't think model builder will help here (I could be wrong though).

Files are named thus (e.g. for a species with several rasters) within one folder:

Lep_jav_N.tif

Lep_jav_R.tif

Best wishes
Mark

0 Kudos
DavidPike
MVP Frequent Contributor
import arcpy
import os
#NB this is messy and a bit lazy/suboptimal but it might give you an idea.

#folder to search
#change this to your folder path
in_folder = r"C:\Users\DaveP\Desktop\tester"
# Set the current workspace
arcpy.env.workspace = in_folder

#create a set of discrete species (and same genus)
species_list = []

rasters = arcpy.ListRasters("*", "")
for raster in rasters:
    split_name = raster.split("_")
    species_name = split_name[0] + "_" + split_name[1]
    species_list.append(species_name)

species_set = set(species_list)

#create dicionary of genus_species with a value of an empty list
species_paths_dict = {}
for species in species_set:
    species_paths_dict[species] = []

#append mathcing paths to the dictionary
for species in species_paths_dict:
    for raster in rasters:
        split_name = raster.split("_")
        if (split_name[0] + "_" + split_name[1]) == species:
            species_paths_dict[species].append(os.path.join(in_folder, raster))

print(species_paths_dict)

#final step will be to iterate of the keys and use the values(raster paths in the list)
#as input into the mosaic raster tool


        
0 Kudos
MarkBalman
Occasional Contributor

Thank you David for taking the time to look at this and provide an example, very kind! As I was trawling the web yesterday evening, I came across some code that I managed to get working for one species hard coded e.g.

import os, arcpy

workspace = r"F:\GIS_data\AOH_2021\test"
list_raster= [] # the list must exist before you can append
walk = arcpy.da.Walk(workspace, type="TIF")
output_location = r"F:\GIS_data\AOH_2021\Mosaic"

for dirpath, dirnames, filenames in walk:
for file in filenames:
if "lep_jav" in file.lower():
list_raster.append(os.path.join(dirpath,file)) # FULL path to each raster

arcpy.MosaicToNewRaster_management (list_raster, output_location, "lep_jav_m.tif", "","", "", "1", "FIRST", "")

So I guess now I need to work out how to get both your code and the above working for a folder containing numerous species.

Once again, many thanks!

Mark

0 Kudos
DavidPike
MVP Frequent Contributor

Should be pretty simple.  My script should already have a dictionary for each species with values being a list of paths of corresponding rasters.

Then something simple such as 

#final step will be to iterate of the keys and use the values(raster paths in the list)
#as input into the mosaic raster tool
out_loc = r'output folder location path'
for species in species_path_dict:
    arcpy.MosaicToNewRaster_management (species_path_dict[species], out_loc, species + '.tif', "","", "", "1", "FIRST", "")
MarkBalman
Occasional Contributor

Hi David

Ran into a new technicality, I now need to clip the species outputs using this script according to their associated raster range files that are stored in a geodatabase (I was clipping all species to an area of interest, but this is causing issues for me whilst reclassifying and performing zonal statistics further down the procesing stream). Both sets of files are using the same naming convention e.g. ant_cor, lep_jav etc.

I have created a simple extract by mask model in modelbuilder to do this manually for one species (ant_cor) and this works fine. I exported this model to python to see if I can work out how to integrate the following code into your script but now scratching my head

 

ant_cor_tif = arcpy.Raster("ant_cor.tif")
ant_cor = arcpy.Raster("F:\\ADB_Tool\\GIS_data\\Raster_range_maps.gdb\\ant_cor")

# Process: Extract by Mask (Extract by Mask) (sa)
ant_cor_ex = "F:\\ADB_Tool\\GIS_data\\AOH_2021\\Mosaic\\Clips\\ant_cor_ex"
Extract_by_Mask = ant_cor_ex
ant_cor_ex = arcpy.sa.ExtractByMask(in_raster=ant_cor_tif, in_mask_data=ant_cor)
ant_cor_ex.save(Extract_by_Mask)

Any tips would be appreciated,

Mark

0 Kudos
curtvprice
MVP Esteemed Contributor
I suggest re-reading the map algebra part of the help, it has some good examples. The model builder export is useful but doesn't make very clean code.
Here are some suggested changes:
ant_cor_tif = arcpy.Raster("ant_cor.tif")
ant_cor = arcpy.Raster("F:\\ADB_Tool\\GIS_data\\Raster_range_maps.gdb\\ant_cor")
# Process: Extract by Mask (Extract by Mask) (sa)
ant_cor_ex = arcpy.sa.ExtractByMask(in_raster=ant_cor_tif, in_mask_data=ant_cor)
ant_cor_ex.save("F:\\ADB_Tool\\GIS_data\\AOH_2021\\Mosaic\\Clips\\ant_cor_ex")
curtvprice
MVP Esteemed Contributor

The Iterate Rasters iterator tool can loop through a wildcard match (Lep_jav_*.tif, for example) of rasters in a folder (and subfolders).

You would do the iteration in a submodel and pass the multi output result to a driver model that creates the mosaic. David is correct that Python is more flexible with things like this, but ModelBuilder has some very powerful iteration capabilities, and if you know a little python you can make it even more powerful by using Python snippets in the Calculate Value tool to do things like manipulate file and path names.

A quick tour of advanced techniques in ModelBuilder (ArcMap)

Examples of using iterators in ModelBuilder (Pro)

UPDATE: I could not resist figuring this out. I managed to do this with four models, a couple including a little bit of python (Calculate Value tool) to for example create the list if unique Genus_species pairs from the list of rasters and write it to to an in_memory table for later use (by the main Run All model). This model runs in Pro 2.8 (should work with all 2.x).

UPDATE: I got some advice from my favorite ModelBuilder expert at Esri that in general iterators should be the first thing in the model, with nothing in front of it. This model did work, but probably this setup is not best practice.

model.jpg

MarkBalman
Occasional Contributor

Hi Curtis,

Wow, what can I say but thank you for taking time to look at this! Not sure what I am doing wrong but if I run the Run All model and selecting the input raster folder and setting the output folder, it starts and trundles along but I am getting Completed with warnings error - specifically "The process did not execute because the precondition is false"

Not sure where to look into this, but will dig around and see if I can resolve.

Best wishes,

Mark

0 Kudos