Iterate workspaces and contained rasters to make workspace specific mosaics

Question

Making as a dedicated thread originated in comments here with @Duncan Hornby and @Curtis Price.

Need:
I have many sites each composed of one to over twenty rasters. They are sorted into folders by site (Atch: "Folder Structure"). I want to iterate through these sites as workspaces in order to mosaic rasters in each site. Output will be in a different folder titled as such: "mosaic_siteX" or "siteX_mosaic".

Model Attempts:
I first tried a 3 part model; one to iterate workspaces, one to iterate rasters in those workspaces, and one to perform the mosaic (Atch: "TestModel_1"). I ran the iterate workspaces feeding the folder containing the sites with "recursive" checked. This, in a way, was the most successful attempt, though it didn't work. It would iterate through all the workspaces and only mosaic the last one, but it mosaic'ed this last correctly.

Changing the above per suggestions in thread linked at top I moved the nest to be m2 inside of m1 (instead of vice versa) and sent that output to a third for mosaicing (Atch: "TestModel_2"). This output iterated through workspaces but only sent the last tile of each site workspace to the Collect Values tool and mosaic'ed the last tile of each site together rather than all tiles from one site together.

I tried a third edit, running the mosaic inside m1 after the collect values, but this had similar results to directly above. It would iterate through and make a mosaic each iteration, each time adding the last tile of the next site added. I.e. first iteration last tile of site 2; second iteration last tile of site 2 and 5 mosaic together; third iteration last tile of site 2, 5, 6 mosaic together.

I need to blend the mistakes, as it were. The iterate rasters to collect values and mosaic results need to be that of the first attempt, but the site folders and workspace iteration needs to do that mosaic to all folders, not just the last one. I want to try and finish this in model builder, as that is where I have had the most successful experimenting thus far. If it comes down to it I may try to use arcpy, but my exposure is limited for the moment; I will be taking a class on it starting next week.

Apologies for the delay in getting this post up. I've had power and internet issues following a local incident in town the other day.

Cheers,
Nate

TestModel_1.JPG

TestModel_2.JPG

FolderStructure.JPG

朱羿 2111084022.jpg

Bud_0-1656204582379.png

IMG_20170330_210524.jpg

NathanSchlagel · Answer

I had given up on trying to do it in model builder and did the Python for Everyone tutorial to get a grasp of it. Then using forums I was able to piece together a code that worked. It was perhaps more complicated than it needed to be trying to get conditionals to work depending on the name of files as I have two sets for each site; the normal dem, and a processed dem to overlay and show depth of holes (essentially wells in the study areas).

Model builder was great for processing the dem's into the overlays I needed. But for mosaicing together, since I had so many conditionals I wanted to impose it was easier to mosaic in arcpy with if statements (which in plotting this out I had several more and realized it was inefficient so I condensed it by the time I made this image). Apologies for the amateur code; I'm still very new to arcpy and python and haven't done any other coding for over six years.

Output looked kind of like this::
Adding rasters to list:
site1_tile1

site1_tile2
site1_tile3
List done; mosaicing tiles from list.
Reset list to empty and start new site (it was important to reset the list "rasters=[]" at the bottom of the code each time otherwise it would have just added site 2 to the end of site 1 list and mosaiced both sites together).
Adding rasters to list:
site2_tile1
site2_tile2>>>>>>

To test it I ran the entire process as print " " statements as above and left the mosaic tool in comment (#arcpy.mapping....) so it wouldn't take as long to test. Once the raster lists contained only raster tiles from a single site, and reset properly only including the next site,  I ran it through with mosaic uncommented.

I realized I deleted the tile sets after the mosaic finished to save hard drive space (as I have several redundant backups of them elsewhere) so I can't run it again and show the exact results.
Apologies if the pathnames are confusing. I have to get permission to publish anything with the actual dems and to avoid any issues I altered the pathnames that included them just incase.

And, now that those are done I'm trying to figure out how to use ArcPy to add layers to an mdx, set them into layer groups, and apply standard symbologies from an IDE; will probably be making a separate post on that soon as I'm having dificulty with it.

Thanks for all your help Curtis!

Cheers,
Nate

curtvprice · Answer

This has been bugging me so I decided to take up the challenge. I set up pair of folders with files in them to test my model: +---wk1 | wk1f1.txt | wk1f2.txt | \---wk2 wk2f1.txt wk2f2.txt‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ First, I created a model that iterates files for a workspace (get files) This model is then embedded in a second model that iterates workspaces (Side note, since Iterate Files requires a folder, I had to use the Calculate Value tool to convert the data type from workspace to folder here. You won't need that for rasters, because you can Iterate Rasters on a all kinds of workspaces, not just folders.) After this model runs Output Values (2) contains all the files: I believe in your case, you would set up the first model to use Iterate Rasters, and in the second model you could put your Mosaic To New Raster tool where I put Collect Values. If that input will not validate into Mosaic To New Raster, the workaround is to hide the tool inside a function inside Calculate Value. (Set the element Output Values as a precondition to this Calculate Value tool to make sure the input is ready to go when you run it.) expr: mtnr(r"%Output Values%", %n%, r"%workspace%") ‍‍‍‍‍‍‍ code block: import os import arcpy def mtnr(vals, n, wk): out_raster = os.path.join(wk, "Mosaic{}".format(n)) arcpy.MosaicToNewRaster_management(vals, out_raster, ....) return out_raster‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ So, as I said, these things are doable, but in the long term this would have been easier in Python (once you have picked up the requisite Python skills).