topic Loop through rasters and extract by masks of polygon fcs with same names in Geoprocessing Questions

Loop through rasters and extract by masks of polygon fcs with same names

KathleenHoenke — Tue, 26 Jan 2021 21:12:51 GMT

I have a gdb of 50 polygon fcs named based on HUC # abbreviation - ie: HUC_002, HUC_003, etc.

I have another gdb of rasters also based on the same state abbreviation - ie: Pluvial_002, Pluvial_003.

I want to create a short scripts that loops through the second gdb and extracts each of the rasters by the features as masks and output to a new gdb. For example, I want to take the raster named pluvial_002 and grab the fc named HUC_002 and use it to extract the raster by mask and output it to floodplain.gdb. Attached is my script - it runs for 2 seconds and says it's complete, but outputs nothing.

TIA!

#fgdb of rasters being clipped source_fgdb = arcpy.GetParameterAsText(0) #fgdb of features doing the clipping clipping_fgdb = arcpy.GetParameterAsText(1) #preferably empy fgdb to put outputs output_fgdb = arcpy.GetParameterAsText(2) #list all the rasters arcpy.env.workspace = source_fgdb source_rasts = arcpy.ListRasters() arcpy.env.workspace = clipping_fgdb clipping_fcs = arcpy.ListFeatureClasses() #make a dictionary of source fc path #to clipping fc and source name #NB relies on names before the '_' being equal source_clip_fc_dict = {} for source_rast in source_rasts: for clipping_fc in clipping_fcs: if source_rast.split("_")[0] == clipping_fc.split("_")[0]: source_rast_path = os.path.join(source_fgdb, source_rast) clipping_fc_path = os.path.join(clipping_fgdb, clipping_fc) source_clip_fc_dict[source_rast_path] = [clipping_fc_path, source_rast] #clip each source feature and output to output_fgdb for source_rast_path in source_clip_fc_dict: clipping_fc_path = source_clip_fc_dict[source_rast_path][0] source_rast_name = source_clip_fc_dict[source_rast_path][1] out_rast_name = source_rast_name + "_Mask" out_rast_path = os.path.join(output_fgdb, out_rast_name) OutExtractbyMask = ExtractByMask(source_rast_path, clipping_fc_path) OutExtractByMask.save(out_rast_path)

Re: Loop through rasters and extract by masks of polygon fcs with same names

DavidPike — Tue, 26 Jan 2021 21:23:32 GMT

It looks ok, but this time the logic for comparison has changed from being what was before the '_' to what's after it, right? That's probably meaning nothing is in the dictionary.

try changing the 3rd line as indicated below.:

for source_rast in source_rasts: for clipping_fc in clipping_fcs: if source_rast.split("_")[1] == clipping_fc.split("_")[1]:

also for your

ExtractByMask()

I'm unsure how you've done your imports, so maybe

arcpy.ExtractByMask()

Also print statements are invaluable for troubleshooting to see what's happening along the script, and where things stop/or you can see a list isn't being populated etc.

Print out all the list and dictionary being created and you'll have a much better idea of what's going on.

Re: Loop through rasters and extract by masks of polygon fcs with same names

DanPatterson — Tue, 26 Jan 2021 22:36:46 GMT

from arcpy.sa import * # ==== or from arcpy.sa import import ExtractByMask

Are your imports missing?

Re: Loop through rasters and extract by masks of polygon fcs with same names

KathleenHoenke — Tue, 26 Jan 2021 23:30:03 GMT

I think my issue is that my dictionaries don't have anything in them.... the lists of fcs and rasters print fine...

#fgdb of rasters being clipped source_fgdb = r'E:\FATHOM\2020FATHOM\Analysis\Pluvial_1in100\Pluvial_1in100_Processed.gdb' #fgdb of features doing the clipping clipping_fgdb = r'E:\Boundaries\WBDHU2_Mar2014.gdb' #preferably empy fgdb to put outputs output_fgdb = r'E:\FATHOM\2020FATHOM\Analysis\Final_Floodplain_Bounds_No_NHD.gdb' #list all the rasters arcpy.env.workspace = source_fgdb rasters = arcpy.ListRasters("*", "GRID") print (rasters) ['FATHOM_pluv_con_002', 'FATHOM_pluv_con_005', 'FATHOM_pluv_con_006', 'FATHOM_pluv_con_012', 'FATHOM_pluv_con_007', 'FATHOM_pluv_con_010', 'FATHOM_pluv_con_011', 'FATHOM_pluv_con_013', 'FATHOM_pluv_con_014', 'FATHOM_pluv_con_015', 'FATHOM_pluv_con_016', 'FATHOM_pluv_con_017', 'FATHOM_pluv_con_008', 'FATHOM_pluv_con_009', 'FATHOM_pluv_con_003'] arcpy.env.workspace = clipping_fgdb clipping_fcs = arcpy.ListFeatureClasses() print (clipping_fcs) ['HUC2_002', 'HUC2_003', 'HUC2_004', 'HUC2_005', 'HUC2_006', 'HUC2_007', 'HUC2_008', 'HUC2_009', 'HUC2_010', 'HUC2_011', 'HUC2_012', 'HUC2_013', 'HUC2_014', 'HUC2_015', 'HUC2_016', 'HUC2_017', 'HUC2_018', 'HUC2_019', 'HUC2_020', 'HUC2_021', 'HUC2_022'] source_clip_fc_dict = {} for raster in rasters: for clipping_fc in clipping_fcs: if raster.split("_")[0] == clipping_fc.split("_")[0]: raster_path = os.path.join(source_fgdb, raster) clipping_fc_path = os.path.join(clipping_fgdb, clipping_fc) source_clip_fc_dict[raster_path] = [clipping_fc_path, raster] print(source_clip_fc_dict) {} {} {} {} {} {} {} {} {} {} {} {} {} {} {}

Re: Loop through rasters and extract by masks of polygon fcs with same names

DanPatterson — Tue, 26 Jan 2021 23:57:14 GMT

are you trying to match the numbers here eg... 002 to 002?

if raster.split("_")[0] == clipping_fc.split("_")[0]:

if so, you should replace 0 with -1 to get the last of the split

if raster.split("_")[-1] == clipping_fc.split("_")[-1]:

dct = {} for f in fcs: for ra in r: if ra.split("_")[-1] == f.split("_")[-1]: dct[f] = ra dct {'HUC2_002': 'FATHOM_pluv_con_002', 'HUC2_003': 'FATHOM_pluv_con_003', 'HUC2_005': 'FATHOM_pluv_con_005', 'HUC2_006': 'FATHOM_pluv_con_006', 'HUC2_007': 'FATHOM_pluv_con_007', 'HUC2_008': 'FATHOM_pluv_con_008', 'HUC2_009': 'FATHOM_pluv_con_009', 'HUC2_010': 'FATHOM_pluv_con_010', 'HUC2_011': 'FATHOM_pluv_con_011', 'HUC2_012': 'FATHOM_pluv_con_012', 'HUC2_013': 'FATHOM_pluv_con_013', 'HUC2_014': 'FATHOM_pluv_con_014', 'HUC2_015': 'FATHOM_pluv_con_015', 'HUC2_016': 'FATHOM_pluv_con_016', 'HUC2_017': 'FATHOM_pluv_con_017'}

Re: Loop through rasters and extract by masks of polygon fcs with same names

KathleenHoenke — Wed, 27 Jan 2021 02:35:47 GMT

Thank you! That worked! Now, the script works but only for one of the items in the dictionary (HUC2_003 and FATHOM_pluv_con_003). It outputs that faster clipped perfectly, but then stops....

for source_raster in source_rasters: for clipping_fc in clipping_fcs: if source_raster.split("_")[-1] == clipping_fc.split("_")[-1]: source_raster_path = os.path.join(source_fgdb, source_raster) clipping_fc_path = os.path.join(clipping_fgdb, clipping_fc) source_clip_fc_dict[source_raster_path] = [clipping_fc_path, source_raster] print(source_clip_fc_dict) #clip each source feature and output to output_fgdb for source_raster_path in source_clip_fc_dict: clipping_fc_path = source_clip_fc_dict[raster_path][0] source_raster_name = source_clip_fc_dict[source_raster_path][1] out_raster_name = source_raster_name + "_Mask" out_raster_path = os.path.join(output_fgdb, out_raster_name) ExtractByMask(source_raster_path, clipping_fc_path) OutExtractByMask.save(out_raster_path)

Re: Loop through rasters and extract by masks of polygon fcs with same names

DanPatterson — Wed, 27 Jan 2021 04:44:53 GMT

clipping_fc_path = source_clip_fc_dict[raster_path][0]

where is raster_path defined? do you source_raster_path???

Re: Loop through rasters and extract by masks of polygon fcs with same names

KathleenHoenke — Wed, 27 Jan 2021 14:58:10 GMT

I think I did in line 37 where it says source_raster_path = os.path.join(source_fgdb, source_raster) below

import arcpy import os from arcpy.sa import ExtractByMask #not a fan of stating the obvious, but change the paths below to your fgdbs #also keep the r then surround the path in quotes #fgdb of rasters being clipped source_fgdb = r'E:\FATHOM\2020FATHOM\Analysis\Pluvial_1in100\Pluvial_1in100_Processed.gdb' #fgdb of features doing the clipping clipping_fgdb = r'E:\Boundaries\WBDHU2_Mar2014.gdb' #preferably empy fgdb to put outputs output_fgdb = r'E:\FATHOM\2020FATHOM\Analysis\Final_Floodplain_Bounds_No_NHD.gdb' #list all the rasters arcpy.env.workspace = source_fgdb source_rasters = arcpy.ListRasters("*", "GRID") print (source_rasters) arcpy.env.workspace = clipping_fgdb clipping_fcs = arcpy.ListFeatureClasses() print (clipping_fcs) #make a dictionary of source raster path #to clipping fc and source name #NB relies on names after the '_' being equal source_clip_fc_dict = {} for source_raster in source_rasters: for clipping_fc in clipping_fcs: if source_raster.split("_")[-1] == clipping_fc.split("_")[-1]: source_raster_path = os.path.join(source_fgdb, source_raster) clipping_fc_path = os.path.join(clipping_fgdb, clipping_fc) source_clip_fc_dict[source_raster_path] = [clipping_fc_path, source_raster] print(source_clip_fc_dict) #clip each source feature and output to output_fgdb for source_raster_path in source_clip_fc_dict: clipping_fc_path = source_clip_fc_dict[raster_path][0] source_raster_name = source_clip_fc_dict[source_raster_path][1] out_raster_name = source_raster_name + "_Mask" out_raster_path = os.path.join(output_fgdb, out_raster_name) ExtractByMask(source_raster_path, clipping_fc_path) OutExtractByMask.save(out_raster_path)

Re: Loop through rasters and extract by masks of polygon fcs with same names

DanPatterson — Wed, 27 Jan 2021 15:14:39 GMT

getting confused as to why you need to do the dictionary thing, why not do the extractbymask if a match is found, basically following this idea

The dictionary is only useful if you plan to reuse it later. You can keep the dictionary but do the extraction while you are there so you don't have to go back and do the match thing. (ps... I didn't pay much attention to the name matching so check!)

Re: Loop through rasters and extract by masks of polygon fcs with same names

KathleenHoenke — Wed, 27 Jan 2021 16:00:45 GMT

Thank you! It got mad because I didn't define OutExtractByMask, so I did it below, and it then output the first pair to my default gdb with that name, and stopped. I feel like I'm so close.... something to do with how I'm doing the extraction to make sure it repeats itself.....

Re: Loop through rasters and extract by masks of polygon fcs with same names

DanPatterson — Wed, 27 Jan 2021 17:01:34 GMT

I don't have anything to test on, so I would suggest a well speckled interjection of print statements before and within your for and if statements to see what is going on

Re: Loop through rasters and extract by masks of polygon fcs with same names

DavidPike — Wed, 27 Jan 2021 21:00:49 GMT

You need to define the out raster path within the loop (just below clipping_fc_path)

out_raster_name = source_rast_name + "_Mask"
out_raster_path = os.path.join(output_fgdb, out_raster_name).

Re: Loop through rasters and extract by masks of polygon fcs with same names

KathleenHoenke — Fri, 29 Jan 2021 02:42:44 GMT

Thank you. I was able to figure out the issue in defining things that weren't defined! What is the general rule for the matching of things before and after the "_"? If I'm looking to match string before the _ then it's [1] but after it is [-1]? Is there ever a [0]?