topic Re: Using shutil to copy rasters in Python Questions

Using shutil to copy rasters

MathewCoyle — Thu, 14 Jun 2012 19:41:56 GMT

Is there any problems with doing this? I take all files associated with a asc or tif and use shutil.copy2. They seem to come out alright but just wondering if I am missing anything behind the scenes?

I went this route as I was having trouble using the ListRasters tool efficiently, it would take minutes just to list the rasters I want in my source directory of 14K rasters. I switched to os.listdir to get all the files matching the names I want and is much faster.

Re: Using shutil to copy rasters

curtvprice — Fri, 15 Jun 2012 01:30:29 GMT

just wondering if I am missing anything behind the scenes?

One thing to consider: raster image files often have ancillary files associated with them (auxiliary files, metadata files, pyramids) that would be left behind by an OS copy. The Copy Raster tool would always take those ancillary files along. You may be able to work around this by doing a wildcard copy, since these ancillary files are named the same as the raster.

In the case of .tif files, the TIFF image format includes internal georeferencing and sometimes even coordinate system, so you may not have an issue. If you lose the coordinate system in the system copy, you could reinstate it on the output using the Define Projection tool.

So I guess my advice is - check your output to make sure all the ancillary information you need about the raster traveled with the OS copy.

To avoid someone getting into trouble I should mention that copying a an Esri grid format raster with shutil would be a bad idea because grids include the information in the grid folder plus *some* of the files in the associated "info" folder. Copy Raster should be used with grids so you get the INFO tables along for the ride.

Re: Using shutil to copy rasters

FabianBlau — Fri, 15 Jun 2012 07:09:59 GMT

You could comine os.listdir with Copy Raster.

Re: Using shutil to copy rasters

MathewCoyle — Sat, 11 Dec 2021 18:20:21 GMT

Thanks both of you for your help. I tried the copy raster method and took much longer than the shutil. It works well for the formats I am exporting (.asc and .tif). Here is the code if anyone is interested.

import arcpy
import os
import shutil

source_dir = r"source\dir"
arcpy.env.workspace = source_dir
table = r"selection_table"
outDirBase = r"output\folder"
if not os.path.exists(outDirBase):
    os.mkdir(outDirBase)

def GetList(table):
    name_list = []
    s_curs = arcpy.SearchCursor(table)
    for row in s_curs:
        name = row.Name
        name_final = "XX_"+name.split("_")[1]
        name_list.append(name_final)
    return name_list

nts_list = GetList(table)

dir_list = [("bare_earth_DEM_ascii","be"),
            ("bare_earth_hillshade_tif","be"),
            ("full_feature_DSM_ascii","ff"),
            ("full_feature_hillshade_tif","ff"),
            ("intensity_image_tif","ii")]
# Get directory name and associated file prefix from tuple list
for directory,pref in dir_list:
    # Create a variable to hold the full path to the export directory
    new_dir = os.path.join(outDirBase,directory)
    # If this folder doesn't exist, create it
    if not os.path.exists(new_dir):
        os.mkdir(new_dir)
    # Create a variable to hold the full path to the source directory
    in_dir = os.path.join(source_dir,directory)
    # Loop through grid locations in the list of selected grids
    for nts in nts_list:
        # Assign the associated prefix to the current grid in the list
        var_in = pref+nts[2:]
        # Create a list of files in the source directory to test against
        file_list = os.listdir(in_dir)
        # Loop through all files in the source directory
        for filename in file_list:
            # If the current file matches the
            if filename.startswith(var_in):
                print "copying {0}".format(filename)
                shutil.copy2(os.path.join(in_dir,filename),os.path.join(new_dir,filename))