topic Re: Batch process multiple ASCII to Raster files in Python Questions

Batch process multiple ASCII to Raster files

JenB — Mon, 05 Mar 2012 19:46:03 GMT

I have multiple (~50) ASCII files that I need to convert to raster. I???m very new to Python but have managed to create a simple script to do this for one file (and it worked- so exciting!). How can I modify this to process ALL of the files? They are all located in the same directory. Any advice would be appreciated. Thanks very much!

# Import system modules
import arcgisscripting

# Create the Geoprocessor object
gp = arcgisscripting.create()

# Set local variables
InAsciiFile = "C:/Data/data_2001.asc"
OutRaster = "C:/Data/data_2001"
gp.outputCoordinateSystem = "Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj"

# Process: ASCIIToRaster_conversion
gp.ASCIIToRaster_conversion(InAsciiFile, OutRaster, "FLOAT")

Re: Batch process multiple ASCII to Raster files

BruceNielsen — Sat, 11 Dec 2021 11:59:16 GMT

I think this will work for you (it's untested):

# Import system modules
import arcgisscripting, os, glob

# Create the Geoprocessor object
gp = arcgisscripting.create()

# Set local variables
gp.workspace = "C:/Data"
gp.outputCoordinateSystem = "Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj"

# Process: ASCIIToRaster_conversion
for x in glob.glob(os.path.join(gp.workspace, "*.asc")):
    gp.ASCIIToRaster_conversion(x, os.path.split(os.path.basename(x))[0], "FLOAT")

Re: Batch process multiple ASCII to Raster files

MathewCoyle — Sat, 11 Dec 2021 11:59:19 GMT

Also untested

# Import system modules
import arcgisscripting, os

# Create the Geoprocessor object
gp = arcgisscripting.create()

# Set local variables
InAsciiFile = None
inDir = "C:/Data"
OutRaster = "C:/Data/data_2001"
gp.outputCoordinateSystem = "Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj"

for InAsciiFile in os.listdir(inDir):
    if InAsciiFile.rsplit(".")[-1] == "asc":
        print InAsciiFile
        try:
            # Process: ASCIIToRaster_conversion
            gp.ASCIIToRaster_conversion(os.path.join(inDir,InAsciiFile), OutRaster, "FLOAT")
        except:
            pass

Re: Batch process multiple ASCII to Raster files

curtvprice — Mon, 05 Mar 2012 22:42:14 GMT

One of the things that makes Arc 10 worth installing is model iterators, which avoid you all this drudgery.

Re: Batch process multiple ASCII to Raster files

JenB — Sat, 11 Dec 2021 11:59:22 GMT

Thanks for the suggestions. I should have mentioned I'm using Python 2.5 and ArcGIS 9.3 (curtvprice, I can't upgrade to 10 yet as I am using a tool that was built exclusively for 9.3).

I tried both suggestions but unfortunately, I was not able to get either code to work.
mzcoyle- the ascii files read properly and printed in the log, but then nothing happened- I didn't get an error, but no files were converted. Any ideas?
bruce.nielsen- I got the following error- I'm not sure in this code where to specify the output?

ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000859: The required parameter Output raster is empty, or is not the type of Raster Dataset.
Failed to execute (ASCIIToRaster).

Thanks again for your help.

Re: Batch process multiple ASCII to Raster files

MathewCoyle — Sat, 11 Dec 2021 11:59:25 GMT

I tested this one and it works. Make sure your file names are less than 13 characters.

# Import system modules
import arcgisscripting, os

# Create the Geoprocessor object
gp = arcgisscripting.create()

# Set local variables
InAsciiFile = None
inDir = r"C:\GIS\ascii"
OutRaster = "C:/GIS/Exports"
gp.outputCoordinateSystem = r"Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj"

for InAsciiFile in os.listdir(inDir):
    if InAsciiFile.rsplit(".")[-1] == "asc":
        print InAsciiFile
        # Process: ASCIIToRaster_conversion
        gp.ASCIIToRaster_conversion(os.path.join(inDir,InAsciiFile), os.path.join(OutRaster,InAsciiFile.rsplit(".")[0]), "FLOAT")

Re: Batch process multiple ASCII to Raster files

BruceNielsen — Sat, 11 Dec 2021 11:59:27 GMT

bruce.nielsen- I got the following error- I'm not sure in this code where to specify the output?
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000859: The required parameter Output raster is empty, or is not the type of Raster Dataset.
Failed to execute (ASCIIToRaster).
Thanks again for your help.

The ASCIIToRaster command as you wrote it required a valid workspace for creating GRID files. Try this instead:

gp.ASCIIToRaster_conversion(x, os.path.split(os.path.basename(x))[0] + ".tif", "FLOAT")

This will create TIF files, which can reside anywhere.

Re: Batch process multiple ASCII to Raster files

JenB — Sat, 11 Dec 2021 11:59:30 GMT

I tested this one and it works. Make sure your file names are less than 13 characters.

# Import system modules
import arcgisscripting, os

# Create the Geoprocessor object
gp = arcgisscripting.create()

# Set local variables
InAsciiFile = None
inDir = r"C:\GIS\ascii"
OutRaster = "C:/GIS/Exports"
gp.outputCoordinateSystem = r"Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj"

for InAsciiFile in os.listdir(inDir):
    if InAsciiFile.rsplit(".")[-1] == "asc":
        print InAsciiFile
        # Process: ASCIIToRaster_conversion
        gp.ASCIIToRaster_conversion(os.path.join(inDir,InAsciiFile), os.path.join(OutRaster,InAsciiFile.rsplit(".")[0]), "FLOAT")

This worked perfectly- thanks!!

Re: Batch process multiple ASCII to Raster files

RumanaReaz — Sun, 28 Oct 2012 19:19:57 GMT

Hello mzcoyle,
I tried your code for batch processing of ASCII to Raster. It works but I got one problem.
How do I rename ASCII files or output Raster files, cause my ASCII files have more than 13 characters?

mzcoyle:
    I tested this one and it works. Make sure your file names are less than 13 characters.
    Code:

    # Import system modules
    import arcgisscripting, os

    # Create the Geoprocessor object
    gp = arcgisscripting.create()

    # Set local variables
    InAsciiFile = None
    inDir = r"C:\GIS\ascii"
    OutRaster = "C:/GIS/Exports"
    gp.outputCoordinateSystem = r"Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj"

    for InAsciiFile in os.listdir(inDir):
        if InAsciiFile.rsplit(".")[-1] == "asc":
            print InAsciiFile
            # Process: ASCIIToRaster_conversion
            gp.ASCIIToRaster_conversion(os.path.join(inDir,InAsciiFile), os.path.join(OutRaster,InAsciiFile.rsplit(".")[0]), "FLOAT")

Re: Batch process multiple ASCII to Raster files

MathewCoyle — Mon, 29 Oct 2012 12:27:39 GMT

Hello mzcoyle,
I tried your code for batch processing of ASCII to Raster. It works but I got one problem.
How do I rename ASCII files or output Raster files, cause my ASCII files have more than 13 characters?

To change the output name you could do something like this. Changing the output to only take the first 13 characters of the input file name. If you want it to be more dynamic, splitting on a specific string etc then you could incorporate that also. At that point I would probably create an output raster variable to create the name.

 gp.ASCIIToRaster_conversion(os.path.join(inDir, InAsciiFile), os.path.join(OutRaster, InAsciiFile.rsplit(".")[0][:13]), "FLOAT")

Re: Batch process multiple ASCII to Raster files

RumanaReaz — Mon, 29 Oct 2012 15:28:29 GMT

To change the output name you could do something like this. Changing the output to only take the first 13 characters of the input file name. If you want it to be more dynamic, splitting on a specific string etc then you could incorporate that also. At that point I would probably create an output raster variable to create the name.
 gp.ASCIIToRaster_conversion(os.path.join(inDir, InAsciiFile), os.path.join(OutRaster, InAsciiFile.rsplit(".")[0][:13]), "FLOAT")

Thank you very much for reply.
I tried your code, only changed "FLOAT" to "INTEGER", cause I need integer values.
Got following error:

2011_001_glsea.asc

Traceback (most recent call last):
File "C:\Users\rumana\Desktop\My EFDC\Data\AVHRR\BatchAscii2Raster2.py", line 18, in <module>
gp.ASCIIToRaster_conversion(os.path.join(inDir, InAsciiFile), os.path.join(OutRaster, InAsciiFile.rsplit(".")[0][:13]), "INTEGER")
RuntimeError: NotInitialized

I never got this error before and I tested your code with an ascii file which has name length less than 13 characters. It worked then.

Re: Batch process multiple ASCII to Raster files

MathewCoyle — Mon, 29 Oct 2012 15:49:20 GMT

Try reducing the name splitting from 13 to 9 characters. It may be that it is a multi band raster and doesn't support 13 character file names. You can read about the formatting required for Esri Grids here.

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//009t0000000w000000

I seem to recall some issues when a file name starts with numeric characters, but that may be a different format I am thinking of.

Re: Batch process multiple ASCII to Raster files

RumanaReaz — Mon, 29 Oct 2012 15:54:53 GMT

Try reducing the name splitting from 13 to 9 characters. It may be that it is a multi band raster and doesn't support 13 character file names. You can read about the formatting required for Esri Grids here.

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//009t0000000w000000

I seem to recall some issues when a file name starts with numeric characters, but that may be a different format I am thinking of.

I checked with 9 but getting the same error.
I have attached a sample data file.

Thanks a lot for helping me out, please take your time.

Re: Batch process multiple ASCII to Raster files

MathewCoyle — Mon, 29 Oct 2012 16:11:09 GMT

I checked with 9 but getting the same error.
I have attached a sample data file.

Thanks a lot for helping me out, please take your time.

Both worked fine for me. What version of ArcGIS are you using?

Re: Batch process multiple ASCII to Raster files

RumanaReaz — Mon, 29 Oct 2012 16:46:53 GMT

I am using ArcGIS 9.3 and Python 2.5 does it have a problem with version?

Re: Batch process multiple ASCII to Raster files

MathewCoyle — Mon, 29 Oct 2012 17:10:23 GMT

There shouldn't be a problem, but I don't have 9.3 to check it on. Check to make sure your output and input directories are proper and exist.

Re: Batch process multiple ASCII to Raster files

RumanaReaz — Mon, 29 Oct 2012 17:36:52 GMT

I have found out two things:

Error1: Run Time Error Not Initialized was due to license problem, ArcGIS license server was down, now it is fixed.

Error2: when the ascii file starts with number say 2011_001.asc it doesn't work, whereas it works with glsea1.asc.

2011_001.asc

Traceback (most recent call last):
File "C:/Users/rumana/Desktop/My EFDC/Data/AVHRR/BatchAscii2Raster3.py", line 18, in <module>
gp.ASCIIToRaster_conversion(os.path.join(inDir,InAsciiFile), os.path.join(OutRaster,InAsciiFile.rsplit(".")[0][:9]), "INTEGER")
ExecuteError: ERROR 999999: Error executing function.
Failed to rename the specified file
Failed to execute (ASCIIToRaster).

What will be the most effective thing to do in this case?

File naming was done by some third party (from where I downloaded the files) and I have around 200 files which needed to be converted to Raster format.

Re: Batch process multiple ASCII to Raster files

RumanaReaz — Sat, 11 Dec 2021 11:59:32 GMT

I have found out two things:

Error: when the ascii file starts with number say 2011_001.asc it doesn't work, whereas it works with glsea1.asc.

I tried renaming the files in the code as below: But also getting error message.

# Import system modules
import arcgisscripting, os

# Create the Geoprocessor object
gp = arcgisscripting.create(9.3)

# Set local variables
InAsciiFile = None
inDir = r"C:/Users/rumana/Desktop/My EFDC/Data/AVHRR/SST_Feb11"
OutRaster = "C:/Users/rumana/Desktop/My EFDC/Data/AVHRR/SST_Feb11/RasterUp_Feb11"

#gp.outputCoordinateSystem = r"Coordinate Systems\Projected Coordinate Systems\UTM\NAD 1983\NAD 1983 UTM Zone 17N.prj"

for InAsciiFile in os.listdir(inDir):
    if InAsciiFile.rsplit(".")[-1] == "asc":
        if InAsciiFile.startswith("2011"):
            InAsciiFilename = "SST" + InAsciiFile.replace("_glsea","")
            os.rename(InAsciiFile, InAsciiFilename)

            print InAsciiFilename
        # Process: ASCIIToRaster_conversion
        gp.ASCIIToRaster_conversion(os.path.join(inDir,InAsciiFilename), os.path.join(OutRaster,InAsciiFilename.rsplit(".")[0][:11]), "FLOAT")

Error Message:

Traceback (most recent call last):
File "C:\Users\rumana\Desktop\My EFDC\Data\AVHRR\BatchAscii2Raster3.py", line 18, in <module>
os.rename(InAsciiFile, InAsciiFilename)
WindowsError: [Error 2] The system cannot find the file specified

Can you help me out please?

Thanks.

Re: Batch process multiple ASCII to Raster files

RaphaelR — Mon, 05 Nov 2012 06:10:53 GMT

os.rename() needs a full path to the filename, or it won´t work.
Try this:

os.rename(os.path.join(inDir,InAsciiFile), os.path.join(inDir,InAsciiFilename)

Re: Batch process multiple ASCII to Raster files

RumanaReaz — Sat, 11 Dec 2021 11:59:35 GMT

os.rename() needs a full path to the filename, or it won´t work.
Try this:
os.rename(os.path.join(inDir,InAsciiFile), os.path.join(inDir,InAsciiFilename)

Thanks a lot for the help. I replaced os.rename with your code.
But I got Token Error: EOF in multi-line statement.

How do I fixed that?

# Import system modules
import arcgisscripting, os

# Create the Geoprocessor object
gp = arcgisscripting.create(9.3)

# Set local variables
InAsciiFile = None
inDir = r"C:/Users/rumana/Desktop/My EFDC/Data/AVHRR/SST_Feb11"
OutRaster = "C:/Users/rumana/Desktop/My EFDC/Data/AVHRR/SST_Feb11/RasterUp_Feb11"

#gp.outputCoordinateSystem = r"Coordinate Systems\Projected Coordinate Systems\UTM\NAD 1983\NAD 1983 UTM Zone 17N.prj"

for InAsciiFile in os.listdir(inDir):
    if InAsciiFile.rsplit(".")[-1] == "asc":
        if InAsciiFile.startswith("2011"):
            InAsciiFilename = "SST" + InAsciiFile.replace("_glsea","")
            os.rename(os.path.join(inDir,InAsciiFile), os.path.join(inDir,InAsciiFilename)
            InAsciiFile = InAsciiFilename
            print InAsciiFile
            
        # Process: ASCIIToRaster_conversion
        gp.ASCIIToRaster_conversion(os.path.join(inDir,InAsciiFilename), os.path.join(OutRaster,InAsciiFilename.rsplit(".")[0][:11]), "FLOAT")