topic Re: Cut a column, then write to a new file, different suffix in Python Questions

Cut a column, then write to a new file, different suffix

PaulHuffman — Wed, 24 Oct 2012 17:35:40 GMT

I need a little hint because I see now I don't understand enough about file input and output in Python.

I have some ascii lidar files that need to have the first column in each line removed to get them ready for the XYZI format of the ASCII 3D To Feature Class tool. I wanted to make a list of all the files in a folder with the suffix .gnd and output the modified data to files with the same name but a txt suffix so the original ascii files were not modified. What I got started with was:

env.workspace = "G:/elevation/lidar2010/points/asc_grounds" fileSuffix = ".gnd" txtList = arcpy.ListFiles("*" + fileSuffix)     for file in txtList:         for line in open("my file"):         parts = line.split(" ")         print " ".join(parts[1:4])

Then I was stuck. I guess I need a new line inside the first loop to write the file to a new file name with a .txt suffix, and I haven't even figured out how to get the file name stripped of the suffix.

What I used to do was I had an XP machine with Unix services for Windows installed that somehow let me use the Linux cut command inside a dos bat file:

for %%f in (*.gnd) do call :sub %%f %%~nf  GOTO:EOF  :sub cut -d" " -f2-4 %1 > %2.txt

The DOS %~ can give you a lot of useful file name manipulations. I installed Unix Services for Windows on a new Win 7 64 machine, and I don't understand why, but cut is not available to bat files on the new machine, although I can start a Unix C or K shell where cut is available but the %~n trick is not.

Re: Cut a column, then write to a new file, different suffix

JamesCrandall — Sun, 12 Dec 2021 04:10:40 GMT

and I haven't even figured out how to get the file name stripped of the suffix.

You could use a variation of this in your loop to strip the extension off the file name...


import os

NameMinusExt = os.path.splitext("TheFileName.gnd")[0]

## the result would be: TheFileName

Re: Cut a column, then write to a new file, different suffix

JamesCrandall — Sun, 12 Dec 2021 04:10:43 GMT

Maybe something like this... (completely untested)

txtList = arcpy.ListFiles("*" + fileSuffix)
for file in txtList:
   newFile = os.path.splitext(file)[0] + ".txt"

   ###...or maybe?

   newFile = str(os.path.splitext(file)[0]) + ".txt"

Re: Cut a column, then write to a new file, different suffix

PaulHuffman — Wed, 24 Oct 2012 21:19:56 GMT

This worked really well when I ran it in the Win 7 machine's C shell:

for f in *.gnd; do cut -d' ' -f2-4 "$f" > "${f%.*}.txt"; done

The resulting txt files had unix line endings, but that didn't seem to bother the ASCII 3D To Feature Class tool.

However, it would be nice to figure out how to make this work in python so I could loop through the files, chop off the first column, and run them through the ASCII 3D To Feature Class tool in one pass. I'll keep testing your suggestion.

Re: Cut a column, then write to a new file, different suffix

SendhilKolandaivel — Sun, 12 Dec 2021 04:10:45 GMT

Paul,

Try this code - Hopefully it should work.

import arcpy
import os

arcpy.env.workspace = r"G:\elevation\lidar2010\points\asc_grounds"
fileSuffix = ".gnd"
fileList = arcpy.ListFiles("*" + fileSuffix)

for f in fileList:
    print "Processing file", f, "..."
    # Text File to Write Out
    txtFile = os.path.splitext(f)[0] + ".txt"
    fobj = open(txtFile, 'w')
    
    for line in open(f):
        parts = line.split(" ")
        fobj.write(" ".join(parts[1:4]))

    fobj.close()

Regards,

Sendhil

Re: Cut a column, then write to a new file, different suffix

PaulHuffman — Thu, 25 Oct 2012 15:32:20 GMT

Thanks for the help. While you were posting this I came up with this that's almost the same. I need to read up on glob. Does arcpy.ListFiles do anything better than glob? Is ListFiles superfluous?

I suppose I'll run the ASCII 3D To Feature Class tool with a new file list and in a new if loop.

import glob import os.path  workdir = "G:/elevation/lidar2010/points/asc_grounds/" fileSuffix = ".gnd" outSuffix = ".txt" fileList = glob.glob( workdir+"*"+fileSuffix )  for f in fileList:   name, ext = os.path.splitext( f )   infile = open( f, "r" )    outfile = open( name+outSuffix, "w" )   for line in infile:     print >> outfile, " ".join( line.strip().split(" ")[1:4] )   outfile.close()   infile.close()

It was hard to import glob into a python window. The window would try to autocorrect it to "import global". I finally figured out I could type in "import glob" and then a space character to over ride it.