Moving files with same initial name to the match folder

BrunoDeus · ‎02-10-2017

Hi,

I would like move automatically files witch are scenes of Landsat 8 (each scene has 9 bands/files + MTL) inside a folder, to folders which have the same name from initial name's files (these directories need be created).

Each band file is Ziped separately, beyond the metadata (.txt). Next line shows a instance from the files names that already exist and a good name for a directory.

e.g. LO82270682016229CUB00_B1.tif.zip, LO82270682016229CUB00_B2.tif.zip to directory LO82270682016229.

I tried too a script which needs imports the module RE, see web page <moving multiple files>. But, unsuccessfully, the python doesn't works - It not even shows RE module when I type import RE + enter.

The CPU's softwares are Arcmap 10.1 - Python 2.7.2 - Windows 8.

>>> import shutil
... import os
...
... source = 'C:\Teste_Auto_CLEIA\Ima_compactadas\Landsat'
... dest1 = 'C:\Teste_Auto_CLEIA\Ima_compactadas\L82270682016229'
... dest2 = 'C:\Teste_Auto_CLEIA\Ima_compactadas\L82270692016229'
...
... files = os.listdir(source)
...
... for f in files:
... if (f.startswith("LO82270682016229") or f.startswith("LC82270682016229")):
... shutil.move(source, dest1) #12 line
... elif (f.startswith("LO82270692016229") or f.startswith("LC82270692016229")):
... shutil.move(source, dest2)
...
Runtime error
Traceback (most recent call last):
File "<string>", line 12, in <module>
File "C:\Python27\ArcGIS10.1\Lib\shutil.py", line 298, in move
copy2(src, real_dst)
File "C:\Python27\ArcGIS10.1\Lib\shutil.py", line 127, in copy2
copyfile(src, dst)
File "C:\Python27\ArcGIS10.1\Lib\shutil.py", line 81, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'C:\\Teste_Auto_CLEIA\\Ima_compactadas\\Landsat'

Thanks in advance!

MitchHolley1 · ‎02-10-2017

You have to specify a complete file path to be moved.

For example, you're saying to copy 'source' which is a folder path. You want to include the filename which resides in that folder.

folder = r'C:\Test\Path\to\folder'
destination r'C:\Test\Path\to\destination'
file = '12540382834.zip'

shuitl.move(folder+'/'+file, destination)‍‍‍‍‍

BrunoDeus · ‎02-10-2017

Mitch,

I'm grateful for your help. I tried do it, but the same line (now is 14) is with trouble.

>>> import shutil
... import os
...
... source = r'C:\Test\Landsat'# I shorted it to easier - The folder with all files.
... dest1 = r'C:\Test\L8227068'# I shorted it to easier
... dest2 = r'C:\Test\L8227069'# I shorted it to easier
... file1 = 'LO82270682016229*'# I changed it to work with all bands
... file2 = 'LO82270692016229*'# I changed it to work with all bands
...
... files = os.listdir(source)
...
... for f in files:
... if (f.startswith("LO8227068") or f.startswith("LC8227068")):
... shutil.move(source+'/'+file1, dest1)#14 line
... elif (f.startswith("LO8227069") or f.startswith("LC8227069")):
... shutil.move(source+'/'+file2, dest2)
...
Runtime error
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "C:\Python27\ArcGIS10.4\Lib\shutil.py", line 302, in move
copy2(src, real_dst)
File "C:\Python27\ArcGIS10.4\Lib\shutil.py", line 130, in copy2
copyfile(src, dst)
File "C:\Python27\ArcGIS10.4\Lib\shutil.py", line 82, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 22] invalid mode ('rb') or filename: 'C:\\Test\\Landsat/LO82270682016229*'

Thanks!

JoshuaBixby · ‎02-11-2017

You are creating invalid file names, i.e., 'C:\\Test\\Landsat/LO82270682016229*' . I recommend using os.path when building file system path names. That said, I believe it is your use of asterisks/wildcards that is causing the error here.

BrunoDeus1 · ‎02-12-2017

I aready replaced those lines for:

... file1 = 'LO82270682016229CUB00_B1.tif.zip'# I changed it to work with 1 band
... file2 = 'LO82270692016229CUB00_B1.tif.zip'# I changed it to work with 1 band

But, the script stops after remove the "file1" to a file without extension with the same name from the folder that It should go. i.e. the scripts needs be shifted to create folders from the last piece of directory's destine.

Runtime error
Traceback (most recent call last):
   File "<string>", line 14, in <module>
   File "C:\Python27\ArcGIS10.4\Lib\shutil.py", line 302, in move
      copy2(src, real_dst)
   File "C:\Python27\ArcGIS10.4\Lib\shutil.py", line 130, in copy2
      copyfile(src, dst)
   File "C:\Python27\ArcGIS10.4\Lib\shutil.py", line 82, in copyfile
      with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'C:\\Test\\Landsat/LO82270682016229CUB00_B1.tif.zip'

I'm looking for the python page from your link and think and trying the best way. Also, I should try do a script without the name from each file to avoid update (re-script) it every time that I will use it.

I pretend put the best try here, when I find the best function. I'm considering too others python's pages.

JoshuaBixby · ‎02-13-2017

The last 3 lines of the error message are telling you the copyfile function, which is being called by the move function, cannot find the file 'C:\\Test\\Landsat/LO82270682016229CUB00_B1.tif.zip' to open and read. I don't have anything else to offer, the Python interpreter isn't finding the file using the path you are passing it.

BrunoDeus1 · ‎02-13-2017

I ran the script and looks that later him remove the file, It doesn' find the file at the original path. I checked the file's name and the basename at statement, both of them match.

BrunoDeus1 · ‎02-15-2017

My best shot was do 1 script to create the folders and run it first. I avoid script it with the second one for precaution, I think that is possible the script moves the band's file to a directory and later, that this directory be replaced for a new empty folder.

In the first script, I just wanna create path with the same basename onetime. Is there anyone who knows how to do it?

>>> import fnmatch #This script create the folders
...
... source = r'C:\Test\Landsat'
...
... for file in os.listdir(source):
... if fnmatch.fnmatch(file, 'L?8*'):
... arcpy.CreateFolder_management(test, file[:16])
... print file[:16]#This line is just to know which folders have been created
...
LO82270682016229
LO82270682016229
LO82270692016229
LO82270692016229

And at the last script, it fails after move the first file to the right path.

>>> import shutil
... import os, fnmatch #put fnmatch
...
... source = r'C:\Test\Landsat'
... dest1 = r'C:\Test\LO82270682016229'#put the same name from the folders created one step before
... dest2 = r'C:\Test\LO82270692016229'#put the same name from the folders created one step before
... file1 = 'LO82270682016229CUB00_B1.tif.zip'# I changed it to work with 1 band
... file2 = 'LO82270692016229CUB00_B1.tif.zip'# I changed it to work with 1 band
...
... files = os.listdir(source)
...
... for f in files:#12 line - This line is OK
... if (f.startswith("LO8227068") or f.startswith("LC8227068")):
... shutil.move(source+'/'+file1, dest1)
... elif (f.startswith("LO8227069") or f.startswith("LC8227069")):
... shutil.move(source+'/'+file2, dest2)
...
Runtime error
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "C:\Python27\ArcGIS10.4\Lib\shutil.py", line 292, in move
raise Error, "Destination path '%s' already exists" % real_dst
Error: Destination path 'C:\Test\LO82270682016229\LO82270682016229CUB00_B1.tif.zip' already exists

*I did a script that joined both of them, and It created the folders like the first. But it failed so soon it moved one file, by the same way that these two scripts above did. You can see the message error below:

Runtime error
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "C:\Python27\ArcGIS10.4\Lib\shutil.py", line 292, in move
raise Error, "Destination path '%s' already exists" % real_dst
Error: Destination path 'C:\Test\LO82270682016229\LO82270682016229CUB00_B1.tif.zip' already exists

So, if has someone to help, I will be thankful!

JoshuaBixby · ‎02-16-2017

The error message indicates a copy of the file you are trying to move already exists in the location you are trying to move it to. If you open Windows Explorer and check, are files with the same name in both locations?

If the file already exists, and you don't want to overwrite it, a simple check to see if the file exists in the new location before moving/copying will prevent the error message.

BrunoDeus1 · ‎02-16-2017

Before script runs, it's all right. Is the script that creates the folder and moves the file. But, the script looks the path with the file that himself moved and see an error.