topic Re: Script for Making Individual Zip file for Each Shapefile? in Python Questions

Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Wed, 29 May 2013 12:47:40 GMT

I posted this in the Geoprocessing forum and then I thought I should post it here - sorry if this is redundant.

I need to take all of the shapefiles in a directory and compress them into individual zip files. I need to take, for example, the roads shapefile and get all of the files named roads.* into roads.zip, the schools shapefile, schools.* into schools.zip, and so on, for hundreds of shapefiles.

Re: Script for Making Individual Zip file for Each Shapefile?

Anonymous User — Sun, 12 Dec 2021 16:22:59 GMT

This worked for me:

import arcpy, os
from os import path as p
import zipfile
arcpy.overwriteOutput = True


def ZipShapes(path, out_path):
    arcpy.env.workspace = path
    shapes = arcpy.ListFeatureClasses()

    # iterate through list of shapefiles
    for shape in shapes:
        name = p.splitext(shape)[0]
        print name
        zip_path = p.join(out_path, name + '.zip')
        zip = zipfile.ZipFile(zip_path, 'w')
        zip.write(p.join(path,shape), shape)
        for f in arcpy.ListFiles('%s*' %name):
            if not f.endswith('.shp'):
                zip.write(p.join(path,f), f)
        print 'All files written to %s' %zip_path
        zip.close()

if __name__ == '__main__':

    path = r'C:\Shape_test\Census_CedarCo'
    outpath = r'C:\Shape_outputs'

    ZipShapes(path, outpath)

Just fill in your own variables for the ones highlighted in red. The "path" variable is the workspace containing all shapefiles to be zipped. The "outpath" variable is the output location for all your zipped folders for each shapefile.

Edit: I added an additional part to get rid of the subfolders created in the zip (the arcname argument for the zip.write).

Re: Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Wed, 29 May 2013 17:53:19 GMT

Thanks, Caleb, that worked!

One thing I noticed, though is that it did not actually compress the files (they were archived in zip files but not compressed).
I changed a line
from this:
        zip = zipfile.ZipFile(zip_path, 'w')

to this:
        zip = zipfile.ZipFile(zip_path, 'w',
        compression=zipfile.ZIP_DEFLATED)

and now the zipped files are compressed.

Thanks again for your help.

Jacob

Re: Script for Making Individual Zip file for Each Shapefile?

Anonymous User — Wed, 29 May 2013 18:07:35 GMT

One thing I noticed, though is that it did not actually compress the files (they were archived in zip files but not compressed).

Ahh, good eye! I did not realize that they were not getting compressed. I suppose I just assumed that automatically happened when writing the zip files. That's good to know. Glad it did the trick tho!

Re: Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Thu, 30 May 2013 20:29:55 GMT

So the script above works perfectly, with ArcGIS 10. But I just realized that it has to run in ArcGIS 9.3.1 instead! I tried importing arcgisscripting instead of arcpy and changed code from, for example,
shapes = arcpy.ListFeatureClasses()
to
shapes = gp.ListFeatureClasses()
but I get errors.

It seems like it does not know what the workspace is, so at least the first error is in here where the workspace is defined:
def ZipShapes(path, out_path):
gp.env.workspace = path
shapes = gp.ListFeatureClasses()

I appreciate any suggestions.

Re: Script for Making Individual Zip file for Each Shapefile?

JasonScheirer — Thu, 30 May 2013 21:12:08 GMT

Set gp.workspace, not gp.env.workspace

Re: Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Thu, 30 May 2013 21:22:23 GMT

I tried gp.workspace = path and I get a "object is not iterable" error. Below is my attempt at 9.3.1 code and below that is the error. I am sure someone will see something obvious that I am missing. Edit: Here the indents are missing - the indents don't show up when I pasted the code here but you get the idea.

#Begin code
import sys, string, arcgisscripting, os
from os import path as p
import zipfile

# Create the Geoprocessor object
gp = arcgisscripting.create()

# arcpy.overwriteOutput = True
gp.overwriteoutput=1

def ZipShapes(path, out_path):
    gp.workspace = path
    shapes = gp.ListFeatureClasses()

    # iterate through list of shapefiles
    for shape in shapes:
        name = p.splitext(shape)[0]
        print name
        zip_path = p.join(out_path, name + '.zip')
        zip = zipfile.ZipFile(zip_path, 'w',
        compression=zipfile.ZIP_DEFLATED)
        zip.write(p.join(path,shape), shape)
        for f in gp.ListFiles('%s*' %name):
            if not f.endswith('.shp'):
                zip.write(p.join(path,f), f)
        print 'All files written to %s' %zip_path
        zip.close()

if __name__ == '__main__':

    path = r'T:\\cotiss\\CobleJ\\shape2zip\\address'
    outpath = r'T:\\cotiss\\CobleJ\\shape2zip\\Shape_outputs'

    ZipShapes(path, outpath)
#End of code

HERE is the ERROR:
Traceback (most recent call last):
File "T:\cotiss\CobleJ\shape2zip\Shape2Zip_931.py", line 34, in <module>
    ZipShapes(path, outpath)
File "T:\cotiss\CobleJ\shape2zip\Shape2Zip_931.py", line 16, in ZipShapes
    for shape in shapes:
TypeError: 'geoprocessing list object' object is not iterable

Re: Script for Making Individual Zip file for Each Shapefile?

JasonScheirer — Thu, 30 May 2013 21:56:25 GMT

The list functions don't return lists if your geoprocessor is set above 9.3. To make it run in 9.3 mode, do this:

gp = arcgisscripting.create(9.3)

This will ensure the same behavior of this script even in 10.0/10.1. Then, use this idiom:

for shape in iter(shapes.next, None):

Since it returns a geoprocessing list object and not a Python list in 9.3.

Re: Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Fri, 31 May 2013 14:29:18 GMT

I don't think ListFiles is recognized in 9.3.1. I seem to have a problem with that.

Re: Script for Making Individual Zip file for Each Shapefile?

Anonymous User — Sun, 12 Dec 2021 16:23:01 GMT

You can use the os.walk() and fnmatch to get the file names:

import fnmatch, os

for shape in shapes:
    name = p.splitext(shape)[0]:
    zip_path = p.join(out_path, name + '.zip')
    zip = zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED)
    zip.write(p.join(path,shape), shape)
    for path, dirs, files in os.walk(path):
        for f in files:
            if fnmatch.fnmatch(f, '%s*' %shape):
                zip.write(p.join(path,f), f)
    print 'All files written to %s' %zip_path
    zip.close()

By using fnmatch you can provide a pattern for it to search for when looking through files so if your shapefile name is roads.shp, it can search all files that are like 'roads*'.

Re: Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Sat, 11 Dec 2021 16:55:51 GMT

Sorry to keep bugging the forum about this. I get the error, "NameError: global name 'zipflie' is not defined". And I am running this in ArcGIS 9.3.1. Unfortunately our shop is still using 9.3.1 on some machines and 10.0 on others so I have to support some scripts for both.

import sys, os, string, arcgisscripting, fnmatch
from os import path as p
import zipfile

# Create the Geoprocessor object
gp = arcgisscripting.create()

# arcpy.overwriteOutput = True
gp.overwriteoutput=1

def ZipShapes(path, out_path):
    gp.workspace = path
    shapes = gp.ListFeatureClasses("*")

    # iterate through list of shapefiles
    #for shape in shapes:
    for shape in iter(shapes.next, None):
        name = p.splitext(shape)[0]
        print name
        zip_path = p.join(out_path, name + '.zip')
        zip = zipfile.ZipFile(zip_path, 'w', zipflie.ZIP_DELFLATED)
        for path, dirs, files in os.walk(path):
            for f in files:
                if fnmatch.fnmatch(f, '%s*' %shape):
                    zip.write(p.join(path,f), f)
        print 'All files written to %s' %zip_path
        zip.close()

if __name__ == '__main__':

    path = r'T:\\cotiss\\CobleJ\\shape2zip\\address'
    outpath = r'T:\\cotiss\\CobleJ\\shape2zip\\Shape_outputs'

    ZipShapes(path, outpath)

Re: Script for Making Individual Zip file for Each Shapefile?

Anonymous User — Sat, 11 Dec 2021 16:55:53 GMT

I get the error, "NameError: global name 'zipflie' is not defined".

Apparently you mistyped 'zipflie.ZIP_DEFLATED' when it was supposed to be zipfile.ZIP_DEFLATED. Fix this line:

zip = zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DELFLATED)

Re: Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Sat, 11 Dec 2021 16:55:56 GMT

Well, it's very close! Just one problem. It will only make one complete zip file out of the first shapefile in my source directory.

Suppose my folder has four shapefiles (this is the folder called "address"). Let's say the shapefiles are named parcels.* and parcel_join.* and pointaddr.* and zipcodes.* - in my output folder "Shape_outputs" there will be four zip files. The first zip file of the first shapefile (parcels.zip will be okay, it has all of the files making up the parcels shapefile). The next two zip files will be empty. The last zip file, zipcodes.zip, will only contain one file from the zipcodes shapefile - it has the zipcodes.shp file but not the zipcodes.dbf, .sbn, .sbx, etc. It takes a few seconds to make the first zip file judging from the message it prints, then in a flash it prints the names of the other three files and the script stops without an error message.

import sys, os, string, arcgisscripting, fnmatch
from os import path as p
import zipfile

# Create the Geoprocessor object
gp = arcgisscripting.create()

# arcpy.overwriteOutput = True
gp.overwriteoutput=1

def ZipShapes(path, out_path):
    gp.workspace = path
    shapes = gp.ListFeatureClasses("*")

    # iterate through list of shapefiles
    #for shape in shapes:
    for shape in iter(shapes.next, None):
        name = p.splitext(shape)[0]
        print name
        zip_path = p.join(out_path, name + '.zip')
        zip = zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED)
        for path, dirs, files in os.walk(path):
            for f in files:
                if fnmatch.fnmatch(f, '%s*' %shape):
                    zip.write(p.join(path,f), f)
        print 'All files written to %s' %zip_path
        zip.close()

if __name__ == '__main__':

    path = r'T:\\cotiss\CobleJ\shape2zip\address'
    outpath = r'T:\\cotiss\CobleJ\shape2zip\Shape_outputs'

    ZipShapes(path, outpath)

Re: Script for Making Individual Zip file for Each Shapefile?

Anonymous User — Sun, 12 Dec 2021 16:23:04 GMT

Okay, I think I got it now. I cannot test on 9.3 but this did work on 10.1 with arcgisscripting. I went with the glob method instead and it got all the files:

import arcgisscripting, os, glob, zipfile
from os import path as p

# Create the Geoprocessor object
gp = arcgisscripting.create()

# arcpy.overwriteOutput = True
gp.overwriteoutput=1

def ZipShapes(path, out_path):
    gp.workspace = path
    shapes = gp.ListFeatureClasses("*")

    # iterate through list of shapefile
    for shape in iter(shapes.next, None):
        name = p.splitext(shape)[0]
        print name
        zip_path = p.join(out_path, name + '.zip')
        zip = zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED)
        os.chdir(path)
        for files in glob.glob('%s*' %name):
            zip.write(p.join(path,files), files)
        print 'All files written to %s' %zip_path
        zip.close()

if __name__ == '__main__':

    path = r'T:\\cotiss\CobleJ\shape2zip\address'
    outpath = r'T:\\cotiss\CobleJ\shape2zip\Shape_outputs'

    ZipShapes(path, outpath)

Re: Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Sun, 02 Jun 2013 04:25:57 GMT

Caleb, thanks for your help! There is just one more quirk. If I have shapefiles with similar names, it will make one zip file for each shapefile but the zip files will have all of the shapefiles with similar names instead of just the shapefile with the exact name. I have two shapefiles, called "parcel" and "ParcelAd" and it makes two zip files with both names. But if I open parcel.zip it has everything for both parcel.* and ParcelAd.* and if I open ParcelAd.zip it will have everything for both parcel.* and ParcelAd.*

Re: Script for Making Individual Zip file for Each Shapefile?

T__WayneWhitley — Sun, 12 Dec 2021 16:23:06 GMT

...minor clarification on this line I think will fix your problem:

for files in glob.glob('%s.*' %name):

For this minor but annoying quirk, you almost answered your own question when you said "...it has everything for both parcel.*..." --- notice you inserted the period. Well you have to do the same in the code. If you leave off the period it not only 'globs' the component files making up your shapefile but if other shapefiles contain the same root name i.e. root + file ext or root + suffix + file ext, then both sets of file components are returned -- in other words, the wildcard allows it. So to 'tighten' the wildcard filter you need the period. Also, careful with the xml component file...

Powerful subtlety... I've stumbled on this as well.

Enjoy,
Wayne

Re: Script for Making Individual Zip file for Each Shapefile?

JacobCoble — Sun, 02 Jun 2013 22:54:16 GMT

Success! Thanks so much Caleb, Jason and Wayne. I ran both the 10.0 and the 9.3.1 scripts and everything is fine - they no longer get "confused" by files with similar names, and it makes sense.

So on the 9.3.1 script the line should be
for files in glob.glob('%s.*' %name):
and in the 10.0 script the equivalent line should be
for f in arcpy.ListFiles('%s.*' %name):

Thanks!