topic How can I modify zip output from arcpy.ExtractData_server sample in Python Questions

How can I modify zip output from arcpy.ExtractData_server sample

RebeccaStrauch__GISP — Sat, 11 Dec 2021 16:55:21 GMT

Q: Is there a simple python way to reoganize a zip?

In an attempt to simplify the conversion from our regular updates from a feature class in a FGDB to zipped shapefile for uploading into ArcGIS Online for users to download, I'm modifying the model

(converted to python snippets ). This sample, per the help Extract Data—Help | ArcGIS for Desktop

 Extracts selected layers in the specified area of interest to a specific format and spatial reference. The extracted data is then written to a zip file.

Caution:

This tool is intended primarily for use as part of a geoprocessing service. When using this tool as part of a geoprocessing service, copy the tool into a custom toolbox, edit the model, and reconfigure it as necessary. For step-by-step instructions on how to make, use, and configure a geoprocessing service using this tool, see Geoprocessing service example: Clip And Ship.

Although it is meant to be used as a GP tool, I have been able to get my script to work, but the output it not AGOL friendly. That is, it adds another layer/folder in the zip structure... e.g. myfile.zip\zipfolder#\myshapefile.* This "zipfolder#" seems to be hard coded into the ExtractData_server. I know there are other scripts to open/read/write to a .zip, but this actually works nice, except for that added folder.

Is there a simple python way to reoganize a zip. If it requires an open-cursor-write-close like the samples I've seen, I may just go that route and skip the ExtractData command, but other than that extra layer, this custom version works great for my needs (just one shapefile to a zip)

I've testing in 10.2.2 and 10.3.1, Catalog and map (python command window, not as a tool yet)

outputted format:

would like the "zipfolder#" level removed.

my test script:

# Import arcpy module
import arcpy
import os
import time
from time import localtime

def myMsgs(message):
    arcpy.AddMessage("{0}".format(message))
    print("{0}".format(message)  )

# Script arguments...
sourceGDB = arcpy.GetParameterAsText(0)
if not sourceGDB:
    sourceGDB = r"d:\data\wc\Master_update.gdb"

sourceFDS = arcpy.GetParameterAsText(1)
if not sourceFDS:
    sourceFDS = "DWCMasters"   
myMsgs(sourceFDS)
sourceFDS = os.path.join(sourceGDB, sourceFDS)
myMsgs(sourceFDS)

GMULayers = arcpy.GetParameterAsText(2)
if not GMULayers:
    GMULayers = os.path.join(sourceGDB, "GMULayers")
myMsgs(GMULayers)

# export as shape files...will create folder for pubYear     
targetFolder = arcpy.GetParameterAsText(3)   
if not targetFolder:
    targetFolder = r"d:\_forAGOLdist"
pubYear = localtime().tm_year
targetFolder = os.path.join(targetFolder, ("{0}_test".format(pubYear)))
if not os.path.isdir(targetFolder):
    os.mkdir(targetFolder)
    myMsgs("Creating target folder: {0}".format(targetFolder))
else:
    myMsgs("Output shapes will be written to: {0}".format(targetFolder))

Feature_Format = "Shapefile - SHP - .shp"
Raster_Format = '#'
Spatial_Reference = "Same As Input"
Custom_Spatial_Reference_Folder = ""

# subs
inSubsFC = os.path.join(GMULayers, "Subunits")   
inSubFL = "Subunits_Layer"
Layers_to_Clip = inSubFL
Area_of_Interest = Layers_to_Clip
Output_Zip_File = os.path.join(targetFolder, "Subunits.zip")
arcpy.MakeFeatureLayer_management(inSubsFC,inSubFL)
arcpy.ExtractData_server(Layers_to_Clip, Area_of_Interest, Feature_Format, Raster_Format, Spatial_Reference, Custom_Spatial_Reference_Folder, Output_Zip_File)

# Spec Areas
inSAFC = os.path.join(sourceFDS, "SpecialAreas")
inSAFL = "SpecArea_Layer"
qryActive = "status = 'a'"
Layers_to_Clip = inSAFL
Area_of_Interest = Layers_to_Clip
Output_Zip_File = os.path.join(targetFolder, "SpecAreas.zip")
field_info = ("OBJECTID OBJECTID VISIBLE NONE;Shape Shape VISIBLE NONE;TYPE TYPE VISIBLE NONE;AREANAME AREANAME VISIBLE NONE;SHORTNAME SHORTNAME VISIBLE NONE;GMU GMU VISIBLE NONE;REGION REGION VISIBLE NONE;gisNotes gisNotes HIDDEN NONE;created_user created_user HIDDEN NONE;created_date created_date HIDDEN NONE;last_edited_user last_edited_user HIDDEN NONE;last_edited_date last_edited_date HIDDEN NONE;status status HIDDEN NONE;Shape_Length Shape_Length HIDDEN NONE;Shape_Area Shape_Area HIDDEN NONE;SqMi SqMi VISIBLE NONE")

arcpy.MakeFeatureLayer_management(inSAFC, inSAFL,qryActive,"#",field_info)
arcpy.ExtractData_server(Layers_to_Clip, Area_of_Interest, Feature_Format, Raster_Format, Spatial_Reference, Custom_Spatial_Reference_Folder, Output_Zip_File)

I've found a couple py-zip type samples in my searches, but again, they all require the O-R-Wr-Cl operation, which will be ok, but then I would skip the command, since this is a simple shapefile to .zip I'm trying to do.

Thanks.

edit: added line 58, so output changed

more edits....lots of typos...must be getting tired.

Re: How can I modify zip output from arcpy.ExtractData_server sample

DanPatterson_Retired — Tue, 28 Jun 2016 01:46:04 GMT

So this 12.4. zipfile — Work with ZIP archives — Python 2.7.12 documentation is no good? haven't looked at it too deeply though

Re: How can I modify zip output from arcpy.ExtractData_server sample

RebeccaStrauch__GISP — Tue, 28 Jun 2016 02:02:39 GMT

I looked at that, but it gets into the open-loop-write-close process. I liked the simplicity of the Extract tool understanding shapefiles off the bat.

I found GeospatialPython.com: Introducing the Python Shapefile Library or a link to the same library on another page, but was a bit confused by what I got on the other page...but plan to look more at this in the am. Some of the links on that page are broken (2010) but this works https://pythonhosted.org/Python%20Shapefile%20Library/

...actually this github version looks more promising (updated more recently) GitHub - GeospatialPython/pyshp: This library reads and writes ESRI Shapefiles in pure Python. I don't think it does .zip files direct, but does understand r/w or shapefiles somewhat.

Again, the extract worked great, and I know the extra folder structure is there for a reason, just not what I need or can use for my AGOL updates.

For my immediate needs, I just did it manually, but would be nice to have it as part of my addin.

edit: btw Dan, thanks for the response. Never hurts to ask me if I've seen it since sometimes it is easy to miss something.

Re: How can I modify zip output from arcpy.ExtractData_server sample

DanPatterson_Retired — Tue, 28 Jun 2016 02:39:52 GMT

Rebecca, I haven't got time to check and I am working on the iThing, but there are some examples of zipfile and shutil to create zips... but again, I don't know if they do the inside folder thing.

python - How to create a zip archive of a directory - Stack Overflow

Create .zip in Python? - Stack Overflow

The zipfile module amongst others

good luck

Re: How can I modify zip output from arcpy.ExtractData_server sample

FC_Basson — Sat, 11 Dec 2021 16:55:24 GMT

The zipfile module works well. You can zip the files in a folder like so:

import zipfile
outputSHPzip = arcpy.env.scratchFolder + "\\shpzip.zip"
shpZip = zipfile.ZipFile(outputSHPzip, "w")
for root, dirs, files in os.walk(ShapefileFolder):
   for f in files:
      shpZip.write(os.path.join(root, f), f)
shpZip.close()

Re: How can I modify zip output from arcpy.ExtractData_server sample

RebeccaStrauch__GISP — Tue, 28 Jun 2016 21:44:22 GMT

I'm giving FC Basson the credit for the right answer, even thought that wasn't the answer I was hoping for, so I can close this out for now. It's simialr to all the links I had found and Dan had pointed to. I'll keep working on it, but will be off this project for a few weeks.

In the long run, I would have liked a way to override the folder creation in the ExtractData tool, and may inquire about options when at the UC (although my guess is I will be pointed to same answer).

Re: How can I modify zip output from arcpy.ExtractData_server sample

AlexP_ — Fri, 25 Aug 2017 12:24:03 GMT

Is there a opposite way? I am trying to write a python script about zip file (“TEST.gbd.zip”) containing a File Geodatabase with both feature classes from FTP County and I need it to download.

Re: How can I modify zip output from arcpy.ExtractData_server sample

RebeccaStrauch__GISP — Mon, 28 Aug 2017 23:22:16 GMT

Is there a opposite way? I am trying to write a python script about zip file

I'm not exactly sure what you are asking, but if you haven't already you may want to ask this as a new question.

See https://community.esri.com/community/help-and-feedback/blog/2016/08/15/community-news-and-tips-how-to-ask-questions-and-get-answers?sr=search&searchId=c97f3da8-ebe1-41cd-9fce-970939e028f9&searchIndex=0‌ if you haven't posted a question before, and if you have code to post, take a look at https://community.esri.com/people/curtvprice/blog/2014/09/25/posting-code-blocks-in-the-new-geonet?sr=search&searchId=c7660506-40c1-4a6d-96f3-9bb8f1c54281&searchIndex=2‌

Re: How can I modify zip output from arcpy.ExtractData_server sample

AlexP_ — Sat, 11 Dec 2021 16:55:26 GMT

Rebecca,

Thank you for your reply. I posted it last week but no one has answer yet. I can't post full script because it is not public. Is there a way to show private to someone and someone might know what I am talking about?

I posted this question on geonet:

There is a zip file (“TEST.gbd.zip”) containing a File Geodatabase with both feature classes from FTP County and I need it to download. What is the python script for this case? Please advise. Thank you.

For example, I pulled this part of script like this: but what about zip file? It has compressed zip file.

# List of Other files to download
# Syntax: County file name with extension separated by colon, then
# the FTP Folder name
CountyOtherFiles = {
 'pa_parceldata.dat':r'GISdata/Pa_data/',
 'pa_parceldata.txt':r'GISdata/Pa_data/',
 'pa_legal_ln.dat':r'GISdata/Pa_data/',
 'pa_legal_ln.txt':r'GISdata/Pa_data/',
}

I am new to python script. I have been researched thru here and having difficult to find a right answer. I have compressed zipfile in FTP WinSCP.

I am trying to extract or unzip in python from FTP Winscp? It has compressed zip file in FTP. I have Winscp connect to FTP. I am not sure what is the right script for this case. I already have script set up that has been created by someone else. Please advise. Thank you.

I tried to use this script but I got an error.

ZipFile.extractall(path='GISData/Pubshp', members= 'PaFiles.gdb.zip')NameError: name 'ZipFile' is not defined

Re: How can I modify zip output from arcpy.ExtractData_server sample

RebeccaStrauch__GISP — Tue, 29 Aug 2017 16:31:43 GMT

I found where you posted it, and it's good that you tagged a couple other spaces. If no one else takes a look or answers it, I can take a look at it in a few hours (during lunch). I followed you, so if you follow me you can send the .zip in a direct message (DM) thru geonet.