nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Uploading and using data to a geoprocessing service

TomSchuller

Hy,
I'm trying to build a simple geoprocessing service which should buffer an uploaded shapefile.

This is the scenario:
1) uploading shapefile to geoprocessing service
2) buffering the shapefile with a given parameter
3) returning the buffered shapefile

The buffering is not the problem, but the handling of the upload and reusing it in the model or python part.
As far as I read, I should have to use a Datafile parameter for upload, but I'm always getting this error while running it:
ExecuteError: Failed to execute. Parameters are not valid. ERROR 000735: input: Value is required Failed to execute

Can anybody help me to use uploads on geoprocessing services?

Thanks,
Tom

タグ付きの投稿をもっと見る

Accepted answers

All comments

KevinHibma

Upload to a GP Service is sort of a multistep process.

You're correct, you do want to use a "file" (GPDataFile) as the input parameter.
If you're using Destkop to consume the service, desktop will send the file to the server.
If you're using a webapp, you're responsible for getting the file to the server. (Server at 10.1 has some upload capabilities in REST you can leverage. If using 10.0 or prior you'll need to devise a way to get the file to the server your self)

Once the file has been upload, because you're talking shape file, you'll have to unzip it. (shapefile being multiple files to make up a single one).
With it unzipped, you can then pass the file to your service.

Based on your error - your file didnt upload, or you never sent it. Whats the client to consume the GP Service - a web app?

TomSchuller

thanks for the quick answer.
Yes, I know it's a multistep process.

I have the problem that I don't see howto handle an uploaded file over gpservice in a model or python script.
Did you have an example of your "unzipModel" scenario, like:
user is uploading data to gpservice (10.1 upload capabilities in rest)
user is calling the unzipModel with the upload-id as parameter (over rest url)
the model is unzipping the zip file into some directory

I just need to see how I can use the uploaded file in a python script or model directly.

Thanks,
Tom

TomSchuller

thanks for the quick answer.
Yes, I know it's a multistep process.

I have the problem that I don't see howto handle an uploaded file over gpservice in a model or python script.
Did you have an example of your "unzipModel" scenario, like:
user is uploading data to gpservice (10.1 upload capabilities in rest)
user is calling the unzipModel with the upload-id as parameter (over rest url)
the model is unzipping the zip file into some directory

I just need to see how I can reuse the uploaded file in a python script or model directly.

Thanks,
Tom

LaurynasGedminas

Hello,
Having the similar issue here, I created python script to upload text files to SDE,
published the script as GP service and it works in ArcMap and ArcGISExplorer.
In Flexviewer added this GP service with input parameter as "datafile" and i cant upload the files

Thanks,

Laurynas

TomSchuller

Hy,
I didn't yet used it in the FlexViewer but only manually over rest.

But it should point you to the right direction:
enable "upload" on your geoprocessing service
upload your file to the gp-service and remember the upload-id
call your gp-service with the datafile paramater as the text below:

For GPDatafile parameter, you have to indicate this full json representation of the upload item:
parameter = {"itemID":"i80dfa12f-52ed-4841-94ff-37f9c3f5dd6f"}

Tom

LaurynasGedminas

Thanks Tom,
I guess I was not clear about my problem
i dont know why flexviewer doesnt display "upload" button on its widget when input data type is set to "datafile"

Laurynas

BjornSvensson

If ESRI will update the geoprocessing widget so it will take a local file, I, and many other users will be very happy!

Thanks for the suggestion to support local file uploads in the Geoprocessor widget. We will take a look at this for version 3.2 of the Flex Viewer. It would most likely just follow the steps outlined by Dasa in the thread you listed: http://forums.arcgis.com/threads/64114-Uploads-capability-on-Geoprocessor.

As for the ArcGIS API for Flex - since the Flash API already supports file upload, I don't think this is something we would add to the ArcGIS API for Flex.

meriyalootka

Hy
I have a model for uploading datafile, every thing is ok and user's zip file extract correctly. But I can not use extracted shapefile. Is there any py code for using feature class in a scratchfolder? Can I have this part of your model?

Hy,
I'm trying to build a simple geoprocessing service which should buffer an uploaded shapefile.

This is the scenario:
1) uploading shapefile to geoprocessing service
2) buffering the shapefile with a given parameter
3) returning the buffered shapefile

The buffering is not the problem, but the handling of the upload and reusing it in the model or python part.
As far as I read, I should have to use a Datafile parameter for upload, but I'm always getting this error while running it:
ExecuteError: Failed to execute. Parameters are not valid. ERROR 000735: input: Value is required Failed to execute

Can anybody help me to use uploads on geoprocessing services?

Thanks,
Tom

meriyalootka

The datafile parameter input for the geoprocessing widget in the Viewer for Flex does not allow the user to upload a file to the server from their local machine. It only accepts paths that begin with http:// or https://

I was under the impression that this was going to be fixed at the 3.1 release, but the widget still won't accept a local file.

This thread indicates all the pieces are available if you have the capability of creating your own custom widget:

http://forums.arcgis.com/threads/64114-Uploads-capability-on-Geoprocessor

There are many of us that cannot create custom widgets. If ESRI will update the geoprocessing widget so it will take a local file, I, and many other users will be very happy!

http://forums.arcgis.com/threads/54441-Uploading-to-SDE-from-Flex-Viewer
http://forums.arcgis.com/threads/41089-datafile-input-broken-in-Geoprocessing-widget
http://forums.arcgis.com/threads/36560-Geoprocessing-widget-has-me-stumped
http://forums.arcgis.com/threads/60510-Geoprocessing-with-File-Input

KevinHibma

The following code (10.1 + only) walks through a directory and finds FeatureClasses (this example assumes a folder of "zip" files).
It then copies each one into a fGDB. You can change what it does, but the logic in it should start you off.
You'll notice that a directory is created inside the scratchFolder (the variable "ZipFolder") - this is where the uploaded ZipFile is extracted to. Then the code looks in there.

import arcpy
import zipfile
import os

inFile = arcpy.GetParameterAsText(0) 

# Create a folder in the scratch directory to extract zip to
zipFolder = os.path.join(arcpy.env.scratchFolder, "zipContents")
os.mkdir(zipFolder)


# Extract the zip contents
zip2Extract = zipfile.ZipFile(inFile, 'r')
zip2Extract.extractall(zipFolder)
zip2Extract.close()

# Create a folder in the scratch directory to hold the fgdb which will be downloaded
fgdbFolder = os.path.join(arcpy.env.scratchFolder, "fgdbOutput")
os.mkdir(fgdbFolder)

# Work through all the FeatureClasses inside the extracted zip folder
for dirpath, dirnames, filenames in arcpy.da.Walk(zipFolder, datatype="FeatureClass"):
    
    for filename in filenames:
        
        # You could replace the code below here with your own code to do what you want....
        arcpy.AddMessage("Copying: {}".format(filename))
        
        # Strip .shp from the filename when merging shapefiles
        if filename.endswith("shp"):
            outFilename = filename[:-4]
        else:
            outFilename = filename

        # Copy each featureclass into the output.gdb
        arcpy.CopyFeatures_management(os.path.join(dirpath, filename),
                                      os.path.join(fgdbFolder, "output.gdb", outFilename))

meriyalootka

import arcpy
import zipfile
import os

inFile = arcpy.GetParameterAsText(0) 

# Create a folder in the scratch directory to extract zip to
zipFolder = os.path.join(arcpy.env.scratchFolder, "zipContents")
os.mkdir(zipFolder)


# Extract the zip contents
zip2Extract = zipfile.ZipFile(inFile, 'r')
zip2Extract.extractall(zipFolder)
zip2Extract.close()

# Create a folder in the scratch directory to hold the fgdb which will be downloaded
fgdbFolder = os.path.join(arcpy.env.scratchFolder, "fgdbOutput")
os.mkdir(fgdbFolder)

# Work through all the FeatureClasses inside the extracted zip folder
for dirpath, dirnames, filenames in arcpy.da.Walk(zipFolder, datatype="FeatureClass"):
    
    for filename in filenames:
        
        # You could replace the code below here with your own code to do what you want....
        arcpy.AddMessage("Copying: {}".format(filename))
        
        # Strip .shp from the filename when merging shapefiles
        if filename.endswith("shp"):
            outFilename = filename[:-4]
        else:
            outFilename = filename

        # Copy each featureclass into the output.gdb
        arcpy.CopyFeatures_management(os.path.join(dirpath, filename),
                                      os.path.join(fgdbFolder, "output.gdb", outFilename))

Hy
Many thanks
Please help me about input and output parameters.

meriyalootka

import arcpy
import zipfile
import os

inFile = arcpy.GetParameterAsText(0) 

# Create a folder in the scratch directory to extract zip to
zipFolder = os.path.join(arcpy.env.scratchFolder, "zipContents")
os.mkdir(zipFolder)


# Extract the zip contents
zip2Extract = zipfile.ZipFile(inFile, 'r')
zip2Extract.extractall(zipFolder)
zip2Extract.close()

# Create a folder in the scratch directory to hold the fgdb which will be downloaded
fgdbFolder = os.path.join(arcpy.env.scratchFolder, "fgdbOutput")
os.mkdir(fgdbFolder)

# Work through all the FeatureClasses inside the extracted zip folder
for dirpath, dirnames, filenames in arcpy.da.Walk(zipFolder, datatype="FeatureClass"):
    
    for filename in filenames:
        
        # You could replace the code below here with your own code to do what you want....
        arcpy.AddMessage("Copying: {}".format(filename))
        
        # Strip .shp from the filename when merging shapefiles
        if filename.endswith("shp"):
            outFilename = filename[:-4]
        else:
            outFilename = filename

        # Copy each featureclass into the output.gdb
        arcpy.CopyFeatures_management(os.path.join(dirpath, filename),
                                      os.path.join(fgdbFolder, "output.gdb", outFilename))

I need a model for extracting a zip file (contain shape file) to a feature class. I need to use from extracted shape file in my model. please help me more.

meriyalootka

I need a model for extracting a zip file (contain shape file) to a feature class. I need to use from extracted shape file in my model. please help me more.

KevinHibma

You have to use a script tool. Theres no tool in the toolbox (to put into modelbuilder) to extract a zipfile.
You can wrap the above script as a script tool and put that into a model if it serves your purpose.

meriyalootka

You have to use a script tool. Theres no tool in the toolbox (to put into modelbuilder) to extract a zipfile.
You can wrap the above script as a script tool and put that into a model if it serves your purpose.

I,m trying more than 1 month, but result is very bad. I tested last script, but can not use it!!
Last script return this error:
IndentationError: unexpected indent YourScript.py, line 1)
Failed to execute (Your script)

please give me a model if you can.
This is my email:meriyaloot@gmail.com

Regards

meriyalootka

You have to use a script tool. Theres no tool in the toolbox (to put into modelbuilder) to extract a zipfile.
You can wrap the above script as a script tool and put that into a model if it serves your purpose.

Hi esri
I,m a bad luck user. There isn't no one to solve my problem:mad:

meriyalootka

You have to use a script tool. Theres no tool in the toolbox (to put into modelbuilder) to extract a zipfile.
You can wrap the above script as a script tool and put that into a model if it serves your purpose.

Hi
The script return this error:

Traceback (most recent call last):
File "C:\Users\administrator\Documents\importzip.py", line 22, in <module>
for dirpath, dirnames, filenames in arcpy.da.Walk(zipFolder, datatype="FeatureClass"):
AttributeError: 'module' object has no attribute 'Walk'

What is wrong in my setting?

KevinHibma

My apologies, the da.Walk command was added at 10.1 Service Pack 1. Based on the message I'm guessing you have 10.1 final (no service packs). If you're able to install the service pack it'll fix that.

meriyalootka

My apologies, the da.Walk command was added at 10.1 Service Pack 1. Based on the message I'm guessing you have 10.1 final (no service packs). If you're able to install the service pack it'll fix that.

Hi
Esri published his Unzip.py about 8 years ago. Why I should use a script that just can work with AGS 10.1 SP1?
I have a Server that has AGS Server 10, and another server has AGS Server10.1, I need a solution that can work with AGS Server 10. please help me more.
Regards

meriyalootka

My apologies, the da.Walk command was added at 10.1 Service Pack 1. Based on the message I'm guessing you have 10.1 final (no service packs). If you're able to install the service pack it'll fix that.

Hi
Any solution?
I heard Khimba is a geoprocessing manager. I saw your project about airlines. Solving my problem is not hard for you:)
Regards

meriyalootka

My apologies, the da.Walk command was added at 10.1 Service Pack 1. Based on the message I'm guessing you have 10.1 final (no service packs). If you're able to install the service pack it'll fix that.

Any solution?

meriyalootka

import arcpy
import zipfile
import os

inFile = arcpy.GetParameterAsText(0) 

# Create a folder in the scratch directory to extract zip to
zipFolder = os.path.join(arcpy.env.scratchFolder, "zipContents")
os.mkdir(zipFolder)


# Extract the zip contents
zip2Extract = zipfile.ZipFile(inFile, 'r')
zip2Extract.extractall(zipFolder)
zip2Extract.close()

# Create a folder in the scratch directory to hold the fgdb which will be downloaded
fgdbFolder = os.path.join(arcpy.env.scratchFolder, "fgdbOutput")
os.mkdir(fgdbFolder)

# Work through all the FeatureClasses inside the extracted zip folder
for dirpath, dirnames, filenames in arcpy.da.Walk(zipFolder, datatype="FeatureClass"):
    
    for filename in filenames:
        
        # You could replace the code below here with your own code to do what you want....
        arcpy.AddMessage("Copying: {}".format(filename))
        
        # Strip .shp from the filename when merging shapefiles
        if filename.endswith("shp"):
            outFilename = filename[:-4]
        else:
            outFilename = filename

        # Copy each featureclass into the output.gdb
        arcpy.CopyFeatures_management(os.path.join(dirpath, filename),
                                      os.path.join(fgdbFolder, "output.gdb", outFilename))

Hi
I have this error:
Traceback (most recent call last):
File "C:\Users\user1\Documents\getfeature.py", line 13, in <module>
zip2Extract = zipfile.ZipFile(inFile, 'r')
File "C:\Python27\ArcGIS10.1\Lib\zipfile.py", line 699, in __init__
self.fp = open(file, modeDict[mode])
IOError: [Errno 13] Permission denied: u'C:\\Users\\user1\\Documents\\ArcGIS\\scratch\\ZipFolder'

ShingLin

Hi,

Have you resolved your problem? If not, try to change your scratch folder to a different place on Desktop when running your script tool. sometimes the default folder in the User directory has permission issue.

You can also use model tool, but I am not 100% sure if it works if you are using web app as Client. Give it a try anyway. You can use Calculate Value with absolute path (String data type) of the shape file as the input. Then use Copy Feature to copy the input file so Server will save the uploaded input shape file in the scratch gdb, and finally do the Buffer or any other Analysis with the uploaded file.

I attached two gpk files for your reference - calcbuffer100.gpk (for 10.0 Server) and calcbuffer101.gpk (For 10.1 Server). Both models are almost the same, except in 10.1 model you have to set the output of Calculate Value as an output parameter, otherwise you won't get the input shape on the server. I have to figured out if this is a bug or as design. Let me know if this helps.

Best,

Shing

meriyalootka

Hi,

Have you resolved your problem? If not, try to change your scratch folder to a different place on Desktop when running your script tool. sometimes the default folder in the User directory has permission issue.

You can also use model tool, but I am not 100% sure if it works if you are using web app as Client. Give it a try anyway. You can use Calculate Value with absolute path (String data type) of the shape file as the input. Then use Copy Feature to copy the input file so Server will save the uploaded input shape file in the scratch gdb, and finally do the Buffer or any other Analysis with the uploaded file.

I attached two gpk files for your reference - calcbuffer100.gpk (for 10.0 Server) and calcbuffer101.gpk (For 10.1 Server). Both models are almost the same, except in 10.1 model you have to set the output of Calculate Value as an output parameter, otherwise you won't get the input shape on the server. I have to figured out if this is a bug or as design. Let me know if this helps.

Best,

Shing

Many thanks for your sincerely helps, but I didn't see any attachment. Where is your attachments??

ShingLin

Sorry for the attachment did not get through. I realize if you adopt the model approach, the input shape file has to be located in a folder the server has access. If you want the users to provide the shape file from their machine, that is not going to work. So for both 10.0 and 10.1, Unzip the shape file is still the way to go.

Also, if you are going to create a web application. ArcGIS Viewer for Flex 3.2 (To be released early next week) now provides the upload feature, so you do not need to write a customized code like it used to. Hope this helps.

Regards,

Shing

Nelly_CristalArellano_García

Hy,
I didn't yet used it in the FlexViewer but only manually over rest.

But it should point you to the right direction:
enable "upload" on your geoprocessing service
upload your file to the gp-service and remember the upload-id
call your gp-service with the datafile paramater as the text below:

For GPDatafile parameter, you have to indicate this full json representation of the upload item:
parameter = {"itemID":"i80dfa12f-52ed-4841-94ff-37f9c3f5dd6f"}

Tom

Hi, I have to do a geoprocessing to validate a excel file with Arcgis 10.1.
It's the first time I have to working with geoprocessing.
Do you know how I can read the excel file with ID using python and toolbox parameters?

I defined my parameter like this:

excel_file = arcpy.Parameter(
displayName = "ID_Excel",
name = "excel_file",
datatype = "GPDatafile",
parameterType = "Optional",
direction = "Input")

But, when I run my script the parameter the ID that I wrote disappear... I am really confused. ¿How I can get my file after uploaded into a feature?

The GPdatafile what will return? A zip or the excel file?

HNoakes

This is an extremely useful python script, but I have one question, does the arcpy.da.Walk function validate the datatype based on the parameter that is passed in (ie. FeatureClass). I just wanted clarification on this so I do not add additional code for handling invalid datatypes if it is not required. The documentation on this function does not appear to be complete.

Just for your information I am using it to upload a Shapefile to server that is within a zipped folder. And want to know if this will error if the Shapefile is missing one of its required associated files (ie. .prj, .dbf)

Thanks,
-h

import arcpy
import zipfile
import os

inFile = arcpy.GetParameterAsText(0) 

# Create a folder in the scratch directory to extract zip to
zipFolder = os.path.join(arcpy.env.scratchFolder, "zipContents")
os.mkdir(zipFolder)


# Extract the zip contents
zip2Extract = zipfile.ZipFile(inFile, 'r')
zip2Extract.extractall(zipFolder)
zip2Extract.close()

# Create a folder in the scratch directory to hold the fgdb which will be downloaded
fgdbFolder = os.path.join(arcpy.env.scratchFolder, "fgdbOutput")
os.mkdir(fgdbFolder)

# Work through all the FeatureClasses inside the extracted zip folder
for dirpath, dirnames, filenames in arcpy.da.Walk(zipFolder, datatype="FeatureClass"):
    
    for filename in filenames:
        
        # You could replace the code below here with your own code to do what you want....
        arcpy.AddMessage("Copying: {}".format(filename))
        
        # Strip .shp from the filename when merging shapefiles
        if filename.endswith("shp"):
            outFilename = filename[:-4]
        else:
            outFilename = filename

        # Copy each featureclass into the output.gdb
        arcpy.CopyFeatures_management(os.path.join(dirpath, filename),
                                      os.path.join(fgdbFolder, "output.gdb", outFilename))

AlexandreMatveev

Hello, how can I access to this file in Geoprocessing script? I uploaded the .zip file and passed this object like parameter and I want to unzip it in, but what is the step between pass the parameter and unzip? how do we get access to .zip file? thank you

会話に参加する

サインインしたメンバーは投稿、更新のフォローなどができます。初めてですか？無料アカウントを登録してください。

クイックリンク

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.