Geoprocessing service with uploaded zip file that contains an FGDB

1272
5
Jump to solution
03-21-2018 08:16 AM
MollyFoley
Regular Contributor

I would like to use a user-uploaded zip file in my geoprocessing service. The GP service is called from a Javascript web app, so I need to switch on "uploads" for the GP service. In doing so, it's my understanding that the file will now be uploaded to the server for me, but then I need to get the itemID to pass as a parameter to my GP service. Then in my python script tool, I can do the logic for unzipping and getting the FGDB and go about my merry way.

I feel like I understand the basics but I'm having a really hard time without a sample to look at. Does anybody have a good example of doing this? I'm working with AGS 10.4.1 and Javascript API 4.6. I would really like to see both an example of how the itemID is retrieved and passed to the GP service in the Javascript and then how it is used in the subsequent python script (e.g. is it just "GetParameterAsText()" or "sys.argv[]"?).

While I'm not totally clear on how to retrieve the itemID in the Javascript, I do know you end up passing something like this as a parameter to the service.

{itemID:"i83fa38d5-69e8-40c0-bd9c-30beb643e522}

How is that then consumed in the python script? If I pass the itemID, will it be translated into the file I actually need? So say I had this script - would the GetParameterAsText actually retrieve the zip file and the below code would execute as expected, or would it just be that itemID and then I have to do something with that itemID to get the actual file associated with it?

import arcpy, os, sys, zipfile

myZipFile = GetParameterAsText(0)
zip_ref = zipfile.ZipFile(myZipFile, 'r')
zip_ref.extractall(r"C:\someDirectory")
zip_ref.close()‍‍‍‍‍‍
1 Solution

Accepted Solutions
MollyFoley
Regular Contributor

When I was trying this, ArcGIS Javascript API wasn't supporting the file upload function yet, which was why it didn't work. I had to switch back to 3.x to proceed with the functionality I needed. I don't know if 4.x has been updated or not yet to include a file upload function, so the below code is 3.x code.

Regardless, this is the code I ended up with. The key is that you assign the itemID to a DataFile object and pass the DataFile object as a parameter to the service. Then get your parameter in python and it will recognize it as a file instead of a nonsensical itemID:

Python code for getting the file and unzipping it. I did not have to do anything with the itemID in here, after assigning the itemID to a DataFile object in JS, it comes in as a zip file instead of an itemID:

#Get the uploaded zip file
uploadedZip = arcpy.GetParameterAsText(0)

#Unzip the file into the scratch folder
try:
zip_ref = zipfile.ZipFile(uploadedZip, 'r')
zip_ref.extractall(scratchFolder)
zip_ref.close()
arcpy.AddMessage("Successfully extracted zip file...")‍‍‍‍‍‍

#Get the path to the unzipped geodatabase file
for file in os.listdir(scratchFolder):
if file.endswith(".gdb") and file != "scratch.gdb":
uploadedGDB = os.path.join(scratchFolder, file)‍‍‍‍‍‍‍‍‍‍‍

try:
#Get the feature class out of the user uploaded geodatabase. Make sure it exists and the user did not alter
#the name.
fcName = "{0}_MgmtTracts_Template".format(stateID)
mgmtTractFC = os.path.join(uploadedGDB, fcName)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

JS passing itemID as parameter:

mgmtGP = new Geoprocessor("https://www.stuff.com/myServer/rest/services/HabitatManagement/HabitatManagement/GPServer/HabitatManagement");

function mgmtUpload() {
//I did a bunch of form validation here in if/else statements. Took it out for ease of reading
else {
//upload the zip file and get back the itemID (via uploadSucceeded)
var upload = esri.request({
url: "https://www.stuff.com/myServer/rest/services/HabitatManagement/HabitatManagement/GPServer/uploads/upload",
form: dojo.byId("mgmtUploadForm"),
content: { f: 'json' },
handleAs: 'json',
}).then(mgmtUploadSucceeded, mgmtUploadFailed);
}
}

function mgmtUploadSucceeded(response) {
//After you upload your file, you have to index into your response object
//using "item" and grab the itemID from the itemID property
var itemID = response["item"].itemID;
//Create the DataFile object you will assign your itemID to
var dataFile = new esri.tasks.DataFile();
//The itemID points to the zip file, which is why I assign it the dataFile object
dataFile.itemID = itemID;
//The Input_Zip_File parameter is assigned the dataFile object which is really the itemID
var params = { "Input_Zip_File": dataFile };
console.log('Input parameters: ' + dojo.toJson(params));
mgmtGP.submitJob(params, gpJobComplete, gpJobStatus, function (error) {
console.error('[(jobStatus: ' + gpError.jobStatus + ') (jobId: ' + gpError.jobId + ') ' + '(messages: ' + gpError.messages + ')]');
}).then(function (promiseResult) {
//Re-enable submit mgmt data habitat button
$("#mgmt").button('enable').removeClass('ui-state-disabled');
});
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

View solution in original post

5 Replies
SigamGeo
New Contributor II

Did you solve your problem? My getparameter is receiving none after I pass the itemID.

0 Kudos
MollyFoley
Regular Contributor

When I was trying this, ArcGIS Javascript API wasn't supporting the file upload function yet, which was why it didn't work. I had to switch back to 3.x to proceed with the functionality I needed. I don't know if 4.x has been updated or not yet to include a file upload function, so the below code is 3.x code.

Regardless, this is the code I ended up with. The key is that you assign the itemID to a DataFile object and pass the DataFile object as a parameter to the service. Then get your parameter in python and it will recognize it as a file instead of a nonsensical itemID:

Python code for getting the file and unzipping it. I did not have to do anything with the itemID in here, after assigning the itemID to a DataFile object in JS, it comes in as a zip file instead of an itemID:

#Get the uploaded zip file
uploadedZip = arcpy.GetParameterAsText(0)

#Unzip the file into the scratch folder
try:
zip_ref = zipfile.ZipFile(uploadedZip, 'r')
zip_ref.extractall(scratchFolder)
zip_ref.close()
arcpy.AddMessage("Successfully extracted zip file...")‍‍‍‍‍‍

#Get the path to the unzipped geodatabase file
for file in os.listdir(scratchFolder):
if file.endswith(".gdb") and file != "scratch.gdb":
uploadedGDB = os.path.join(scratchFolder, file)‍‍‍‍‍‍‍‍‍‍‍

try:
#Get the feature class out of the user uploaded geodatabase. Make sure it exists and the user did not alter
#the name.
fcName = "{0}_MgmtTracts_Template".format(stateID)
mgmtTractFC = os.path.join(uploadedGDB, fcName)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

JS passing itemID as parameter:

mgmtGP = new Geoprocessor("https://www.stuff.com/myServer/rest/services/HabitatManagement/HabitatManagement/GPServer/HabitatManagement");

function mgmtUpload() {
//I did a bunch of form validation here in if/else statements. Took it out for ease of reading
else {
//upload the zip file and get back the itemID (via uploadSucceeded)
var upload = esri.request({
url: "https://www.stuff.com/myServer/rest/services/HabitatManagement/HabitatManagement/GPServer/uploads/upload",
form: dojo.byId("mgmtUploadForm"),
content: { f: 'json' },
handleAs: 'json',
}).then(mgmtUploadSucceeded, mgmtUploadFailed);
}
}

function mgmtUploadSucceeded(response) {
//After you upload your file, you have to index into your response object
//using "item" and grab the itemID from the itemID property
var itemID = response["item"].itemID;
//Create the DataFile object you will assign your itemID to
var dataFile = new esri.tasks.DataFile();
//The itemID points to the zip file, which is why I assign it the dataFile object
dataFile.itemID = itemID;
//The Input_Zip_File parameter is assigned the dataFile object which is really the itemID
var params = { "Input_Zip_File": dataFile };
console.log('Input parameters: ' + dojo.toJson(params));
mgmtGP.submitJob(params, gpJobComplete, gpJobStatus, function (error) {
console.error('[(jobStatus: ' + gpError.jobStatus + ') (jobId: ' + gpError.jobId + ') ' + '(messages: ' + gpError.messages + ')]');
}).then(function (promiseResult) {
//Re-enable submit mgmt data habitat button
$("#mgmt").button('enable').removeClass('ui-state-disabled');
});
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

View solution in original post

FayuLai
New Contributor II

I am trying to do the similar thing too.  But I would like to do in ArcGIS Pro.  Any example code will be very helpful.

0 Kudos
SigamGeo
New Contributor II

The same user who made this question has a sample of code: https://community.esri.com/thread/211943-invalid-url-error-for-geoprocessing-service 

0 Kudos
MollyFoley
Regular Contributor

Not sure if you get the notification of a reply or not since I replied to the other person, sorry! See the above code for what I ended up doing. Though I'm not using Pro.