m.glasesri-de-esridist

How to process uploaded files in a GPService

Blog Post created by m.glasesri-de-esridist Employee on Mar 5, 2015

Why this post?

Well, it was quite difficult to do what I wanted to do: uploading a Shapefile and processing it afterwards in an python script which was published as GP service. I was surprised that it was that difficult, because every GP service has an upload operation as standard equipment. But I could not find any extensive description, how to upload any data and to use it in an GP service.

 

Therefore this should help all people who have the same problem like me.

 

First I wrote a little JavaScript client who should upload a file and call the gp service afterwards:

require(["esri/urlUtils","esri/map", "esri/tasks/Geoprocessor", "esri/arcgis/utils", "esri/request", "dojo/dom", "dojo/on", "dojo/domReady!"], function(urlUtils, Map, Geoprocessor, arcgisUtils, esriRequest, dom, on) {

    var gpServiceFold= "GPServices";
    var gpServiceName= "MyFirstGPService/GPServer";
    var gpServerToolName= "MyFirstGPTool";

    gp = new Geoprocessor("http://"+server+":"+port+"/arcgis/rest/services/"+gpServiceFold+"/"+gpServiceName+"/"+gpServerToolName);
    on(dom.byId("upload"), "click", upload);

    function upload() {  
        var upload = esriRequest({  
            //// upload URL depends on polyline/polygon/point GP selected  
            url: "http://"+server+":"+port+"/arcgis/rest/services/"+gpServiceFold+"/"+gpServiceName+"/"+ "uploads/upload",  
            form: dojo.byId("uploadForm"),   
            content: {f: "pjson"},  
            handleAs: "json",  
        }).then(uploadSucceeded, uploadFailed);  
    }  
    function uploadSucceeded(response, io) {  

        console.log(dojo.toJson(response));  
          
        var itemID= response["item"].itemID;  
        console.log(itemID);    
      
        var inParams = {};    
        inParams.ShapeZip = '{"itemID":"' + itemID + '"}';  
        inParams.To = 'anymailaddress@esri.de';  
        console.log('Input Parameters: ' + dojo.toJson(inParams));  
          
        console.log('submitting job');    
        gp.setOutSpatialReference({wkid:102100});  
        gp.submitJob(inParams,   
            function(result){    
                console.log(result.jobStatus + '(' + result.jobId + ')')    
            },     
            function(message){    
                console.log(message.jobStatus + '(' + message.jobId + '):' + dojo.toJson(message));  
            },     
            function(error){    
                console.error(error.jobStatus + '(' + error.jobId + '):' + dojo.toJson(message));  
            });    
    }  

This embedded in a little html site:

<body>
    <form id="uploadForm" method="post" enctype="multipart/form-data">
        <input type="file" name="file" />
        <input type="hidden" name="description" />
    </form>
    <input type="button" value="Upload" id="upload"/>
</body>

 

This is very similar to this, where I got the information how to upload files.

Take a look to the parameter definition (26ff).

While the "To"-Paramater is a simple string assignment, the "ShapeZip"-Parameter looks quite different. It is a syntax like "{itemid:if8e76deb-5718-4dbe-bee7-6b780b8d51f0}". This is a definition in the ArcGIS REST API for the Submit GP Job operation (GPDataFile).

 

What happens if you call the GP service by submitting the item ID in this way?

Well, you do not get the item id on server side. Instead, you really get the uploaded file.

See this python snippet, which I had published in a complete python script as a GP service:

 

# Import system modules
import sys, os, arcpy, zipfile, datetime

##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
## main program routine starts here
##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
if __name__ == '__main__':

    # read the uploaded file
    infile = arcpy.GetParameterAsText(0)
    (outpath, outfileext) = os.path.splitext(infile)
    filename = outpath.split('\\')[-1]
    arcpy.AddMessage("Processing zip " + infile)

    # unzip file
    zippedShapeFile = zipfile.ZipFile(infile, 'r')
    zippedShapeFile.extractall(outpath)
    zippedShapeFile.close()
    shpPath = outpath + '\\' + filename + '.shp'
    arcpy.AddMessage('Finished unzipping file ' + shpPath)

    # continue with processing the file ...

 

The magic is somewhere in the middleware.

So, the content of arcpy.GetParameterAsText(0) - which is defined in the service as my parameter ShapeZip - is not the "{itemid:if8e76deb-5718-4dbe-bee7-6b780b8d51f0}" as I defined in the client. It is the file I has uploaded before.

Therefore I can treat the "infile" variable like a file.

That's the way, and it was stony.

 

 

One more little hint:

How could you see the messages created by the gp tool on client side? By default you just get some minor job status information, not really informative.

But you can change the message level of the gp service! By increasing it to "Info", you will get all messages you generate in your tool.

messagelevel.png

Outcomes