Zip File to GDB Service

[BrianLeroux]

I am trying to create a service that allows a user to upload a zip file that contains a shape file, unzip it, and finally append to an existing GDB Feature Class. The fields in the shape file do not match the FC so I need to performa field mapping. However, since the input is a zip file I do not have the capabilty to set up the field mapping in the model. So when I run this it adds the elements of the shape file but all the attributes are blank because the missing mapping. Is there any way around this? [ATTACH=CONFIG]19948[/ATTACH]

[DaleHoneycutt]

The only way I can think of is to have the user supply the field mapping; they'll have to give you the name of a field on the shapefile and the corresponding field on your output FC.  You'll then have to create your own field mapping object in a script to use in the append tool.  Another way is to force your user to rename their fields to known names/types.  There are no easy solutions to this I'm afraid.

[BrianLeroux]

Thanks for the reply. i was able to get this working by creating the mapping in a script.

 Create FieldMappings object and load the target dataset
#
    fieldmappings = arcpy.FieldMappings()
    fieldmappings.addTable(WildFire_Table_Target)

    inputfields = [field.name for field in arcpy.ListFields(shpPath) if not field.required]
    for inputfield in inputfields:
        # Iterate through each FieldMap in the FieldMappings
        #
        for i in range(fieldmappings.fieldCount):
            fieldmap = fieldmappings.getFieldMap(i)
            #arcpy.AddMessage(fieldmap.getInputFieldName(0))
            # If the field name from the target dataset matches to a validated input field name
            #
            if fieldmap.getInputFieldName(0) == inputfield.replace("", ""):
                # Add the input field to the FieldMap and replace the old FieldMap with the new
                #
                fieldmap.addInputField(shpPath, inputfield)
                fieldmappings.replaceFieldMap(i, fieldmap)
                break

[TomSchuller]

Brian,
I'm trying to create a similar geoprocessing service.
But I'm currently blocked in the upload part.

How can I reuse the uploaded file in the GPModel or Python script?
Should I have to build on my own the path the the uploaded file?

I'm working on 10.1, so my uploaded file is:
http://localhost:6080/arcgis/rest/services/UploadTest/GPServer/uploads/i83fa38d5-69e8-40c0-bd9c-30be...
C:\arcgisserver\directories\arcgissystem\arcgisuploads\services\ListVersions2.GPServer\i83fa38d5-69e8-40c0-bd9c-30beb643e522

Could you please share that part of your code with me or give me some hints?

Thanks,
Tom

[BrianLeroux]

Hi Tom,

I am a little stuck on authentication issues with the upload protion but I have some sample Siverlight code to use an uploaded file in a geoprocessing tool. You will see I have an ItemID hard coded in for now as I am not able to get the ItemID results from the file upload in silverlight. You need to pass the ItemID of the uploaded file to the geoprocessing task. I got the hang of it by using the rest endpoint to upload files and pass to the GP Service. You can use fiddler to see what is happening during each step.

Geoprocessor geoprocessorTask = new Geoprocessor("http://mygisserver/arcgis/rest/services/GPTools/WildfireUpload/GPServer/ZipImportTest");
            geoprocessorTask.ExecuteCompleted += GeoprocessorTask_ExecuteCompleted;
            geoprocessorTask.Failed += GeoprocessorTask_Failed;

            List<GPParameter> parameters = new List<GPParameter>();

            parameters.Add(new GPItemID("Input_File", "iec5a4db4-33c9-40b9-ab82-cd27ecef279f"));

            //http://gis/arcgis/rest/services/GPTools/WildfireUpload/GPServer/uploads/iec5a4db4-33c9-40b9-ab82-cd27ecef279f old file link
            geoprocessorTask.ExecuteAsync(parameters);

Hope this helps.

[TomSchuller]

Brian,
thanks for the quick answer.
I'm not yet on the client side.

My problem is the server side:

I have no idea how I can access the uploaded file.
Could you share your unzip model specially the "unzipfile" script part?
Which parameter type does your "ZIP-Input" have?

Tom

[BrianLeroux · Accepted solution]

Sorry I got ahead of you there.

Because I needed to perform the field mapping on the fly i switched from using a model to using a python script. The input parameter type is going to be "File" in any case. In the script below, "infile = arcpy.GetParameterAsText(0)" is going to get the input file as a File type parameter. Then you can use the file as you please.

You will see when you run this in desktop you have a file browser that you can use to browse to a file of your choice. Once you get that to run succesfully you can publish as a service. With the service you will need to get the itemID of the uploaded file and use that as an input parameter like itemID:"i83fa38d5-69e8-40c0-bd9c-30beb643e522".

import arcpy, os, zipfile  infile = arcpy.GetParameterAsText(0) outpath, outfileext = os.path.splitext(infile) filename = outpath.split('\\')[-1]  try:     # unzip file     fireZip = zipfile.ZipFile(infile, 'r')     fireZip.extractall(outpath)     fireZip.close()     shpPath = outpath + "\\" + filename + ".shp"     arcpy.AddMessage("Finished unzipping file.")  # Local variables:     WildFire_Table_Target = "Database Connections\\SQL_DB(TEST Server).sde\\ArcSDE.dbo.WILDFIREPERIM"  # Create FieldMappings object and load the target dataset      fieldmappings = arcpy.FieldMappings()     fieldmappings.addTable(WildFire_Table_Target)      inputfields = [field.name for field in arcpy.ListFields(shpPath) if not field.required]     for inputfield in inputfields:         # Iterate through each FieldMap in the FieldMappings         #         for i in range(fieldmappings.fieldCount):             fieldmap = fieldmappings.getFieldMap(i)             #arcpy.AddMessage(fieldmap.getInputFieldName(0))             # If the field name from the target dataset matches to a validated input field name             #             if fieldmap.getInputFieldName(0) == inputfield.replace("", ""):                 # Add the input field to the FieldMap and replace the old FieldMap with the new                 #                 fieldmap.addInputField(shpPath, inputfield)                 fieldmappings.replaceFieldMap(i, fieldmap)                 break               # Process: Append       arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)  except Exception as e:       print e.message       arcpy.AddError(e.message)

[TomSchuller]

thanks,
got it running.

For GPDatafile parameter, I have to indicate this json representation like this:

{"itemID":"i80dfa12f-52ed-4841-94ff-37f9c3f5dd6f"}

Thanks.
Tom

[meriyalootka]

Sorry I got ahead of you there. 

Because I needed to perform the field mapping on the fly i switched from using a model to using a python script. The input parameter type is going to be "File" in any case. In the script below, "infile = arcpy.GetParameterAsText(0)" is going to get the input file as a File type parameter. Then you can use the file as you please. 

You will see when you run this in desktop you have a file browser that you can use to browse to a file of your choice. Once you get that to run succesfully you can publish as a service. With the service you will need to get the itemID of the uploaded file and use that as an input parameter like itemID:"i83fa38d5-69e8-40c0-bd9c-30beb643e522". 

import arcpy, os, zipfile

infile = arcpy.GetParameterAsText(0)
outpath, outfileext = os.path.splitext(infile)
filename = outpath.split('\\')[-1]

try:
    # unzip file
    fireZip = zipfile.ZipFile(infile, 'r')
    fireZip.extractall(outpath)
    fireZip.close()
    shpPath = outpath + "\\" + filename + ".shp"
    arcpy.AddMessage("Finished unzipping file.")

# Local variables:
    WildFire_Table_Target = "Database Connections\\SQL_DB(TEST Server).sde\\ArcSDE.dbo.WILDFIREPERIM"

# Create FieldMappings object and load the target dataset

    fieldmappings = arcpy.FieldMappings()
    fieldmappings.addTable(WildFire_Table_Target)

    inputfields = [field.name for field in arcpy.ListFields(shpPath) if not field.required]
    for inputfield in inputfields:
        # Iterate through each FieldMap in the FieldMappings
        #
        for i in range(fieldmappings.fieldCount):
            fieldmap = fieldmappings.getFieldMap(i)
            #arcpy.AddMessage(fieldmap.getInputFieldName(0))
            # If the field name from the target dataset matches to a validated input field name
            #
            if fieldmap.getInputFieldName(0) == inputfield.replace("", ""):
                # Add the input field to the FieldMap and replace the old FieldMap with the new
                #
                fieldmap.addInputField(shpPath, inputfield)
                fieldmappings.replaceFieldMap(i, fieldmap)
                break

            
# Process: Append
      arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)

except Exception as e:
      print e.message
      arcpy.AddError(e.message)

Hi
I tested your code, but it return this error:

IndentationError: unindent does not match any outer indentation level


How should I set script input and output parameters?

[BrianLeroux]

Hi 
I tested your code, but it return this error: 

IndentationError: unindent does not match any outer indentation level


How should I set script input and output parameters?

Just be sure to check your python code to make sure none of the lines are wrapping on you. Does it say what line the indentation error is on?

There is one input parameter scripted in this sample. infile = arcpy.GetParameterAsText(0). If you need more you can just take a similar appraoch.