topic Re: Zip File to GDB Service in Geoprocessing Questions

Zip File to GDB Service

BrianLeroux — Fri, 14 Dec 2012 17:52:57 GMT

I am trying to create a service that allows a user to upload a zip file that contains a shape file, unzip it, and finally append to an existing GDB Feature Class. The fields in the shape file do not match the FC so I need to performa field mapping. However, since the input is a zip file I do not have the capabilty to set up the field mapping in the model. So when I run this it adds the elements of the shape file but all the attributes are blank because the missing mapping. Is there any way around this? [ATTACH=CONFIG]19948[/ATTACH]

Re: Zip File to GDB Service

DaleHoneycutt — Tue, 18 Dec 2012 02:10:13 GMT

The only way I can think of is to have the user supply the field mapping; they'll have to give you the name of a field on the shapefile and the corresponding field on your output FC. You'll then have to create your own field mapping object in a script to use in the append tool. Another way is to force your user to rename their fields to known names/types. There are no easy solutions to this I'm afraid.

Re: Zip File to GDB Service

BrianLeroux — Sun, 12 Dec 2021 01:12:02 GMT

Thanks for the reply. i was able to get this working by creating the mapping in a script.

 Create FieldMappings object and load the target dataset
#
    fieldmappings = arcpy.FieldMappings()
    fieldmappings.addTable(WildFire_Table_Target)

    inputfields = [field.name for field in arcpy.ListFields(shpPath) if not field.required]
    for inputfield in inputfields:
        # Iterate through each FieldMap in the FieldMappings
        #
        for i in range(fieldmappings.fieldCount):
            fieldmap = fieldmappings.getFieldMap(i)
            #arcpy.AddMessage(fieldmap.getInputFieldName(0))
            # If the field name from the target dataset matches to a validated input field name
            #
            if fieldmap.getInputFieldName(0) == inputfield.replace("", ""):
                # Add the input field to the FieldMap and replace the old FieldMap with the new
                #
                fieldmap.addInputField(shpPath, inputfield)
                fieldmappings.replaceFieldMap(i, fieldmap)
                break

Re: Zip File to GDB Service

TomSchuller — Thu, 27 Dec 2012 03:38:07 GMT

Brian,
I'm trying to create a similar geoprocessing service.
But I'm currently blocked in the upload part.

How can I reuse the uploaded file in the GPModel or Python script?
Should I have to build on my own the path the the uploaded file?

I'm working on 10.1, so my uploaded file is:
http://localhost:6080/arcgis/rest/services/UploadTest/GPServer/uploads/i83fa38d5-69e8-40c0-bd9c-30beb643e522
C:\arcgisserver\directories\arcgissystem\arcgisuploads\services\ListVersions2.GPServer\i83fa38d5-69e8-40c0-bd9c-30beb643e522

Could you please share that part of your code with me or give me some hints?

Thanks,
Tom

Re: Zip File to GDB Service

BrianLeroux — Sun, 12 Dec 2021 01:12:05 GMT

Hi Tom,

I am a little stuck on authentication issues with the upload protion but I have some sample Siverlight code to use an uploaded file in a geoprocessing tool. You will see I have an ItemID hard coded in for now as I am not able to get the ItemID results from the file upload in silverlight. You need to pass the ItemID of the uploaded file to the geoprocessing task. I got the hang of it by using the rest endpoint to upload files and pass to the GP Service. You can use fiddler to see what is happening during each step.

Geoprocessor geoprocessorTask = new Geoprocessor("http://mygisserver/arcgis/rest/services/GPTools/WildfireUpload/GPServer/ZipImportTest");
            geoprocessorTask.ExecuteCompleted += GeoprocessorTask_ExecuteCompleted;
            geoprocessorTask.Failed += GeoprocessorTask_Failed;

            List<GPParameter> parameters = new List<GPParameter>();

            parameters.Add(new GPItemID("Input_File", "iec5a4db4-33c9-40b9-ab82-cd27ecef279f"));

            //http://gis/arcgis/rest/services/GPTools/WildfireUpload/GPServer/uploads/iec5a4db4-33c9-40b9-ab82-cd27ecef279f old file link
            geoprocessorTask.ExecuteAsync(parameters);

Hope this helps.

Re: Zip File to GDB Service

TomSchuller — Thu, 27 Dec 2012 11:33:02 GMT

Brian,
thanks for the quick answer.
I'm not yet on the client side.

My problem is the server side:

I have no idea how I can access the uploaded file.
Could you share your unzip model specially the "unzipfile" script part?
Which parameter type does your "ZIP-Input" have?

Tom

Re: Zip File to GDB Service

BrianLeroux — Thu, 27 Dec 2012 11:59:28 GMT

Sorry I got ahead of you there.

Because I needed to perform the field mapping on the fly i switched from using a model to using a python script. The input parameter type is going to be "File" in any case. In the script below, "infile = arcpy.GetParameterAsText(0)" is going to get the input file as a File type parameter. Then you can use the file as you please.

You will see when you run this in desktop you have a file browser that you can use to browse to a file of your choice. Once you get that to run succesfully you can publish as a service. With the service you will need to get the itemID of the uploaded file and use that as an input parameter like itemID:"i83fa38d5-69e8-40c0-bd9c-30beb643e522".

import arcpy, os, zipfile  infile = arcpy.GetParameterAsText(0) outpath, outfileext = os.path.splitext(infile) filename = outpath.split('\\')[-1]  try:     # unzip file     fireZip = zipfile.ZipFile(infile, 'r')     fireZip.extractall(outpath)     fireZip.close()     shpPath = outpath + "\\" + filename + ".shp"     arcpy.AddMessage("Finished unzipping file.")  # Local variables:     WildFire_Table_Target = "Database Connections\\SQL_DB(TEST Server).sde\\ArcSDE.dbo.WILDFIREPERIM"  # Create FieldMappings object and load the target dataset      fieldmappings = arcpy.FieldMappings()     fieldmappings.addTable(WildFire_Table_Target)      inputfields = [field.name for field in arcpy.ListFields(shpPath) if not field.required]     for inputfield in inputfields:         # Iterate through each FieldMap in the FieldMappings         #         for i in range(fieldmappings.fieldCount):             fieldmap = fieldmappings.getFieldMap(i)             #arcpy.AddMessage(fieldmap.getInputFieldName(0))             # If the field name from the target dataset matches to a validated input field name             #             if fieldmap.getInputFieldName(0) == inputfield.replace("", ""):                 # Add the input field to the FieldMap and replace the old FieldMap with the new                 #                 fieldmap.addInputField(shpPath, inputfield)                 fieldmappings.replaceFieldMap(i, fieldmap)                 break               # Process: Append       arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)  except Exception as e:       print e.message       arcpy.AddError(e.message)

Re: Zip File to GDB Service

TomSchuller — Thu, 27 Dec 2012 16:38:59 GMT

thanks,
got it running.

For GPDatafile parameter, I have to indicate this json representation like this:

{"itemID":"i80dfa12f-52ed-4841-94ff-37f9c3f5dd6f"}

Thanks.
Tom

Re: Zip File to GDB Service

meriyalootka — Sun, 12 Dec 2021 01:12:08 GMT

import arcpy, os, zipfile

infile = arcpy.GetParameterAsText(0)
outpath, outfileext = os.path.splitext(infile)
filename = outpath.split('\\')[-1]

try:
    # unzip file
    fireZip = zipfile.ZipFile(infile, 'r')
    fireZip.extractall(outpath)
    fireZip.close()
    shpPath = outpath + "\\" + filename + ".shp"
    arcpy.AddMessage("Finished unzipping file.")

# Local variables:
    WildFire_Table_Target = "Database Connections\\SQL_DB(TEST Server).sde\\ArcSDE.dbo.WILDFIREPERIM"

# Create FieldMappings object and load the target dataset

    fieldmappings = arcpy.FieldMappings()
    fieldmappings.addTable(WildFire_Table_Target)

    inputfields = [field.name for field in arcpy.ListFields(shpPath) if not field.required]
    for inputfield in inputfields:
        # Iterate through each FieldMap in the FieldMappings
        #
        for i in range(fieldmappings.fieldCount):
            fieldmap = fieldmappings.getFieldMap(i)
            #arcpy.AddMessage(fieldmap.getInputFieldName(0))
            # If the field name from the target dataset matches to a validated input field name
            #
            if fieldmap.getInputFieldName(0) == inputfield.replace("", ""):
                # Add the input field to the FieldMap and replace the old FieldMap with the new
                #
                fieldmap.addInputField(shpPath, inputfield)
                fieldmappings.replaceFieldMap(i, fieldmap)
                break

            
# Process: Append
      arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)

except Exception as e:
      print e.message
      arcpy.AddError(e.message)

Hi
I tested your code, but it return this error:

IndentationError: unindent does not match any outer indentation level

How should I set script input and output parameters?

Re: Zip File to GDB Service

BrianLeroux — Wed, 20 Feb 2013 11:48:54 GMT

Hi
I tested your code, but it return this error:

IndentationError: unindent does not match any outer indentation level

How should I set script input and output parameters?

Just be sure to check your python code to make sure none of the lines are wrapping on you. Does it say what line the indentation error is on?

There is one input parameter scripted in this sample. infile = arcpy.GetParameterAsText(0). If you need more you can just take a similar appraoch.

Re: Zip File to GDB Service

meriyalootka — Thu, 21 Feb 2013 03:40:34 GMT

Just be sure to check your python code to make sure none of the lines are wrapping on you. Does it say what line the indentation error is on?

There is one input parameter scripted in this sample. infile = arcpy.GetParameterAsText(0). If you need more you can just take a similar appraoch.

The error is for Line 41.

Re: Zip File to GDB Service

BrianLeroux — Thu, 21 Feb 2013 11:19:43 GMT

The error is for Line 41.

I am not exactly sure what your line 41 would be. i would assume you changed the code as least a little bit so my line 41 is not the same as yours. If you can post your code and state the text of the line that is giving you issues I can see if I can identify the issue. When posting your code make sure it posts with the correct indents and line breaks or it could make it look like you have more problems that aren;t necesarily in your code.

Re: Zip File to GDB Service

meriyalootka — Thu, 21 Feb 2013 19:24:58 GMT

I am not exactly sure what your line 41 would be. i would assume you changed the code as least a little bit so my line 41 is not the same as yours. If you can post your code and state the text of the line that is giving you issues I can see if I can identify the issue. When posting your code make sure it posts with the correct indents and line breaks or it could make it look like you have more problems that aren;t necesarily in your code.

Hi
I guess the error related to this line:

arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)

I did not change your code, I got it from your answer to TOM.

I wonder if you would mind to email your model (meriyaloot@gmail.com)

regards

Re: Zip File to GDB Service

BrianLeroux — Fri, 22 Feb 2013 11:53:38 GMT

Honestly it is probably just an issue with the indentation due to copying and pasting. I would just go though and make sure everything is indented properly. Using a program like notepad++ will make it easier to spot issues. When I copied my code below into NP++ you can see that line is indented too much as well as 2 other below it. Just delete the white space before that following 3 lines and tab them each over 1 time.

arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)
print e.message
arcpy.AddError(e.message)

Re: Zip File to GDB Service

meriyalootka — Fri, 22 Feb 2013 13:28:16 GMT

Honestly it is probably just an issue with the indentation due to copying and pasting. I would just go though and make sure everything is indented properly. Using a program like notepad++ will make it easier to spot issues. When I copied my code below into NP++ you can see that line is indented too much as well as 2 other below it. Just delete the white space before that following 3 lines and tab them each over 1 time.

arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)
print e.message
arcpy.AddError(e.message)

I changed it with your solution and it return this error:
SyntaxError: invalid syntax

This is my new code:
import arcpy, os, zipfile

infile = arcpy.GetParameterAsText(0)
outpath, outfileext = os.path.splitext(infile)
filename = outpath.split('\\')[-1]

try:
# unzip file
fireZip = zipfile.ZipFile(infile, 'r')
fireZip.extractall(outpath)
fireZip.close()
shpPath = outpath + "\\" + filename + ".shp"
arcpy.AddMessage("Finished unzipping file.")

# Local variables:
WildFire_Table_Target = "Database Connections\\SQL_DB(ks204060).sde\\ArcSDE.dbo.gis"

# Create FieldMappings object and load the target dataset

fieldmappings = arcpy.FieldMappings()
fieldmappings.addTable(WildFire_Table_Target)

inputfields = [field.name for field in arcpy.ListFields(shpPath) if not field.required]
for inputfield in inputfields:
# Iterate through each FieldMap in the FieldMappings
#
for i in range(fieldmappings.fieldCount):
fieldmap = fieldmappings.getFieldMap(i)
#arcpy.AddMessage(fieldmap.getInputFieldName(0))
# If the field name from the target dataset matches to a validated input field name
#
if fieldmap.getInputFieldName(0) == inputfield.replace("", ""):
# Add the input field to the FieldMap and replace the old FieldMap with the new
#
fieldmap.addInputField(shpPath, inputfield)
fieldmappings.replaceFieldMap(i, fieldmap)
break

# Process: Append
arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)
print e.message
arcpy.AddError(e.message)

Re: Zip File to GDB Service

BrianLeroux — Fri, 22 Feb 2013 17:43:49 GMT

You are going to have to post your code to this thread using code blocks. Without that I will not be able to properly view your code with the correct indenting.

Re: Zip File to GDB Service

meriyalootka — Sat, 23 Feb 2013 04:26:58 GMT

You are going to have to post your code to this thread using code blocks. Without that I will not be able to properly view your code with the correct indenting.

I attached pyton script. please help me.

Re: Zip File to GDB Service

BrianLeroux — Sun, 12 Dec 2021 01:12:11 GMT

I attached pyton script. please help me.

The last few statements you have a incorrect. You need to use with the appropriate indentation.

# Process: Append
      arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)

except Exception as e:
      print e.message
      arcpy.AddError(e.message)

Re: Zip File to GDB Service

meriyalootka — Sun, 12 Dec 2021 01:12:14 GMT

The last few statements you have a incorrect. You need to use with the appropriate indentation.
# Process: Append
      arcpy.Append_management(shpPath, WildFire_Table_Target, "NO_TEST", fieldmappings)

except Exception as e:
      print e.message
      arcpy.AddError(e.message)

Hi
I tested your code, but it return this error:
IndentationError: unindent does not match any outer indentation level (import_zip_91.py, line 40)

What is appropriate indentation?
Please help me more. Is it possible to send me your model or your code?
Regards

Re: Zip File to GDB Service

BrianLeroux — Tue, 26 Feb 2013 11:23:51 GMT

Hi
I tested your code, but it return this error:
IndentationError: unindent does not match any outer indentation level (import_zip_91.py, line 40)

What is appropriate indentation?
Please help me more. Is it possible to send me your model or your code?
Regards

It is pretty simple at this point. You just need to make sure your indentation is correct on those lines that I posted last. Delete the white space before each line so they are not indented and then tab each line over 1 time. The only exception is "except Exception as e:": should not be indented at all.