field mapping in python for ArcGIS 10

RuthEmerick · ‎06-20-2011

Does anyone have or know of any field mapping code samples for arcpy? I'm using the Append tool, and read that there's a simpler way of doing a field map than creating a field map object. However, I've been unable to find any details or code samples.

Thanks!

curtvprice · ‎06-20-2011

The field map object is the easy way - the string representation can get pretty complicated so it's easier to work the the object.

An "easy" method I've used with append is to create an empty output feature class with just the fields I want and run Append to that dataset with "NO_TEST" - any fields that don't match with the output dataset's fields (schema) are simply ignored. I have found this less painful than dinking with the field map object to change field order or drop fields.

MathewCoyle · ‎06-21-2011

Here's a sample of using field mapping. Not as hard as it looks once you get some practice.

                block_out = str(sel)+"_blocks"
                field_mappings = arcpy.FieldMappings()

                field_mappings.addTable(block_lyr)
                
                fldmap_OBJID = arcpy.FieldMap()
                fldmap_BLOCKSTAGE = arcpy.FieldMap()
                fldmap_AREAGIS = arcpy.FieldMap()
                fldmap_OPTYPE = arcpy.FieldMap()
                fldmap_BLOCKTYPE = arcpy.FieldMap()
                fldmap_HARVSEASON = arcpy.FieldMap()
                fldmap_ROADPERCNT = arcpy.FieldMap()
                #fldmap_SOURCEID = arcpy.FieldMap(8,"","Join")

                fldmap_OBJID.addInputField(block_lyr, "OBJECTID")
                fld_OBJID = fldmap_OBJID.outputField
                fld_OBJID.name = "O_OID"
                fldmap_OBJID.outputField = fld_OBJID
                field_mappings.addFieldMap(fldmap_OBJID)

                fldmap_BLOCKSTAGE.addInputField(block_lyr, "BLOCKSTAGE")
                fld_BLOCKSTAGE = fldmap_BLOCKSTAGE.outputField
                fld_BLOCKSTAGE.name = "BLOCKSTAGE"
                fldmap_BLOCKSTAGE.outputField = fld_BLOCKSTAGE
                field_mappings.addFieldMap(fldmap_BLOCKSTAGE)

                fldmap_AREAGIS.addInputField(block_lyr, "AREAGIS")
                fld_AREAGIS = fldmap_AREAGIS.outputField
                fld_AREAGIS.name = "AREAGIS"
                fldmap_AREAGIS.outputField = fld_AREAGIS
                field_mappings.addFieldMap(fldmap_AREAGIS)

                fldmap_OPTYPE.addInputField(block_lyr, "OPERATIONSTYPE")
                fld_OPTYPE = fldmap_OPTYPE.outputField
                fld_OPTYPE.name = "OPTYPE"
                fldmap_OPTYPE.outputField = fld_OPTYPE
                field_mappings.addFieldMap(fldmap_OPTYPE)

                fldmap_BLOCKTYPE.addInputField(block_lyr, "BLOCKTYPE")
                fld_BLOCKTYPE = fldmap_BLOCKTYPE.outputField
                fld_BLOCKTYPE.name = "BLOCKTYPE"
                fldmap_BLOCKTYPE.outputField = fld_BLOCKTYPE
                field_mappings.addFieldMap(fldmap_BLOCKTYPE)
                
                fldmap_HARVSEASON.addInputField(block_lyr, "HARVESTSEASON")
                fld_HARVSEASON = fldmap_HARVSEASON.outputField
                fld_HARVSEASON.name = "HARVSEASON"
                fldmap_HARVSEASON.outputField = fld_HARVSEASON
                field_mappings.addFieldMap(fldmap_HARVSEASON)

                fldmap_ROADPERCNT.addInputField(block_lyr, "ROADPERCENTAGE")
                fld_ROADPERCNT = fldmap_ROADPERCNT.outputField
                fld_ROADPERCNT.name = "ROADPERCNT"
                fldmap_ROADPERCNT.outputField = fld_ROADPERCNT
                field_mappings.addFieldMap(fldmap_ROADPERCNT)

                for selection in blocklist:
                    
                    if blockPrev <> selection and not arcpy.Exists(block_out+".shp"):
                        block_query = ("\""+str(block_id)+"\" = \'"+str(selection)+"\' AND (BLOCKSTAGE = 'APPR')")
                        arcpy.FeatureClassToFeatureClass_conversion(block_lyr, exportdir, block_out, block_query, field_mappings)

Or here is the alternative

                        dropFields = list()
                        fieldList = arcpy.ListFields(block_out+".shp")                        
                        keep_list = ["O_OID","SOURCEID","BLOCKTYPE","ROADPERCNT","BLOCKSTAGE","AREAGIS","OPERATIONS","HARVSEASON","OBJECTID","PLANNEDVOL","OID","Geometry"]
                        for f in fieldList:
                            #print f.name
                            if f.name not in keep_list and f.type not in keep_list:
                                dropFields.append(f.name)

                        #print dropFields
                        arcpy.DeleteField_management(block_out+".shp", dropFields)

RuthEmerick · ‎07-19-2011

Thanks everyone!

JoelCalhoun · ‎07-19-2011

I just want to preface this post with the fact that I'm not a programmer and don't have much knowledge of Python or for that matter proper programming techniques.

However, with that in mind, I have hacked together a little script (for ArcGIS 10) that I use for table to feature class joins.

I often run into the situation where I need to create a subset of (for example) our parcels dataset based on parcel numbers in a dbf or excel spreadsheet.

It seems that the input table attribute names are usually altered in the join output dataset.
I have a difficult time correlating the new altered names with what data is stored in each field.

For this reason (in the attached script) I build my own field map string from each of the input sources and apply it to the output dataset to maintain the original input field names. I have added a few other options which can be seen in the included script tool interface.

Part of this code has also been used in some of our other processes where the join output code from modelbuilder would specify hard coded field lengths, this script is dynamic since it builds the output fieldmap from the input sources instead of forcing the data into a predefined format.

One of my future plans is to recode the case handling section with a simple dictionary.
I also plan to expand the error checking and I might look into adding SDE output capability.

If there is a much more simple way to achieve this same result without manually entering in the correct input field names, I would love to hear it.

Also, if there are any other comments regarding this script or topic I would love to discuss it further.

*Note: the attached zip file includes the python script and a toolbox with a script tool that is meant as the inteface for the script.

Thanks,

Joel

MarcNakleh · ‎11-22-2011

Hey guys,

I was thinking about this very problem today.
I tried to create a simpler, cleaner interface for straightforward field mapping situations. Using Mathew's code and example as a reference, I came up with the following:

def GetFieldMappings(fc_in, fc_out, dico):
    field_mappings = arcpy.FieldMappings()
    field_mappings.addTable(fc_in)

    for input, output in dico.iteritems():
        field_map = arcpy.FieldMap()
        field_map.addInputField(fc_in, input)        
        field = fieldmap.outputField        
        field.name = output
        field_map.outputField = field
        field_mappings.addFieldMap(field_map)
        del field, field_map

    return field_mappings

With this in place, you can simply make a call as follows:

    fc_in = r'C:/input.shp'
    fc_out = r'C:/output_blocks.shp'

    dico = {'OBJECTID':               'O_OID',
                'BLOCKSTAGE':          'BLOCKSTAGE',
                'AREAGIS':                 'AREAGIS',
                'OPERATIONSTYPE':   'OPTYPE',
                'BLOCKTYPE':             'BLOCKTYPE',
                'HARVESTSEASON':    'HARVSEASON',
                'ROADPERCENTAGE':  'ROADPERCNT'} 

    Mapper = GetFieldMappings(fc_in, fc_out, dico)

and your GeFieldMappings subroutine will return the proper FieldMappings object, properly populate. The relationships are stored within a simple Python dictionary, which seemed to me the most obvious way of structuring the information.
Clearly, this doesn't support more complicated relationships for merge rules, but you can either handle those manually, or create a nested dictionary or list that specifies the rule to use.

Hope this helps!

MathewCoyle · ‎11-22-2011

I always meant to come back to this to write something along these lines, but I figured if it ain't broke don't fix it, so I let it slide. Very nice though, tested and approved. One small thing though. Missing underscore on one line.

def GetFieldMappings(fc_in, fc_out, dico):
    field_mappings = arcpy.FieldMappings()
    field_mappings.addTable(fc_in)

    for input, output in dico.iteritems():
        field_map = arcpy.FieldMap()
        field_map.addInputField(fc_in, input)        
        field = field_map.outputField      # Missing underscore  
        field.name = output
        field_map.outputField = field
        field_mappings.addFieldMap(field_map)
        del field, field_map

    return field_mappings

MarcNakleh · ‎11-22-2011

Thanks for the correction.

Been copy-pasting code back and forth between development workstations, which probably caused the typo. Sorry about that!

Cheers,

curtvprice · ‎11-22-2011

What a great idea! I've needed something like this for a while.

Here's my take. I attached a 9.3-compatible version too.

def FieldNameMap(tbl, maps):
    """Create a field mappings object to rename, drop, or re-order fields.

    Arguments

      tbl - input feature class, table, or table view
      maps - field names map list (';'-delimited string)
            If a field is left out, it will not be included in the field map
      
    Example

      import arcpy
      Maps = "Shape_Area AREA;BID BID2;AREASQMI AREAMI2"
      Mapper = FieldNameMap("temp.dbf", Maps)
      # to debug: print out the field mapping
      print Mapper.exportToString().replace(";","\n")      
      # copy table, keeping only renamed fields: AREA, BID2, AREAMI2
      arcpy.Merge_management("temp.dbf","temp2.dbf",Mapper)

    Author

    Curtis Price, U.S. Geological Survey, cprice@usgs.gov
    Not reviewed/approved use at your own risk
    """
    
    field_mappings = arcpy.FieldMappings()   
    mapList = Maps.split(';')

    for rec in mapList:
        fromName,toName = rec.split()
        # create a new field map
        field_map = arcpy.FieldMap()
        # populate it and add to field_mappings
        try:
            field_map.addInputField(fc_in, fromName)        
            field = field_map.outputField        
            field.name = toName
            field_map.outputField = field
            field_mappings.addFieldMap(field_map) 
        except:
            raise Exception, "Cannot not map fields (%s) in %s" % (rec,tbl)
  
    return field_mappings

DanaDiotte1 · ‎04-30-2012

I've never found info so fast. Thanks for the code samples. Exactly what i was looking for.

Dana