Append Tool and Field Mapping ... Help? Examples?

23390
21
Jump to solution
11-03-2016 08:19 AM
Brownschuh
Occasional Contributor II

Having a bit of trouble with the field mapping component of the append tool; I cannot seem to get it to work properly as I am left with a feature class that is populated with all the features, but the attribute table remains blank. 

Background info: I have two feature classes in an enterprise geodatabase with address point data.  One is the authoritative source edited by our address coordinator and the other is used as to host a map service.  The data is essentially the same, but the schemas are slightly different; half of the fields are the exactly the same, but others have different names.  My workflow has been to update the 'map service' feature class on a daily basis by deleting all the records and loading the data manually, selecting the NO_TEST option, and manually mapping the fields where the names are different ... but this is where I run into trouble with my script.

I've ran the append process manually using the geoprocessing tool and copied the python snippet of the field mapping component and am left with an output attribute table that has all null values.  Ideally I'd like to move away from this style of coding as I am aware it can be rather clunky/lengthy ... and it just does not work (even though it throws no errors).

I'd like to do 'real' field mapping, but so far the examples I've found have a bit more complicated or focus mainly on the Merge tool.  I'd like to see a nice, simple Append tool example if possible.

Also, how does field mapping work if some fields are the same and others are different? I haven't run across a good example that deals with this problem.

Here are some of the sources I've looked at so far:

Field mapping and Python scripting | ArcGIS Blog

Mapping input fields to output fields—ArcPy Get Started | ArcGIS for Desktop 

0 Kudos
1 Solution

Accepted Solutions
LukeWebb
Occasional Contributor III
__author__ = 'WebbL'

import arcpy

append_layer = r'C:\a_layer.shp'
target_layer = r'C:\target_layer.shp'


#This object looks like the empty grid of fields 
#you see when you first open the append tool in the toolbox
fieldmappings = arcpy.FieldMappings()

#Like when you manually choose a layer in the toolbox and it adds the fields to grid
fieldmappings.addTable(target_layer)
fieldmappings.addTable(append_layer)

#####Lets map fields that have different names!
list_of_fields_we_will_map = []
#Lets chuck some tuples into the list we made
list_of_fields_we_will_map.append(('TaxPIN', 'Permanent_Identifier'))
list_of_fields_we_will_map.append(('SwissBankNo', 'RetirementFundAccNo'))
list_of_fields_we_will_map.append(('Balance', 'Amount'))

for field_map in list_of_fields_we_will_map:
    #Find the fields index by name. e.g 'TaxPin'
    field_to_map_index = fieldmappings.findFieldMapIndex(field_map[0])  
    #Grab "A copy" of the current field map object for this particular field
    field_to_map = fieldmappings.getFieldMap(field_to_map_index)
    #Update its data source to add the input from the the append layer
    field_to_map.addInputField(append_layer, field_map[1])
    #We edited a copy, update our data grid object with it
    fieldmappings.replaceFieldMap(field_to_map_index, field_to_map)

#Create a list of append datasets and run the the tool
inData = [append_layer]
arcpy.Append_management(inData, target_layer, field_mappings=fieldmappings)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Here you go

View solution in original post

21 Replies
MitchHolley1
MVP Regular Contributor

I would try to build a model in ModelBuilder first, then export it to a python script. 

0 Kudos
Brownschuh
Occasional Contributor II

Ah yes, I should have noted that I did attempt this but ended up with the same result ... an attribute table with all null values.

0 Kudos
MitchHolley1
MVP Regular Contributor

In my minimal experience with using the Append tool in Python, there really is no simple way to do it.  I've given up on field mappings, because it's so clunky and confusing.  Sorry, but I can't help you anymore

LukeWebb
Occasional Contributor III
import arcpy

append_layer = r'C:\a_layer.shp'
target_layer = r'C:\target_layer.shp'


#This is like defining an empty grid of fields you see when you run it manually in the toolbox
fieldmappings = arcpy.FieldMappings()

#Add the target datasets fields to the field map table  (Like when you manually choose a tagert layer in the toolbox)
fieldmappings.addTable(target_layer)

#Add the append datasets fields to the field map table  (Like when you manually choose a tagert layer in the toolbox)
fieldmappings.addTable(append_layer)

#At this point, you have a grid like when you run it manually saved in your field mappings.




#####Lets map a field that have different names!


#Find which "Index" the field has as we cant refer to them by name when editing the data only index
field_to_map_index = fieldmappings.findFieldMapIndex("Field_name_in_target")  #Field name that exists in the target layer but not append data source!

#Grab "A copy" of the field map object for this particular field
field_to_map = fieldmappings.getFieldMap(field_to_map_index)

#Update its data source to add the input from the the append layer
field_to_map.addInputField(append_layer, "Field_name_in_append")


#####Lets update the master field map using this updated copy of a field

fieldmappings.replaceFieldMap(field_to_map_index, field_to_map)


#Create a list of append datasets and run the the tool
inData = [append_layer]
arcpy.Append_management(inData, target_layer, field_mappings=fieldmappings)

I havent tested this code, but I think it should work

Brownschuh
Occasional Contributor II

Excellent, tried this code and it worked for my test script.  However, I was only able to get it to work for one instance of a mismatched field ("TaxPIN" vs "Permanent_Identifier").  Not sure how to tweak the code to map the other mismatched fields; is this done findFieldMapIndex component? 

I should also note that all the matching fields populated just fine; pretty happy about that. 

0 Kudos
LukeWebb
Occasional Contributor III
__author__ = 'WebbL'

import arcpy

append_layer = r'C:\a_layer.shp'
target_layer = r'C:\target_layer.shp'


#This object looks like the empty grid of fields 
#you see when you first open the append tool in the toolbox
fieldmappings = arcpy.FieldMappings()

#Like when you manually choose a layer in the toolbox and it adds the fields to grid
fieldmappings.addTable(target_layer)
fieldmappings.addTable(append_layer)

#####Lets map fields that have different names!
list_of_fields_we_will_map = []
#Lets chuck some tuples into the list we made
list_of_fields_we_will_map.append(('TaxPIN', 'Permanent_Identifier'))
list_of_fields_we_will_map.append(('SwissBankNo', 'RetirementFundAccNo'))
list_of_fields_we_will_map.append(('Balance', 'Amount'))

for field_map in list_of_fields_we_will_map:
    #Find the fields index by name. e.g 'TaxPin'
    field_to_map_index = fieldmappings.findFieldMapIndex(field_map[0])  
    #Grab "A copy" of the current field map object for this particular field
    field_to_map = fieldmappings.getFieldMap(field_to_map_index)
    #Update its data source to add the input from the the append layer
    field_to_map.addInputField(append_layer, field_map[1])
    #We edited a copy, update our data grid object with it
    fieldmappings.replaceFieldMap(field_to_map_index, field_to_map)

#Create a list of append datasets and run the the tool
inData = [append_layer]
arcpy.Append_management(inData, target_layer, field_mappings=fieldmappings)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Here you go

deleted-user-zpcJw1u0IXiO
Occasional Contributor

Thank you, Luke, for this. It works great, but for anyone who gets an error on the last line, try changing it to:

arcpy.Append_management(inData, target_layer, "NO_TEST", fieldmappings)

ModernElectric
Occasional Contributor III

This is a great script that initial testing has been successful. However, I am having another problem. 

A little background first... I have a WORKING GDB on my local drive that is used to run analysis and edits. We have a separate PRODUCTION GDB that is on our network that is used to "run" the .PMF on ArcReader. I successfully built an ArcGIS Model to Truncate and Append data from the WORKING GDB to the PRODUCTION GDB without requiring ArcReader users from exiting out. It has been my goal to build a group of Python scripts to do this automatically with the help Windows Task Scheduler. Here is the problem. I have a few Feature Class(s) that has attachment(s) and the only way I have found the ArcGIS Model to work successfully is by opening the .MXD for the PRODUCTION GDB and run the ArcGIS Model. All of the attachments append/copy over without a problem. Now, doing this with the Field Mapping Python script isn't working properly. Any advice to duplicate the process I have been doing, but with the Field Mapping script with attachments?

Thanks

Again, GREAT script!!

0 Kudos
Matt_Trebesch
New Contributor III

Luke, or anyone else...

I too am learning to append and user fieldmap, flieldmappings to get my fields squared away.

basically I'm appending data into a "shell" featureclass that is empty.  There's only a few fields I'm appending and trying to match.

I'm able to add ONE field change with this script:

(append and target layers are defined earlier in the script as well as importing the arpy library

# field map and append

append_layer = outputclip
target_layer = out_name


#This is like defining an empty grid of fields you see when you run it manually in the toolbox
fieldmappings = arcpy.FieldMappings()

#Add the target datasets fields to the field map table  (Like when you manually choose a tagert layer in the toolbox)
fieldmappings.addTable(target_layer)

#Add the append datasets fields to the field map table  (Like when you manually choose a tagert layer in the toolbox)
fieldmappings.addTable(append_layer)

#At this point, you have a grid like when you run it manually saved in your field mappings.




#####Lets map a field that have different names!


#Find which "Index" the field has as we cant refer to them by name when editing the data only index
field_to_map_index = fieldmappings.findFieldMapIndex("PropertyAddress")  #Field name that exists in the target layer but not append data source!

#Grab "A copy" of the field map object for this particular field
field_to_map = fieldmappings.getFieldMap(field_to_map_index)

#Update its data source to add the input from the the append layer
field_to_map.addInputField(append_layer, "Addy_Concat")



#####Lets update the master field map using this updated copy of a field

fieldmappings.replaceFieldMap(field_to_map_index, field_to_map)


#Create a list of append datasets and run the the tool
arcpy.Append_management(append_layer, target_layer, "NO_TEST", fieldmappings, "1")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

So, this works for me.  I end up appending the "Addy_Concat" field into the "PropertyAddress" field.   Great.

But I have 3 other fields to do this with.  So I used this:

# field matching for append
# Name the Append and Target Layer (target layer is named in the ArcGIS Pro Map)
append_layer = outputclip
target_layer = out_name

fieldmappings = arcpy.FieldMappings()

# Like when you manually choose a layer in the toolbox and it adds the fields to grid
fieldmappings.addTable(target_layer)
fieldmappings.addTable(append_layer)

# Lets map fields that have different names!
list_of_fields_we_will_map = []

# list_of_fields_we_will_map.append(( append layer field name, target layer field name)
list_of_fields_we_will_map.append(('Addy_Concat', 'PropertyAddress'))
#list_of_fields_we_will_map.append(('CMUNITY', 'PropertyCity'))
#list_of_fields_we_will_map.append(('ZipCode', 'PropertyZip'))

print(*list_of_fields_we_will_map, sep = ", ")


for field_map in list_of_fields_we_will_map:
    # Find the fields index by name.
    field_to_map_index = fieldmappings.findFieldMapIndex(field_map[0])
    # Grab "A copy" of the current field map object for this particular field
    field_to_map = fieldmappings.getFieldMap(field_to_map_index)
    # Update its data source to add the input from the the append layer
    field_to_map.addInputField(append_layer, field_map[1])
    # We edited a copy, update our data grid object with it
    fieldmappings.replaceFieldMap(field_to_map_index, field_to_map)

# Create a list of append datasets and run the the tool - the "1" at the end is for the subtype.
inData = [append_layer]
arcpy.Append_management(inData, target_layer, "NO_TEST", fieldmappings, "1")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I keep on getting this error:

Traceback (most recent call last):
File "<string>", line 119, in <module>
File "c:\program files\arcgis\pro\Resources\arcpy\arcpy\arcobjects\arcobjects.py", line 666, in addInputField
return convertArcObjectToPythonObject(self._arc_object.AddInputField(*gp_fixargs(args)))
RuntimeError: FieldMap: Error in adding input field to field map

Specifically, it's this line (119) that's causing issues:

field_to_map.addInputField(append_layer, field_map[1])

If you noticed, I remarked out my multi fields to let the list just take the one field match combo that worked in the simpler, non-list script.

Still throws this error.

I'm confused how this list loops works.   Not sure how it's figuring out position 0,1 with the  case like

('Addy_Concat', 'PropertyAddress')

Any ideas what I'm doing wrong here? Or what I can do to try accomplish what I want to do?

0 Kudos