Community

Append Tool and Field Mapping ... Help? Examples?

23755
21
Jump to solution
11-03-2016 08:19 AM
Brownschuh
by
Occasional Contributor II
‎11-03-2016 08:19 AM

Having a bit of trouble with the field mapping component of the append tool; I cannot seem to get it to work properly as I am left with a feature class that is populated with all the features, but the attribute table remains blank. 

Background info: I have two feature classes in an enterprise geodatabase with address point data.  One is the authoritative source edited by our address coordinator and the other is used as to host a map service.  The data is essentially the same, but the schemas are slightly different; half of the fields are the exactly the same, but others have different names.  My workflow has been to update the 'map service' feature class on a daily basis by deleting all the records and loading the data manually, selecting the NO_TEST option, and manually mapping the fields where the names are different ... but this is where I run into trouble with my script.

I've ran the append process manually using the geoprocessing tool and copied the python snippet of the field mapping component and am left with an output attribute table that has all null values.  Ideally I'd like to move away from this style of coding as I am aware it can be rather clunky/lengthy ... and it just does not work (even though it throws no errors).

I'd like to do 'real' field mapping, but so far the examples I've found have a bit more complicated or focus mainly on the Merge tool.  I'd like to see a nice, simple Append tool example if possible.

Also, how does field mapping work if some fields are the same and others are different? I haven't run across a good example that deals with this problem.

Here are some of the sources I've looked at so far:

Field mapping and Python scripting | ArcGIS Blog

Mapping input fields to output fields—ArcPy Get Started | ArcGIS for Desktop 

21 Replies
AngelaBlanco
by
New Contributor
‎03-03-2019 07:52 AM

I tried to create a pyhton script using FieldMapping following the examples  on https://pro.arcgis.com/en/pro-app/arcpy/classes/fieldmappings.htm.

When I use exportToString () for the FieldMapping in the python console I got different syntax than from Python Snippet

Python console

" FREQUENCY " true true false 4 Long 0 0 ,First,#,D:\Processed_Data\03_Geodatabase\07_20170226_Afternoon.gdb\Afternoon_20170226_Average,FREQUENCY,-1,-1,

Python Snippet

FREQUENCY "FREQUENCY" true true false 4 Long 0 0 ,First,#, Afternoon_20170226_Average,-1,-1;

I don’t understand why one must set properties of the output name. https://preply.com/en/question/can-anyone-give-me-some-examples-please-of-the-correct-way-to-use-ust...

Can anyone give more examples, please?

0 Kudos
DylanHarwell
by
Occasional Contributor
‎04-30-2019 01:45 PM

I just ran into a situation where I needed to figure out field mapping with Python, and my situation is slightly different than the examples posted here so I thought I'd share my solution. 

The Problem: We have a web map that has what we call 'compiled layers'. These are layers built from features joined to SQL Server views, which can be incredibly slow and impractical on a web map. I built a python module to reuse in scripts that run every night and perform the join, delete and append to target feature classes in our database. Originally these were set up with models, but now we are going to create a lot more and that is the reason behind scripting this. Whenever these models were first created, the target feature class only contained a subset of fields from the joined tables - so field mapping came into play. 

The examples above assume that you know what fields you are mapping, or that you already have a field map string. The method I came up with is generic enough to create a field map with no knowledge of the append fields or target fields. And since we are an organization, some trickery is involved to match the fields created after the join since they aren't just a field name but are something like <DB name>.DBO.<feature name>.

For info on what the EmailService module is and how to get email notifications when scripts fail, check this out. 

#-------------------------------------------------------------------------------
# Name:        CompiledLayers.py
#
# Author:      Dylan Harwell - GIS Tech - City of GJ
#
# Created:     25/04/2019
# Copyright:   (c) dylanh 2019
#
#-------------------------------------------------------------------------------
import arcpy
arcpy.env.overwriteOutput = True


class CompiledLayers(object):
    """Custom class to create compiled layers in lieu of joined layers on
    web maps."""
    def __init__(self):
        self.ignore = ['OBJECTID', 'Shape', 'Shape.STArea()',
                       'Shape.STLength()', 'STArea()', 'STLength()',
                       'GlobalID', 'ESRI_OID', 'SHAPE' ]

    def createCompiledLayer(self, join_fc, in_field, join_table,
                            join_field, join_type, target_fc):
        """Makes feature layer, joins to sql server table, sets up fieldmapping,
        deletes and appends target feature class in gjgisweb.sde."""
        arcpy.MakeFeatureLayer_management(join_fc, 'temp_lyr')
        arcpy.AddJoin_management('temp_lyr', in_field, join_table, join_field,
                                 join_type)

        target_fields = [f for f in arcpy.ListFields(target_fc)
                         if f.name not in self.ignore]
        append_fields = [f for f in arcpy.ListFields('temp_lyr')
                         if f.name.split('.')[-1] not in self.ignore]

        fields_to_map = []
        for a in append_fields:
            for t in target_fields:
                if a.name.split('.')[-1].lower() == t.name.lower():
                    fields_to_map.append((a.name, t.name))

        fieldmapping = arcpy.FieldMappings()
        for fields in fields_to_map:
            field_map = arcpy.FieldMap()
            field_map.addInputField('temp_lyr', fields[0])
            field_map.addInputField(target_fc, fields[1])
            fieldmapping.addFieldMap(field_map)

        arcpy.DeleteFeatures_management(target_fc)
        arcpy.Append_management('temp_lyr', target_fc, 'NO_TEST', fieldmapping)


def main():
    pass
if __name__ == '__main__':
    main()


################### Example of using this module in a script ########################
import CompiledLayers as CL
import EmailService as ES

def getTime():
    """Function to fetch current date and time."""
    date_time = time.strftime("%Y%m%d") + "_" + time.strftime("%H%M%S")
    return date_time

# Set path to log file
log = 'T:...\\Log\\CompileNorthstarTrashParcels' + getTime() + '.txt'

# Set variables
join_fc    = 'T:...\\dbConnections\\gjgisprod.sde\\gjgisprod.DBO.Parcel'
in_field   = 'PARCEL_NUM'
join_table = 'T:...\\dbConnections\\ns.sde\\Northstar64_Live.dbo.gj_trashdata'
join_field = 'PARCEL'
join_type  = 'KEEP_COMMON'
target_fc  = 'T:...\\dbConnections\\gjgisweb.sde\\gjgisweb.DBO.CompiledLayers\\gjgisweb.DBO.northstarTrashParcels'

with open(log, 'w') as log_file:
    try:
        log_file.write('Calling CompiledLayers module at ' + getTime() + '\n')
        compiler = CL.CompiledLayers()
        compiler.createCompiledLayer(join_fc, in_field, join_table, join_field,
                                     join_type, target_fc)
        log_file.write('Succeeded creating compiled layer at ' + getTime())

    except Exception as e:
        log_file.write('Failed at ' + getTime() + '\n')
        log_file.write(e.message)

        # Send email to server admin if script fails
        service = ES.EmailService()
        subject = 'Server Script Failure!'
        body = 'CompiledNorthstarTrashParcels.py failed'
        recip = 'jacksont@gjcity.org'

        service.configMessage(subject, body)
        service.sendEmail(recip)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
0 Kudos