topic Re: How to specify fields to use in a merge? in Python Questions

How to specify fields to use in a merge?

JustinBridwell2 — Sat, 11 Dec 2021 23:35:16 GMT

I am using arcpy to merge multiple feature classes from a list of feature classes. They all have the same fields. Let's say they all have the fields = ['CO_FIPS', 'MILES', 'FLOOD_ZONE', 'VALID', 'REGION']. However, I only want to include 'CO_FIPS', 'MILES', and 'FLOOD_ZONE' in the output merged feature class. It seems I need to use field mapping to limit the fields in my output, but I am not 100% how to do that:

working_dir = r"C:\Projects\MyProj\stream_lines.gdb"
arcpy.env.workspace = working_dir 
s_studies_for_merge = ['stream1', 'stream2', 'stream3']
fieldMappings = arcpy.FieldMappings()
merged_output = os.path.join(working_dir, "AllStreams")
fieldMappings.addTable('stream1')
for field in fieldMappings.fields:  
    if field.name not in ['CO_FIPS', 'MILES', 'FLOOD_ZONE']:   
        fieldMappings.removeFieldMap(fieldMappings.findFieldMapIndex(field.name))     
arcpy.Merge_management(s_studies_for_merge, merged_output, fieldMappings)

My confusion is mainly that most examples show only 2 features being merged and that the mapping involves settings that I don't need for certain fields. I only want to exclude the fields I don't want and keep the fields I do. What do I need to do here?

Re: How to specify fields to use in a merge?

CCWeedcontrol — Sat, 11 Dec 2021 23:35:19 GMT

If you just want to keep certain fields i would just use drop/delete fields. I use filed mapping if I want to change a field or do more complex things.

Here is the example

import arcpy,os, sys

#drop feilds
fc = r"D:\GIS Folder\Roads.gdb\Roads"

keep_fields = ['STREET','ROAD_TYPE'] #fields to keep

fieldNameList = []
fieldObjList = arcpy.ListFields(fc)

for field in fieldObjList:
    if (not field.name in keep_fields) and (not field.required):
        fieldNameList.append(field.name)


arcpy.DeleteField_management(fc, fieldNameList)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: How to specify fields to use in a merge?

JustinBridwell2 — Wed, 30 Oct 2019 18:21:55 GMT

OK, lets say you want to make sure that the char length of all string fields called 'REGION' are 4 characters using field mapping? This needs to be done during the merge.

Re: How to specify fields to use in a merge?

CCWeedcontrol — Sat, 11 Dec 2021 23:35:21 GMT

I've never been able to really understand that filed mapping syntax.

Here is what i use.

import arcpy,sys
from datetime import datetime as d
startTime = d.now()
start_time = time.time()

Fet1 = "blah1"
Fet2 = "Blah2"
Fet3 = "Blah3"

FieldMapString = '''
DXF_TEXT "DXF_TEXT" true true false 11 Text 0 0 ,First,#, {0}, DXF_TEXT,-1,-1;
ACCOUNT "ACCOUNT" true true false 11 Text 0 0 ,First,#, {0}, ACCOUNT,-1,-1;
PIN "PIN" true true false 13 Text 0 0 ,First,#, {0}, PIN,-1,-1;
ACRES "ACRES" true true false 4 Double 0 0  ,First,#, {0}, ACRES,-1,-1;
Instrument "Instrument" true true false 10 Text 0 0 ,First,#, {0}, Instrument,-1,-1;
SiteAddres "SiteAddres" true true false 106 Text 0 0 ,First,#, {0}, SiteAddres,-1,-1;
SiteCity "SiteCity" true true false 32 Text 0 0 ,First,#, {0}, SiteZip,-1,-1;
SiteZip "SiteZip" true true false 10 Text 0 0 ,First,#, {0}, SiteZip,-1,-1
'''
def Layers1(Fet1):
    """pass the variable and the constant into
    the function
    """
    fieldmappings = arcpy.FieldMappings()
    fieldmappings.loadFromString(FieldMapString)
    return fieldmappings

def main(args=None):
        if args is None:
                args = sys.argv

# Process: Feature Class to Feature Class
arcpy.Append_management(Fet1,Fet2,Fet3 "C:\Temp\Blah.gdb\Fe_ALL","NO_TEST", Layers1(Fet1))

try:
    print ('(Elapsed time: ' + str(d.now() - startTime)[:-3] + ')')

except Exception as e:
    # If an error occurred, print line number and error message
    import traceback, sys
    tb = sys.exc_info()[2]
    print ("Line %i" % tb.tb_lineno)
    print (e.message)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I had trouble using this field mapping with Merge until Randy Burton posted some code and can be found here.

Merge with field mapping

Re: How to specify fields to use in a merge?

NobbirAhmed — Thu, 31 Oct 2019 00:01:02 GMT

Hi @Justin Bridwell,

You want to use the fieldmap in Append tool? or in Merge tool? However, field map works same in all tools. Your code example shows Append - let's go with it.

As far as I understood - you want to keep only three fields -

'CO_FIPS', 'MILES', 'FLOOD_ZONE'

And exclude all other fields. I assume these three fields are present in all of the input feature classes ('stream1', 'stream2', 'stream3'), right?

Also, when you run your code what errors you are getting back?

Here is a nice example on Esri's help page: you can play around and see whether it can be of any use to you.

FieldMappings—ArcPy classes | ArcGIS Desktop

Re: How to specify fields to use in a merge?

NobbirAhmed — Thu, 31 Oct 2019 16:27:03 GMT

Here is a way to get it done:

import arcpy
import os
import sys

# Make data path relative (not relevant unless data is moved here and paths modified)
wks = os.getcwd() # location of the script file
in_gdb = os.path.join(wks, "GeonetUserQuery.gdb") # geodatabase is in the same folder as the script's

arcpy.env.overwriteOutput = True

try:

   in_fc1 = os.path.join(in_gdb, "Eid2few")
   in_fc2 = os.path.join(in_gdb, "Eid4few")
   in_fc3 = os.path.join(in_gdb, "ElDoradoFew")

   out_fc = os.path.join(in_gdb, "merge_outfc2b")

# construct the fieldmap
fieldMappings = arcpy.FieldMappings()

   fieldMappings.addTable(in_fc1) # add all fields of the table
   fieldMappings.addTable(in_fc2)   # add all fields of the table
   fieldMappings.addTable(in_fc3)    # add all fields of the table

fields = fieldMappings.fields
fields_to_exclude = ['Extra1', 'Extra2', 'Extra3']

   for field in fieldMappings.fields:
      if field.name in ['Extra1', 'Extra2', 'Extra3']:
         print(field.name)
         fieldMappings.removeFieldMap(fieldMappings.findFieldMapIndex(field.name))

#print(fieldMappings.exportToString())

arcpy.Merge_management("{};{};{}".format(in_fc1, in_fc2, in_fc3), out_fc, fieldMappings)

except arcpy.ExecuteError as aex: # get arcpy-specific exception messages
print(aex.args[0])

except Exception as ex: # get generic exception messages
print(ex.args[0])