Append Field Mapping Not Working

Question

Using the ArcGIS API for Python in a desktop Jupyter notebook. I am trying to automate a process of appending new data from a survey123 feature service into a feature service used by a dashboard. I am trying to use the Append method, but the field mapping does not work for fields that don't have matching names. There are no errors. The append works fine otherwise. Every field that has a matching name in the source and the target data appends and updates with upsert just fine. I seem to have similar or other issues with the Append method in the API for the last three years.

The process I have chosen is to query the records I need from the source data with the result being a dataframe. That dataframe is exported to a GDB. That GDB is used to append the records.

Is this a bug? Am I missing something? Does append just still not work properly?

from arcgis.gis import GIS
gis = GIS("PRO")

sourceData = gis.content.get({itemid})
targetData = gis.content.get({itemid})

targetData_lyr = targetData.layers[0]
sourceData_lyr = sourceData.layers[0]

sourceData_df = sourceData_lyr.query(
    where="Contract_Well_Status = 'Plugged' OR Contract_Well_Status = 'In Progress'",
    as_df=True,
    out_fields="Contract,FARM_NAME,county,Municipality,API,LAT_DEC,LON_DEC,Coal_Area,Bid_Package_to_Construction,Issue_Date,Pre_Bid_Meeting,Bid_Open_Date,Bid_Winner,Contracted_Amount,Recommendation_Memo,Contract_Award_Statement,Actual_Start_Date,Actual_End_Date,Final_Invoice_Amount,Plug_Cert,Final_Inspection_Date,wellStatus,Attainable_Bottom,Solid_Plug,Soil_Remediation,Ecological_Restoration,Infrastructure_Removal,Land_Type,wellScore,Environmental_Impact,Human_Impact,Leaking_Gas,Water_Contamination,environmentalJustice,Contract_Well_Status,Notice_to_Proceed,GlobalID"
)

sourceData_fc = gis.content.import_data(sourceData_df)
tempGDB = gis.content.search(query="title:" + sourceData_fc.title, item_type="File Geodatabase")
itemGDB = tempGDB[0]

targetData_lyr.append(item_id=itemGDB.id,
                 upload_format = "filegdb",
                 field_mappings=[{"name" : "Contract","sourceName" : "Contract"},
                                  {"name" : "Name","sourceName" : "FARM_NAME"},
                                  {"name" : "County","sourceName" : "county"},
                                  {"name" : "Municipality","sourceName" : "Municipality"},
                                  {"name" : "API","sourceName" : "API"},
                                  {"name" : "Lat","sourceName" : "LAT_DEC"},
                                  {"name" : "Long","sourceName" : "LON_DEC"},
                                  {"name" : "Coal_Area","sourceName" : "Coal_Area"},
                                  {"name" : "Bid_Package_to_Construction","sourceName" : "Bid_Package_to_Construction"},
                                  {"name" : "Issue_Date","sourceName" : "Issue_Date"},
                                  {"name" : "Pre_Bid_Meeting","sourceName" : "Pre_Bid_Meeting"},
                                  {"name" : "Bid_Open_Date","sourceName" : "Bid_Open_Date"},
                                  {"name" : "Bid_Winner","sourceName" : "Bid_Winner"},
                                  {"name" : "Contracted_Amount","sourceName" : "Contracted_Amount"},
                                  {"name" : "Recommendation_Memo","sourceName" : "Recommendation_Memo"},
                                  {"name" : "Contract_Award_Statement","sourceName" : "Contract_Award_Statement"},
                                  {"name" : "Actual_Start_Date","sourceName" : "Actual_Start_Date"},
                                  {"name" : "Actual_End_Date","sourceName" : "Actual_End_Date"},
                                  {"name" : "Final_Invoice_Amount","sourceName" : "Final_Invoice_Amount"},
                                  {"name" : "Plug_Cert","sourceName" : "Plug_Cert"},
                                  {"name" : "Final_Inspection_Date","sourceName" : "Final_Inspection_Date"},
                                  {"name" : "Status","sourceName" : "wellStatus"},
                                  {"name" : "Attainable_Bottom","sourceName" : "Attainable_Bottom"},
                                  {"name" : "Solid_Plug","sourceName" : "Solid_Plug"},
                                  {"name" : "Soil_Remediation","sourceName" : "Soil_Remediation"},
                                  {"name" : "Ecological_Restoration","sourceName" : "Ecological_Restoration"},
                                  {"name" : "Infrastructure_Removal","sourceName" : "Infrastructure_Removal"},
                                  {"name" : "Land_Type","sourceName" : "Land_Type"},
                                  {"name" : "AO_Final_Score","sourceName" : "wellScore"},
                                  {"name" : "Environmental_Impact","sourceName" : "Environmental_Impact"},
                                  {"name" : "Human_Impact","sourceName" : "Human_Impact"},
                                  {"name" : "Leaking_Gas","sourceName" : "Leaking_Gas"},
                                  {"name" : "Water_Contamination","sourceName" : "Water_Contamination"},
                                  {"name" : "EJ_Zone","sourceName" : "environmentalJustice"},
                                  {"name" : "Contract_Well_Status","sourceName" : "Contract_Well_Status"},
                                  {"name" : "Notice_to_Proceed","sourceName" : "Notice_to_Proceed"},
                                  {"name" : "GlobalID","sourceName" : "globalid"}
                                ],
                 upsert=True,
                 skip_updates=False,
                 use_globalids=False,
                 update_geometry=True,
                 rollback=False,
                 skip_inserts=False,
                 upsert_matching_field="GlobalID")

GT_UTCO · Answer

This is still a problem with the field mapping not working. The real bad part about it is you still get a good response, even if it didn't work.

AnthonyAtkins2 · Answer

I'm wondering if you were able to find a solution or a work around (other than renaming the destination fields to match)? I'm having a similar issue, I'm trying to append from a shapefile, and the shortened fields aren't matching.
The problem with the "work around" provided above and using the converted python code is that the append function used in that scenario is a different function than what is needed to append to a feature layer.

Thanks

JoelRogers · Answer

Thanks @JakeSkinner . However, I did not say I am using Pro. I want a stand-alone script that I can run in ArcGIS Online as a task or a script where I don't have to rely on Pro. I just didn't want to show my admin credentials for the example above. Also, I am using Pro with a script for this exact workflow on a few other ETL processes because I haven't been able to get this process to work as expected. I don't like doing that. Using Pro presents its own problems. One of which is the user account automatically logs out randomly on the Pro instance on the server used for scripts.

I was hoping the API would work as advertised for this scenario, but I understand if I have to add this process to the list of Esri things that just aren't working yet.