Append Tool does not respect field map

oharlacker_verita

I recently encountered strange behavior upon attempting to append features from a FGDB to an AGOL Feature Service using the Append GP Tool. The two layer schemas do not match so I was using a field map to reconcile any differences. The tool then returned a set of warnings for each feature talking about how the value from one field in the input layer was not allowed in a different field in the target layer. (For example, it was warning about a value from a Type (TEXT) field not being able to be inserted into the Longitude (DOUBLE) field. These fields were not mapped together in the field map, but this warning went away when removing any mappings to the target layer's field. This is a new behavior that has only happened after the latest bugfix release, and was not an issue last week. Running the append tool in arcpy with the schema_type parameter set to "NO_TEST" resulted in the features being appended correctly and without any warnings.

GP Tool OutputField Map for Longitude FieldField Map for Type Field

KevinSaavedra

Hi, I just want to clarify: the warning was raised when the input StructureType (text) field was not explicitly mapped to the target longitude (double) field? After the problem run, if you reset the field map in the field properties window, and re-map to reconcile field schema, does the problem still occur?

If you are able to share, I would be curious to see the field map string generated from the tool run that raised the warning.

In the history pane->geoprocessing tab->Append tool run that had the warning. Right click->copy python command. Paste the output of the field_mapping parameter.

e.g.:

field_mapping=r'FacilityID "FacilityID" true true false 30 Text 0 0,First,#,C:\Path\to\gdb\redlands_buildings.gdb\Redlands_buildings_inpt,FacilityID,0,29; etc.

oharlacker_verita

The problem occured when the StructureType field of the input layer was mapped to the StructureType field of the target layer and the Longitude of the input was mapped to the Longitude of the target. When all mappings were removed from the Longitude field, it no longer showed a warning about Longitude, but still showed a warning about a different field having the same issue. Resetting the field map resulted in the same mapping string.

The field map string from that specific tool run is:

# Raw string directly from copy command:
field_mapping=r'Enabled "Enabled" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Enabled,-1,-1;Comments "Comments" true true false 255 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Comments,0,254;Name "Name" true true false 255 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Name,0,254;SymbolRotation "RotationAngle" true true false 0 Double 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,SymbolRotation,-1,-1;StructureType "StructureType" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,StructureType,0,49;Workflow_Status "WorkflowStatus" true true false 25 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Workflow_Status,0,24;GlobalID "GlobalID" true true false 38 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,GlobalID,-1,-1;ResidentialCount "ResidentialCount" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,ResidentialCount,-1,-1;CommercialCount "CommercialCount" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,CommercialCount,-1,-1;IsInMdu "IsInMdu" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,IsInMdu,-1,-1;NetworkId "NetworkId" true true false 38 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,NetworkId,-1,-1;VacantCount "VacantCount" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,VacantCount,-1,-1;FACILITYID "FACILITYID" true true false 255 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,FACILITYID,0,254;OWNER "OWNER" true true false 100 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,OWNER,0,99;Latitude "Latitude" true true false 0 Double 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Latitude,-1,-1;Longitude "Longitude" true true false 0 Double 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Longitude,-1,-1;ConstructionStatus "ConstructionStatus" true true false 20 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,ConstructionStatus,0,19;JobNumber "JobNumber" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,JobNumber,0,49;Material "Material" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Material,0,49;StructureSize "StructureSize" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,StructureSize,0,49;Manufacturer "Manufacturer" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Manufacturer,0,49;Model "Model" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Model,0,49;MDUCount "MDUCount" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,MDUCount,-1,-1;Market "Market" true true false 255 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Market,0,254'

# Formatted with newlines after semicolon:

field_mapping=r'
Enabled "Enabled" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Enabled,-1,-1;
Comments "Comments" true true false 255 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Comments,0,254;
Name "Name" true true false 255 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Name,0,254;
SymbolRotation "RotationAngle" true true false 0 Double 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,SymbolRotation,-1,-1;
StructureType "StructureType" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,StructureType,0,49;
Workflow_Status "WorkflowStatus" true true false 25 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Workflow_Status,0,24;
GlobalID "GlobalID" true true false 38 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,GlobalID,-1,-1;
ResidentialCount "ResidentialCount" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,ResidentialCount,-1,-1;
CommercialCount "CommercialCount" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,CommercialCount,-1,-1;
IsInMdu "IsInMdu" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,IsInMdu,-1,-1;
NetworkId "NetworkId" true true false 38 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,NetworkId,-1,-1;
VacantCount "VacantCount" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,VacantCount,-1,-1;
FACILITYID "FACILITYID" true true false 255 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,FACILITYID,0,254;
OWNER "OWNER" true true false 100 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,OWNER,0,99;
Latitude "Latitude" true true false 0 Double 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Latitude,-1,-1;
Longitude "Longitude" true true false 0 Double 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Longitude,-1,-1;
ConstructionStatus "ConstructionStatus" true true false 20 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,ConstructionStatus,0,19;
JobNumber "JobNumber" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,JobNumber,0,49;
Material "Material" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Material,0,49;
StructureSize "StructureSize" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,StructureSize,0,49;
Manufacturer "Manufacturer" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Manufacturer,0,49;
Model "Model" true true false 50 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Model,0,49;
MDUCount "MDUCount" true true false 0 Long 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,MDUCount,-1,-1;
Market "Market" true true false 255 Text 0 0,First,#,C:\Users\oharlacker\Documents\Projects\WOW\Append_Issues\GIS\Port A.gdb\UndergroundStructure,Market,0,254'

KevinSaavedra

Thanks. I can't immediately see any problems with the field map string that's being generated. A few more questions:

When you run the tool, is the "Optimize performance for Feature Services" parameter available? If so, do you run it checked or unchecked?
Is the target feature service a hosted feature service or is it published to reference an eGDB? (Edit: I see you said AGOL)
What is the schema difference between the target StructureType/Longitude fields and the input?

oharlacker_verita

This behavior happened both when the "Optimize performance for Feature Services" parameter was enabled and disabled. The target layer is a feature service hosted on AGOL. The input layer is an export from ArcFM but this hasn't been an issue in the past.

Target Layer Schema:

Field Name	Field Type	Domain
OBJECTID	OID
Enabled	Integer
Comments	String
Name	String
GlobalID	String
IsInMdu	Integer
NetworkId	String
SpecSymbol	String
FACILITYID	String
OWNER	String
Latitude	Double
Longitude	Double
JobNumber	String
Material	String
Model	String
MDUCount	Integer
Market	String
Permit_No	String
GlobalID_2	GlobalID
CreationDate	Date
Creator	String
EditDate	Date
Editor	String
Released_to_Construction	String	UndergroundStructure_Released_to_Construction_...
Structure_Type	String	UndergroundStructure_Structure_Type_307a90b9-f...
Construction_Status	String	UndergroundStructure_Construction_Status_0e8e7...
Cabinet	String
OLT	String
SymbolRotation	Double
StructureType	String
ResidentialCount	Integer
CommercialCount	Integer
VacantCount	Integer
ConstructionStatus	String
StructureSize	String
Workflow_Status	String
Manufacturer	String
Shape	Geometry

Input Layer Schema:

Field Name	Field Type	Domain
OBJECTID	OID
SHAPE	Geometry
AncillaryRole	SmallInteger
Enabled	SmallInteger	EnabledDomain
CreationUser	String
DateCreated	Date
DateModified	Date
LastUser	String
Comments	String
Name	String
SymbolRotation	Double
StructureType	String	Structure Type
Workflow_Status	String	Workflow_Status
GlobalID	GlobalID
ResidentialCount	Integer
CommercialCount	Integer
IsInMdu	Integer	Yes/No
NetworkId	Guid
CatalogId	String
IsLong	String
VacantCount	Integer
SpecSymbology	String
FACILITYID	String
OWNER	String
Latitude	Double
Longitude	Double
ConstructionStatus	String	Fiber Construction Status
JobNumber	String
Material	String	UndergroundStructureMaterial
StructureSize	String	UndergroundStructureSize
Manufacturer	String	Fiber Structure Manufacturer
Model	String
FacilityDiagram	Integer	Yes/No Indicator
MDUCount	Integer
Market	String	F_Market
RT_GlobalID	Guid
Ref_autocad_block_name	String
Ref_autocad_entity_handle	String
Ref_autocad_layer	String
Ref_autocad_source_filename	String
RT_ID	String
Ref_UniqeID	String
Ref_AmpName	String
Ref_Relationship	String
SourceID	String
RT_MARKET	String
RT_Networkid	Guid

KevinSaavedra

I'm unable to reproduce the issue with a local feature class using the schema you provided. This could be specific to the ArcFM export; if you are able to share, please send it to us at ksaavedra@esri.com.