Error using append method from ArcGIS API for Python

[JosephHilliard94]

Hi there,

I have been trying to append from one feature layer to another using the ArcGIS API for Python. I get the below error message? 

Exception: Object reference not set to an instance of an object.
(Error Code: 400)

My code is below:

archive_trees.append(item_id=lyr,upload_format='featureCollection',source_table_name='Felled_Trees',upsert=False,update_geometry=True,field_mappings=[{"name" : "OBJECTID","sourceName" : "OBJECTID"},{"name" : "PROPERTY","sourceName" : "property"},{"name" : "LOCATION_DETAILS","sourceName" : "location_details"},
{"name" : "SPECIES","sourceName" : "species"},{"name" : "USAGEZONE","sourceName" : "usagezone"},{"name" : "TREEREF","sourceName" : "treeref"},
{"name" : "AGE","sourceName" : "age"},{"name" : "GIRTH","sourceName" : "girth"},{"name" : "TREETYPE","sourceName" : "treetype"},{"name" : "PROPREF","sourceName" : "propref"},
{"name" : "SITEINFORMATION","sourceName" : "siteinformation"},{"name" : "TREEINFORMATION","sourceName" : "treeeinformation"},{"name" : "TREECOMMENTS","sourceName" : "treeinformation"},
{"name" : "GLOBALID","sourceName" : "globalid"},{"name" : "CREATED_USER","sourceName" : "created_user"},{"name" : "CREATED_DATE","sourceName" : "created_date"},
{"name" : "LAST_EDITED_USER","sourceName" : "last_edited_user"},{"name" : "LAST_EDITED_DATE","sourceName" : "last_edited_date"}])

Any help would be great as this never works for me!

Joe

[MCameron]

Hi Joe, 

Commonly, this error can occur when trying to Append data from one feature layer to another based on one of the following: 

• There are duplicate field values in the field used as the matching field to append the hosted feature layer.
• The field values in the dataset exceed the character limit of the matching field of the hosted feature layer.
• The file geodatabase appended in ArcGIS Online is not recognised.

Could you please review the following Technical Article Solutions and confirm if any of these options help to resolve the issue you are seeing? --> https://support.esri.com/en/Technical-Article/000025195.

[JosephHilliard94]

Hi there,

I have checked:

•  The target dataset is empty so there can't be an a duplicate value in the FID. 
• Field values do not exceed character limit (same append works using ArcGIS Pro just not on API). 
• File Geodatabase has successfully created the feature layer. 

Upsert is set to False as sync is enabled, can you offer any other solutions? I tried with a one feature append using the same schema and didn't work.

Thanks!

[MCameron]

Hi Joe,

When you are calling "lyr", is this an item in your ArcGIS Online? If so, could you update item_id=lyr.id in the following line to include .id: 

archive_trees.append(item_id=lyr.id,upload_format='featureCollection',source_table_name='Felled_Trees',upsert=False,update_geometry=True,field_mappings=[....

Based on the ArcGIS API for Python Append documentation this would be required. Or you could use the item id reference which can be found in the URL of the layers item description page: 

[JosephHilliard94]

Hi again,

Still not working sadly, it is possible for me to PM you my script for you to have a look?

Thanks!

[JosephHilliard94]

This is what I got up to and same error message: 

#Selection of Felled Trees

item = tsm_trees.query(where="TREETYPE='FEL'")

sdf = item.sdf

#Convert SDF to Feature Layer
lyr = sdf.spatial.to_featurelayer(sdf, folder='TSM')

NOTE - Tried GDB and Feature Layer here 

GDB = "194etc"

geodatabase = gis.content.get(GDB)

new_input = geodatabase.id

FEATURE_LAYER = "f14etc"

fcl = gis.content.get(FEATURE_LAYER)

new_fcl = fcl.id

#Append Data Appended Feature Layer -

archive_trees.append(item_id=new_input,upload_format='filegdb',source_table_name='Felled_Trees',upsert=False)

NOTE: Upsert has to be false as sync is enabled and doesn't work when I turn it off. 

A solution would be grand as this is a common workflow for us. 

[Clubdebambos]

Hi @JosephHilliard94 

Make sure the item_id parameter is set to an actual item id and not just pointing to an object like a layer. 

[JosephHilliard94]

I have tried that as well, no dice, thanks though!

[Clubdebambos]

I think I've only ever got the append to work with a CSV 😞

Alternatively, if the source data being appended had the same schema as the target you can convert a FeatureCollection to a FeatureSet and use  FeatureLayer.edit_features(add=feature_set) 

from arcgis import GIS
from arcgis.features import FeatureCollection

agol = GIS("home")

## Feature Collection object
fc = agol.content.get("**FC_Item_ID**")
fc = FeatureCollection.fromitem(fc)

## get as FeatureSet
fc_set = fc.query()

## feature service containing the target layer
fs = agol.content.get("**FS_Item_ID"**)

## target layer
lyr = fs.layers[0]

## add in the feature set, will return a dictionary with successful/unsuccessful results
print(lyr.edit_features(adds=fc_set))

I know that doesn't help you now, but might help plan for/be useful in the future.

[JakeAdamsUGRC]

We were running into this problem as well when trying to append a spatially-enabled dataframe to an existing layer in a feature service. I don't know if this will help Joe, but I figured I'd put it here for anyone else who gets here from Google.

Our workflow involved getting the live data from the feature service as a dataframe, inserting new rows, truncating the existing data from the feature service, and then using .append to write the new, enlarged dataframe to the feature service using the following code. We kept hitting the "Object reference not set..." error (which, digging through the API source, is coming from the method that keeps checking the status of the operation).

geojson = dataframe.spatial.to_featureset().to_geojson
result, messages = target_featurelayer.append(
    upload_format='geojson',
    edits=geojson,
    upsert=False,
    return_messages=True,
    rollback=True
)

My coworker had the thought of checking for and removing all Nulls/NaNs in the dataframe, and this seems to have solved the problem. Any numeric columns with np.nan were set to 0 (which makes sense in this one use case; in the future, I may use -999 or something like that) and any text values were set to empty strings.

His dataset is now uploading as it ought to, so it may be that somewhere in the featureset/geojson/append pipeline it really doesn't like null values.