topic Re: Error using append method from ArcGIS API for Python in ArcGIS Online Questions

Error using append method from ArcGIS API for Python

JosephHilliard94 — Thu, 07 Jul 2022 13:07:19 GMT

Hi there,

I have been trying to append from one feature layer to another using the ArcGIS API for Python. I get the below error message?

Exception: Object reference not set to an instance of an object.
(Error Code: 400)

My code is below:

archive_trees.append(item_id=lyr,upload_format='featureCollection',source_table_name='Felled_Trees',upsert=False,update_geometry=True,field_mappings=[{"name" : "OBJECTID","sourceName" : "OBJECTID"},{"name" : "PROPERTY","sourceName" : "property"},{"name" : "LOCATION_DETAILS","sourceName" : "location_details"},
{"name" : "SPECIES","sourceName" : "species"},{"name" : "USAGEZONE","sourceName" : "usagezone"},{"name" : "TREEREF","sourceName" : "treeref"},
{"name" : "AGE","sourceName" : "age"},{"name" : "GIRTH","sourceName" : "girth"},{"name" : "TREETYPE","sourceName" : "treetype"},{"name" : "PROPREF","sourceName" : "propref"},
{"name" : "SITEINFORMATION","sourceName" : "siteinformation"},{"name" : "TREEINFORMATION","sourceName" : "treeeinformation"},{"name" : "TREECOMMENTS","sourceName" : "treeinformation"},
{"name" : "GLOBALID","sourceName" : "globalid"},{"name" : "CREATED_USER","sourceName" : "created_user"},{"name" : "CREATED_DATE","sourceName" : "created_date"},
{"name" : "LAST_EDITED_USER","sourceName" : "last_edited_user"},{"name" : "LAST_EDITED_DATE","sourceName" : "last_edited_date"}])

Any help would be great as this never works for me!

Joe

Re: Error using append method from ArcGIS API for Python

MCameron — Thu, 07 Jul 2022 13:33:29 GMT

Hi Joe,

Commonly, this error can occur when trying to Append data from one feature layer to another based on one of the following:

There are duplicate field values in the field used as the matching field to append the hosted feature layer.
The field values in the dataset exceed the character limit of the matching field of the hosted feature layer.
The file geodatabase appended in ArcGIS Online is not recognised.

Could you please review the following Technical Article Solutions and confirm if any of these options help to resolve the issue you are seeing? --> https://support.esri.com/en/Technical-Article/000025195.

Re: Error using append method from ArcGIS API for Python

JosephHilliard94 — Thu, 07 Jul 2022 14:59:04 GMT

Hi there,

I have checked:

The target dataset is empty so there can't be an a duplicate value in the FID.
Field values do not exceed character limit (same append works using ArcGIS Pro just not on API).
File Geodatabase has successfully created the feature layer.

Upsert is set to False as sync is enabled, can you offer any other solutions? I tried with a one feature append using the same schema and didn't work.

Thanks!

Re: Error using append method from ArcGIS API for Python

Clubdebambos — Thu, 07 Jul 2022 15:27:01 GMT

Hi @JosephHilliard94

Make sure the item_id parameter is set to an actual item id and not just pointing to an object like a layer.

Re: Error using append method from ArcGIS API for Python

JosephHilliard94 — Thu, 07 Jul 2022 15:39:25 GMT

I have tried that as well, no dice, thanks though!

Re: Error using append method from ArcGIS API for Python

MCameron — Thu, 07 Jul 2022 16:02:42 GMT

Hi Joe,

When you are calling "lyr", is this an item in your ArcGIS Online? If so, could you update item_id=lyr.id in the following line to include .id:

archive_trees.append(item_id=lyr.id,upload_format='featureCollection',source_table_name='Felled_Trees',upsert=False,update_geometry=True,field_mappings=[....

Based on the ArcGIS API for Python Append documentation this would be required. Or you could use the item id reference which can be found in the URL of the layers item description page:

Re: Error using append method from ArcGIS API for Python

Clubdebambos — Thu, 07 Jul 2022 16:23:17 GMT

I think I've only ever got the append to work with a CSV 😞

Alternatively, if the source data being appended had the same schema as the target you can convert a FeatureCollection to a FeatureSet and use FeatureLayer.edit_features(add=feature_set)

from arcgis import GIS from arcgis.features import FeatureCollection agol = GIS("home") ## Feature Collection object fc = agol.content.get("**FC_Item_ID**") fc = FeatureCollection.fromitem(fc) ## get as FeatureSet fc_set = fc.query() ## feature service containing the target layer fs = agol.content.get("**FS_Item_ID"**) ## target layer lyr = fs.layers[0] ## add in the feature set, will return a dictionary with successful/unsuccessful results print(lyr.edit_features(adds=fc_set))

I know that doesn't help you now, but might help plan for/be useful in the future.

Re: Error using append method from ArcGIS API for Python

JosephHilliard94 — Fri, 08 Jul 2022 14:10:49 GMT

Hi again,

Still not working sadly, it is possible for me to PM you my script for you to have a look?

Thanks!

Re: Error using append method from ArcGIS API for Python

JosephHilliard94 — Tue, 12 Jul 2022 15:25:39 GMT

This is what I got up to and same error message:

#Selection of Felled Trees

item = tsm_trees.query(where="TREETYPE='FEL'")

sdf = item.sdf

#Convert SDF to Feature Layer
lyr = sdf.spatial.to_featurelayer(sdf, folder='TSM')

NOTE - Tried GDB and Feature Layer here

GDB = "194etc"

geodatabase = gis.content.get(GDB)

new_input = geodatabase.id

FEATURE_LAYER = "f14etc"

fcl = gis.content.get(FEATURE_LAYER)

new_fcl = fcl.id

#Append Data Appended Feature Layer -

archive_trees.append(item_id=new_input,upload_format='filegdb',source_table_name='Felled_Trees',upsert=False)

NOTE: Upsert has to be false as sync is enabled and doesn't work when I turn it off.

A solution would be grand as this is a common workflow for us.

Re: Error using append method from ArcGIS API for Python

JakeAdamsUGRC — Thu, 06 Oct 2022 16:11:18 GMT

We were running into this problem as well when trying to append a spatially-enabled dataframe to an existing layer in a feature service. I don't know if this will help Joe, but I figured I'd put it here for anyone else who gets here from Google.

Our workflow involved getting the live data from the feature service as a dataframe, inserting new rows, truncating the existing data from the feature service, and then using .append to write the new, enlarged dataframe to the feature service using the following code. We kept hitting the "Object reference not set..." error (which, digging through the API source, is coming from the method that keeps checking the status of the operation).

geojson = dataframe.spatial.to_featureset().to_geojson result, messages = target_featurelayer.append( upload_format='geojson', edits=geojson, upsert=False, return_messages=True, rollback=True )

My coworker had the thought of checking for and removing all Nulls/NaNs in the dataframe, and this seems to have solved the problem. Any numeric columns with np.nan were set to 0 (which makes sense in this one use case; in the future, I may use -999 or something like that) and any text values were set to empty strings.

His dataset is now uploading as it ought to, so it may be that somewhere in the featureset/geojson/append pipeline it really doesn't like null values.

Re: Error using append method from ArcGIS API for Python

dns — Wed, 12 Jul 2023 16:50:47 GMT

Doesn't work for me either. Have checked every parameter and compared to how AGO does this on the backend, where it works fine. I am using the exact same parameters as a web browser would, which leads me to believe it's a bug in Esri's Python API. I hope they are actually looking at a thread like this. I can confirm I am using the actual item id string and that is not the problem.