Copy features from SDE to AGOL hosted feature layer

[dstrigl]

Dear Community,

I want to copy the features from a SDE to a hosted feature layer on AGOL.

My sample so far looks like:

import arcgis
import arcpy
from arcgis import GIS
from arcgis.features import FeatureLayer, FeatureSet

fc = r"C:\Sample.sde\SAMPLE.POLY_SAMPLE"
fields = [field.name for field in arcpy.ListFields(fc)]
fields[fields.index("SHAPE")] = "SHAPE@"

user = "???"
password = "???"
item_id = "???"

gis = GIS(r"https://???.maps.arcgis.com/", user, password)
data_id = gis.content.get(item_id)
print(data_id)
layer = FeatureLayer.fromitem(item=data_id, layer_id=0)
print(layer)
layer.manager.truncate()

feature_set = arcpy.FeatureSet()
feature_set.load(fc)
print(feature_set.JSON)

result = layer.edit_features(adds=???)
print(result)

First of all I delete all features from the hosted feature layer on AGOL.

Next I want to add the features from the SDE feature layer to the hosted feature layer ...

My problem is, that I couldn't find a real working sample for this.

Does anyone have a working sample how to do it?

Regards,

Daniel.

[RhettZufelt]

I'm sure there are other ways, but using arcpy append is working for me.

arcpy.management.Append(r"\\pathtosde\signs.sde\Posts", "https://urltofeatureservice/SSA_Map_Data/FeatureServer/1", "TEST", None, '', '')

R_

[dstrigl]

Thanks for your fast response, but I can't use the URL as the target (second parameter), because the hosted feature layer on ArcGIS Online is only available after login (see sample above).

I tried to pass the "layer" from here

layer = FeatureLayer.fromitem(item=data_id, layer_id=0)

 but this failed 😞

Regards,

Daniel.

[RhettZufelt]

But you are logging in on line 14.

This works for me, though I pass it the item id of my hosted feature layer.

It will truncate the layer, then append the data from the SDE feature class to the newly truncated hosted feature layer ( I use the "TEST" option as my fields/schema match):

import arcpy
from arcgis.gis import GIS

fc = r"C:\Sample.sde\SAMPLE.POLY_SAMPLE"  # FC holding data to append to FeatureLayer

gis = GIS(url='https://????.maps.arcgis.com/', username='AdminUser', password='UserPass') #sign in to AGOL

feature_layer = gis.content.get('63644981afa0438e97sgd68fs79gs19ec')  # Establish the working dataset by itemid

FeatLay = feature_layer.layers[0] # Grab the first layer in the dataset
FeatLayUrl = FeatLay.url # grab URL as arcpy append uses string value

FeatLay.manager.truncate()

arcpy.management.Append(fc, FeatLayUrl, "TEST", None, '', '')

R_

[dstrigl]

Thanks for your hint, I will try it ...

[MikeVolz]

Rhett:

I used your code to get FeatLayUrl, but I then get the following error message when appending to AGOL even though I have already connected to AGOL to truncate the records in the hosted feature layer.

Traceback (most recent call last):
File "C:\GIS\AGOL_Update.py", line 40, in <module>
arcpy.management.Append(fc, FeatLayUrl, "TEST", None, '', '')
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\management.py", line 7545, in Append
raise e
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\management.py", line 7542, in Append
retval = convertArcObjectToPythonObject(gp.Append_management(*gp_fixargs((inputs, target, schema_type, field_mapping, subtype, expression, match_fields, update_geometry), True)))
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 512, in <lambda>
return lambda *args: val(*gp_fixargs(args, True))
arcgisscripting.ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000732: Target Dataset: Dataset https://server URL/FeatureServer/0 does not exist or is not supported
Failed to execute (Append).

Any idea why the code would be throwing this error?

From further testing, it looks like this is caused by running the python script on a computer where ArcGIS Pro had not been logged into AGOL and editing had not been enabled on the hosted feature layer.

When editing did get enabled, the python based Append process was very slow (several days for a million records).

When I logged into AGOL in ArcGIS Pro and closed out of ArcGIS Pro, the python based Append process was relatively fast (15-18 minutes for a million records).

I do not understand why I needed to log into ArcGIS Pro in order to get much better speed as the python script requires AGOL credentials itself to access the hosted feature layer.