topic Re: Truncated and appended data to existing feature layer and features disappeared?! in ArcGIS API for Python Questions

Truncated and appended data to existing feature layer and features disappeared?!

maya_fromstein — Wed, 29 May 2024 19:30:36 GMT

Hi,

I have a single feature layer that I've symbolized in a few different webmaps that all feed into an experience builder. In order to not lose the symbolization when updating the data, I understood (?) that I would need to truncate and append new data to the original feature layer. I tried this many times with this code and this data (but a backup) and it has repeatedly worked.

However!! I just tried to update the original feature layer now (the one that comprises my whole experience builder) after several test runs on backup layers, and all the data has transferred except all the features are invisible. It doesn't seem to help if I change the field being symbolized (which sometimes has 'refreshed' things i guess and made it work). I am really hoping I didn't just throw out a ton of time down the drain by losing all these products! Is anyone able to help?

Here is the code (after I have pulled the original and new feature layer items from my portal):

## flc = the original layer in feature layer collection format
## fLyr is the original layer in feature layer (item.layers[0])
print("Truncating Feature Service")
fLyr.manager.truncate()
print("target layer has " + str(len(fLyr.query()))+ " clients")
################################
##### UPDATING THE SCHEMA #####
################################
# these fields are in the original data layer I want to update
featureServiceFields = {}
for field in fLyr.manager.properties.fields:
if field.type != "esriFieldTypeOID" and "Shape_" not in field.name and 'GlobalID' not in field.name:
featureServiceFields[field.name] = field.type
# these are the fields in the new/updated data layer
featureClassFields = {}
for field in newFC.manager.properties.fields:
if field.type != "esriFieldTypeOID" and "Shape_" not in field.name and 'GlobalID' not in field.name:
featureClassFields[field.name] = field.type
## identifying which fields are missing from which layers and need updating in the original layer
minusSchemaDiff = set(featureServiceFields) - set(featureClassFields)
addSchemaDiff = set(featureClassFields) - set(featureServiceFields)
## deleting fields
update_dict={}
if len(minusSchemaDiff) > 0:
print("Deleting removed fields")
for key in minusSchemaDiff:
print(f"\tDeleting field {key}")
remove_field = {
"name": key,
"type": featureServiceFields[key]
}
update_dict = {'fields': [remove_field]}
print(update_dict)
fLyr.manager.delete_from_definition(update_dict)
print("Finished removing fields")
## adding new fields
if len(addSchemaDiff) > 0:
print("Adding additional fields")
for key in addSchemaDiff:
print(f"\tAdding field {key}")
new_field = {
"name": key,
"type": featureClassFields[key]
}
update_dict = {'fields': [new_field]}
print(update_dict)
fLyr.manager.add_to_definition(update_dict)
print("Finished adding additional fields")
#####
####################################
##### APPENDING THE NEW DATA #####
####################################
print("Adding data to the target layer now")
# shp_item is the shapefile format of the new data layer
fLyr.append(item_id=shp_item.id, upload_format="shapefile", upsert=False, field_mappings=[])
print("target layer now has " + str(len(fLyr.query()))+ " clients")

Here is what the feature layer in one of the webmaps looks now:

And here is the data table in the original feature layer, showing that the data did infact change and so did the necessary field names:

Thank you for your help!! Much appreciated!!

Re: Truncated and appended data to existing feature layer and features disappeared?!

EarlMedina — Wed, 29 May 2024 19:47:49 GMT

Did you view the layer's data? It sounds like the Shape values may be missing.

Re: Truncated and appended data to existing feature layer and features disappeared?!

maya_fromstein — Thu, 30 May 2024 03:38:03 GMT

Hm, that's so odd that they are missing but you're right. Is there a reason the shape values wouldn't be saved in the geojson? I exported it from the feature layer's page (before I messed it up and there still were shape values) Is there anything I need to do or can do to add them back?

Re: Truncated and appended data to existing feature layer and features disappeared?!

EarlMedina — Thu, 30 May 2024 13:23:14 GMT

Ah, so you are appending from a geojson file then? My guess is there may be an issue parsing the geometry data. Is the source data only available in geojson?

Re: Truncated and appended data to existing feature layer and features disappeared?!

maya_fromstein — Tue, 04 Jun 2024 21:22:46 GMT

Sorry, I was really struggling with this and worked my way in circles.

I wasn't updating from a geojson, I had just tried to make a copy of my target data layer with one and forgot how it was involved. Sorry about that. I was updating it from another hosted feature layer using the code I listed in the question. For whatever reason, it works perfectly for point layers but the shape columns get dropped when the data is polygons. I hacked a workaround by just adding a new visible layer on top, but I feel like the more hacks there are the more room for error as well. For now it is working at least.

I would really love to understand how to append polygon features and not just point features.

Thanks.

Re: Truncated and appended data to existing feature layer and features disappeared?!

maya_fromstein — Wed, 05 Jun 2024 02:28:18 GMT

I think I've understood that I can't update the geometry after truncating the whole feature layer because the polygons are so huge, my server times out before it can update each feature's geometry again.

It is also the issue if I convert to feature set and use the .edit_features() route.

Is there a way to:

Selectively truncate/clear data, without removing the geometry properties or the join fields
Update the cleared fields with new data

Every time I think I've figured out a way of playing with the feature.attribute dictionaries, I still have to update the features, including their geometries, and time out again.

Thank you!