Append selected records and calculate value for one field

[AmyRoust]

I'm new to using the ArcGIS API for Python in an ArcGIS Online Notebook and am looking for help to identify the right function(s) for a data maintenance script I'm migrating from a scheduled task on a server.

The start of the script works fine: I select a small number of records from a hosted feature layer. (If it matters, the hosted feature layer is part of a Survey123 survey).

I'm struggling with finding the function(s) to:

1. Append those selected records to another hosted feature layer, and

2. On those newly appended records, update the value in the status field to 'Not Reviewed'. The status field does not exist in the source layer, but it does in the target layer. 

These two steps don't have to be in this order. I'm fine with exporting the records, adding a status field, calculating that 'Not Reviewed' value and then appending the updated records to my target layer.

Appreciate any hints that the community can offer!

[JakeSkinner]

Hi @AmyRoust,

You could do something like below:

from arcgis.gis import GIS

# Connect to your GIS
gis = GIS("https://www.arcgis.com", "username", "password")

# Access source feature service
item = gis.content.get("source item id")
feature_layer = item.layers[0]

# Define your query
query = "POPULATION > 10000"

# Query the feature layer and create a spatial dataframe
sdf = feature_layer.query(where=query).sdf

# Calculate the status field
sdf["status field name"] = sdf.calculate("'Not Reviewed'", inplace=True)

# Access the hosted feature service to update
item = gis.content.get("target item id")
feature_layer = item.layers[0]

# Append the data from the spatial dataframe to the hosted feature service
added_features = feature_layer.edit_features(adds=sdf.to_featureset())

[Clubdebambos]

Hi @JakeSkinner,

Just an observation... in line 14 you convert the feature set (returned from the query) to a sdf, and then in line 21 you are converting the sdf back to a feature set with no operations on the sdf in-between.

Would the operation in-between be to add the "status" field to the sdf, populate the status attributes to Not Reviewed and then append into the target? (omitting the calculate method on line 24)

@AmyRoust do you also need to account for attachments? 

Cheers,

Glen

[JakeSkinner]

Yes, you are correct @Clubdebambos, thanks for pointing that out.  I went ahead and updated my previous reply.

[AmyRoust]

I don't have any attachments, @Clubdebambos.

Is there a way to do field mapping with the edit_features function? Some of the field names in the input feature layer don't match the target layer.

[JakeSkinner]

@AmyRoust try the following:

# Define the field mapping between the source and target dataframes
field_mapping = {'ID': 'ID_new', 'Name': 'Name_new', 'Value': 'Value_new'}

# Append data with field mapping
feature_layer.edit_features(adds=sdf.to_featureset(), field_mapping=field_mapping)

[AmyRoust]

I still have something wrong, but I don't know how to print error messages because arcpy.GetMessages() doesn't return anything. Is there another way to get the error messages from failed functions?

Here's the portion of code that is failing:

field_mapping = {'ObservationDate':'ReportObservationDate',
                         'CampsiteDescription':'ReportCampsiteDescription',
                         'FollowUpRequest':'ReportFollowUpRequest',
                         'SubmitterName':'ReportSubmitterName',
                         'PreferredContact':'ReportSubmitterPreferredContact',
                         'EmailAddress':'ReportSubmitterEmailAddress',
                         'PhoneNumber':'ReportSubmitterPhoneNumber',
                         'submission_date_and_time':'ReportDate'
                        }
        sdf = feature_layer.query(where=query).sdf
        feature_layer.edit_features(adds = sdf.to_featureset(), field_mapping=field_mapping)

I think line 10 is working because I added a print statement for the sdf variable and got this (confidential information blurred in the screenshot):

[AmyRoust]

Just for clarification, I do have the variable feature_layer defined earlier in the script and it's working properly.

item = gis.content.get("999999999") # actual item id has been removed for this chat
feature_layer = item.layers[0]

[JakeSkinner]

Try the following:

try:
    feature_layer.edit_features(adds = sdf.to_featureset(), field_mapping=field_mapping)
except Exception as e:
    print(e)
    arcpy.AddError(str(e))

[AmyRoust]

Thank you! That worked for error trapping. I fixed the issue, but field mapping doesn't work.

    if len(selected_features.features)>0:
        field_mapping = {'ObservationDate':'ReportObservationDate',
                         'CampsiteDescription':'ReportCampsiteDescription',
                         'FollowUpRequest':'ReportFollowUpRequest',
                         'SubmitterName':'ReportSubmitterName',
                         'PreferredContact':'ReportSubmitterPreferredContact',
                         'EmailAddress':'ReportSubmitterEmailAddress',
                         'PhoneNumber':'ReportSubmitterPhoneNumber',
                         'submission_date_and_time':'ReportDate'
                        }
        sdf = feature_layer.query(where=query)
        feature_layer.edit_features(adds = sdf, field_mapping=field_mapping)

Line 12 generates the error:

FeatureLayer.edit_features() got an unexpected keyword argument 'field_mapping'
ERROR: FeatureLayer.edit_features() got an unexpected keyword argument 'field_mapping'

I tried using field_mappings (plural) since that's listed in the FeatureLayer append method as a keyword, but no luck.

So then I thought I'd try the append method, but I'm getting Unknown Error (Error Code: 500) in response to this:

    if len(selected_features.features)>0:
        field_mapping = [{'ObservationDate':'ReportObservationDate',
                         'CampsiteDescription':'ReportCampsiteDescription',
                         'FollowUpRequest':'ReportFollowUpRequest',
                         'SubmitterName':'ReportSubmitterName',
                         'PreferredContact':'ReportSubmitterPreferredContact',
                         'EmailAddress':'ReportSubmitterEmailAddress',
                         'PhoneNumber':'ReportSubmitterPhoneNumber',
                         'submission_date_and_time':'ReportDate'
                        }]
        sdf = feature_layer.query(where=query)
        feature_layer.append(sdf,
                            field_mappings = field_mapping,
                            return_messages = True)

My guess is that the error code is the result of not specifying an item_id, since sdf is a selection of records and not an entire item.

Any ideas on what to try next?

I wish the ArcGIS API for Python documentation consistently provided sample code like the arcpy documentation does.