@Jennifer_Parmeley this prompted me to finally work on a Python script to sync the v2.0 schema with the v1.0 schema. This may not be the most optimal way to do this, but here is the Python script we are now using. The script takes the v2.0 service and translates the data to match the v1.0 service. Then it compares the content of the v2.0 service to that of the v1.0 service to see if an update is needed. If the data matches the script exits, but if there is a difference then the v1.0 service is truncated and the new data is added.
For all of this to work the v1.0 service needs to have supportsApplyEditsWithGlobalIds set to True and Sync cannot be enabled on the service. The GlobalID setting is needed to copy the GlobalIDs from v2.0 to v1.0 otherwise new GlobalID are created. The GlobalIDs are needed for the comparison. Sync must be disabled or the truncate operation will fail.
I am running this on ArcGIS Enterprise using ArcGIS Notebook Server Standard, but it should work with ArcGIS Online and/or a standalone Python file running as a scheduled task with some adjustments to the code.
from arcgis.gis import GIS
from arcgis.features import GeoAccessor
import pandas as pd
import numpy as np
gis = GIS("home")
# Get the Esri road closures layer using its Item ID
item_esri_closures = gis.content.get('<InsertEsriItemIDHere>')
# Get just the closure lines layer from the Esri road closure item
layer_esri_closures = item_esri_closures.layers[1]
# Convert the feature layer to a spatial Pandas dataframe
sdf_esri = GeoAccessor.from_layer(layer_esri_closures)
# Create a column to store the Waze "type" value and default it to ROAD_CLOSED
sdf_esri['type'] = 'ROAD_CLOSED'
# Create a column to store the Waze "reference" value and default it to organization ID
sdf_esri['reference'] = '<InsertWazeOrganizationIDHere>'
# Replace the Esri "reason" values with the Waze equivalent
sdf_esri['reason'] = sdf_esri['reason'].replace({'Construction':'ROAD_CLOSURE_CONSTRUCTION', 'Event':'ROAD_CLOSURE_EVENT', 'Hazard':'ROAD_CLOSURE_HAZARD'})
# Replace the Esri "direction" values with the Waze equivalent
sdf_esri['direction'] = sdf_esri['direction'].replace({'Both Directions':'BOTH_DIRECTIONS','One Direction':'ONE_DIRECTION'})
# Rename Esri columns to match Waze column names
sdf_esri = sdf_esri.rename(columns={'reason':'subtype', 'closureid':'identifier'})
# Keep only the columns that match the Waze schema
sdf_esri = sdf_esri[['street','subtype','description','direction','altroute','pocname','pocemail',
'pocphone','url','identifier','activeincid','type','reference','globalid',
'created_user','last_edited_user','created_date','starttime','last_edited_date',
'endtime','SHAPE','SHAPE__Length','objectid']]
# Get the Waze road closure service using its item ID
item_waze_closures = gis.content.get('InsertWazeItemIDHere')
# Create a feature layer object of the closure lines layer from the Waze road closure item
layer_waze_closures = item_waze_closures.layers[0]
# Convert the feature layer to a spatial dataframe
sdf_waze = GeoAccessor.from_layer(layer_waze_closures)
# Compare just a subset of the fields from both services
# Apparently the SHAPE and SHAPE__Length are modified slightly when applying the edits
# so they will not match
compare_fields = ['street','subtype','description','direction','altroute','pocname','pocemail',
'pocphone','url','identifier','activeincid','type','reference','globalid',
'created_user','last_edited_user','created_date','starttime','last_edited_date',
'endtime']
sdf_waze_compare = sdf_waze[compare_fields]
sdf_esri_compare = sdf_esri[compare_fields]
# Make sure the null values are the same between the dataframes
sdf_waze_compare = sdf_waze_compare.replace(np.nan,'')
sdf_waze_compare = sdf_waze_compare.replace(np.nan,'')
# Make sure the dataframes are sorted the same way using the GlobalID
sdf_esri_sort = sdf_esri_compare.sort_values(by='globalid', ignore_index=True)
sdf_waze_sort = sdf_waze_compare.sort_values(by='globalid', ignore_index=True)
# Set the index of both dataframes to use the GlobalID
sdf_esri_sort = sdf_esri_sort.set_index('globalid')
sdf_waze_sort = sdf_waze_sort.set_index('globalid')
# Check if the number of rows are the same
equal_indexes = sdf_esri_sort.index.equals(sdf_waze_sort.index)
print('Row Count Equal: ' + str(equal_indexes))
# Check if the number of columns are the same
equal_columns = sdf_esri_sort.columns.equals(sdf_waze_sort.columns)
print('Column Count Equal: ' + str(equal_columns))
# Check if the data in the dataframes already match
exists = sdf_esri_sort.isin(sdf_waze_sort.to_dict(orient='list')).all(axis=1)
equal_values = False
if all(exists):
print('Data Equal: True')
equal_values = True
else:
print('Data Equal: False')
equal_values = False
# A list of the check values
checks = [equal_indexes,equal_columns,equal_values]
if all(checks):
# If all the checks are True then everything is
# exactly the same and nothing needs to be updated
print('Nothing to update')
else:
# If one of the checks is False then update the Waze service
# Convert the translated dataframe of the Esri service to a FeatureSet
featureset_esri = sdf_esri.spatial.to_featureset()
# Remove all existing rows from the Waze feature layer
layer_waze_closures.manager.truncate()
# Add the road closures from the translated Esri dataframe to the Waze service
# The supportsApplyEditsWithGlobalIds must be set to True on the Waze service
# or the edit operation will fail
layer_waze_closures.edit_features(adds=featureset_esri, use_global_ids=True)
print('Updated Waze service')
"Not all those who wander are lost" ~ Tolkien
