nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Distributed Collaboration Sync Error

KevinPiraino2

I am looking for some guidance or confirmation on an error I have been encountering when attempting to sync a workspace that participates in a distributed collaboration between our Orgs. Portal and AGOL sites.

Basically, we are attempting to share Portal feature services (where the data is hosted in our MSSQL Server) to AGOL as hosted feature services; the AGOL hosted feature service would be a "copy" of the Portal feature service.

Some additional background:

The feature service / data we are attempting to share is originates from a Parcel Fabric enabled database.
Our MSSQL server has two instance, a production (editing) and publication instance. Every night our production instance is replicated/copied and overwrites our publication instance.
The feature services I am trying to share to the collaboration are built using data from the publication instance.

When we try to sync the workspace immediately after the feature service is published and shared (same day), the sync completes successfully. When we try to sync the workspace the next day, the sync fails. I am wondering if the nightly replicate / copy routine, where the production instance of the database is replicated / copied and overwrites the publication instance of the database, is breaking/blowing away the "replica" that is created by the collaboration/sync process. Thus when the workspace sync is executed the same day the feature service is published/added to the workspace, the workspace has no issue syncing the content, but when the sync is executed the following day (or any day after) the sync fails due to no reference of the "replica" in the database.

The specific error I am receiving from the Portal Admin logs is: "Failed to obtain sync/replica info for Feature Service item 'ITEMID', URL 'PORTALURL/arcgis/rest/services/FeatureServiceName/FeatureServer' while collecting item list for collaboration workspace id 'WORKSPACEID'. Item exists but will not be synchronized."

Is there a way to confirm that a replica has been created at the database level (in SQL Server)? Is there a way to confirm my theory about the replica being broken/blown away by our nightly SQL replicate/copy routine?

Infrastructure Configuration:

Using ArcGIS Enterprise 11.3
Enterprise Geodatabase version 11.3.0.52636
SQL Server version 2022

Collaboration & Workspace Configuration:

Distributed Collaboration with AGOL site as HOST and Portal site as GUEST.
Workspace set as SEND ONLY (from Portal to AGOL).
Workspace set with "Feature layers and views in my portal are sent as 'COPIES'".
1. "If unable to share as copies share as references" is UNCHECKED.

Portal Feature Service Configuration:

Geodatabase workspace set to "Branch Version".
Version Management UNCHECKED.
Editing "Enabled" (Add, Delete, Update).
Enabled Sync CHECKED.
Sync - Version Creation --> NONE.

Find more posts tagged with

Administration

Sharing and collaboration

Data and services

Accepted answers

All comments

dcoley

Ok Kevin, like I said I

". . . extract our MSSQL features (including some UN network features) to file gdb. Those layers are shared to portal using the ArcPy sharing module from Pro. Each week I then overwrite the service definition file, un-pause the sync and then re-sync the layers. "

All I am doing here is using ArcPy to ExportFeatures from SDE to a file gdb. If the layer exists in the file gdb, it is deleted first. I then use the file gdb sourced layers to publish either a hosted feature layer or a feature layer collection from Pro to Portal and then share the layers with the Open Data groups.

Once the hosted feature layer is in place in portal, I use the arcpy sharing module to overwrite the service definition file as a 3-step process: create sd draft, stage sd and overwrite sd:

def stageSDDraft(mp,lyrName,summary,lyrList,descript,folder,tags,sdfolder): # mp,lyrName,summary,lyrs,descript,folder,tags,sdfolder
sddraft_out = "C:/ArcProProjects3x/PortalUpdates/" + sdfolder + "/" + lyrName + ".sddraft"
draft = mp.getWebLayerSharingDraft("HOSTING_SERVER", "FEATURE", lyrName, lyrList) # if don't include lyrList arg here the draft will include all map layers.
draft.portalFolder = folder
draft.tags = tags
##draft.allowExporting = True
draft.summary = summary
draft.description = descript
draft.overwriteExistingService = True
draft.exportToSDDraft(sddraft_out)
return

try:
portal = arcpy.SignInToPortal("https://yourorg/portal", "username", "password")
aprx = arcpy.mp.ArcGISProject('C:/ArcProProjects3x/PortalUpdates/PortalUpdates.aprx')
for m in aprx.listMaps("Land Management"):
print("Map: " + m.name)
for lyr in m.listLayers():
if lyr.name == 'wdFireHydrant':
lyrList = []
lyrList.append(m.listLayers(lyr.name)[0])
print(lyr.name)
stageSDDraft(m, lyr.name, "Points representing Fire Hydrants.",lyrList,
"Points representing Fire Hydrants.","LandManageLayers","Utilities,Water Device","ParksRec")
print(lyr.name + " Draft Created")
arcpy.StageService_server('C:/ArcProProjects3x/PortalUpdates/ParksRec/' + lyr.name + '.sddraft', 'C:/ArcProProjects3x/PortalUpdates/ParksRec/' + lyr.name + '.sd')
print(lyr.name + " Service Staged")
arcpy.UploadServiceDefinition_server('C:/ArcProProjects3x/PortalUpdates/ParksRec/' + lyr.name + '.sd', 'My Hosted Services', "","","","","","OVERRIDE_DEFINITION","SHARE_ONLINE","PRIVATE",
"NO_SHARE_ORGANIZATION","Land Management Group")
print(lyr.name + " Service Uploaded")
message = message + "\n" + "1. " + (lyr.name) + " Service Uploaded"

That's it. You can loop through as many layers as you need per map by adding more if layer.name = "AnotherLayer" if blocks....

KevinPiraino2

David - This is an interesting workflow. Would you mind going into more detail about this workflow?

The basics of what I am trying to achieve is send updates that are made to certain datasets internally (cadastral data) to an AGOL hosted feature layer regularly via the distributed collaboration workspace. I would like to avoid sharing services from the production SQL server into the workspace and instead share services from the publication SQL server, but this may not be feasible based on your comments.

Basically, I am trying to leverage AGOL hosting certain datasets that are in high demand to reduce load on our standalone ArcGIS Server site that would normally host this data.

dcoley

Hello @KevinPiraino2 - anytime you are trying to work with an 'As-Copies' distributed collaboration workspace model, the source feature cannot be overwritten because the workspace ID is changed. As-Copies distributed collaboration workflows expect that the source feature on your guest portal will remain in place, and not be overwritten in any way, because as 'As Copies' distribution thinks that you are going to be making changes, edits etc directly to the portal hosted feature layer.

It doesn't matter if the source feature is a portal-hosted feature layer or an RDBMS hosted feature service. I have some 13 distributed collaboration workspaces setup for portal hosted feature layers to sync across to our AGOL org. Our workflow extracts our MSSQL features (including some UN network features) to file gdb. Those layers are shared to portal using the ArcPy sharing module from Pro. Each week I then overwrite the service definition file, un-pause the sync and then re-sync the layers.

This only works because our collaborations are all set up as 'By Reference'.

Hope this helps

KevinPiraino2

Timo,

The only other thing I will add regarding this is that we do not currently have "replica tracking" enabled on the feature datasets/feature classes we are trying to share into the workspace. While I'm not entirely sure that this is the reason for the issue, it is another configuration option I need to look into. Upon initial testing, it didn't appear that not having "replica tracking" enabled mattered, but reading into the ESRI documentation a bit further makes it sound like it may be necessary.

I also found a different ESRI Community post discussing what happens behind the scenes when "replica tracking" is enabled.

Replica Tracking GP tool: https://pro.arcgis.com/en/pro-app/3.4/tool-reference/data-management/enable-replica-tracking.htm

Replica Tracking behind the scenes info: https://community.esri.com/t5/arcgis-pro-questions/enable-replica-tracking/td-p/1382804

TimoT

Unfortunately, not too much more I can add. Looking back at your original post, your theory certainly sounds plausible at this stage.

I hope someone else can chime in and provide you with further insight or some sort of SQL query to confirm the replica state at geodatabase level.

KevinPiraino2

Timo,

A couple answers to your questions and comments:

I believe your Version Creation setting should be set to “Create a version for each downloaded map.”
- I have tested this and I do not believe the configuration is necessary even for a branch versioned database. I was able to successfully publish, share to the collaboration (with AGOL creating the hosted "copy" layer), make an edit in the Portal layer, and sync the workspace successfully.

With your initial successful syncs, are they only for newly published feature services that haven’t been shared to the collaboration workspace?
1. Yes, I did not test this workflow using another already published feature service. I did not want to use the production Parcel Fabric feature service to test this against and we do not have any other feature services in our publication side to test against.
What happens if you sync the feature layer twice on the same day, before the nightly replicate/copy process runs? If your theory is correct, the sync should work both times since the replica would still be in place. Ensure there is a change in the feature layer before doing the re-sync test.
1. If I sync the feature service twice on the same day, it works fine and is able to successfully sync. Even when performing an edit on the Portal side, the edit does come through on the AGOL side (as long as its within the same day).
What error messages appear in the sync status collaboration logs? You can find these at /portal/sharing/rest endpoint > Home > Portals > Self > Collaborations > [COLLABORATION-ID] > Workspaces > [WORKSPACE-ID] > Sync Status
1. The message from the "Sync Status" page:
```
Unable to sync item '46bb45cb5a0f4086b40bf2e0e4de77e9'. Error retrieving Sync info or replicas for peer 'neJvtQ4PXvnQ86MJ'.
```
Cross checking your sync status logs, are your initial successful syncs real-time or scheduled syncs?
1. The interesting thing about the real-time vs. scheduled sync, is that even on an initial sync, the workspace tries to perform both the real-time and scheduled sync. I'm not sure if that's intended or if I just set up the workspace incorrectly, but on the initial sync it will attempt both and one will inevitably fail (because its trying to create two AGOL layers with the same ID). After the initial sync though, its fine and the scheduled sync that is performed on the same day succeeds, but any subsequent syncs on following days fails.

TimoT

Hi @KevinPiraino2,

It’s been a while since I've worked with branch-versioned data, especially in a distributed collaboration context, but I can offer a few thoughts.

I believe your Version Creation setting should be set to “Create a version for each downloaded map,” even though your workspace is set to Send Only. I believe this setting is required when working with versioned data in distributed collaborations, but I’m not 100% certain. It would be great if someone else could chime in and confirm or provide additional insight. Although your scenario doesn’t exactly match the one in the document below, this document provides some useful guidance on preparing versioned data for distributed collaborations:

Prepare feature layer data for use in a distributed collaboration—ArcGIS Pro | Documentation

If you're trying to confirm your theory about the replica being broken by the nightly replicate/copy routine, here are a few things you could try or gather further information on:

With your initial successful syncs, are they only for newly published feature services that haven’t been shared to the collaboration workspace?
What happens if you sync the feature layer twice on the same day, before the nightly replicate/copy process runs? If your theory is correct, the sync should work both times since the replica would still be in place. Ensure there is a change in the feature layer before doing the re-sync test.
What error messages appear in the sync status collaboration logs? You can find these at /portal/sharing/rest endpoint > Home > Portals > Self > Collaborations > [COLLABORATION-ID] > Workspaces > [WORKSPACE-ID] > Sync Status
Cross checking your sync status logs, are your initial successful syncs real-time or scheduled syncs?

Regards,

Timo

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.