ArcGIS Enterprise Portal > AGOL Distributed Collaboration | Local Feature Service to Hosted Feature Layer Replication
Background
I've recently done some testing with a Distributed Collaboration between our ArcGIS Enterprise Portal (11.1, with federated Server) and AGOL site, towards the goal of having locally-hosted feature service (LFS) replicated in AGOL as a hosted feature layer (HFL). We use postgres for our enterprise geodatabase and generally everything is traditionally versioned rather than branch versioned at this time. Our Open Data hub provides the public with a variety of datasets. These are currently hosted locally on Enterprise and our AGOL has references to these services – mostly map services, technically, rather than Feature Services.
Our goal is to have the public continue consuming our public data through our Open Data site, but using an HFL on AGOL is preferable than having an AGOL reference a local map/feature service, as the HFL allows extra download options in the Open Data site (i.e. File GDB), and also alleviates potential load on our local system when users query or download these datasets (maybe not a big impact, but it's something).
Our County/major neighboring metropolitan city also streams a few of our layers directly from a LFS and uses it in their 'big' web map that serves a population of 1 million+ metro area. I'd similarly prefer that connection hit an AGOL HFL instead of our LFS.
Testing has shown the process of sharing a LFS (versioned, globalid, editor tracking, sync enabled) with the properly established Distributed Collaboration (DC) workspace/group in our Portal can replicate our LFS data to AGOL as a HFL copy. We have specified settings of the DC to establish one-way replication/sync, rather than two-way, for this purpose. It really does seem to behave like a replication service. It preserves the globalids of the source features, and the item that gets spooled out in AGOL has the same content ID as the item in Portal.
Testing shows that Insert/Update/Delete edits on the LFS side are automatically pushed to the HFL on a schedule that can be customized (although any schema changes on the LFS side would need to be updated manually on the HFL side). Basically… it works!
Some Sync Behavior Observations/Questions/Concerns
Repairing broken LFS > HFS relationship
Once the LFS is shared in the DC collaboration group on the Portal side, the HFL copy is immediately spooled up on the AGOL side.
If the HFL item on the AGOL side does not have delete protection enabled, the HFL on the AGOL will *poof* disappear immediately if the LFS is un-shared with the collaboration group for whatever reason.
If the LFS is then re-shared with the group, the HFL is re-established in AGOL, but loses any customizations that were made on the AGOL HFL side, e.g. summary, description, symbology, popup, etc. Interestingly, it does seems to retain the same AGOL REST endpoint when it gets re-established...
Question: I wonder if that AGOL REST service endpoint would eventually be removed on the backend if enough time passes without the item being re-shared?
If delete-protection is enabled on the HFL, and the LFS is unshared from the Portal side, the HFL item will persist but the sync processing log will indicate there was an error because an item was delete protected and it can’t be removed automatically on the AGOL side.
When the LFS is re-shared with the collaboration group, the sync processing log shows a similar but different error because the previous item already exists, and it indicates that the item must be removed and workspace should be re-synced to correct it. In other words, replication of data edits from LFS will no longer function with the existing HFL item, so it has to be wiped out to let AGOL recreate the HFL item.
Question: is there any way to repair this relationship or ensure it persists if -- for any reason (woopsie mistake?) -- the LFS gets unshared with the collaboration group?
If we assume there’s not a way to keep that AGOL item persistent in this case, then that means that any customization to Summary, Description, tags, symbology, popups, etc should be modified on the Portal side, and AGOL side should not be touched in order to avoid potentially losing those kinds of customizations.
This is something I will have to drill into my head early and often because I would otherwise be tempted to customize the AGOL item, since that is what is directly shared to the AGOL group that exposes the item to my Open Data hub. I did some tests just now and actually it does look like changes to summary, description, etc immediately start a sync and push those changes to the AGOL item within a few minutes.
Limited Sync Log Info?
Although there is a slightly more verbose ‘sync log' on AGOL side than Portal side of the collaboration, I wish it had it’s own menu to review these logs and history since it does not seem to provide comprehensive information about what is failing vs. what is succeeding in each sync, and only shows the issues from the last sync and no other history. In our case, we would ideally be making 40+ LFS replicated to AGOL as HFS using this method... I'm worried we would not have comprehensive understanding of what has failed if there was more than one issue at a time.
Warning/notification of errors?
There does not seem to be any way to set a notification/email to provide a heads up if there is an error. That would be a nice addition. Otherwise, I'll probably never know unless I check a specific sub-menu of the distributed collaboration workspace.
Your experiences?
I'm curious if anyone else has the same use case and has had success (or not) with using this strategy?
