Why are my hosted feature layers failing to copy in my distributed collaboration?

KevinChristy1 · ‎07-18-2019

This is my current distributed collaboration setup:

AGOL - Host, uses standard logins

Enterprise Portal 10.6.1 - Guest, uses IWA

Did not set up web-tier authentication
created groups during the collaboration process
The guest Portal is set up to only receive data from the AGOL host.
The workspace is set up to copy data.

So I have a survey created in Survey123 that feeds a hosted feature layer in AGOL. I ensured that sync was enabled on that hosted feature layer. I shared said hosted feature layer with the workspace group. The first sync takes place and the hosted feature layer is copied over to the guest Portal like a dream. I can fully access and interact with the layer. The layer is still set to allow sync.

Everything is wonderful, then I run a test.

On the guest Portal side, I add the copied layer to a web map and enable the refresh interval. I then go back to my host AGOL environment and fill out another survey to add a record. The hosted feature layer on the AGOL side is immediately updated without issue. I then go back into the guest Portal side, and schedule the sync for the workspaces to occur within the next two mins or so. I am simply testing to see if the feature layer updates properly at sync...

It does not.

I get the following screen message:

Clicking it reveals...

And to top it all off I get a notification on my AGOL site that "The hosted feature layer could not be sent as a copy, so it was sent as a reference instead."

Now, my nice original hosted feature layer copy seems to have lost some of its functionality and has not updated.

It is my understanding from everything I could find and read that when performing a collaboration between AGOL and Enterprise 10.6.1 that hosted feature layers can be copied.

Can anybody tell me why this could be occurring?

Anonymous User · ‎10-12-2020

Hi Ingrid,

This technical article covers the steps to ensure SSL certificates are trusted across environments: Unable to create collaboration between ArcGIS Enterprise portals due to SSL certificate error.

Looking at the error you received, it appears something may have happened with the copy of the data. Can you unshare and reshare the layer to the collaboration group (ensure it is sync enabled)? This well re-trigger the sync. If you are seeing issues after trying that, I'd recommend reaching out to Esri support who can help further troubleshoot.

GregCarlino2 · ‎08-10-2022

@IngridMans

I'm having the same issue with enterprise-referenced feature services being downgraded to reference after 'copy' fails. Did you ever find a solution to this?

Thanks,

Greg

Carl_Flint · ‎01-29-2021

Good morning all,

Had a similar issue with a different resolution. When troubleshooting and looking at the portal/sharing/rest api for the syncStatus and sync details I found that in a 1-way sync we had roughly half of the feature services that were replicated would fail to sync. So clearly it shouldn't be an issue with SSL's.

Did a deep dive into the REST API docs (didn't help provide any resolution and seems to be out of date as it appears to be missing some methods like "Import Replication Package" and "Export Replication Package"). After deciding to peer behind the curtain at the JSON versus the HTML I stumbled upon a undocumented limitation to replication for the collaboration workspaces.

"config": {
    "maxItemSizeInMB": 1024,
    "maxReplicationPackageSizeInMB": 5120,
    "copyByRefIfCopyFail": true,
    "bidirectionalSyncCapable": true
  },

There is a 1GB per item and 5GB per replication package (sqllite db export from our portal to AGOL) limit that I can find no documentation on. I instantly knew this was the issue because all of the feature services initially published as copies from portal to AGOL.

The solution was to create smaller groups of Collaboration workspaces. Initially I had created 3 in total with one for each transport type (Push,Pull,Two-Way). Now modeling the workspaces more closely with our feature dataset schema, the scheduled sync's all appear to be succeeding without errors. Just a heads up for anyone troubleshooting with similar issues as OP, SSL isn't always the problem.

Carl Flint, GISP

SolomonPulapkura2 · ‎02-20-2021

@Carl_Flint can you elaborate on your solution? I am having the same issue. What do you mean by "create smaller groups of Collaboration workspaces" and "modeling the workspaces more closely with our feature dataset schema".

Do you mean creating a workspace for only one feature layer or grouping fewer feature layers to be synced in each workspace?

What if you have one feature layer that is more than 5GB in size? Which is what I am trying to do.

Carl_Flint · ‎02-22-2021

@SolomonPulapkura2 to your first question yes. Instead of syncing 75 feature layers in one Collaboration workspace try ramping down to only 20 and see if that is more manageable. Have you checked your logs to see how long it takes to perform a scheduled sync of the entire Collaboration workspace?

I can't exactly answer the best practice for that particular use case (on feature layer over 5 GB), but perhaps creating smaller table views of the very large feature layer would be best.

Carl Flint, GISP

SolomonPulapkura2 · ‎02-22-2021

Thank you @Carl_Flint . Something to try for sure.

Can you point me to where you see -

"maxItemSizeInMB": 1024,
"maxReplicationPackageSizeInMB": 5120,

Are these adjustable?

Thanks!

LakshmiKanth · ‎09-13-2022

https://<portalurl>/<webadaptor>/sharing/rest/portals/self
Navigate to "collaborations" and select the workspace and then updateInfo

The browser url looks like below:
https://<portalurl>/<webadaptor>/sharing/rest/portals/0123456789ABCDEF/collaborations/8a482df51ae04c1f99e0c8d45efeac63/workspaces/fc8f515a83db440388edcd22fb1e468d/updateInfo

Documentation is here:
https://<portalurl>/<webadaptor>/portalhelp/apidocs/rest/index.html?root.html#/workspaceID_updateInfo/02t60000008p000000/

SolomonPulapkura2 · ‎02-23-2021

https://www.esri.com/arcgis-blog/products/arcgis-enterprise/sharing-collaboration/administering-your...

Thanks @MattTwietmeyer

Mondi_GIS · ‎05-12-2021

Hi,

We are having similar issues, but only our one larger dataset seems to fail. It is around 400mb, so not exceeding the 1gb limit. Every time we manage to push the file from ArcGIS Enterprise up to ArcGIS online for the first load, but when we want to sync, it fails with the error:

"java.net.SocketException: Connection reset by peer (Write failed)" in the collaboration sync status logs.

The ports are open, but when I perform a Tracert to www.esri.com, it succeeds but with a number of requests timedout. Could this be causing the issue? Our smaller files seem to sync fine.

ahargreaves_FW · ‎01-06-2022

Hello,

I'm facing a similar issue. Here's my setup:

The Feature Service we need to participate in the collaboration is published as part of Survey123. The collaboration is set to 'copies' only. AGOL is the host, our internal 10.6.1 portal is the guest with send and receive permissions.
If the feature service is initially published to our internal portal it is hosted on our portal hosting server (https://portalhostingserver.ourdomain.org/portal/rest/xxxxxxx). It successfully replicates itself over to AGOL, where it is an AGOL hosted service (https://services.arcgis.com/rest/xxxxxxx).
- This is our preferred architecture to ensure a true air-gap between the public facing AGOL feature service and our internal portal feature service.
- However under this architecture any edits or new records to the AGOL replica DO NOT COPY BACK to the portal replica. No errors are reported by the portal. In fact it shows the “sync workspace” task as “succeeded”.
If the feature service is initially published to AGOL it is hosted within AGOL as a hosted service (https://services.arcgis.com/rest/xxxxxxx). It successfully replicates any edits or new records back and forth to our internal portal.
- However under this architecture the portal feature service is only a referenced service, still referencing the exact same URL as within AGOL of https://services.arcgis.com/rest/xxxxxxx.
- Our desire is to not have a publicly accessible URL referenced inside of our internal portal.

Any advice on how to get this to work as described in the doc?