Collaboration with Federated Feature Layer

[Anonymous User]

We have a portal site with a federated ArcGIS server. We will soon create an ArcGIS Online site in order to allow users who do not have accounts on the portal to view some of our map services.

We need to decide the best way to share the data and ensure it remains up to date on both sites. Editing of data will only happen on the portal side, so synchronization will only be one way (for now).

Collaboration looks promising, but there are a couple of details I need to nail down before I can report whether or not this is a viable option.

• What I think will work is to create a synchronized collaboration that pulls over copies of the data onto the ArcGIS Online site, because those who do not have an account on the portal will not be able to access the actual data referenced by the service. Is this correct?
• One thing that confuses me a bit is terminology. It sounds like what the portal would be sharing with the ArcGIS online host is a federated feature layer. The services (layers?) we would share are published on a federated server. Is this correct? If so, does "feature" in "feature layer" refer to a feature-enabled layer as opposed to a map image layer? I would much rather work with map image layers for now.

Thank you for any information you may be able to provide

Randy McGregor

[Anonymous User]

Hey Randy!

So I think your two questions are essentially asking:

1. When you collaborate content, where does the underlying data go?
2. What type of content can be collaborated?

I'll answer with an annoying level of detail, as my coworkers have come to expect

Question 1: 

Short Answer: You can choose whether to make copies of your data that get hosted in AGOL or to add references to your portal services via AGOL.

So when you establish a one-way collaboration between AGOL (host) and Portal for ArcGIS (guest), there're a few moving parts. Firstly, I'd confirm your deployment and data meet the requirements (Create a collaboration—ArcGIS Online Help | ArcGIS , http://enterprise.arcgis.com/en/portal/latest/administer/windows/common-questions-for-distributed-co...). Then you have options when configuring your collaboration: you can either choose the option to share content as copies or references. 

If you choose the option to share content as copies, the following happens:

1. The Hosting GIS Server creates the initial replica (i.e. creates a copy of the dataset behind the service being collaborated);
2. The Portal machine then runs the 'sync' jobs to identify updates and pass them at an interval (fastest interval is one-hour);
3. If the Portal machine's sync jobs find differences, the Portal sends the delta tables to AGOL as updates to the hosted dataset.

When collaborated content is copied to AGOL org, a hosted item is created within your AGOL org. This means you can then dictate how your publicly accessible content is shared (separate from how it is shared / accessible in your Portal).

If you choose the option to share content as references, then an item is created in your AGOL org that is just a redirect to your internal service. So, if your Portal item is internal only or secured (requires a login), then these will also be enforced in the collaborated content in AGOL.

Question 2:

Short Answer: Feature layers (feature services in AGS) are the only service-type that can currently be collaborated. As I've been informed, there are no plans for AGOL to support GP services (so these won't be collaborated), no immediate plans for map services, or locator services. 

Given that collaboration is basically versioning and replication, using this methodology for cached services or map services has some complications. 

[DavidColey]

Hi Andrew -  do you know if Image Services can participate in a collaboration?

[Anonymous User]

As of this point, no, image services cannot participate in collaboration.

[DavidColey]

Ok thanks for the reply Andrew.  I could not find that info anywhere and I've been trying for like 2 days.

[DavidColey]

An update:  Image services from portal federated servers can participate in collaborations.  I just had to change the workspace sync settings to 'As Copies (if possible)' instead of 'As References (Always)':

[Anonymous User]

Share content with collaboration groups—Portal for ArcGIS (10.6) | ArcGIS Enterprise 

I apologize, I thought you were looking to share the content as copies, not just as references. Yes, you can collaborate image services (Imagery Layers) as references, but this means that the content, if internal only, will remain accessible internally only. 

[Anonymous User]

I'm guessing that David's and my questions have perhaps gotten mixed together. I (Randy) had asked about collaborating as copies, because I thought that that was the only way I could make the data available to those who do not have access to the Portal (but do have access to the AGOL site we want to collaborate with.)

I have gotten both "Yes" and "No" in answer to this question - from Tech Support I got both, but "Yes" was the last answer, after I said, "Really?"

It appears as though I can in fact share the data if I collaborate with the "as copies" option. Also, I can share image services in addition to feature services.

We don't have an AGOL site set up yet, so I am trying to answer this question before I am able to test directly, and that is making it harder to get a solid answer.

Thanks for your time and assistance.

Randy

[AdrienLepoutre]

Hello Andrew,

I've been trying to figure out if a specific setup is possible. In a nutshell, I would like to know if it is possible to have a feature layer, initially published by reference (pointed to a map image layer by reference) to then be collaborated as a copy to leverage the hosting capability and keep it in sync.

When I tried setting up the workspace for the collaboration, you get the following option:

That being said, there seems to be a conditional aspect added to the selection - from what I would understand from the text below, on a layer by reference, since there is no "Enable Sync" option in the settings, then a layer initially published by reference is not become a copy in AGOL but will still point to the intial map image layer - thus not leveraging the hosted/sync capabilities?

Thanks!

Adrien

[DavidColey]

All of this is still really buggy.  I've been trying to develop a workflow where I create a web layer draft, stage the sd,  and then upload the staged sd to my portal.  The layers particpate in a collaboration to be sent to our agol

But right now it seems the very act of synchronization makes the upload portion of the staged sd fail with a 99999 error 'Something unexpected caused the tool to fail' upon the next script run.  Here is my sample, Has anyone seen this behavior before?

# Import system modules
import sys, os, arcpy, time, smtplib
from arcpy import env
try:
 # Sign-in to ArcGIS Online
 portal = arcpy.GetActivePortalURL()
 print (portal)
 if portal != "https://myportal.scgov.net/portal":
 arcpy.SignInToPortal("https://myportal.scgov.net/portal", "myUserName", "myPassword")
 print ("I'm signed in")
 else:
 print ("already signed in")
 aprx = arcpy.mp.ArcGISProject(r'V:\ArcProProjets\PortalUpdates\PortalUpdates.aprx')
 for m in aprx.listMaps("Appraiser Layers"): ############## multi-layer map
 message = message + "\n\n" "Map: " + m.name + "\n"
 print("Map: " + m.name)
 for lyr in m.listLayers():
 desc = arcpy.Describe(lyr)
 if (lyr.name == 'ParcelAppraisal' or lyr.name == 'ParcelSalesAdj') and not desc.editorTrackingEnabled:
 
 arcpy.mp.CreateWebLayerSDDraft(lyr, "V:/ArcProProjets/PortalUpdates/Appraiser/" + lyr.name + ".sddraft", lyr.name, "MY_HOSTED_SERVICES", "FEATURE_ACCESS",
 "AppraiserLayers", True, False, False, True, True, "Property Appaisers Sales and Appraisal Layers", 'Appraisers', "Property Appaisers Sales and Appraisal Layers")
 print(lyr.name + " Draft Created")
arcpy.StageService_server('V:/ArcProProjets/PortalUpdates/Appraiser/' + lyr.name + '.sddraft', 'V:/ArcProProjets/PortalUpdates/Appraiser/' + lyr.name + '.sd')
 print(lyr.name + " Service Staged")
 
 arcpy.UploadServiceDefinition_server('V:/ArcProProjets/PortalUpdates/Appraiser/' + lyr.name + '.sd', 'My Hosted Services', "","","","","",
 "USE_DEFINITION","SHARE_ONLINE","PRIVATE","NO_SHARE_ORGANIZATION","Appraiser Layers Group")
 print(lyr.name + " Service Uploaded")
 message = message + "\n\n" "Service Uploaded: " + lyr.name
 else:
 message = message + "\n\n" "Editor tracking enabled or layer " + lyr.name + " not set for stage and upload"
except Exception:
 # If an error occurred, print line number and error message
 import traceback, sys
 tb = sys.exc_info()[2]
 e = sys.exc_info()[1]
 #print(e.args[0])
 #print "Line %i" % tb.tb_lineno
 message = message + "\n" + "Line %i" % tb.tb_lineno
 message = message + "\n" + e.message‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

It is the arcpy.UploadServiceDefinition_server portion that fails the next night after a collaboration sync, where I am syncing 'As Copies'