Select to view content in your preferred language

Pull Attachments Table from AGOL

299
9
07-31-2024 06:08 AM
Labels (3)
CodyPatterson
Frequent Contributor

Hey All,

I have recently been asked to create a sync between AGOL and a local database, this is a relatively simple task with a combination of ArcGIS API for Python and ArcPy since ArcPy cannot communicate with AGOL.

The issue arises now because there are attachments, when I exported the feature layer to a FGDB I see this here:

CodyPatterson_0-1722431248331.png

I see that there is a relationship and an ATTACH table, I found that the relationship can be entered once and left without issue, but I need to update the ATTACH table for the attachments in the feature classes to be included, how can I extract this ATTACH table or possibly the attachments to format into the table?

Cody

0 Kudos
9 Replies
RhettZufelt
MVP Notable Contributor

To sync from AGOL data to the local database, I have been using the arcpy.management.Append with preserve GlobalIds and maintain Attachments.

Then I get an updated local copy of the data with attachments while preserving the GlobalIds.

R_

0 Kudos
CodyPatterson
Frequent Contributor

Hey @RhettZufelt 

Unfortunately it seems that ArcPy is not reliable with AGOL according to BUG-000144037 which is not planned to be fixed unfortunately, I ended up setting up a macro on a server to make clicks which downloaded the FGDB, and then ran the script after it was export from AGOL, which then is deleted afterward.

Cody

0 Kudos
RhettZufelt
MVP Notable Contributor

Not sure if you are trying to sync both ways, or why you are wanting to use the arcpy.MakeFeatureLayer_management as the bug references, but maybe my process is different enough it won't work for you.

In my case, I have a local eGDB that I published to AGOL for data collection in a Field Maps app.

So, the data is collected in AGOL hosted feature service, but I need to keep the local copy up to date.

I have been using code similar to the following for the last 2 years and have had no issues with it.

Basically, I establish a search cursor on the AGOL and local datasets and load the GlobalIDs into a list.  It also compares the two lists, and if there are GlobalIDs in the AGOL list, but not the local list, it means those features need to be copied (appended) over. (So, we are only coping the 'missing' features, not the entire dataset)

Then I use arcpy.management.Append to append the missing features to the local data using a where clause that is an IN() with the GlobalID's from the missing list like so:

 

import arcpy

Signs = r'\\path_to\\SignDatabase.sde\SignsDatabase.DBO.Signs'  # path to the signs local eGDB FC
L0Sign = r"https://services3.arcgis.com/itemID/ArcGIS/rest/services/HostedFeatureService/FeatureServer/0" # URL to AGOL hosted feature

AGOL_SignTable =  arcpy.FeatureSet()  # define AGOL_SignTable as FeatureSet
AGOL_SignTable.load(L0Sign)                # Load hosted feature into AGOL_SignTable

SignList = []       # empty list for local GDB features
AGOLSignList = []   # empty list for AGOL features


with arcpy.da.SearchCursor(Signs,"GlobalID") as cursor:           # Get list of GlobalID's from SDE data
    for row in cursor:
        SignList.append(row[0])

with arcpy.da.SearchCursor(AGOL_SignTable,"GlobalID") as cursor:  # Get list of GlobalID's from AGOL data
    for row in cursor:                                      
        AGOLSignList.append(row[0].upper())
        if (row[0].upper() or row[0]) not in SignList:             # and if not in local SignList, append to AddSignList
           AddSignList.append(row[0])


ith arcpy.EnvManager(preserveGlobalIds=True, maintainAttachments = True): 
    if AddSignList:                                                 # if any 'missing' features in AddSignList, then append to local eGDB
       print("adding " + str(len(AddSignList)) + " Sign Records")
       Signs = arcpy.management.Append(inputs=[L0Sign], target=Signs, schema_type="TEST", field_mapping="", subtype="", expression="GlobalID IN ('" + '\',\''.join(AddSignList) + "')")[0]
    else:
       print("No Signs to add")

 

I can't say the same for any scripts I run using the API python (seem to get a lot of timeout errors, etc. and have to code accordingly).  However, I have had good success with the API code here to export an AGOL hosted feature service to a FGDB (including attachments) and download to local folder without having to script any mouse clicks.

R_

 

0 Kudos
CodyPatterson
Frequent Contributor

Hey @RhettZufelt 

Thank you for the informative response and code, I attempted to copy the code exactly as you mentioned, but I'm not sure what the SignTable function should represent. The reason behind the makefeaturelayer function is due to geoprocessing tools that need to run on the hosted feature service. Attempting to load the layer into an FGDB does not work as well, currently, the main issue is getting ArcPy and AGOL to talk in any extent, the support technician I worked with stated that it's a known limitation that these aren't working, but work for some just fine, and ESRI is not interesting in pursuing the fix further, as the bug mentions unfortunately.

 

0 Kudos
RhettZufelt
MVP Notable Contributor

If you copied it exactly, not surprising you got an error as I had a mistake in the code.

 

AGOL_SignTable =  arcpy.FeatureSet()
AGOL_SignTable.load(L0Sign)

 

Basically, I define AGOL_SignTable as an  arcpy feature set, then the next line loads the data from the hosted feature server layer URL into the arcpy FeatureSet.

Once loaded, I can treat AGOL_SignTable as any other FGDB featureclass.  I have not tested all functions on it, but works well with search cursors, arcpy calculate and arcpy append.

R_

0 Kudos
CodyPatterson
Frequent Contributor

Hey @RhettZufelt 

This is my output here:

CodyPatterson_0-1722881783887.png

My code:

import arcpy

layer_name = r"https://services9.arcgis.com/{id}/arcgis/rest/services/USA_Counties_6260724028191129561/FeatureServer"

AGOL_SignTable =  arcpy.FeatureSet()
AGOL_SignTable.load(layer_name)

This was another error that was mentioned to only work for some, and not others.

Cody

0 Kudos
RhettZufelt
MVP Notable Contributor

What happens is you use the full path to the feature layer in that feature service (add the correct layer number to the end of the URL.  /0 on my HFS)?

layer_name = r"https://services9.arcgis.com/{id}/arcgis/rest/services/USA_Counties_6260724028191129561/FeatureServer/0"

 

R_

0 Kudos
CodyPatterson
Frequent Contributor

Hey @RhettZufelt 

Unfortunately, the exact same error came through. Not sure why my AGOL and ArcPy are so incompatible, but maybe I'm just missing something!

Cody

0 Kudos
RhettZufelt
MVP Notable Contributor

Not sure what to tell you.  I have tested on my end with polygon and point feature classes with no issue.

Perhaps there is a limitation on the Feature Service you are connecting to.  I published my own HFS with no restrictions and use the code to keep the local data in sync with changes to the HFS.

R_

0 Kudos