Backup AGOL Hosted Feature Layers (aka Hosted Feature Services, aka Hosted Feature Collections)

[NorthSouthGIS]

Here is an example python script that can be run to specifically backup hosted feature layers from ArcGIS Online to a local drive or network drive on your computer or network. Although not ideal to code in credentials, if it is stored and ran from a secure machine, it can be scheduled to run automatically with Windows Task Scheduler. Alternatively, you can run this script manually and alter it to prompt you for credentials. Thanks to Adam Koelker for presenting the core of this script via YouTube. I adapted to query and loop through a specific set, and added some print statements. 

from time import strftime
print (strftime("%c"))
from arcgis.gis import GIS

###Authenticate to ArcGIS Online
gis=GIS("https://yourorganization.maps.arcgis.com","yourUsername","yourPassword")

###Query to for all items to be downloaded. In this case, it's searching for all feature layers marked as authoritative
myFeatureCollections=gis.content.search(query="contentstatus:org_authoritative",item_type="Feature Layer Collection",max_items=1000)

###Output location for downloaded backups
output=r"C:/path/to/output/location"

###Initiate cycle to export, download, and delete backups
for item in myFeatureCollections:
	try:
		print('Exporting '+str(item.title))
		currentItemID=item.itemid
		dataitem=gis.content.get(currentItemID)
		### Create Backup
		tempfile=strftime(dataitem.title+"_backup_%Y%m%d")
		dataitem.export(title=tempfile,export_format="File Geodatabase",parameters=None,wait=True)
	except:
		print("Could not create backup for "+str(item.title)+' ('+str(item.itemid)+')')
		pass
	try:
		### Find and download export
		myexport=gis.content.search(tempfile,item_type="File Geodatabase")
		fgdb=gis.content.get(myexport[0].itemid)
		fgdb.download(save_path=output)
		print("Downloaded "+str(fgdb.title)+" to "+output)
	except:
		print("Could not download export for "+str(myexport[0]))		
		pass
	try:
		###Delete export
		fgdb.delete()
	except:
		print("Could not delete export for "+str(fgdb))
		pass


print("Script completed at {}".format(strftime("%c")))

EDIT: This query no longer retrieves explicitly hosted feature layers, but also stored web layers. See https://developers.arcgis.com/rest/users-groups-and-items/items-and-item-types.htm for updated item types that can be queried

[HarunJoeVellikkara]

Hey @Nick_Creedon sorry to bother you! You have been very helpful.

I had a query since I am running this script for ArcGIS Enterprise is there a way we can tweak this script to get only Hosted Feature services. Since, query_string = "type:Feature Service" includes both Hosted and Referenced services right?

Also, do you think taking a GeoJson backup would be as helpful as taking a File Geodatabase backup? For example if one of my hosted service is corrupted/deleted which one would be more feasible to restore (JSON Vs Fgdb).

[Nick_Creedon]

These are good questions,

Referenced services like versions or "View Layers"? To my understanding it will try to export the versioned layer but it will fail and only export the parent. If this is what you are referring too. I have lots of view layers and they don't make it into my backups.

I left this out of my previous messages, but you could specify "Tags" for your backups so you only backup specific layers and not your whole organization every time. I do weekly backups of specific tagged layers and quarterly backups of the whole organization. 

I have never worked with JSON, I cannot answer that question. I would love to get to understand its uses.

If you deploy a layer from ArcGIS Pro to Portal or AGOL, you create a Service Definition. I have pulled a service definition back to use the original schema and used data from a FGDB to fix a layer. 

Maybe this is the JSON? I don't know.

To download Service Definitions the section would look something like this:

I need to update this script to Retry if one fails to download. 

[HarunJoeVellikkara]

Thanks @Nick_Creedon for the detailed explanation! Appreciate all your help so far. Cheers!!!🥂

[AndrewHankinson]

@Nick_Creedon, thanks for this works great.