Hi Harun,
Happy to hear other people are using this!
I use this for ArcGIS Online Hosted Feature Layers, and I come across some layers that are slow to download.
I do not have a right answer for this, but large data sets and or layers with lots of attachments can definitely slow down the process. If you manually export a layer from AGOL or Portal as a GDB a large dataset will take longer than a small one. So these symptoms are across the board, its not only in the script.
To optimize faster downloads, you could break down group layers, relocate attachments to a web service and embed them as links in your layers to take the heavy lifting off your system. We do this for a few layers or else they would be over 10gb easy.
Since I posted this script I have definitely made some changes because have also had issues with some layers timing out when they are attempting to export. I have added a "Retry" action into the script to try at least 3 times if it were to fail.
import os
import datetime as dt
from arcgis.gis import GIS
import certifi
import urllib3
import time
# Suppress only the single InsecureRequestWarning from urllib3 needed for unverified HTTPS requests
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
# Connect to ArcGIS Online or Portal for ArcGIS with certificate verification
gis = GIS("home", verify_cert=certifi.where())
# Set the parent folder path for backups
parent_folder_path = r"FILE LOCATION HERE"
# Create a new folder for today's date (e.g., "2025-02-11 Backup")
today_date = dt.datetime.now().strftime("%Y-%m-%d Full Backup")
folder_path = os.path.join(parent_folder_path, today_date)
# Create the folder if it doesn't exist
if not os.path.exists(folder_path):
os.makedirs(folder_path)
# Define the number of items to fetch
num_items = 1000
# Search for the most recently modified feature services
query_string = "type:Feature Service"
items = gis.content.search(query=query_string, max_items=num_items, sort_field='modified', sort_order='desc')
# Print the number of items to be backed up and their details
print(str(len(items)) + " items will be backed up to " + folder_path + ". See the list below:")
for item in items:
print(item.title)
# Function to download items as file geodatabases with retry mechanism
def download_as_fgdb(item_list, backup_location, max_retries=3):
for item in item_list:
retries = 0
while retries < max_retries:
try:
if 'View Service' in item.typeKeywords:
print(item.title + " is a view, not downloading")
break
else:
print("Downloading " + item.title)
version = dt.datetime.now().strftime("%d_%b_%Y")
result = item.export(item.title + "_" + version, "File Geodatabase")
result.download(backup_location)
result.delete()
print("Successfully downloaded " + item.title)
break
except Exception as e:
retries += 1
print(f"An error occurred downloading {item.title}: {str(e)} (Retry {retries}/{max_retries})")
if retries < max_retries:
time.sleep(5) # Wait for 5 seconds before retrying
else:
print(f"Failed to download {item.title} after {max_retries} retries")
print("The function has completed")
# Call the function to download items
download_as_fgdb(items, folder_path)
