Hi there,
I have a deployment using Azure blob for the cloud object store @ 12.0. I am seeing ArcGIS Server backups fail once they get over a certain size, or if I attempt to include object store caches in the backups. I get this message in the webgisdr logs:
{
"name": "ArcGIS Server",
"messages": "Cannot access the backup file for ArcGIS Server in webgisdr-backup/12-0-0/backup/20260505-091107-backup\\server\\cd67fdf9jcwpzzk6.",
"URL": "https://xyzxyzxy/hosting",
"status": "failure"
}
Latest job status of async backup operation on server https://xyzxyzx/hosting: {"jobid":"j65e79912-4aaa-46d3-8b8d-0717760d431b","stepInfo":["Export operation started.","Export operation failed."],"status":"EXECUTING"}.
"{"jobid":"j65e79912-4aaa-46d3-8b8d-0717760d431b","status":"COMPLETED","stepInfo":["Export operation started.","Export operation failed."],"operationResponse":{"status":"error","messages":["Export operation failed. Failed to write the file '20260505-091124.agssite' to this location 'webgisdr-backup/12-0-0/backup/20260505-091107-backup/server/cd67fdf9jcwpzzk6'"]}}"
looking at a manual export of that site from the admin API I get:
Failed to write the file '20260505-095011.agssite' to this location 'backup-20260504' Failed to upload backup-20260504/20260505-095011.agssite to object store.. Caused by: Timeout on blocking read for 300000000000 NANOSECONDS
Export operation failed. Failed to write the file '20260505-095011.agssite' to this location 'backup-20260504'
which is an exception in:
Caused by: Timeout on blocking read for 300000000000 NANOSECONDS at com.esri.arcgis.objectstore.clients.azure.AzureBlobStoreClient.putObject(AzureBlobStoreClient.java:986) at com.esri.arcgis.objectstore.clients.azure.AzureBlobStoreClient.putObject(AzureBlobStoreClient.java:915) at com.esri.arcgis.discovery.admin.backuprestore.BackupRestoreUtil.writeSiteFileToBackupStore(BackupRestoreUtil.java:888)
This worked at 11.5, there was a change from staging everything locally first before uploading to writing directly to blob storage at 12.0. We are using private endpoints to access the storage and should have a clear path to it.
I have seen two bugs with 12.0 but don't know if this is related to what I'm seeing:
If anyone has hit this have you got a cause / solution? I'm going to staging locally and uploading to blob in the meantime.
Thanks!
UPDATE: I'm trying to write the backup to a local drive / share now and have updated the webgisdr properties to suit. If I run with no Azure config in the webgisdr.properties file I get this:
{"backupLocation":"","messages":"The value of the property BACKUP_BLOB_ACCOUNT_NAME cannot be empty.","id":"cbcfbc64699144a89586a1bb1383ba80","operation":"backup","results":[],"status":"failure","elapsedTime":"00hr:00min:02sec"}
and if I put in the location of the objstore container I get this in the server logs and the export fails:
Could not connect to the Backup Object Store specified. Check if the specified parameters are valid. JSONObject["objectStore"] not found.
this is with INCLUDE_OBJECT_STORE_CACHES = true
Another update - if I set:
# Specify whether to include object store caches.
INCLUDE_OBJECT_STORE_CACHES = false
then I can get a .agssite file locally.