Select to view content in your preferred language

WEBGISDR not functioning @12.0 with cloud object store?

305
2
2 weeks ago
FraserHand
Frequent Contributor

Hi there,

I have a deployment using Azure blob for the cloud object store @ 12.0. I am seeing ArcGIS Server backups fail once they get over a certain size, or if I attempt to include object store caches in the backups. I get this message in the webgisdr logs:

{
"name": "ArcGIS Server",
"messages": "Cannot access the backup file for ArcGIS Server in webgisdr-backup/12-0-0/backup/20260505-091107-backup\\server\\cd67fdf9jcwpzzk6.",
"URL": "https://xyzxyzxy/hosting",
"status": "failure"
}

Latest job status of async backup operation on server https://xyzxyzx/hosting: {"jobid":"j65e79912-4aaa-46d3-8b8d-0717760d431b","stepInfo":["Export operation started.","Export operation failed."],"status":"EXECUTING"}.
"{"jobid":"j65e79912-4aaa-46d3-8b8d-0717760d431b","status":"COMPLETED","stepInfo":["Export operation started.","Export operation failed."],"operationResponse":{"status":"error","messages":["Export operation failed. Failed to write the file '20260505-091124.agssite' to this location 'webgisdr-backup/12-0-0/backup/20260505-091107-backup/server/cd67fdf9jcwpzzk6'"]}}"

looking at a manual export of that site from the admin API I get:

Failed to write the file '20260505-095011.agssite' to this location 'backup-20260504' Failed to upload backup-20260504/20260505-095011.agssite to object store.. Caused by: Timeout on blocking read for 300000000000 NANOSECONDS
Export operation failed. Failed to write the file '20260505-095011.agssite' to this location 'backup-20260504'

which is an exception in:

Caused by: Timeout on blocking read for 300000000000 NANOSECONDS at com.esri.arcgis.objectstore.clients.azure.AzureBlobStoreClient.putObject(AzureBlobStoreClient.java:986) at com.esri.arcgis.objectstore.clients.azure.AzureBlobStoreClient.putObject(AzureBlobStoreClient.java:915) at com.esri.arcgis.discovery.admin.backuprestore.BackupRestoreUtil.writeSiteFileToBackupStore(BackupRestoreUtil.java:888)

This worked at 11.5, there was a change from staging everything locally first before uploading to writing directly to blob storage at 12.0. We are using private endpoints to access the storage and should have a clear path to it.

 

I have seen two bugs with 12.0 but don't know if this is related to what I'm seeing:

https://support.esri.com/en-us/bug/large-web-gis-disaster-recovery-webgisdr-exports-fail-t-bug-00017...

https://support.esri.com/en-us/bug/after-migrating-hosted-scene-layer-caches-from-the-tile-bug-00017...

If anyone has hit this have you got a cause / solution? I'm going to staging locally and uploading to blob in the meantime.

 

Thanks!

 

0 Kudos
2 Replies
FraserHand
Frequent Contributor

UPDATE: I'm trying to write the backup to a local drive / share now and have updated the webgisdr properties to suit. If I run with no Azure config in the webgisdr.properties file I get this:

{"backupLocation":"","messages":"The value of the property BACKUP_BLOB_ACCOUNT_NAME cannot be empty.","id":"cbcfbc64699144a89586a1bb1383ba80","operation":"backup","results":[],"status":"failure","elapsedTime":"00hr:00min:02sec"}

and if I put in the location of the objstore container I get this in the server logs and the export fails:

Could not connect to the Backup Object Store specified. Check if the specified parameters are valid. JSONObject["objectStore"] not found.

this is with INCLUDE_OBJECT_STORE_CACHES = true

 

 

 

0 Kudos
FraserHand
Frequent Contributor

Another update - if I set:

# Specify whether to include object store caches.
INCLUDE_OBJECT_STORE_CACHES = false

then I can get a .agssite file locally.

0 Kudos