We are trying to set up an Azure Storage Account container to host our ImageServer data (primarily CRFs) and are getting errors about a vsiaz folder both when publishing/starting services and when creating ACS files. The Storage Account is Premium_LRS with BlockBlobStorage. ImageServer is on a Windows VM (following Esri specs) and is federated.
Our first attempt was using the Storage Account’s Access Key for authentication. We are aware that Access Keys are not a long-term solution and were waiting on the bug for registering User-Assigned Managed Identity authenticated Data Stores to be resolved. Steps:
- Create ACS file using “Create Cloud Storage Connection File” tool. Use Storage Account Access Key from Azure Storage Account. Pertinent settings:
- Service Provider: Azure
- Access Key ID: (storage account name)
- Secret Access Key: (storage account access key)
- Bucket (Container) Name: gis
- Folder: (blank)
- Region: Azure Cloud
- Service End Point: blob.core.windows.net
- Provider Options: (none)
- Register Azure Cloud Store in ImageServer using Storage Account Access Key. Settings:
- Type: Microsoft Azure Storage
- Cloud Store Name: storageaccountname_cloudstore
- Authentication type: Shared key
- Account Key: (storage account access key)
- Storage domain: Azure Cloud
- Container Name: gis
- Folder: cloud/arcgis
- Register Cache directory: false
- Register Raster store in ImageServer. Settings:
- Name: storageaccountname_rasterstore
- Type: Cloud Store
- Cloud Store: storageaccountname_cloudstore
- Validated both stores successfully.
- Created CRF. Used “Copy Raster” tool. Added CRF to Storage Account via Azure Storage Explorer.
- Published CRF to ImageServer. Steps:
- Added ACS file as Cloud Store to Pro Catalog.
- Right-click CRF, choose Share as Web Layer. Pertinent settings:
- Layer and Data Type | Imagery | Reference Registered data
- Server: https://ourdomain/imageserver
- Folder: (ourfolder)
- Share with: Enterprise Portal
- Result: Web layer published successfully on (date).
- Viewed layer via Enterprise Portal. Layer works.
- Published several more CRFs using the same publishing process.
At some point, all CRFs stop working. The services stop. When attempting to start the service in Server Administrator, we get:
{
"status": "error",
"messages": ["Service failed to start"],
"detail": {
"fqsn": {
"folderName": " ourfolder ",
"serviceName": "service_name",
"type": "ImageServer",
"description": null
},
"service": " ourfolder.service_name.ImageServer",
"state": "FAILED",
"message": "",
"machines": [{
"machine": "MACHINE.NAME",
"state": "FAILED",
"reports": [{
"state": "FAILED",
"message": " Invalid input data source path: /vsiaz/gis/ourfolder/crf_name.crf."
}]
}]
}
}
I am unclear why ArcGIS Server is looking for a folder named vsiaz. The root would be the “gis” BLOB container in the storage account.
Since we made this attempt, there is now a workaround for registering a Data Store using a User-Assigned Managed Identity (UAMI), so we tried that. We also heard that ImageServer may need write access to the storage account, so we gave the UAMI appropriate permissions. General steps:
- Created a UAMI in Azure.
- Assigned the UAMI to the VM running ImageServer and the VM running Portal.
- Created an Active Directory group with the Storage Blob Data Contributor role.
- Assigned the group to the Storage Account’s resource group.
- Added the UAMI to the group.
- Added Azure Cloud Store to ImageServer authenticating via the UAMI (had to use the Enterprise API for this).
- Successfully validated the Azure Cloud Store.
- Added a Raster Store to ImageServer that points to the Cloud Store.
- Successfully validated the Raster Store.
After setting up the UAMI and adding the Cloud Store / Raster Store, services would still show the same error when attempting to start.
On the assumption that the service needs to be published via the same credentials (using UAMI, not Access Key), we attempted to create a new ACS file that uses the UAMI. Pertinent settings for the Create Cloud Storage Connection File tool:
- Service Provider: Azure
- Access Key ID (Account Name): storageaccountname
- Secret Access Key (Account Key): (blank)
- Bucket (Container) Name: gis
- Region (Environment): Azure Cloud
- Provider Options:
- AZURE_IMDS_OBJECT_ID: (UAMI | Overview | Object (principal) ID)
- AZURE_IMDS_CLIENT_ID: (UAMI | Overview | Client ID)
- AZURE_IMDS_MSI_RES_ID: (UAMI Resource ID)
The UAMI Resource ID was fetched via the Azure CLI:
az identity show --name (UAMI-resource-name) --resource-group (UAMI-resource-group-name) --query "id" -o tsv
When we run the “Create Cloud Storage Connection File”, we get the error:
WARNING 002594: Failed to establish connection to the root of cloud storage, created ACS file may not work. AZURE_STORAGE_ACCESS_KEY or AZURE_STORAGE_SAS_TOKEN or AZURE_NO_SIGN_REQUEST configuration option not defined [Failed cloud operation: Ping, Path: /vsiaz/gis/]
The ACS file gets created but does not work (nothing shows when it is clicked in Catalog). Since we are not able to create the ACS file to be able to publish the CRF from the Storage Account, we are unable to test whether the new UAMI authenticated Cloud Store / Raster Store is working.
Similar to the error when starting services, this error is referencing a vsiaz folder. From web searches, I have learned vsiaz has something to do with GDAL, but I am not finding much more documentation from Esri on how this is specific to implementation of a Storage Account / ImageServer stack.
Unfortunately, we do not know exactly what conditions were present when the CRFs that were published via the Access Key authentication stopped working. Any advice would be greatly appreciated.
