Unable to use dynamic provisioning with Azure Files in ArcGIS Enterprise on Kubernetes

Question

We were able to easily deploy and configure ArcGIS Enterprise on Kubernetes using dynamic provisioning with Azure Disk.
However, when using dynamic provisioning with Azure files, the first pod that comes up fails with a "crashLoopBack" error and the configuration does not proceed forward.

We have used the following storage class YAML:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: arcgis-storage-default
provisioner: kubernetes.io/azure-file
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=0
  - gid=0
  - mfsymlinks
  - cache=strict
  - actimeo=30
parameters:
  skuName: Standard_LRS
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

Errors we are getting during configuration:

Logs from the failing container:

ChristopherPawlyszyn · Answer

Hi Vaibhav,

From the research I've done on Azure Files, it is using a CIFS mount to access the underlying share. Once mounted, CIFS filesystem permissions cannot be changed from the Linux client. With that in mind, the PostgreSQL initdb process executed when the container spins-up attempts to modify the permissions on the PGDATA location, which is not an allowed operation with Azure Files. With that being the case, there is no Azure Files-based storage class that would allow for PostgreSQL to initialize successfully.

Azure Files is listed as a supported storage type, but we do specifically recommend the use of block storage in our documentation:

"It is recommended to use block storage devices which provide low latency such as EBS volumes, Azure Disks, vSphereVolume, etc."
From https://enterprise-k8s.arcgis.com/en/10.9/deploy/system-requirements.htm#ESRI_SECTION1_71FE341CD85F490EB15FBD48A924B3E8

Anonymous User · Answer

Hi Chris,
Thank you for your response. 
Our motivation to use Azure FIles is two-fold; the first is to verify support for Azure Files as it is listed as a supported option in the documentation, and the second is that Azure Files give better options for backup and recoverability especially across regions as compared to Azure Disks.

The help documentation includes an example of a storage class definition using Azure Disks and it has worked well for us.
Can you provide us with a working example of a storage class definition for Azure Files?
It will be a great help.
Thanks in advance

ChristopherPawlyszyn · Answer

Is there any specific motivation to use Azure Files for the relational store persistent volume instead of Azure Disks?

Typically it is not recommended to run a PostgreSQL database on an NFS mount, as latency and client-side caching may affect the performance and/or integrity of the running database. Even though the container would technically be mounting a share from the underlying host when using a PV on an Azure Disk, that is the preferred deployment method for those stateful sets.