nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Azure Files premium file share - SLOW File Geodatabase Performance

danbecker

Has anyone attempted/used an Azure Files premium file share for storing File Geodatabases and Pro projects?

We have an Azure Virtual Desktop deployment for ArcGIS Pro workstations. These workstations are in the same Azure region as the above premium file share. Workstations map the premium file share via a SMB private endpoint connection, it appears like any other on-prem network drive.

We can transfer a 450MB+ file (.zip, .tif, ect) between local managed drive <--> the premium file share using file explorer in seconds. This single, large file transfers either direction in < 1 second. I briefly saw the transfer dialog window and it was at 498MB/sec.

But the same file explorer transfer with a 2GB File Geodatabase (contains 5,527 files) takes 5-10 minutes! Same is true when expanding a feat. dataset in the FGDB, the Pro catalog window spinning wheel appears for 30+ seconds before anything happens. Opening the attribute table, calculating a field, ect. everything is super slow, Pro appears to hang frequently.

Has anyone experienced this? There seems to be a performance issue when thousands of small files are manipulated in Azure Files.

Finde mehr Beiträge mit dem Tag

Geodatabase

Desktop

Accepted answers

All comments

zporteous

I have noticed in our organization slow performance with Azure File Storage when it comes to GDBs after a recent migration. Very unfortunate.

danbecker

I can't believe this isn't documented, or discussed more..?

We abandoned Azure Files and deployed an Azure NetApp Files 2TB smb volume, premium tier. Throughput given that tier and volume size is 128 MiB/s, equivalent to an on-prem 1000 Mbps network.

And, the performance shows, Id say it's faster then our 10k RPM SSD SATA disks on a 1000mbos network.

This is our identical architecture:

https://learn.microsoft.com/en-us/azure/architecture/example-scenario/data/esri-arcgis-azure-virtual-desktop

The only issue we have are random "general function failures" in Pro 3.1.2. Happens when the project is using FGDBs. SDE GDBs and you never get that failure. I'd say that error was happening 1-4 times /hr on Azure Files, and now once per day on NetApp files. It seems to happen when you open an attribute table. There's no way for Pro to recover, nothing draws, ect until you close Pro, then re-open..back to normal.

So close to a REALLY nice, highly scalable virtual platform.

danbecker

We deployed a new Azure Virtual Desktop host pool that contains NVIDIA GPUs and Pro 3.2.1.

This general function failures STILL happen, only when accessing FGDBs stored in Azure NetApp files.

Anyone experience this, ESRI, any advice here?

danbecker

We deployed a new Azure Virtual Desktop host pool that only contained Entra ID joined session host VMs, same NVIDIA GPUs. upgraded to Pro 3.2. Since the VMs were no longer AD joined, we had to migrate all the AD Group Policy Objects to InTune, which really wasn't that difficult given the GPO import tools in InTune.

FsLogix profile containers were switched away from Azure Files Premium and are being stored in an Azure NetApp Volume, just like our GIS file-based data.

Entra ID-joined session hosts can only authenticate with SMB shares using Kerberos authentication, not NTLM (on-prem AD). So, Kerberos auth. and ACLs (permissions) were configured per Azure docs. for both the NetApp \profile and \gis volumes.

For the results: ALL PROBLEMS ARE SOLVED!

Added bonus is that all MS office apps, including the Remote Desktop client are SSO using Entra ID (cloud) credentials. The VMs, Pro, and FGDB access is 100% improved.

I wish I had a better explanation, but our problems with FGDB performance in AVD was 100% caused by authentication issues with traditional on-prem AD (which was a VM in Azure, not really on-prem).

With the sputtering/performance issues resolved, I have no doubt that better performance could be achieved by increasing the NetApp capacity pool quota (i.e. monthly A$ure invoice). Our current cap. pool quota is 2TB which comes with 128 MiB/s throughput, divided how ever we want between \profile and \gis volumes. Another 1TB would add +64 MiB/s throughput.

Azure NetApp is quite amazing, so simple.

JakubPruchnik

I can confirm that "my" infrastructure is experiencing this same issue. We initially used generic Azure Files - Transaction Optimised tier and performance was terrible. We decided to move to Azure Premium Files and performance improved slightly but not significantly. We are currently conducting an investigation if this can be improved before we start tinkering about other solutions. NetApp storage mentioned here by others caught my attention.

danbecker

My only advice is to migrate both FsLogix profile containers AND GIS file share to Azure NetApp files and stop fussing with Azure Files (all storage/transaction tiers).

We have 1 Azure NetApp 3TB capacity pool (premium tier) that's used for /profile and /gis shares and have had zero issues since my above thread reply. We opted for manual throughput, so that we could favor the /gis volume and assign it 132 MiB/Sec. The profiles are a bit slow to sign-out due to the throughput bottleneck, but we just advised users to sign-out of AVD, then minimize the AVD session and go about their business on the local PC. FsLogix eventually completes all communication with the profile container in netapp and signs-out w/out any issues.

We started using this CPU SKU: Standard NC16as T4 v3, and we're able to comfortably fit 4 pooled (multi-session) users on the same VM. I actually think we could increase that to 5, maybe 6... and all of them are using Pro. It's pretty amazing.

JamieMaxwell

Same issue here. We initially tried Azure Files premium with bad results in the Pro Performance Assessment Test. Now we have a Windows Server box in Azure for our file share. Performance is definitely better, but we still get the general function failure issues in Pro. We also have a few people still using ArcMap. In addition to the general function failures, they get errors about saving the mxd. They have to "save as" with a different name.

Nimm an der Diskussion teil

Angemeldete Mitglieder können Beiträge verfassen, Updates folgen und mehr. Neu hier? Registriere ein kostenloses Konto.

Schnellzugriffe

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.