Select to view content in your preferred language

ArcGIS Enterprise and Azure Net App Files, Microsoft Azure

973
5
07-25-2022 04:08 PM
FraserHand
Occasional Contributor III

Hi there,

Sorry for the cross post.

Has anyone in the community used Azure NetApp Files with their ArcGIS Enterprise deployment. If so could you share your experience with and and what the performance was like?

Thanks

0 Kudos
5 Replies
danbecker
Regular Contributor

I have this same question.

Background: we've been using Azure Virtual Desktop (VD; Win10 multi-session host), FsLogix user profiles are stored in Azure Files share. So far, Pro has worked great in the VD environmetn, EXCEPT our "GIS" file share for FGDBs, Pro Projects, ect.

The "GIS" file share was configured as a Premium Azure Files storage account in the same Azure region (Azure GOV Virginia) as the VDs, smb multi-channel enabled. AD DS authentication is enabled, VD machines and storage account are all AD joined, they both have line of sight to Domain Controller.

Opening Pro projects stored in Azure Files is painful, so too are field calculations. Just opening the field calculation tool takes 5-10 seconds for it to fully populate all controls. If the Pro project and FGDB is copied to a local, managed drive, everything is 10x faster.

Our Azure deployment is configured according to this doc, EXCEPT for Azure NetApp files. Our IT consultant told us "it shouldn't matter" if we used Azure Files instead. https://learn.microsoft.com/en-us/azure/architecture/example-scenario/data/esri-arcgis-azure-virtual...

Anyone have recommendations? Should we just switch to NetApp files?

0 Kudos
DevinBartley2
New Contributor III

Looking for an update on this - I am in the exact same situation as described with slow ArcGIS Pro geoprocessing performance of projects in Azure Files.  Considering upgrading to NetApp files from Azure Files but if someone else can confirm that this does or does not fix the issue it would be greatly appreciated. 

0 Kudos
danbecker
Regular Contributor

I had a different thread I was updating: Azure Files premium file share - SLOW File Geodata... - Esri Community

Yes, absolutely upgrade to NetApp files, you will not be disappointed. But, also note that we found performance still lacking until our AVD session hosts were Entra ID joined; and Kerberos authentication to the NetApp volume was configured. 

Traditional AD-joined AVD session hosts still had issues with NetApp performance, the machines would seemingly drop the NetApp volume connection, which caused Pro to display "General Function Failure" when accessing an attribute table of a FGDB layer, that was stored on the NetApp volume. 

0 Kudos
DevinBartley2
New Contributor III

Appreciate the extra provided detail and comments, this saves me an immense amount of time and lets me focus on the heart of the issue. Your comment makes me wonder how the performance would have been if you stuck with Azure Files but had the AVD hosts Entra ID joined and Kerberos auth enabled on Azure Files.  

0 Kudos
danbecker
Regular Contributor

@DevinBartley2 wrote:

Appreciate the extra provided detail and comments, this saves me an immense amount of time and lets me focus on the heart of the issue. Your comment makes me wonder how the performance would have been if you stuck with Azure Files but had the AVD hosts Entra ID joined and Kerberos auth enabled on Azure Files.  


The performance of Azure Files with AVD hosts Entra ID joined would still not match the NetApp volume (Premium performance Tier).

How do I know this?

After we deployed Entra ID joined AVD session hosts, we continued to use Azure Files for our FsLogix profile containers (user_name.vhdx). To do so, we had to create a new Azure Files premium StorageAccount for the "profiles" file share because you can only configure 1 authentication method per storage account, and the old storage account was configured for AD DS authentication. AD DS auth. isn't supported for Entra ID joined AVD session hosts, so the new storage account was configured with Kerberos authentication. 

We tried to migrate the old user_name.vhdx profile containers to the new storage account, but were unsuccessful, because the SID of the user_name changed (AD DS identity vs. syned Entra ID identity). Something internal with how FsLogix looks up the location of the user's profile container prohibited it from working, generally it's unsupported according to MS docs; migrating a profile container after a user's SID is changed. 

Everything seemed to work great, UNTIL the session host failed to retrieve an updated Kerberos ticket from Microsoft (Azure Files) to authenticate the "profiles" share that was hosting the user_name.vhdx profile container. Windows command: klist would show that the "profiles" share kerberos ticket expired 1 hour after it was issued. If that ticket wasn't updated by Windows successfully before that expiration date/time, then it was the equivalent of pulling a hard drive out of a working windows machine, complete chaos ensues for the user's session. I have no clue why Windows wasn't able to retrieve updated Kerberos tickets from Azure Files. It would work fine for all users for hours, then all of a sudden, it wouldn't. There was no rhyme/reason why it would/not work and when. All I can think of is Azure vnet sputtering or packet loss was the cause. Someone on StackOverflow said they cannot think of a worse use-case scenario for Azure Files (profile container hosting)...that's all we had to hear and pulled the plug on Azure Files, especially given our past GIS experience with it. 

Now, remember the Windows command: klist. While we were troubleshooting the above profile container issue with Azure Files, we noticed that the kerberos ticket for the NetApp "GISfiles" volume (stores FGDBs, ect.) was being issued by our AD machine, and had a lifespan of 10 hours. So, why not create another NetApp volume "profiles" and simply move the profile containers to there? That's exactly what we did, and all of our problems are now solved.

Something tells me that the un-reliable Kerberos ticket renewal process with Azure Files would cause problems for GIS data stored on there as well, I can almost guarantee it. Pro would throw the "General Function Failure" error when that ticket expires, if a new one wasn't issued in time.

I recently overheard an analyst last week "has anyone worked on their local machine with Pro lately?" someone then responded "no, the AVD machines are 10x faster". Simple conversation, but very satisfying to hear, we've been put through the paces getting this technology to work. When paired with a high performing NVIDIA GPU, our Pro machines are BLAZING fast. Next step will be to purchase another 1TB of capacity on our NetApp capacity pool, so the total would be 3TB, split between 2 volumes (GIS and Profiles), then we get 3 * 64 MiB/sec =  192 MiB/sec throughput to play with. NetApp lets you manually specify how much throughput to allocate to each volume, its highly configurable and super simple. I would highly recommend NetApp, it's amazing.

0 Kudos