webgisdr super slow with Azure file storage v2

338
6
05-27-2020 10:13 PM
FraserHand
Occasional Contributor II

Hi,

We have webgisdr backing up to a SMB file share in AFS. It completes but is incredibly slow - it's the portal component that takes the longest. The AFS is connected via a private endpoint and uses AADDS authentication so we know it isn't permission issues. Has anyone seen this?

Thanks!

Fraser

Edit: ArcGIS Enterprise 10.8

Jonathan Quinn

0 Kudos
6 Replies
JonathanQuinn
Esri Frequent Contributor

How long does it take to backup? How much content do you have? Can you describe your enviroment, (distributed, HA, etc).

0 Kudos
FraserHand
Occasional Contributor II

Hi Jon,

It's a fresh install of 10.8 - no content except the default. what I see is that the server exports take a few minutes and then the portal component takes about 4 - 6 hours. 

The GIS share for backups is an AFS share provisioned at 512GB - this gives us 


Allowed IO/s : 512
Burst IO/s : 1536
Egress Rate : 90.7 MiBytes / s
Ingress Rate : 60.5 MiBytes / s

Looking at the log file the backup of Server, Data Store and Portal take about 25mins, but the packaging takes forever (about 3 hours). I know we can store in Blob storage, but we still need a common location for staging and packaging so would still need a common share. Have you seen any metrics like this in Azure?

Thanks

Fraser

Jonathan Quinn

0 Kudos
JonathanQuinn
Esri Frequent Contributor

Is that the equivalent of standard or premium shares?

Planning for an Azure Files deployment | Microsoft Docs 

Packaging a backup over a share is definitely an issue right now. You can package locally and then copy to wherever it needs to be stored and see if that's faster.

0 Kudos
FraserHand
Occasional Contributor II

Hi Jonathan Quinn‌,

It's a premium share. We've bumped the tier up and it performs a little better but still very slow. I'll rework to use a local shared location. Why is it so slow packaging to AFS? This is connected as a SMB 3 shared to all the machines via ADDDS auth.

Thanks

Fraser

0 Kudos
JonathanQuinn
Esri Frequent Contributor

It's slow packaging to any share. I think it's due to the underlying mechanism we use to zip files and we're investigating other ways of zipping.

BrandonMol4
New Contributor II

We had the same thing. It has to write everything, then read it all and then write it all again to zip it. On top of that, Azure files performs particularly poorly with regard to file enumeration/listing. There is no real reason to use AzFiles as the temp/staging location anyway. Just use local disk (relative to the webgisdr tools executables, but shared to your other relevant machines) for that and then write to AzFiles in the end. You don't need premium storage for that.

0 Kudos