ArcGIS Server Directories on EFS in AWS

[DavidHoy]

Hi All,

This question has been asked before, but I thought I would get a refresh of current thinking.

The Esri supplied Cloudformation template for an HA AGS server site with multiple AGS machines utilises an EC2 instance in Autorecovery mode acting as a file share for use by the AGS site for the Server Directories. (with S3 & DynamoDB for configuration)

Like this

But Autorecover only works in a single Availability Zone and if the entire AZ is lost – theoretically your site will die and only be recoverable from any snapshot backups you have configured for the EBS volumes attached to the File Server.

The ESRi Australia Managed Cloud Services team Is using ObjectiveFS across two Linux EC2  (in different AZ) to provide a Samba share that can be used for the Server Directories, but this seems like a lot of config overhead.

There is another Esri supplied pattern illustrated when using AGS in Docker (experimental at 10.5.1 and not recommended for Production) that uses EFS as the storage for Server directories (and config)

https://s3.amazonaws.com/arcgisstore1051/7333/docs/ReadmeECS.html

Like this

My question – and I am asked by clients fairly regularly, is – why don’t we recommend EFS as the HA File Store for Server directories?

I am aware that the Esri  recommendation is to have a file store that provides low latency high volume read/write performance.

Is EFS not fast enough? (that is what I have been telling people up till now). Are there any benchmarks that give some performance comparisons?

Why is it OK to provide an EC2 instance with Autorecover as an alternative HA option when this would fail in the event of an AWS availability zone outage?

And, as a bonus question,

What is the equivalent answer for Azure?

[JoshHevenor1]

I'm just starting to look at this same question and came across your post.   The performance of the EFS looks theoretically better than using EBS on a single VM (which is what I understand the cloudformation template uses). https://docs.aws.amazon.com/efs/latest/ug/performance.html    I plan to spin up both configs as sandboxes and I'll get some rough performance numbers.  Hopefully someone else here can give you a more accurate response, but it's been a couple months so maybe not. 

Also,  Azure (premium) files (File Storage | Microsoft Azure ) would be the equivalent.  I've just starting using those on a different project for all shared folders. The initial impression is that they're holding up fine, but I don't have any hard numbers as of yet.  

[JoshHevenor1]

Following up on this, it looks like AWS EFS is not supported by Windows: https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/AmazonEFS.html 

There's a Windows file share service called FSx that looks like it's a little slower, and more expensive than EFS: https://aws.amazon.com/fsx/windows/?nc=sn&loc=1  It's also not available in my region, which is a deal breaker for me. At the moment that means I'm sticking with the VM file server. 

[DavidHoy]

Thanks for doing the experiment Josh.

Oneof my customers may be shifting their environment to RHEL chiefly to be able to take advantage of EFS

another alternative we have been using is Two File Server VMs ((Linux) with ObjectiveFS installed to make them stay synchronised. Then use Samba to provide a SMB / NFS file share that can be mounted as a drive on each AGS VM.

This seems to be working as fast as a directly attached EBS volume and no latency problems on write operations

[JoshHevenor]

The Linux file server seems like a solid option. I assume it's at least twice the price of what using EFS would be though?  Definitely more setup and maintenance to deal with.  

[DavidHoy]

Hi Josh,

I am not certain, but

As OFS is based on using S3 as its backend storage location, and I think EFS is using EBS - I think OFS actually works out cheaper for a small number of fileserver nodes (that only need to be small EC2 Linux instances).

Of course, you will need to do the maths for your own situation.

[FrankieEast]

SoftNAS Cloud NAS - Cloud File Storage - AWS | Azure | vSphere 

[DavidHoy]

Thanks for the link Frankie,

Do you have a running HA ArcGIS Enterprise site using SoftNAS as shared directories file server?

did you do any objective comparison tests ?

[SteveTaylor5]

Hi David,
We are looking at using FSX as a file share for ArcGIS Server config.  If we were to get it working, would this be a supported deployment?
Also, do you know of anyone using FSX.as a file share.
Thanks

[DavidHoy]

Hi Steve,

sorry I missed the notification that you had posted.
We are trialing the use of FSx for another large site in NSW.
But we are using this only for the ArcGIS Server System directories, still using DynamoDB for ArcGIS Config store.
So far, the simple testing we have done shows no significant performance hits for simple map export and feature query.

So far as I am aware, there is not an "unsupported" flag on using this - the doc only specifies that a file store for system directories must provide high performance read/write and not wait until change is committed across all nodes (which is why you cant use a DFS based system, FSx supports acting as a node within a DFS cluster, but doesn't use DFS concepts itself).

As I am sure you are now aware, all AWS instances need to be part of the same Active Directory domain as the FSx share (we are using AWS Directory Service for this)