Advice on Multi-tier ArcGIS Server installation

[KennethLohr]

Trying to make the jump to 10.3 from 10.0 and at the same time scaling up the system, I'm looking for some guidance on best practices for a moderate sized, tiered ArcGIS Server/ArcSDE installation..  Requiring performance balancing and failover, Esri has recommended using ArcGIS Server's built in clustering/load balancing to meet thhose requirements for ArcGIS Server.  Essentially, just having multiple servers participate in the site, and the Web Adaptor would decide which servers would facilitate completion of the requests should a node drop out or be busy processing other requests.  That was great, because I was not excited about stepping into the realm of clustered Windows servers.  For redundancy on ArcSDE, they recommended just placing the databases on a SAN to satisfy performance and availability.  This is only somewhat acceptable I think, since the high availability of the database storage space is not going to stop the mapping services that reference ArcSDE geodatabases from failing, should the single instance of SQL Server go down.  Budgetary constraints could possibly force me to accept this, though.

I'm not even to the point of addressing the ArcSDE redundancy issues yet.  Right now, I'm trying to figure out the best way to plan the ArcGIS Server site.  I have little to no experience with SAN environments, and the only experience I have is poking around in the help documentation for our current production SAN, and searching online forums for solutions.  I was able to get some storage configured from the SAN and visible and writable to the first ArcGIS Server box via an iSCSI connection.  Knowing that all the ArcGIS Server machines need to access the same storage repository, I then learned about the data corruption issues that can result from multiple servers accessing the same LUN.  This also explained why when I mounted the storage on a second server, it would only mount as read-only!  I've seen ideas like sharing the LUN at the Windows level but I don't like that solution because when that box providing the share goes down...well, the data might as well just be locally shared.

Our SAN environment is an EMC Clariion system which I'm currently working with, and the administrator who knew anything about it is no longer with the organization, unfortunately.  I'm poking around in it, figuring out how things are set up.  Sitting idle, ready to deploy is an IBM Storewize system.   I think the Clariion system has some life left in it, and I don't mind setting up the solution there, unless requirements of the ArcGIS Server system are going to prevent it from being feasible, technically or financially.  I think I've read that EMC has a licensed software solution to solve the multiple host issue, but I'm not clear on which software it is, or if it's even the silver bullet.

I've got a fair amount of resources to throw at this solution.  At my disposal are:

• 2x Dell R420 Servers, 32GB RAM, Dual 6 core Xeon E5-2430v2 (not in production)
• Dell R410 Server, 8GB RAM, Single Xeon 4 core X5667 (not in production)
• Dell R710 Server, 16GB RAM, Single Xeon 4 core X5667(partially in production)
• Dell R300 Server, 4GB RAM, Single Xeon 4 core X3323 (production web server, planned to house 10.3 Web Adaptor)
• Dell PE1950 Server, 8GB RAM, Single Xeon 4 core X5460 (currently in production at ArcGIS Server 10.0, probably needs retired)
• Dell PE2950 Server, 16GB RAM, Dual Xeon 4 core X5460 (currently in production at ArcSDE 10.0/SQL Server 2005, probably needs retired)
• EMC Clariion SAN
• IBM Storewize SAN

Currently, my tentative plan is to use the two R420 servers as ArcGIS Server Site nodes, the R300 already has the web adaptor (it's currently hosting our CMS website), and the R710 server for SQL Server/ArcSDE.  Maybe the R410 should be the web host/Web Adaptor instead, I'm not sure. Any comments are welcome, including strategy, hardware allocation/resource upgrades, or anything else that would help to sort this out.  I'm also curious as to how virtualization could play a role here.

I realize this is a lot of information to throw out there, I'm mainly providing all that in case someone is able to see something strategically that I'm overlooking.  I'm mainly concerned right now about how to get the ArcGIS Server boxes to share SAN storage, but all insight is welcome.

[MiguelMorgado]

Hi Kenneth,

We at Exprodat Consulting can help you with this if you needed. I will try to help you the most I can.

Basically for the AGS cluster with 10.2.1  10.2.2. or 10.3 it works in this way.

you install the ags servers software in the servers you want and how many you wish. Just create a site in the main principal server. access the ags manager and check that site is working well.

Find a SAN/NAS where you will store in future all your AGS Site configuration and data (if you will copy data to the server for example). Of course the access to this SAN need to be really quick from all AGS servers (ideaaly) and not located on the same rack as the AGS servers.

Talking about Racks we just use VMs and it works fine. The tricky part is the infrastructure sizing and AGS performance which need some technical input to be successful.

Second part of the job is to point the config-store and directories from the first initial site to the SAN/NAS location.

after checking all is working, just run the manager on the others ags configurations and add the ags servers machines to the main ags site! job done.

Of course you will need to have an ags license for each 4 cores.

SDE cluster will depend of the DBMS. We at Exprodat can help you with that as well.

You can read some of my posts in Linked in relation to AGS High availability here:

Cloud - Building High Availability Applications - Week 1 | Luis Miguel Morgado | LinkedI​

Cloud - Building High Availability Applications - Week 2 | Luis Miguel Morgado | LinkedIn

ESRI AGS - Cloud security and final steps | Luis Miguel Morgado | LinkedIn

Contact me on mmorgado@exprodat.com if you have any questions.

miguel

[KennethLohr]

I think what I was missing was an understanding of the elements which would be required for ArcGIS Server to connect to the SAN.  After some research and learning how SAN actually works, it's evident that the SAN space needs to be connected to a single server that will host a Windows share using that SAN storage, and then ArcGIS Server can use that share for its server directories.  Dedicated space could be carved out on the SAN and presented to a single ArcGIS Server, but that does not work for a multiple ArcGIS Server node installation because the SAN itself cannot share LUNs among multiple servers without special software, licensing and other complicated gyrations.  Not to mention, if there is only a single AGS node, the server directories might as well be stored locally anyway.  So, I think the simplest route is just to attach the LUN to another Windows server and let it handle the file sharing for ArcGIS Server.

[KennethLohr]

Attaching SAN storage to a single server makes sense for the machine running only ArcSDE/SQL Server, since that storage does not need to be shared with other systems.

However, can anyone expound on the advantages of having your database storage on a SAN as opposed to local storage?  SAN devices/systems use the same kind of disks and failure protections (RAID5/10) that we would use for local storage, so outside of the bigger picture reasons of consolidating all enterprise storage; is there a real performance or HA benefit to using SAN vs local storage?  Esri recommended SAN instead of the complexity of redundant/clustered/replicating servers, so I'm wondering exactly why that is?

[MiguelMorgado]

Hi Kenneth,

I cannot understand very well your question, but I will try to bring some clarification.

If you just need to configure a SDE instance then use local disks.

If you want to configure SDE in failover or High Availability then configure an SQL cluster where the database is stored in a common place in a SAN (not in the same rack as sql1 or sql2 servers).

The advantages in a failover configuration is that is sql1 servers goes, sql2 server will automatically take over and user will not be disrupted. of course you need to configure your NLB, and have an primary and secondary (active /passive) sql server. Since it use the same database all data is updated and synchronized (not really, just the 2 applications use the same database).

Clear or clear as mud?

M