topic Re: Configuring ArcSDE HA in Data Management Questions

Configuring ArcSDE HA

Anonymous User — Fri, 27 May 2011 09:29:37 GMT

Original User: tariqchpk75

I want to configure/manage ArcSDE failover. I have read an article found at

www.lincoln.ne.gov/city/pworks/gis/pdf/arcsde.pdf

As per this there should be two separate oracle server sharing common data stores.

My question is if we will have the replica of entire geodatabase or it will be the same single geodatabase for both oracle servers.

Any kind of assistance n help is appreciated in advance.

Re: Configuring ArcSDE HA

Anonymous User — Fri, 27 May 2011 10:59:36 GMT

Original User: vangelo

There isn't any failover capability in ArcSDE (as that seven year old PDF correctly states).

Oracle's high availability enviroment is managed by RAC (Real Application Cluster). Install/
config of RAC is outside the scope of the ArcGIS forums, but the generic description is correct --
there are two (or more) servers accessing one external disk store, which is stored in common
for active/passive or active/active use.

Oracle also has a feature called Data Guard, which permits log shipping from a primary database
to have active/passive or active/active databases with two (or more) disk stores. More exotic forms
can include Data Guarding RAC nodes (many servers, many disk stores).

Since that whitepaper was released, ArcGIS has introduced its own replication capability, which
is captured at the ArcGIS level and doesn't use ArcSDE (beyond versioning).

- V

Re: Configuring ArcSDE HA

MohammadIshfaq — Sun, 29 May 2011 10:45:51 GMT

Hello,
I have worked in setting up a High Availability environment where oracle and arcsde are configured on Microsoft Failover Cluster Manager and Oracle Failsafe. ArcSDE is configured as a generic service and with the failover it resumes on same database instance on another server with NAS architecture. So, I hope it answers your question if you are asking that it is the same single ArcSDE instance for both oracle servers.

Re: Configuring ArcSDE HA

Anonymous User — Mon, 30 May 2011 06:42:14 GMT

Original User: tariqchpk75

In my case data is located on SAN, where as Oracle 11g R2 64 bits is on another physical machine, now presently I have an ArcSDE machine connected with geodatabase in service mode, what I want to do is Have another ArcSDE machine which should have a service to provide connection to same geodatabase in case of any failure of 1st ArcSDE machine using same oracle server.

Re: Configuring ArcSDE HA

MohammadIshfaq — Mon, 30 May 2011 07:52:43 GMT

Well, it is doable. First you need to do to setup Microsoft Failover Cluster Manager and setup a virtual host in it. Then you need to install Oracle database, Oracle Failsafe and ArcSDE on both node machines which you want to be acting as failover nodes (don't execute the post installation for ArcSDE at this point). Then You setup the oracle database instance against which you need to run ArcSDE (Run post installation for ArcSDE here). Now you do the configuration for the other node specifically to share database instance and ArcSDE service. You can test the failover from the Microsoft Failover Cluster Manager.

Re: Configuring ArcSDE HA

Anonymous User — Mon, 30 May 2011 12:54:58 GMT

Original User: vangelo

Adding more ArcSDE servers only adds additional single points of failure (reducing reliability).
Instead, use Direct Connect to connect tot the RAC node directly, and remove the application
server from the equation.

Each session that was active when the Oracle instance fails over would need to be restarted
(and restarted again when the primary returns), but that would be the case with multiple
ArcSDE servers anyway.

- V

Re: Configuring ArcSDE HA

Anonymous User — Tue, 31 May 2011 04:14:40 GMT

Original User: tariqchpk75

Thanks for your replies. Sir Muhammad Ishfaq if I can get your contact number. It will be a favor for me so that I can clear my doubts more easily.

Possible architecture of my application is shown in the attached diagram suggest me how to manage SDE HA in this architecture also suggest me if there is any change required in design.

Re: Configuring ArcSDE HA

MohammadIshfaq — Tue, 31 May 2011 06:39:30 GMT

I am glade if my answer helped you in any way. Please share your email contact and I would respond to you back.

Re: Configuring ArcSDE HA

Anonymous User — Tue, 31 May 2011 08:03:24 GMT

Original User: tariqchpk75

Please see the diagram and consider my requirement and then guide me.

My email address is

tariqchpk@hotmail.com

Thanks for cooperation.

Re: Configuring ArcSDE HA

VinceAngelo — Wed, 01 Jun 2011 09:44:38 GMT

I worked on a project where we had to evaluate the relative performance of a cluster
of VMs in a similar configuration with a single equivalent physical server. The exact
details were under non-disclosure, but we found that a physical server out-performed
a cluster of SOCs running on the same host as VMs, generating 40-120% more maps
per minute over a 10+ minute test period. The details in our test report were sufficiently
glaring that there was no objection when we requested variance from the "all systems
shall be VMs" facility configuration requirement. We made use of dozens of VMs on that
project, but not where production use of ArcGIS Server or ArcSDE was involved.

- V

Re: Configuring ArcSDE HA

Anonymous User — Thu, 02 Jun 2011 08:07:10 GMT

Original User: tariqchpk75

Thanks for reply.

You have raised performance issue regarding virtualized environment. I need to clarify one thing what were RAM and processors specifications of the hosing servers. As I think low performance should be attributed to the hardware resources employed but not to the virtualiztion itself. What do you say?

Re: Configuring ArcSDE HA

VinceAngelo — Thu, 02 Jun 2011 10:58:09 GMT

It's difficult to blame RAM or CPU when two identical hosts are provided for testing. We
spent three weeks in the test lab and, based on the results that we had to present before
an engineering review board, threw out the load-balanced 8 virtual server architecture
in favor of a single physical server with a virtual failover host, and eventually threw out
the failover host because it was unreliable with the needed fibrechannel drivers.

By definition, virtualization has a cost associated with it (I call it the "virtualization tax").
One of the prime benefits of virtualization is that it lets you steal CPU cycles from unbusy
systems in order to reduce the physical host footprint. Yet when the systems are NOT unbusy,
and especially when the busyness is I/O related, there is a greater cost. GIS applications
easily have the highest I/O demand in the business marketplace. Virtualizing production
GIS servers causes them to have the highest taxes. You won't notice when the systems
aren't under heavy load, but when they are, you can't help but notice it.

I had VM administrators pleading with me to speed up physical deployment of an ArcSDE
server because the four virtual servers it was replacing were increasing the data center's
demand numbers to the point where it was endangering their contract performance metrics
(when deployed, the ArcSDE connect time [with scores of concurrent users and thousands
of feature classes] dropped from 40-120 minutes to 5 minutes). There was nothing wrong
with the RAM or CPUs involved, and only two of the servers were really busy. That same
physical server now services 3-4 times the client load of the four virtual hosts, and still has
a similar connection time (with 30% more tables).

- V

Re: Configuring ArcSDE HA

Anonymous User — Fri, 03 Jun 2011 04:44:07 GMT

Original User: tariqchpk75

Can you please elaborate it with the help of diagram that what was your architecture for both the tests (virtual and physical). what arcgis server configuration you used (single som, multiple socs, arcsde failover etc).

Moreover arcgis server is 32 bit. if it is installed on a physical server having 32gb ram, will it be able to use all the ram? virtual architectures are io intensive. did you use the same number of ethernet interfaces for both configurations.Plz post the diagrams.

Regards

Re: Configuring ArcSDE HA

VinceAngelo — Fri, 03 Jun 2011 11:04:38 GMT

As stated previously, the project was done under non-disclosure. The three-member test team
had a combined fifty years of GIS/IT experience, twenty years of rigorous test experience, and
more than ten years with the VM application in use. We were fully aware of the issues and did
our best to optimize VM performance with the hardware previously selected for the project. We
had to present our results to an engineering review board that *really* wanted us to select VMs
for all components. In the end, our results were consistent with Esri support policy on virtual
architecture -- Works okay in test environments, but not recommended for production use.

- V

Re: Configuring ArcSDE HA

Anonymous User — Mon, 06 Jun 2011 11:44:45 GMT

Original User: mishfaq

Thanks Vangelo for the very informative reply. However, I would like to mention here that when we talk about Failover Manager implementation so it is not true virtualization rather you are working with actual system physical resources. As per my understanding and after my discussion with our system administrator, the virtual host on top of your cluster nodes manages control from one node to another. To simply say, it is not similar to virtualization offered by Microsoft Hyper V or VMWare. Please correct me if you have better information as I am not master of this subject.

Your comment stay true regarding the hardware architecture as mentioned by Tariq. Though, I also got a comment saying that virtualization environment/tools are improved with time. So, can you share vintage of this experiment about which shared statistics.

Re: Configuring ArcSDE HA

VinceAngelo — Mon, 06 Jun 2011 12:21:01 GMT

Esri and VMware have a Deployment Guide which contains performance statistics from 2009. Our
disk options were not as performant, and the result was reflected in our evaluation. The ArcGIS
Server Blog has guidelines for ArcGIS Server virtualization.

The way to get higher availablility out of ArcSDE is to use Direct Connect (the MTBF for an unused
component is infinity). A failover configuration for the application server should only be considered
as a short term solution to get all your clients to the same release (or at least to those for which
DC is an option) [and, IMHO, probably isn't worth the bother -- our failover provisions caused more
downtime than any other cause].

- V

Re: Configuring ArcSDE HA

Anonymous User — Wed, 08 Jun 2011 06:09:10 GMT

Original User: mishfaq

Can you kindly explain how can we achieve High Availability using direct connect? Please share if you have any literature or reference URL in this regard.

Re: Configuring ArcSDE HA

VinceAngelo — Wed, 08 Jun 2011 11:24:24 GMT

Mean time between failures (MTBF) is a key component in availability calculation. If you remove
an unnecessary component, you remove any possibility of its failure, which increases reliability.

Using Direct Connect to Oracle will not provide high availability without RAC, but it will provide
higher availability than a redundant ArcSDE application server could, and it does it without
the complexity of a redundant system for a component that isn't designed to support it.

- V