ArcGIS Enterprise HA: Tile cache failover

NicolasGIS

Hello,

Setting in an HA architecture, I wonder how HA for a tileCache datastore in standBy mode is supposed to work ? Checking the HA doc

it is stated for primary and standby mode:

```

In primary-standby mode, your tile cache data store contains two machines. Both machines contain the same scene cache data. Scene layers access the cache data on the primary machine. If the primary machine fails, the standby machine becomes the primary machine, and scene layers access the cache data on the new primary machine. This allows continuous access to scene layers while you, as the ArcGIS Data Store administrator, recover or replace the machine that failed.

```

It's not the case for me. After configured HA in primary-standby mode, if I switch off the VM hosting the primary tileCache datastore, then the standy does not pick-up:

Checking the status on ArcGIS Server admin interface after having the VM offline for 15 min, I receive the following error:

```

Could not connect to the ArcGIS component at URL 'https://PORTAL01.COMPANY.COM:2443/arcgis/datastoreadmin/machines/PORTAL02.COMPANY.COM/validate'. The ArcGIS component on that machine may not be running or the machine may not be reachable at this time.Error: connect timed out

```

What I am doing wrong ? 

When trying the option to "makePrimary", I receive as well an error:

```

Server machine 'https://PORTALQA02.COMPANY.COM:2443/arcgis/datastoreadmin/machines/PORTAL01.COMPANY.COM/makePrimary' returned an error. 'Attempt to make 'PORTAL02.COMPANY.COM' the primary data store machine with role 'PRIMARY' is not allowed.'

```

So far, from my experience, I cannot takes offline the primary tileStore cache for maintenance purpose. Any idea ?

With the relational datastore, I found out that editing the property "failover_on_primary_stop" in C:\Program Files\ArcGIS\DataStore\framework\etc\datastore.properties solves this issue but it only works with relational datastore. 

Quid about tileCache ?

Thanks !

@JonathanQuinn maybe ?

 

ChristopherPawlyszyn

I've found in practice that the Tile Cache Data Store behaves the same as the rules outlined for the Relational failover (if the failover_on_primary_stop property is not enabled). With that being the case, this should apply to the failover process:

"The only human-initiated situations that cause a failover are if the primary data store machine is deliberately taken offline, or the ArcGIS Server site administrator runs the makePrimary REST command on the standby machine."
From: https://enterprise.arcgis.com/en/portal/latest/administer/linux/add-standby-machine.htm#:~:text=Failover%20scenarios%20for%20relational%20data%20stores&text=The%20primary%20data%20store%20stops,app%20on%20the%20primary%20machine.

Prior to shutting-down the primary machine, you should be able to promote the standby to primary via the Admin API, then you can patch/restart the previously-primary machine without interruption of your hosted scene layers. If that is not working as described, then we'd need to take a closer look at your particular configuration to get a better idea of what part of the process is breaking-down.