topic Re: webgisdr not creating backups in us-west-2 region in ArcGIS Enterprise Questions

webgisdr not creating backups in us-west-2 region

GirishYadav1 — Thu, 23 Aug 2018 21:09:31 GMT

Hi,

To fulfill the disaster recovery requirements. I have a deployment of ArcGIS Enterprise 10.6 on us-east-1 and us-west-2 regions. Also setup a cron job to create nightly backups of ArcGIS enterprise using webgisdr utility into the S3 buckets in respective regions. It works just fine in us-east-1 region. But in us-west-2 region it fails with following error:

==================================================

Starting the WebGIS DR utility.

==================================================

Unable to get the S3 endpoint from the specified region us-west-2.

Check to make sure us-west-2 is a valid region.

Exiting the WebGIS DR utility.

I have two S3 buckets deployed in both regions us-east-1 & us-west-2 with cross region replication. In east region ArcGIS Enterprise writes backups to us-east-1 and west region ArcGIS Enterprise deployment writes to us-west-2 region.

From ArcGIS Enterprise EC2 instances in us-west-2 region, I can access s3 buckets through aws s3 cli.

Any thoughts?

Re: webgisdr not creating backups in us-west-2 region

JonathanQuinn — Thu, 23 Aug 2018 22:46:04 GMT

Are you using access keys or an IAMRole? Do you have a regionsforcloudstorage.dat file under <install dir>\framework\etc?

Re: webgisdr not creating backups in us-west-2 region

GirishYadav1 — Fri, 24 Aug 2018 02:33:39 GMT

Using IAMRole. And IAMRole assigned to the instance has full access to S3. However, I DO NOT see "regionsforcloudstorage.dat" file under <portal install dir>/framework/etc. Following are the files inside etc folder:

root@arcgis-ent-qawest-blue:# ls /arcgis/portal/framework/etc/

actions.sh config-store-connection.json

agsportal.sh database-config.properties

arcenv NodeAgentExt.xml

arcgis-framework.properties pids

arcgis-logsettings.json portal-config.properties

arcgis-logsettings.json.rlock portal-ha-config.properties

arcgisportal RestoreJre.txt

arcgisportal.service service-logging.properties

arcgis-rmi.properties

Note: The portal content store is NOT S3. Content store is in EFS.

Re: webgisdr not creating backups in us-west-2 region

JonathanQuinn — Fri, 24 Aug 2018 17:47:52 GMT

Ah, so you're trying to store the backup in S3, not the content? I guess that's irrelevant, it's all the same.

It's fine if you don't see it, we look at that file if we can't get the endpoint for the region you provided. Can the Portal make outbound connections to http://arcgisstore105.s3.amazonaws.com/regionsforcloudstorage.dat? If not, then you may need to download that file and move it into the framework/etc folder.

Re: webgisdr not creating backups in us-west-2 region

GirishYadav1 — Mon, 27 Aug 2018 23:06:02 GMT

No Luck...

The Portal machine can not make outbound connections to http://arcgisstore105.s3.amazonaws.com/regionsforcloudstorage.dat. I downloaded the file and put inside framework/etc folder but still getting the same error.

Also, I am using ArcGIS 10.6 therefore, downloaded http://arcgisstore106.s3.amazonaws.com/regions.dat. put it inside framework/etc. still same error. I also downloaded the http://arcgisstore106.s3.amazonaws.com/endpoints.tar.gz file and put it in the dame directory. Still no success.

Any suggestions?

Re: webgisdr not creating backups in us-west-2 region

GirishYadav1 — Tue, 28 Aug 2018 19:53:01 GMT

After further testing, I found that the Portal machine can access http://arcgisstore105.s3.amazonaws.com/regionsforcloudstorage.dat through proxy. I tested with setting up proxy in the Ec2 and able to wget this url endpoint and it worked. However, The ArcGIS webgisdr is still throwing the same error. Does webgisdr respect the proxy settings in the machine? Is there a separate proxy setting for webgisdr?

Re: webgisdr not creating backups in us-west-2 region

JonathanQuinn — Wed, 29 Aug 2018 17:02:54 GMT

We embed that file in the DR tool and Portal. Placing it there yourself is really meant for users connecting to non-default regions. I was just testing with the us-west-2 region myself and didn't see any problems, so I'm not too sure what's going on. Just to be sure, Site 2 which is in us-west-2 is attempting to save the backup to a bucket in us-west-2?

Re: webgisdr not creating backups in us-west-2 region

GirishYadav1 — Wed, 29 Aug 2018 18:56:40 GMT

Yes, the site in us-west-2 region is attempting to save backup in a bucket in the same region. There is no VPC endpoint for cross region bucket access (i.e. the EC2 in us-west-2 do not have direct access to a bucket hosted in us-east-1 region. Access to the bucket in us-east-1 is only through proxy). Therefore, this url http://arcgisstore105.s3.amazonaws.com/regionsforcloudstorage.dat (from bucket in us-east-1) is only accessible to the ArcGIS Portal machine through proxy. Is this the reason why the webgisdr utility is throwing this error? Is there a way to set proxy for webgisdr?

As a workaround, I can create a script and set it as a cron job to backup locally and upload it to the S3 bucket using aws S3 cli.