topic Re: Using Aws S3 for cache tiles for arcgis server. in ArcGIS Enterprise Questions

Using Aws S3 for cache tiles for arcgis server.

SnowWong — Wed, 19 Apr 2017 10:49:22 GMT

Hi, anyone faces the issue of unable to use their Aws S3 storage as their cache tiles directory?
I used arcmap to make the cache tiles with the cache tiles inside the Aws EBS.
Everything works fine, i'm able to load the cache tiles for my map service which i published
and the arcgis server manager works fine.
However when i copied the cache tiles into my S3 bucket and changed the directory of the cache from EBS to S3.
It doesn't seems to detect the map cache tiles. Anyone have ideas on what went wrong? Require assistance, thanks in advance.
Link as reference : Cache creation in Amazon EC2—ArcGIS Enterprise on AWS | ArcGIS Enterprise

Re: Using Aws S3 for cache tiles for arcgis server.

RebeccaStrauch__GISP — Wed, 19 Apr 2017 15:25:47 GMT

This may be unrelated, but will mention it since I ran into it (but using local ArcGIS Server, not AWS, etc). with different compact cache formats and different ArcGIS versions. I'm not sure if this could be an issues on your platforms.

If you are using the compact format for your caches, make sure that the versions of the other ArcGIS client software are compatible. I ran into an issue with copying a compact version of a cache created in 10.2.x to the cache folder for a 10.5.x service, and it could not see it (nor could 10.5.x Desktop see it directly).

I had to copy my cache to the proper cache folder, then create the 10.5.x service so I could upgrade the cache format. once I did that I could access/see the cache. With 10.3.1 I could use either compact fomat (.bundle and bundlex .vs the combined .bundle files), but 10.5.x dropped the 10.2.2 format compatibility.

Re: Using Aws S3 for cache tiles for arcgis server.

Rudy_Stricklan — Wed, 19 Apr 2017 15:32:44 GMT

It sounds like the permissions on your S3 bucket aren't being set to allow read access by All Users. By default, there aren't any permissions granted to users beyond the authorized bucket creator.

Re: Using Aws S3 for cache tiles for arcgis server.

SnowWong — Thu, 20 Apr 2017 01:23:38 GMT

Rebecca Strauch, GISP‌
Erms, It's running 10.5 currently for Arcgis Desktop / Arcmap as well as the Ec2 instance which is my Arcgis server launched using the 10.5 Esri Arcgis Server AMI from marketplace. Maybe about the folder directory issue? I followed the esri guide as well as some other guides online, but all seems to not be working.

Rudy Stricklan‌
I granted full permission to my S3 bucket though, for testing purpose i allowed anyone to access it.

I have the following bucket permission though. What else am i missing? I have a feeling is some sort of folder directory issue? But I'm not really sure what went wrong though.

Bucket Policy :
{ "Sid": "", "Effect": "Allow", "Principal": "*", "Action": [ "s3:GetObject" ], "Resource": "arn:aws:s3:::examplebucket/*" } ] }

As well as my CORS configuration :

<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<AllowedHeader>*</AllowedHeader>

Hope that someone is able to help me.

My S3 bucket directory :

My bucket > arcgiscache > my service name folder > Layers > Have all the service files + _alllayers > L (Number)

Re: Using Aws S3 for cache tiles for arcgis server.

AdamDrackley — Tue, 31 Oct 2017 19:53:10 GMT

We're thinking of giving this a try with our cache as well. A quick question for anyone who's tried; Does the tile cache need to be uploaded to S3 as an exploded cache, or can the service properly feed out the .bundle files of a compact cache as well? Does anyone have any steps on how to set up a MapService to reference a cache hosted in S3?

Re: Using Aws S3 for cache tiles for arcgis server.

GarethWalters3 — Sun, 05 Nov 2017 23:30:27 GMT

I am just looking at setting up a bucket for image caches and I see the only document that talks about a policy is the cloud builder doc. Build an ArcGIS Server site on Amazon Web Services using Cloud Builder—ArcGIS Enterprise on AWS | ArcGIS Enterprise

In that they specify the action as "Action": "s3:*", perhaps that will help.

Cheers,

Gareth

Re: Using Aws S3 for cache tiles for arcgis server.

Anonymous User — Mon, 23 Jul 2018 22:19:33 GMT

Maybe investigate BUG-000106057 - I have had just had Esri "support" tell me this is my issue with our cache on S3 not working using export map or WMS calls (although it does work using rest tile calls through the JS API). It seems this doc saying it was addresses at 10.5.1 was premature. https://support.esri.com/en/download/7514

We tried switching to a key which our info sec guys aren't very happy with but that also failed.

Frustrating as now we are so far down the migration we will be swapping out to EBS storage at 10x the price - and we are talking 10-20TB.