Community
Select to view content in your preferred language

ArcGIS Server site: poor cache efficiency due to different "etag" header sent according to machine

399
10
2 weeks ago
NicolasGIS
by
Honored Contributor
2 weeks ago

Hello,

When refreshing my web application, I was surprised not to only hit '304 Not Modified' responses when querying my ArcGIS Server services for the layer properties with the following URL:

https://gis.company.com/arcgis/rest/services/General/Data/MapServer/86?f=json&token=Foo

What I noticed is that if you query your ArcGIS Server VM individually:

https://gisVM01.company.com:6443/arcgis/rest/services/General/Data/MapServer/86?f=json&token=Foo

https://gisVM02.company.com:6443/arcgis/rest/services/General/Data/MapServer/86?f=json&token=Foo

and compare the responses, they are identical (to notepad at least) but they both have a different "Etag" header.

NicolasGIS_0-1765724844792.png

As a consequence, cache cannot be leverage.

It looks like ArcGIS Server machine is creating the Etag header by taking into account specific properties to the VM. As I am hitting my servers in a round robin way behind a load balancer, I always have 50% layers responses with 304 Not modified and 200. 

 

Do you observe the same phenomena ?

 

Thanks

10 Replies
JoshuaBixby
by MVP Esteemed Contributor
MVP Esteemed Contributor
2 weeks ago

The way you worded your question, it makes me believe you are not using ArcGIS Web Adaptors, is that correct?

 

0 Kudos
NicolasGIS
by
Honored Contributor
2 weeks ago

Hi,

No, I am using webadaptor on each VM and there is a front load balancer.

Don’t think the issue is related to WebAdaptor as you can replicate when by passing it it.

Hope it is clearer 

 

0 Kudos
JoshuaBixby
by MVP Esteemed Contributor
MVP Esteemed Contributor
2 weeks ago

A quick check on a couple of the multi-machine ArcGIS Server sites that I manage show the same ETAG being generated from different machines in the same site when the same REST API calls are made to the different machines.  Without knowing more about the specific REST API calls being made (is f=json&token=Foo really the complete URL?), I can't offer any additional comments.

0 Kudos
ArchitSrivastava
by
Frequent Contributor
2 weeks ago

Hello @NicolasGIS,

Yes, as per my understanding, this is expected behavior in a multi-machine ArcGIS Server site.

How ArcGIS Server uses ETags

  • ArcGIS Server generates ETag headers per machine.
  • Even when the REST response payload is identical, different machines return different ETags.
  • In a load-balanced setup, this leads to alternating 200 and 304 responses.

When accessing services through the Web Adaptor, this is also seems to be by design. The Web Adaptor distributes requests in a round-robin manner across ArcGIS Server machines. If a request is routed to the same machine where the cached response is still valid and the ETag has not expired, the server returns 304 Not Modified. If it is routed to a different machine, the ETag does not match and a fresh 200 response is returned.

When a request is repeated and the maximum age of the cache has not expired, the browser uses the cached response without sending the request to the server. If the maximum age has expired, the browser must send the request to the server and set an IF-NONE-MATCH header with an associated ETag value corresponding to the response in its cache. ArcGIS Server evaluates the request and uses the ETag value to determine if the response has changed. If the response from the server is different than the copy on the browser, the server sends a new response to the browser. If the response is identical to the copy on the browser, the server alerts the browser to continue to use the response in its cache. (Quoted from Document) 

How to influence caching behavior

  • Rather than relying on ETags, Esri recommends controlling client-side caching using the cacheControlMaxAge property.
  • This defines how long (in seconds) a response can be cached before revalidation, providing predictable behavior in load-balanced environments.

It’s best to test different cacheControlMaxAge values to determine what works best for your usage patterns.

For more details, see the official ArcGIS Enterprise documentation on setting up caches for client consumption: 

https://enterprise.arcgis.com/en/server/latest/publish-services/linux/setup-caches-for-client-consum... 

cacheControlMaxAge property details : https://enterprise.arcgis.com/en/server/latest/publish-services/linux/setup-caches-for-client-consum... 

Hope it helps!

0 Kudos
JoshuaBixby
by MVP Esteemed Contributor
MVP Esteemed Contributor
2 weeks ago

Does your understanding come from documentation, experience, or somewhere else?  If documentation, please share links to specific Esri documentation to back up the information.  If experience, how much have you tested this specific situation?  A quick 5-min check on a couple of the multi-machine ArcGIS Server sites I manage, so the opposite behavior you describe as expected.  If you can share a reproducible test others can try to generate the results you expect, I would be interested.  

0 Kudos
ArchitSrivastava
by
Frequent Contributor
a week ago

Hello @JoshuaBixby ,


This understanding comes primarily from hands-on experience managing multiple ArcGIS Enterprise environments over time. I’ve seen this behavior across several multi-machine ArcGIS Server sites, configured with Web Adaptors and load balancers, using different browsers and client applications. It’s not based on a single observation, but repeated testing while troubleshooting client-side caching and REST performance.

That said, this behavior is not driven by ArcGIS Server alone. It’s the interaction between ArcGIS Server headers, the Web Adaptor’s request routing, and browser caching behavior that produces what you see. The cacheControlMaxAge property works in combination with how the browser honors caching and validation headers.

If someone wants to reproduce and validate this, two simple test scenarios help isolate the behavior (assuming a Web Adaptor–based site):

Test scenario 1: Browser-side caching behavior (isolate browser caching and ETag handling.)

This confirms the browser cache is a contributing factor.

Test scenario 2: ArcGIS Server caching behavior (observe the effect of ArcGIS Server cache headers.)

In a Web Adaptor setup, requests are distributed in a round-robin manner by design. When caching is enabled, hitting the same machine with a valid ETag may return 304, while hitting another machine may return 200. This is expected given machine-specific ETag generation and client-side cache validation.

Different environments, browsers, and request patterns can show slightly different results, which may explain why behavior appears inconsistent across sites.

Keep in mind that changing these caching properties can have a direct impact on service performance. Disabling or reducing client-side caching may cause the service to fetch data or tiles on every request instead of reusing cached responses, which can increase load on ArcGIS Server and lead to slower response times.

0 Kudos
NicolasGIS
by
Honored Contributor
a week ago

Hello ,

Thanks for your reply @ArchitSrivastava.

this is expected behavior in a multi-machine ArcGIS Server site.

> Don't you think it is a pitty ? Shouldn't there be something cleverer that prevent penalizing multi machines setups ? Should an idea be logged or do you think it is pointless ? To me, multi machines setup should be transparent client side specially when responses are identical !

Rather than relying on ETags, Esri recommends controlling client-side caching using thecacheControlMaxAgeproperty.

> The problem is that I configured "cacheControlMaxAge" property on this service and for a reason unknown to me, it does not apply to request in the form of:

https://gis.company.com/arcgis/rest/services/General/Data/MapServer/86?f=json&token=Foo

It does work on 'query' request:

NicolasGIS_0-1766041722260.png

returns:

NicolasGIS_1-1766041736573.png

Good, this request will be cached for 30 minutes.

But the request:

https://gis.company.com/arcgis/rest/services/General/Data/MapServer/86?f=json&token=Foo

even if "cacheControlMaxAge" is configured always returns "max-age=0" and as the 'etag' headers changes depending on machine replying, this is never cached nor getting a 302:

NicolasGIS_2-1766041770532.png

 

Any reason why  @ArchitSrivastava ?

@JoshuaBixby I think it may depend on your service data source. On one service consumming data from SDE, I do have the same reply accross machines but for services referencing a file geodatabase data source, I am getting different Etag Header even if file geodatabase is identical. Might be interesting to know what is actually taken into account to generate this ETag header.

 

 

 

0 Kudos
ArchitSrivastava
by
Frequent Contributor
a week ago

@NicolasGIS 


Thanks for the added screenshots, they help a lot.

Additionally, I agree it’s worth logging an Idea. Multi-machine behavior being more transparent client-side is a reasonable expectation, and even if it’s “by design,” it’s still a good enhancement request.
https://community.esri.com/t5/arcgis-ideas/ct-p/arcgis-ideas 

0 Kudos
JoshuaBixby
by MVP Esteemed Contributor
MVP Esteemed Contributor
a week ago

I have a hard time believing the data source itself is a factor, at least the difference between different types of geodatabases, but it is definitely worth looking into some more.  If I can find some time I will run some additional tests.

Regarding one of my earlier questions, what is the full REST call?  Is f=json&token=Foo really the full URL that is causing changing ETAGs? 

0 Kudos