Requests via the proxy fail after first attempt -- something with cache control

2052
4
08-14-2018 05:15 PM
FredSpataro
Occasional Contributor III

Hi All, 

I've got an odd issue that I can easily reproduce in a specific environment.  It's slightly complicated to explain ...

Short version: JavaScript API requests to a map service (legend, query and export) fail after the first successful attempt.  The response is a 504.  Using fiddler to trace these request, I've narrowed it down to the IF-NONE-MATCH header in the request.  Using fiddler, I reproduce the failures, then remove this header value and the requests will process successfully.  It appears to me that some interaction between the proxy, firewalls, arcgis server, etc are having a mis-communication for these request that would probably result in a 304 type response.  

I don't control the network, firewall environment so I might be missing some details but this is my view of the setup:

INTERNET

         |

        V

PUBLIC FIREWALL

         |

        V

LOAD BALANCER 

         |

        V

DMZ WEB SERVER (x2 i think)

   * web app/site + proxy

   * arcgis web adapter site 

         |

        V

INTERNAL FIREWALL 

         |

        V

ARCGIS SERVER (v10.5)

   * secure map service

Here's the sequence of events I've traced so far and can reliably and consistently reproduce:

1. Start with a cleared browser cache or in-private browser window

2. Load web app/site in browser..

   * all the web site resources (html, css, js etc) load correctly

   * AGS REST API requests kick off and are successful:

      * https://{mywebserver/site}/proxy.ashx?https://{myarcgiswebadpater/service}/MapServer/3/query?f=json.....

      * https://{mywebserver/site}/proxy.ashx?https://{myarcgiswebadpater/service}/MapServer/export?f=image.....

      * https://{mywebserver/site}/proxy.ashx?https://{myarcgiswebadpater/service}/MapServer/legend?f=json...

   NOTE: none of these requests have an IF-NONE-MATCH header in the request and all have a ETAG header with a specific value in the response

3. Refresh the web app using the default browser button

   * all the web site resources (html, css, js etc) load correctly

   * AGS REST API requests kick off and all FAIL - status 504 

   NOTE: in each request there is now a IF-NONE-MATCH header in the request with a value that corresponds to the ETAG from the equivalent request in step #2.  When I trap these requests in Fiddler and remove the IF-NONE-MATCH header, they'll execute successfully.  

A few more details: 

* Step #3 will continue to fail until a full browser cache clear or new in-private window...obvisouly since the caching is reset.  It does appear to reset after some minutes interval but I haven't narrowed down the time window.  

* When I run the website from my local dev machine or test server (web app and proxy) directed to the live map service, the refreshes will correctly load, the request headers are identical and the responses have 304 status codes. 

This seems like normal browser/internet caching stuff and it works correctly from an outside web server so I'm guessing something in the hosting environment isn't configured correctly to rematch these caching requests? 

Anyone have suggestions on what settings or tweaks I can pass along to the network ops staff that might remedy the current situation? 

Thanks!!! 

Tags (2)
0 Kudos
4 Replies
FredSpataro
Occasional Contributor III

Little more testing details:

  • We configured the "proxy logging" with verbose level...
    • from the log it appears the traffic makes it all the way through the sequence of the proxy... I see the requests come into the proxy and appear to be returned from the map service as 304 (logged as "server error, response 304")...
    • This seems to indicate that the proxy is working correctly but something at the public firewall/load balancer is killing the requests on the way back out.
0 Kudos
FredSpataro
Occasional Contributor III

Looks like it's an issue within the Esri proxy resource code.  See this github issue ticket: 

304 Responses should not contain a body · Issue #488 · Esri/resource-proxy · GitHub 

Essentially the firewall/load balance clips the body of the 304 response as per the spec but since the ags/proxy were returning a body ...

0 Kudos
KirkChesla
New Contributor III

Hi Fred,

We are having the same issues with 304 Not Loaded on our public web services which is causing end users in AGOL and other configurable apps not be be able to use our services.  We basically have the same server site configuration as you.

The question we have, where does the fix you outline above go...on the load balancer configuration or does it go in the javascript on the client side?  Or in the arcgis proxy config on our server site?

0 Kudos
FredSpataro
Occasional Contributor III

Hi Kirk:

The fix goes in the proxy.ashx code.  A pull request was already accepted and merged:

Merge pull request #503 from sardeenz/sardeenz/#448 · Esri/resource-proxy@74725a6 · GitHub 

You should be able to pull the latest version of the proxy from github, deploy and see proper results come thru.  

Fred

0 Kudos