504 gateway timeout when printing through utility.arcgis.com
Hi - we have a stand-alone ArcGIS Server (v11.5) that we are hosting for a customer to enable them to do custom prints from ArcGIS Online apps. The print service is set as synchronous (required for printing from Web App Builder apps) and secured with user credentials and these are embedded within an item in the customer's ArcGIS Online account and the item is then shared and used across ArcGIS Online (e.g. in the print widgets etc). This approach has been in place and working fine for a couple of years.
For information - we cannot currently change the print service to asynchronous, because this mode is not supported by older apps (e.g. Web App Builder) that the customer is still using.
Recently the customer has started experiencing failed prints due to intermittent 504 gateway timeouts. We have investigated and found that there are no timeouts happening on the ArcGIS Server (the timeouts are set quite long and the load is minimal) and when a print fails, there are no errors in ArcGIS Server and the output has been generated in the output directory.
We have tested printing directly using the ArcGIS Server URL (user needs to enter credentials each time) and this consistently works without errors. However, we can't use this in production.
Our investigation suggests the timeout is coming from the utility.arcgis.com proxy that is used when using an item with embedded credentials:
The error in the browser also states "Error from CloudFront". CloudFront is not part of our infrastructure, so this error must therefore surely be coming from the utility service proxy.
Further testing has revealed that successul prints tend to complete in <30s, whereas failed prints tend to fail around the 30s mark. I personally believe this indicates there is a timeout setting (possibly 30s) somewhere in the utility.arcgis.com proxy, probably in CloudFront. This would explain why longer prints would fail but there are no errors in ArcGIS Server, because the proxy is timing out before the request is completed in ArcGIS Server.
We have raised a support call but are not getting very far. Can anyone confirm if there is indeed a timeout setting for the utility service proxy if so, what it is set to?
