We are running ArcGIS Enterprise 10.8 in Azure behind App GW. We have a base deployment and a federated GIS Server. Both GIS servers have contexts of /server (base deployment) and /server1 (federted GIS server).
Server1 has a dynamic map service with imagery which is cached.
When managing offline areas and attempting to create a offline map service image cache, we can see the output map cache get created in the e:\arcgisserver\directories\arcgisoutput\aerialmapimageservice directory (on server1).
However the packaging fails. We see this in the portal logs:
Item 'd4ff17894c764c4ea5c6bab170dc38d9' added from URL 'https://.../arcgis/rest/directories/arcgisoutput/../Aerial_Imagery_MapServer/_ags1282724663_ch/Map.tpk?token='.
Failed to add item 'd4ff17894c764c4ea5c6bab170dc38d9' from URL 'https://.../arcgis/rest/directories/arcgisoutput/.../Aerial_Imagery_MapServer/_ags1282724663_ch/Map.tpk?token=b4sE98g0vDiOCWgiftcu6F7ieOqL4KiOZhPdqxsgPZMm6whuBnpKfVt_oiBD-zlFvqOipQ9rmkb4ZxY09u6G949kc-NmNT4hSYuY5nqo8Qg3rbz4A2yMCBocARuSKaTV3XK3kMwQQ2N6rzX_XnrLVHmz-mGMxplESHZh6w5-In6-WZw28NGoTMv8jejoFSHJo7dNQiMDloJZgOUyrwVoLbkqdkkocx-L-J98UvdFmf7jG9zbhkvCKdWuwIDMT30C'. Data length is zero
Which is expected as the server context is /server1/ not /arcgis/
Following the requests going into the submitJob for the SetUpMapArea everything looks right and the outputs in the jobs directory (result etc) all have the right urls in it. So it looks like the Offline packaging tools are creating everything as expected but something somewhere is writing arcgis instead of portal.
Has anyone had a similar issue?
EDIT: So it looks like there is a bug in the REST API
The tpk works for the map service and is available in the output directory - it looks like the wrong url is returned by a call to the REST API.
When this URL is called
the API returns this:
folders: [ ],
So the response being written seems to be ignoring the WebContextURL and defaulting to ArcGIS.
This is easily patched in the Offline Areas tools but can't be patched for collector as Collector calls that url directly.
Bruce Harold - Seen this before Bruce? I'm assuming the calls to /arcgisoutput/someservice/..MapServer/_ags<ran>_ch are handled by the API
Solved! Go to Solution.
If it's only OfflinePackaging that's a problem, then it does seem like it's something specific in that workflow. Are you using a web adaptor as well? I'm not familiar with that workflow enough to know what the traffic should look like, but it seems as though Server will tell Portal what URL to use in the item when it's creating it. Does the admin URL use "arcgis", or do they both use "server1"?
It's just not offline packaging but also Collector when trying to download a map for offline - as this also calls the https://myurl/server1/rest/directories/arcgisoutput/service/Aerial_Imagery_MapServer/_ags1066105292_...f=json url to get the tpk for the cached map service.
The deployment pattern is pretty much what Cloud Builder 10.8 (Azure) uses.
Portal web context url and private portal url are the same - pointing to external dns
Federation is external dns of both servers (hosted server and GIS server)
Security config for the GIS servers has external DNS settings for portalUrl, privatePortalUrl. serverUrl and privatehostingserverUrl
not a /arcgis/ to be seen anywhere in the configuration of any of the 3 servers.
Web context url set correctly on both GIS servers.
Collector is probably looking at the body of the request, and the body contains the URL to download whatever it needs. If specific workflows are the only ones causing a problem and the rest of the workflows work as expected, then it looks like it could be a bug with those tools and utilities. I'd suggest reaching out to Support and have them test to see if they can repro.
I have found a solution to this. I kept circling back to the web adaptors as this was the only difference between a site that the offline packaging was working and the Azure site which wasn't. The App gateway sets X-Forwarded-Host as described here: Configure your portal to use a reverse proxy server—Portal for ArcGIS (10.8) | Documentation for Arc... - however when looking at the web adaptor you can see that it sets further custom headers when handling requests mainly:
I added these to my App GW and the offline packaging began working - including being able to create offline areas in Collector. Note I could see the response from the GIS Server change from something like
So it looks like the GIS Server uses the X-Forwarded-Request-Context to write the response rather than the WebContextURL system setting. It appears that if the X-Forwarded-Request-Context header isn't set then it falls back to /arcgis/ - I don't know if this is intended or not but - I had a quick look through the online documentation and couldn't find these headers documented. I naturally assumed that the GIS Server would use the WebContextURL setting for any responses (including responses in GP results or API calls) before any custom headers as not all users use web adaptors - and in the Esri Azure cloud automation tools web adaptors aren't used - and these additional headers aren't set in the App GW configuration (at 10.8 anyway).
Bruce Harold - looping you in as you were tagged in the original post
I can repro this behavior; I suggest you reach out to Support to get this tested and logged. Server should only need to depend on the X-Forwarded-Host header, as documented in the help.