topic Re: SEVERE errors and Issues with Tiles in ArcGIS Enterprise Questions

SEVERE errors and Issues with Tiles

DevinLavigne — Thu, 21 May 2020 15:15:24 GMT

I am getting a ton of SEVERE errors in my server logs -

They are essentially 2 different ones: This exception was thrown after the response was committed. Access to this resource is not allowed - Code 9002 and Response already committed. Cannot forward to error page. - Code 9001.

I went looking in the logs after I was experiencing problems publishing static/pre-generated tiles. It looks like the tiles are generating on my server - I can view the "bundles" in my cache directory but they do not show up in the browser - I only see the basemap. I can click and query the tiles - so even though they aren't visible I can see a popup. If I select generate tiles dynamically it works.

I'm hoping the two issues are related.

Re: SEVERE errors and Issues with Tiles

DevinLavigne — Tue, 09 Jun 2020 14:48:02 GMT

I'm still receiving these errors. Does anyone have any idea what is happening?

Re: SEVERE errors and Issues with Tiles

TownOfGroton — Mon, 13 Jul 2020 13:36:51 GMT

Did you ever figure out this issue? I'm experiencing the same issue.

Re: SEVERE errors and Issues with Tiles

ScottRutzmoser1 — Wed, 26 Aug 2020 18:49:42 GMT

I am having this issue as well, any update on resolution.

Re: SEVERE errors and Issues with Tiles

NiekGoorman1 — Tue, 22 Sep 2020 14:20:35 GMT

I'm seeing the same error messages - a great amount of Severe errors, same as the topic starter. However I'm not doing any tile work, rather we are having issues with users trying to submit responses to a Survey123 form, and one of my colleagues is not able to re-publish the survey. This is on ArcGIS Enterprise 10.7.1, fully patched, spread across three machines (Server/Portal/Data Store), all of which have plenty of resources.

Re: SEVERE errors and Issues with Tiles

KeithOlsen — Wed, 20 Jan 2021 20:23:44 GMT

I'm getting this error as well, but I can't tell what's causing it. Any idea how I can trace the source of this error?

I'm running Enterprise 10.8.1 with server/portal/datastore on different machines; although, I was getting the same error when I was on 10.7.1.

--Keith

Linn County, OR

Re: SEVERE errors and Issues with Tiles

KrystaHouseknecht — Fri, 26 Feb 2021 00:26:34 GMT

Same errors here at Clatsop County. We are using 10.7.1. I can't seem to find any issues that it's causing though, but I'm still not liking these SEVERE ERRORS occurring nearly every minute, every day.

-Krysta Houseknecht

Clatsop County, OR

Re: SEVERE errors and Issues with Tiles

Brian_Wilson — Tue, 02 Mar 2021 22:53:23 GMT

Whatever causes that can't be that SEVERE since it does not seem to affect anything except generating log messages! I looked at it and gave up and opened a support ticket. When I learn anything new I will post it back here for posterity. Hi KeithOlsen!

Re: SEVERE errors and Issues with Tiles

KeithOlsen — Wed, 03 Mar 2021 17:52:06 GMT

Hi Brian! Yeah, it seems silly to post an error that can't be traced. Someone on slack suggested looking at the web adapter machine server logs and see if there were errors there. Haven't done that yet.

Right Krysta? If it's so SEVERE then what is the cost? Performance?

Re: SEVERE errors and Issues with Tiles

Todd_Metzler — Wed, 03 Mar 2021 19:01:10 GMT

Hello,

My environment: ArcGIS Enterprise 10.8.1 Full Stack Windows Server 2012R2.

I get many severe errors every day on my server. Now that I've been using ArcGIS Monitor for about 4 months, and can really analyze performance, I can report that the majority of the 9000 series errors are from consumer requests to use our services and the construct of the queries isn't correct. Rarely do any of these severe 9000 series codes result in catastrophic failure of our ArcGIS Enterprise. What's more, since most of them are getting generated from services I share to the public, we've nearly zero ability to stop the bad requests from coming in. Moral of the story: If your ArcGIS Enterprise is serving the public, then make sure the environment is resourced enough to keep on going despite these bad requests.

Todd

Re: SEVERE errors and Issues with Tiles

VirenderRawat — Mon, 17 Jan 2022 10:35:36 GMT

does any one has solution to fix this error "This exception was thrown after the response was committed. Access to this resource is not allowed" generated in server log

Re: SEVERE errors and Issues with Tiles

JervinAlbeso — Wed, 25 May 2022 06:51:30 GMT

same here, i got tons of it on server logs but not sure on what GP or service is it pointing?

Re: SEVERE errors and Issues with Tiles

Brian_Wilson — Wed, 25 May 2022 15:47:10 GMT

@Todd_Metzler wrote "What's more, since most of them are getting generated from services I share to the public, we've nearly zero ability to stop the bad requests from coming in."

So it sounds like people are being malicious constructing bad queries intentionally? If that's the case running something like fail2ban might help. I see something called "ipban" for Windows, I will look at that.

Basically fail2ban watches log files and when it sees patterns matched it will put a block into your firewall. For example, "Bad Login". It can be set to remove a block after a time delay, so that you don't permanently ban everyone coming from a big service provider like Comcast.

Maybe we can watch for SEVERE coming in from the Internet and block further requests from that IP?

Re: SEVERE errors and Issues with Tiles

Brian_Wilson — Tue, 07 Jun 2022 17:39:56 GMT

Finally I decided to deal with this so I opened a ticket and followed some suggestions from my support tech.

I figured out by checking IIS logs that it's just the app requesting non-existent tiles. The Server logs are useless. The IIS server is returning 404 errors that include the information on the layer name, the ip address etc.

I use a basemap that is an empty tile vector layer with visibility range set to keep it from ever rendering.It turns out that the "map viewer" ignores the visibility range in the basemap and it requests tiles anyway. 100% transparency did not fix it. I had to set the visibility range in the map itself. Once I did that, the browser app stopped asking for tiles and the problem went away.

On our lightly loaded server I was seeing thousands of errors every day and it had no discernible impacts but it made our log files uselessly clogged with meaningless SEVERE messages. The fact that it was not hurting anything created a situation where we were forced to just ignore the log files until finally I had time to address it.

The Esri tech was helpful but constrained by the useless log messages. I wish Esri developers actually had to chase errors when the messages are "a problem happened". Maybe they'd be more sympathetic then?

Re: SEVERE errors and Issues with Tiles

Todd_Metzler — Sun, 21 May 2023 09:57:33 GMT

We did consider a strategy as you've proposed at our Interent domain guard/request aggregator level outside the GIS tier. The level of effort and administrative overhead wasn't worth it.

Re: SEVERE errors and Issues with Tiles

nkiabee — Mon, 05 Jun 2023 10:22:37 GMT

Hi,

we recently discovered that whenever a client making a REST call to a cached map service (https://myserver/arcgis/rest/services/MyService/MapServer/export?bbox=...) and gets a return of 304 code (indicating no change to the content since last accessed), our ArcGIS Server will log the 9001 & 9002 errors. You may want to check if yours is of the same issue.

Re: SEVERE errors and Issues with Tiles

AL_77 — Mon, 05 Aug 2024 20:40:29 GMT

@nkiabee, did you have a solution to this? I am seeing the same.

Re: SEVERE errors and Issues with Tiles

nkiabee — Tue, 06 Aug 2024 03:51:30 GMT

Nope unfortunately. We are just accepting it at least is not a real error.

Re: SEVERE errors and Issues with Tiles

AL_77 — Tue, 06 Aug 2024 11:59:30 GMT

Yeah, us too. Thanks!

Re: SEVERE errors and Issues with Tiles

TimHaverlandNOAA — Thu, 13 Mar 2025 17:30:56 GMT

Hi thank you for this tip. I just confirmed that we are getting the 9001 & 9002 errors when the /export endpoint results in a 304. In addition to these SEVERE errors, with DEBUG on, I'm seeing messages like "RHandler is null, Access to this resource is not allowed" and "403: Access to this resource is not allowed at com.esri.rf.RServlet.service." Did you ever find a way to prevent these errors from occurring?

Btw we are on ArcGIS Enterprise 11.4 running on Ubuntu22.04.