There's a whole host of factors here. Let me illustrate. I once worked on a project where I wasn't able to touch the database, it was managed, and well outside my sphere of influence. We put a good sized ArcGIS Server in place for development and experienced the issues you're describing. We tried every tweak in the book to try and get some good performance from that ArcGIS Server. Nothing worked.
In the end I created a Query Layer in ArcMap (it's a while ago) and asked for a single polygon to be returned to me, using a point in polygon SQL statement. It took 16 seconds.
The project team said to me, "see, it doens't matter what Esri product you use, it's slow". So I gave them the SQL Statement and asked them to run it from a SQL Server Management Studio equivalent. It took 15.5 seconds. No Esri tech.
When they optimised the database, ArcGIS Server started going fast!
Ultimately when I implement ArcGIS Server, a common non-functional requirement is that it must be as fast as Google Maps (or faster). If the data source is slow you will never do it!
I would expect map requests to (typically) draw sub-second. If you give it a data source that takes 16 seconds, instead of 0.5 seconds, then you sort of need to ask yourself, do I want my server to produce 32 maps in that 16 second period, or just one. If I do need to produce 32, then do I need 32 instances/services.
In the above example, the database server was in the same data centre as the ArcGIS Server, but I've seen optimised databases in remote data centres, and seen the resultant latency impacts on performance.
So from what you've shown us, I think there is work to do on the backend database, rather than the ArcGIS Server. It would appear to be timing out because the DB is slow. The impact of that is that you're delaying production of other maps as a result, think of it as a multi-tenanted environment and one tenant has just bottlenecked it for everyone else.
Scott Tansley
https://www.linkedin.com/in/scotttansley/
