A few quick questions:
a. Since you mention .mxd, I assume you're using ArcMap and not PRO?
b. Are the layers hosted feature services or published directly to ArcGIS Server?
c. Is it possible to install and run a tool like Fiddler2 on your client machine, and while Fiddler is running check the traffic? My guess is that each layer is responding to some request.
There are a couple of potential culprits. This could be an issue with the geodatabase/datastore, or potentially network related, but my bet is on some query being executed against every layer in the map doc not being quickly serviced.
a. Currently using ArcMap 10.6
b. We're bring in the layers through the "database connections"
c. I will be able to try this Wednesday and update our findings.
Concerning the last piece of information, can you elaborate on the query's not being executed against every layer in the map? Does this have to do with the network firewall possibly blocking certain access request?