Increase Enterprise Environment Speed

CodyPatterson

Hey all,

I'm wanting to increase the speed of our Enterprise environment, but I'm having a decently hard time figuring out where I'm hitting a bottleneck, but I have some ideas.

My environment:

ArcGIS Server - 8 Cores 2.30GHz, 32GB Ram, 500GB Virtual SCSI, 3.6GB pipe
ArcGIS Portal - 4 Cores 2.30 GHz, 16GB Ram, 500GB Virtual SCSI, 3.6GB pipe
ArcGIS Datastore - 2 Cores 2.30 GHz, 16GB Ram, 500GB Virtual SCSI, 3.6GB pipe
ArcGIS Notebook Server - 2 Cores 2.30 GHz, 8GB Ram, 500GB Virtual SCSI, 3.6GB pipe

We've had to make some adjustments, but this is what we're running with now. We have around 40 concurrent users during regular business hours, and rising to upwards of 80 on especially busy days.

All of these servers are hosted through VMware.

I've not noticed any extreme issues with Monitor, the most I've seen is that when we were attempting to integrate a load balancer, the issues exponentially rose, so we removed that and everything returned to normal.

The current issues are: Commits take around 10-20 seconds a piece, normally loading for that amount of time and with hundreds to thousands of commits needing made a day, this is a substantial amount of time to wait.

What should we upgrade/do next? I imagine a load balancer correctly configured may help the issues, but I'm not sure.

Thanks in advance!

Cody

ryanuthoff

I'd recommend using Esri's System Log Parser to get some more granular results than what ArcGIS Monitor provides. There are a lot of rabbit holes you can go down when trying to figure out what to do next. I'm of the opinion that adding resources should be the last option to do after we've explored all other options first. 

You didn't include your DB servers (assuming you're using some type of DBMS and not solely using ArcGIS Data Store). Your DB server will absolutely have an impact on your Enterprise performance. If it is maxed out on resources, then that's probably your problem. But it might not be your entire DB server, it could be some inefficient DBs and/or tables.

That is the beauty of System Log Parser. It will give you statistics down to the individual layer level on the average response time, number of requests, etc. Hypothetically, for a feature service, you might see 1-2 second response times for all layers except for one, which takes 15 seconds. With that information, you can target that layer (and underlying table in the DB) for optimization. Also, at the feature service level, it will also give you wait time, which is the amount of time is spent waiting for a feature service instance to become available to process requests because the max number of instances has been reached. You can also use wait time to potentially increase the maximum number of instances allocated to a feature service.

You also didn't include your server resource usage. Are all your servers resources alright (like CPU/RAM)? Or are they maxing out when your issues are happening? If resources are maxed out, that could be a sign that that is where you're hitting your limit and you need to increase your resources on the server machines.

There's definitely a lot of things to check. But in my experience, adding resources is often not the "correct" solution, and instead, it's poorly optimized DBs, tables, etc.