Suggested Hardware Upgrades

[CodyPatterson]

Hey all,

Our Enterprise environment has been running well for a while, but due to an increase of users, the site is starting to struggle.

ArcGIS Portal has started showing signs of growing pains, the feature layers typically take upwards of 2 minutes to load fully, then lag when the view is moved around. These are pretty large feature layers, with quite a lot of symbology though.

In terms of hardware upgrades, what should be upgraded? CPU, RAM, etc. Along with this, what servers should receive these upgrades?

I have ArcGIS Monitor installed on all servers, ArcGIS Server will regularly top out at 100% CPU usage, this is after an additional two cores were added in. ArcGIS Portal is relatively tame, never really topping out, but actively spiking.

TLDR; What component should I upgrade on what sever.

Server: 16GB RAM, 2 sockets 4 cores 2.30GHz

Portal: 16GB RAM, 1 socket 2 cores 2.30Ghz

SQL: 32GB RAM, 1 socket 2 cores 2.30GHz

Thanks in advance!

Cody

[George_Thompson]

Not an architect, but many people have at least 4 cores on Portal and 8 cores on Server. I would probably go higher than that if a production system. As you mention, using Monitor to help validate the changes.

For SQL Server, I would think at least 4 cores for a production system. But this would be dependent on the load on the SQL machines.

Is your Datastore on the Server machine?

[CodyPatterson]

Hey @George_Thompson 

Thank you for the response, these all would be a production system, so I'll definitely look into applying some increases. The datastore would be located on a separate server rather than being on the same server.

Would the datastore possibly hit some bottlenecks while being in this configuration?

Cody

[ThomasHoman]

Hi Cody,

Every result is different and is highly dependent on your environment. Have you looked through the Architecture Center's Performance and Scalability section - https://architecture.arcgis.com/en/framework/architecture-practices/performance-and-scalability/over... 

Portal is likely not your culprit unless the OS is grumpy. It is mostly the pass-through for requests.

Server - Start looking here - Every layer service wants to consume a processor and memory. If you have many layers with complex symbology it has to work hard do deliver the result and really needs the extra horsepower. For example my 10.9.1 ArcGIS server is running 2 sockets/16 cores for 32 processors.

SQL - Again will depend, especially if that is what is hosting the Data Store that the layers are dynamically building from.

You don't mention the network interconnect between Server and SQL but it can be a concern. If you are pulling data dynamically from SQL through an effectively small pipe then the server could be in a position of just standing around waiting for data.

To make things simpler, is there a way to push some of the more background stuff into tile layers so stuff is already rendered and just has to be displayed? This can unload the server running at 100% tremendously.

Just some quick thoughts.

Tom

[CodyPatterson]

Hey @ThomasHoman 

Thank you for your reply and the resources! I'll look into the link more as I've already spotted some useful information.

I'll look into the additional core allocations for the server, these do have complex symbology so displaying them may cause the server to come to a halt. Your server sounds like a powerhouse!

The network interconnect may lack, I'll have to check out the specific speeds that the server is operating at to see if there's a significant bottleneck, personally I would like a 10Gb connection at least, hopefully I don't find out it's just 100Mb or something similar! This could definitely be the culprit.

I've not pushed items over to the tile layers, I'll have to check that out to see if I can get those working, many people use complex layers constantly, so preloading and keeping a render may be useful!

I'll check out what's all happening and return with results!

Thank you,

Cody

[CodyPatterson]

Hey @ThomasHoman  and @George_Thompson 

Might as well update both, the speed of the transfers from the Data Store to ArcGIS Server are around 2.6Gbps so I don't believe that is the hold.

I believe the consensus so far is that increasing the CPU resources would be the best move from here?

Cody

[berniejconnors]

Nobody else commented on ArcGIS Monitor being installed on all 3 servers.  Isn't that overkill?  I suspect Monitor can be installed on just one server and still collect stats on all of the servers.  We are not using Monitor.

Bernie.

[CodyPatterson]

Hey @berniejconnors 

ArcGIS Monitor consists of the ArcGIS Monitor Server and ArcGIS Monitor Agents, the server is the host that contains and writes information to the database while acting as a hub for the information. The other servers have the ArcGIS Monitor Agent installed on them to monitor their systems and return information back to the Monitor Server.

I've had no issues so far in this configuration, standalone monitoring that took place before the installation has not found any noticeable changes in performance after these installations.

Cody

[berniejconnors]

@CodyPatterson That makes more sense! My reading of your original post made me think you had 3 complete installs of ArcGIS Monitor.