High CPU load from lsass.exe related to webadaptors?

[JoëlHempenius3]

My current production setup has two Server2012R2 Virtual machines 4CPU, 32GB RAM with Arcgis Server 10.3.1 installed. The servers are configured with Integrated Windows Authentication.

Recently, a new GIS viewer was released and the number of users increased a lot and this caused performance problems. The arcgis server statistics show 500.000 requests daily. 

The zabbix monitoring showed that lsass.exe on average used 25% of CPU resources. After some digging on the internet I found out this was related to SSL and/or Authentication from IIS and I assumed the webadaptors were causing the high CPU load from lsass.exe.

I therefore moved the webadaptors to two separate virtual machines and the lsass.exe CPU load moved with them, so I was able to solve the performance issue because the main Arcgis Server processes now have more CPU resources available. 

Is it normal to have such CPU load from lsass.exe with 500.000 requests daily? 

Are there ways to reduce the CPU load from lsass.exe?

If somebody has more information on this, please share, because VM's are not cheap within my current organization and I want to make sure I made the right decisions here.

[JacobBoyle412 · Accepted solution]

Hey Joël Hempenius‌,

You're a victim of your own success, the increased usage of the environment you've set up is most likely the root cause. Looking into this further, I'd suggest taking a look at: https://support.microsoft.com/en-us/help/2550044/how-to-troubleshoot-high-lsass-exe-cpu-utilization-... 

I'd also take a look at: https://support.esri.com/en/technical-article/000010869

This seems to align with your symptoms.

[JoëlHempenius3]

Thank you for your response.

I'm not sure whether the linked Microsoft article applies here, because the server I'm using does not have the Active Directory role, but I'll ask server support to look into this. 

The esri support artice looks a bit old where it comes to solving the issue, but it's good to see it's a known issue. 

Can I expect lower lsass.exe usage by upgrading to Windows Server2016 and or Arcgis Server 10.7?

My organization is not yet ready to migrate to Arcgis Enterprise, which will solve this problem when I can remove Integrated Windows Authentication from my webadapters and shift the authentication and authorization part to Portal.

[JacobBoyle412]

It's still worth looking at as IWA authentication can cause this too.

If you switch to ArcGIS Enterprise(Portal), I'd seriously look into whether or not your organization supports a SAML like ADFS or Azure AD.  That would push the authentication burden to the SAML provider and take that burden off the Web Adapter host box.

Let me know if you have any further questions. 

[JoëlHempenius3]

My organization has a Portal with Enterprise login using SAML. It's only used for the apps for a small user group and still on 10.4.1, so it doesn't have the free viewing role users. We're planning on upgrading everything to 10.7, which will hopefully end the talks about the named user costs.