I agree Michael's comment above. You ould have 100 SQL Server instances, that are used intensively or sporadically, that will impact the CPU usage, so lots of 'use' expect to need more cores.
Each datastore, is going to need at least 1 Map Service/Map Image Layer to be published, and this will be the limiting factor. I typically work to 100 services per ArcGIS Server, as an arbitratry rule of thumb, BUT you need RAM to support that number. Each single instance of a SOC consumes 100-200MB. So times that by 100 services, and you're going to need a fair amount of RAM if those services are published as dedicated.
You may want to look to use Shared Service Instances instead of dedicated to give the server sufficient headroom to run the services your suggesting, and you may need to be prepared to have more than one ArcGIS Server if there's a lot of usage.
Final thought. Have you got the Enterprise Portal, Server, and ArcGIS Data Store all running on one 4-CPU host? If so then that would be quite constrained.
Scott Tansley
https://www.linkedin.com/in/scotttansley/