Linux or Windows server?
This will likely depend on many outside factors, like what are you looking for, what platform do you have set up already, what are you (or your IT group) familiar with, etc.
Here is a thread that discusses some pros/cons here:
Linux versus windows ArcGIS install
You also need to consider what backend database you will be using. Some minor discussion of that is here:
What is the best database for arcgis? Oracle, Postgres, SQL Server, other ..?
Retrieving data ...