You just need to authorize the software
Authorizing ArcGIS for Server—Installation Guides (10.4.1) | ArcGIS Enterprise
Authorizing ArcGIS Server—Installation Guides (10.5) | ArcGIS Enterprise
With pre-10.5, it is installed on the same machine as Server. With 10.5.x you can install it on a separate server if desired.
EDIT: just verified that 10.5 just requires you to go thru the authorization process using the provision file created for Image Server. We are keeping ArcGIS Server and Image Server (and Portal) 10.5 all on one server machine for our testing purposes. After authorizing Server, and Image Server, I was able to create an image service of a mosaic database (using Desktop 10.5).
