Yes, Geoportal Server (not Geoportal Toolkit) is the solution for your need.
No, you don't need to use ArcGIS Server Services. But you will have to connect to ArcGIS Online to access some maps for your embedded viewer.
Just to let you know, you can download the OpenSource version of Geoportal Server here: http://sourceforge.net/projects/geoportal/