Hi Gagag,
> I am able to access this ArcGIS Service from ArcMap without any problems. The ArcMap gives the usual complain about the security certificate - but I can add the layer to the map and can query it. It all works perfect.
This is likely because you probably already signed into ArcGIS Server in ArcMap (e.g., you opened a connection to Server from ArcMap to add the service).
> While working in the Portal, when I create a Map and add this service as a layer to it, it only shows up in the map if I have logged into the ArcGIS Server manager in the same browser. If I log out of the ArcGIS Server Manager, the layer doesn't render in the map in Portal.
This behavior suggests that your issue is an authentication problem. When you've already signed into ArcGIS Server in the web browser you can access the service, but when you haven't then Portal can't see it.
FYI, ArcGIS Server and Portal for ArcGIS both have separate security models (unless you federate your Server site with Portal, and it sounds like you did not; also Server 10.1 does not support federation). For some background watch this UC tech session,
ArcGIS Server and Portal for ArcGIS: An Introduction to Security | Esri Video
Instead of adding the web services to the Portal map viewer directly, can you please add the web services a new Portal items, follow the help:
Add items—Portal for ArcGIS (10.4.1) | ArcGIS for Server
(see Add items from the web section)
When you add the item, you will be asked to save credentials with the item. Save the credentials with the item.
Then, in the Portal map viewer add the web services as Portal items, instead of adding the service directly - in this way, you'll also add and save the credentials for the web services to the web map.
Hope this helps,
