Hey @hesmebye
I have a similar setup to yours, hopefully I can provide some insight!
We have a Server, Notebook Server, Workflow Manager, Portal, and Datastore all on separate servers, with Webadaptors on those servers through Windows IIS. We also have a load balancer setup that acts as a reverse proxy, we setup the servers to have a WebContext URL as the load balancer's URL, in that case, we federated the servers with the load balancer URL, here are some example pictures if it helps:
I would personally use the loadbalancer URL for most if not all as it keeps everything consistent.
Hope that helps!
Cody