Your load balancer should be configured to have each machine within it's server pool, so requests are sent to each machine through the load balancer URL.
Multiple-machine deployment with third party load balancer—Installation Guides (10.5) | ArcGIS Enterprise
The image above is taken from the following resource:
Deployment scenarios for a highly available ArcGIS Enterprise—Portal for ArcGIS (10.5.x) | ArcGIS Enterprise
It describes that federation is done using the LB URL for the services URL as well as admin URL:
Retrieving data ...