Fraser,
The ArcGIS Administrator is composed of 2 parts. The graphical user interface and the services that the user creates. When a user creates a service in ArcGIS Administrator, the service is composed of the test interface for the service and the collector for the service. The service test interface is associated with the required port when the user defines the service. The test interface is used by ArcGIS Administrator to validate each configured item before adding the new item to a list of validated definitions. Once an item is validated, the service will observe the target and send the results to ArcGIS Monitor Server. When the user clicks test on an item the GUI calls localhost:port. If the machine cannot resolve localhost or the port is not available on the machine the connection error is reported.
Feel free to send me an email and I can guide you to review the service test logs.
