adding a few rules to Windows Firewall corrected this 'none connection' issue for me.
4 rules total: Inbound port 6080 TCP and UDP; Outbound port 6080 TCP and UDP
after those ports were open; connections could be established (in my case)
Win Server 2008 R2 - ArcGIS Server 10.2.2 Web Adapter configured but not SSL yet.
I just upgraded to 10.4 and I secured my services like yours using https on port 6443. Did you find a resolution to this? I'm getting the same errors.
Thanks,
Brent