Ayyaz Mahmood-Paracha,
thank you so much for your insights, as soon as you said that I had a feeling that it just had to be exactly that as I had been streamlining both test and production environments to utilise common application, widget, logs and backup locations and as you say you can update the logs to a shared folder for which I had not anticipated such a catastrophe, as it's the log files and not the applications themselves and the admin portal allows you to change it:
Once I amended the log path back to the local directory in Portal Admin the postgres.exe that I was running with a local testuser account on the failover server stopped and the postgres.exe services started running with the correct service account:
Failover Server:
Both portals are now up and running as they should be.
and obviously Jonathan Quinn Thanks so much for helping out with the troubleshooting, it was incredibly useful to run through troubleshooting this one with you. It's not so obvious that the logs create such an issue, and as you say you can't set as a network share in the configuration and this is one that slipped past me.
These decision to migrate the logs paths to a network share were actually kicked into action by the webgisdr utility filling up the backup\walarchive folder causing HDD space issues and Portal downtime, ironically the opposite effect intended creating the backups.
As a workaround we'll write a script that clears the local log folders once they reach a particular age / size and will keep the webgisdr running at weekly intervals, though I'm sure there is a better internal solution that could be put into Portal to protect against such things happening.
Thanks again so much for your help guys!
Dean