Lisa, Robert - thanks for your replies.
I can't make the service public unfortunately because the data is sensitive, but I'm going to try with a different 'test' layer tomorrow and see if I get the same problems.
The only other information I can give is that the service is hosted on AGOL using Feature Layers that have been uploaded to the AGOL server - so there is nothing held outside of AGOL.
Any other suggestions on what I can try would be appreciated, I'll try posting on the Web AppBuilder forum tomorrow if I'm not having any more luck.
Thanks again