Log out of AGOL first. Open blank doc. Then open the mxd in question. Do not log in to AGOL. You will get login messages for as many layers as you have loaded in the mxd that are linked to your online hosting. Keep hitting cancel until the entire mxd is loaded. Remove the suspected bad layer file and save the document. Close the doc, then sign in to AGOL and load the mxd. This should resolve the issue.