There isn't any way to cancel the opening of the document. You could probably run code in the OpenDocument event to challenge with a login and if it fails load a blank document. Any solution you implement through code would need to have the code saved in the document and executed when that document is opened. Code written in C# or VB.NET would have to be installed, so any user who didn't have your extension installed would completely bypass the login. A better approach would probably be to use file permissions set through the OS. Just create a group that has permissions you want and add qualified users to that group. Then set the permissions on the document so that only that group (and maybe admins) can do anything with it.
