You can certainly deauthorize the geodatabase (by manually removing the authorization key).
The best way to revert a backup-less RDBMS instance into which ArcSDE has been added is
to drop all tables using SDE.ST_GEOMETRY or ST_RASTER datatypes, create a new backup
of the instance, drop and recreate the instance, and restore all users except the SDE user.
I have manually deleted the SDE login from SQL-Server, but it was preceded by a long and
painful sequence of object deletion (sometimes multiple times before foreign key constraints
were removed).
Making and verifying backups should always the first step before any major database
change (and both ArcSDE install and upgrade qualify). It's certainly recommended as
"best practice".
- V