I experienced the same problem but with a different error message and solution.  
A 10.4.1 GDB that I inherited passed the 'pre-req' checks but produced the following error message on the dialog window after trying to upgrade the GDB. The same message was in the sde_setup.log.
 
ERROR in creating GDB_Locks table.
Error: -25
  DBMS error code: 0
 
Upon investigation, I discovered that the <My GDB name>.sde.GDB_Locks table was missing.  Why?  No idea, it's still a mystery.
The solution was to go into SSMS (SQL Server Management Studio) and add the 'db_owner' role to the 'sde' user. Then I ran the upgrade and revoked the 'db_owner' role from the 'sde' user.  
This ESRI doc helped point me in the right direction, Privileges for geodatabases in SQL Server—ArcGIS Pro | Documentation 
This solution may help another GIS DBA experience a similar problem.
Ciao!