Just getting back to this.
I did a complete reinstall, Python too. I uninstalled Python and got back the version that comes with the ArcGIS installation. I am currently running 10.5.1 as of tonight, and Python 2.7.13. Still, same problem of crashing on startup. ArcCatalog still works fine.
Seems really bizarre that this could be happening after starting from a clean slate. I guess it must not be so clean.
I've tried going into the registry and changing the folder under PythonCore to 2.7-32, 2.7-13, 2.7-12, 2.7.32, 2.7.13, 2.7.12. All of those resulted in no crash message, the loading document stage ran extra long, and eventually ArcMap just closed with no message. Changing it back to "2.7" brought it back to crashing faster with the error message.
I'll contact Esri next week.