It would be helpful to see what exception is causing ArcMap to crash. You can use Visual Studio to run ArcMap and can catch these exceptions in order to view them. Just break in the debugger before the crash and tell Visual Studio to stop when an exception is thrown. Note that you can get false positives so leaving this on will get very confusing.
I have not used add-in's but perhaps the problem lies in ArcMap finding assemblies that your add-in references? Just a guess.