I doubt that you can use them directly.
ArcObjects can be accessed in several different contexts including ArcMap, ArcEngine, or ArcGIS Server. The way that you inject custom business logic into each of contexts is different.
What you can do is split your add-in such that you can use the heart of your business logic regardless of context. You then develop a small, context specific, layer on top of that which allows you to inject your logic.