Experience Builder and Calcite have separate implementations for theming. So for example, if you build a widget that you'd like to use within the Builder experience, it won't work correctly with the theme of the embedded Calcite components. However, you should be able to use Calcite components within your custom Exp Builder widgets that are added directly to an individual app (outside of the Builder environment).
We are also investigating the possibility of adding a Calcite theme to Experience Builder so you could choose to build an app that looks like Calcite (and your custom widgets that use Calcite would blend well with this). Would this be of interest to you?