Are you talking about the addins that live in Local/ESRI/ArcGISPro/Toolboxes? If so that's actually the first step of initialization of arcpy:
# find add-ins toolbox modules
local_appdata = os.getenv('LOCALAPPDATA')
uuid_paths = []
module_paths = []
if local_appdata and os.path.exists(local_appdata):
addin_toolbox_path = pathlib.Path(local_appdata).joinpath('ESRI', 'ArcGISPro', 'Toolboxes')
if addin_toolbox_path.exists():
uuid_paths = [pth.resolve() for pth in addin_toolbox_path.iterdir() if pth.is_dir()]
module_paths = [f.glob('arcpy/*.py') for f in uuid_paths]
These are all deleted after they are checked for, but you could definitely just use this exact code to find them at any time.
