Once you Rebuild the locators that the Composite is reading the Composite will update as well. Like you said the Composite is reading to the other locators.
Can we get confirmation from Esri staff? I'm thinking about automating the process of updating our address locators and I'd rather not have to create a new composite address locator every time I do so.
You only need to rebuild the individual locators that comprise the composite address locator. You will need to stop the composite address locator to remove locks. The are some python script examples on the geonet that you should be able to find with a keyword search. Hope this helps.
The workflows in the following topic should help in keeping your locators and services updated.
https://pro.arcgis.com/en/pro-app/latest/help/data/geocoding/keep-your-locator-up-to-date.htm