I have an ags service. I add the service to an agol web map. i remove a few of the layers i don't want exposed. i create a web app using the Template (Not Web AppBuilder) from the web map. i see the layers i removed. I don't want to see the layers i removed.
The only thing I can think of is a browser cache problem. Can you see it if you open the web app in a private/incognito window, or a different browser than what you normally use?
be sure you saved your web map after removing unwanted layers. refresh the web app afterwards. If you still see the unwanted layers in the app (but not in the web map after saving) try opening the web app in a different browser (to confirm if it is/inst a cache issue)