We run our own AGS Services and add them to an ArcGIS Online webmap, configure the popups and other properties and then consume the webmap in a WAB app --- that's the goal anyway. If you add a specific layer from a dynamic map service, it is added to the webmap as a FeatureLayer (in JSAPI speak), so it is essentially a graphics layer with attributes that are displayed when you click a feature.
However, if you add an entire dynamic map service to a webmap, then it uses an identify query task on the service when you click on the map. In the ArcGIS Online viewer, as I turn on an off individual layers it affects which popups are displayed (only for visible layers). But when I put that same webmap into a WAB app, the identify task doesn't seem to honor which layers are turned on and off.
Is this just me? have a set something up wrong? Here's my example:
When it first opens, two layers are visible and if you click on the map you will get the popup for each. However, turn those two layers off and then click on the map and the popup still displays. And if you turn on other layers, their popups do not begin to display.