Chris,
I think you are talking about BUG-000084319.
"The pop-up menus from the ArcGIS for Server services layers in Web
AppBuilder persist when either the layer or the service is unchecked
from the table of contents"
I've downloaded the WAB version 1.2, and this bug seems to be fixed.
Unfortunately, the following bug has not been fixed:
BUG-000087423The Attribute Table widget does not update when changes are made in the
Layer List widget using secured services in a Web AppBuilder application