Erik:
Is there any chance your layers appear in the app from another source such as ESRI basemaps? I ask because my organization submitted GIS layers to ESRI to add to the public basemap layer (e.g. Parcels, Roads, Contours), so if I loaded in my specific layers as well as the ESRI basemap and my layers did not load it would still appear as though my layers are in the map.