I've seen this on several arcgis online apps, maybe it's something that can't be duplicated on our inhouse enterprise 10.8.1 stack.
I have visited several county websites showing data without restriction, wide open to the public. When I go to check information about the layers, I get the 'you are not a member of this organization' type restriction. I'm not interested in getting the data directly, but am interested in protecting our data from general sniffing around, while making it available through public web apps. I know it's not the same, but kind of like disabling services directories on the services boxes. We know they are there, but just can't see them???