By the looks of it, since you have a 'grouped' layer in classic viewer, it appears as if you are adding a Mapservice and not a feature service (hosted or otherwise).
In this case, the Turn all layers on/off will toggle visibility of the layer, but, not the sublayers beneath it. So, will toggle the main layer but the sub layers will remain in the state that the are when you toggle all.
If you want all the layers to toggle, one option would be to add each sublayer from the mapservice individually. However, if you add them this way, you will be limited in their order as they must reside above any mapservices in the TOC ( It won't let you move them below a mapservice).
R_