The Select widget, when 'Interact with a Map widget' is selected, will list all layers from the map, without group layers. When a map has many layers, and in some case different layers used in different groups can have the same name, this list gets messy.
The configuration panel gives the possibility to make individual layers not selectable but the possibility to keep the group layer stucture is missing.