I realize this is an old thread but I was just going crazy with this issue and agree it seems buggy but have found a successful workaround (and this was literally the only thing I could find anywhere on this issue and am sure other people have had this problem). Open up the map that is in the layout, go to the Selection Options and ensure "Selection combination" is set to Create a new selection and not Add to the current selection. This solved the issue for me even in an empty layout with graphic elements but no map. It is somehow "remembering" previous selections of multiple elements for every time you try to select anything that was within the group selection.
... View more