I can't count all the times I have lost or had difficulties selecting graphics, legends, or scalebars in a layout view. All objects in the layout view should be viewable and selectable in the TOC -- not just feature classes and groups.
I don't think the TOC is the right place to put this but I think a separate window that lists all elements in the layout and enables you to select, open and delete them would be very useful.
I see where you're going with this, and perhaps the TOC isn't the right place, but you're right, there is no definitive list of all graphic elements on the page. I've managed to lose an element or two in the past, and had to pull up VBA and parse through them all to find them.
