This can be accomplished via the Python API if you want to write a quick notebook to do the trick, but I agree - an option within the GUI would be beneficial.
@KoryKramer One of my favorite Esri people...any updates on this idea? This is a giant time waste having to remove each layer individually. Thanks in advance.