I don't know how workable this idea is, but sometimes I would find it very useful to delete a layer (i.e. both the feature layer as well as in the layer in the Table of Contents). Would a delete and delete permanent be an option? Preferably, "delete permanent" would be somewhere in a sub-folder, when right-clicking an Image/Layer on the table of contents (a bit hidden away).