Programmatically Remove Empty Symbol Classes

[MarkHammond4]

I'm looking for a way to programmatically remove empty symbol classes from the Contents pane. Although it's easy to remove empty symbol classes by going to the Symbology pane and removing classes with a count of 0, I need to automate this with ArcPy. Maybe using the CIM is necessary?  I'm trying to do this on a View Service with symbol classes based on domains. 

[MSus4]

Did anyone have a solution for this? Many times there are bunch of symbol classes with count of 0, changing the order according the count will mess up the order of the symbology and finding hundreds of count 0 -items in between used values is a hassle.