This looks like it's already logged as a bug. The version found states 10.2, so I can see the process showing similar behavior at 10.1 SP1. I imagine if I tried a couple more times I could reproduce it at 10.2.1 then.
I can't reproduce this at 10.2.1; if you republish a service and choose the option to update cache automatically, it'll recreate tiles in the same cache folder. If you choose the option to update cache manually, it should only create a new folder within the cache directory with the service name and timestamp. This behavior is described here.
At 10.2, the options were a bit different. If you overwrite a service you have two options through the publishing wizard to keep the cache or overwrite the cache. Unchecking the option to keep the cache will create a new folder within the cache directory with the service name and timestamp, per the information here.
Any behavior other than that, for example a new service is created with the timestamp appended to it when overwriting, could be a bug.
The option to keep an existing cache should be added back at 10.3.