If the service is rendered dynamically, do the symbols show up? If not, you know it's not a problem with caching. I would expect the same problem with a dynamic service. It's likely that the handicap symbol font isn't installed on the Server machine, which is surprising because I thought that was a default symbol.
Thanks
I after setting it up a few times, this time I left it on rendering dynamically, when I was intending to cache...
I set it to a cached tile service and all is well.