I'm more familiar with JavaScript, so. I apologize if you have already tried this. In JavaScript, I use the proxy file which contains my token information which I then reference in my web doc. You may want to check out these links, including one that was having issues that was then resolved. Might help you track down your issue.
How to use secure ArcGIS Server services | ArcGIS Blog
ArcGIS API for Silverlight Help doc
since I've been dealing with the same issues of getting my secure services to be accessed seamlessly, I know it can take a while to get everything tested and working, so good luck with it all. FYI I do have mine working finally.