What part of the Silverlight API 2.4 make calls to http://www.arcgis.com/sharing/tools/legend? I occasionally see a call to this service in my proxy.ashx file but they fail because my instance of ArcGIS server is not accessible outside of my domain.
what version of ArcGIS Server are you using? If you have any services from < 10 sp 1 then they will try to use the Esri legend service as that capability is not available prior to that service pack.