I'm also having the same problem with a Silverlight application accessing map services after upgrading to ArcGIS Server 10.4. Here is the full stack trace:
System.InvalidCastException: Unable to cast object of type 'System.String' to type 'System.Collections.ICollection'.
at ESRI.ArcGIS.Client.Layer.OnInitializationFailed(EventArgs e)
at ESRI.ArcGIS.Client.Layer.Initialize()
at ESRI.ArcGIS.Client.ArcGISDynamicMapServiceLayer.<>n__FabricatedMethod33()
at ESRI.ArcGIS.Client.ArcGISDynamicMapServiceLayer.<>c__DisplayClass31.<MapServiceInfoInitFailed>b__2f()
Thanks,
David