Where is the fGDB located, on the network or local? If on the network, is it in the same location as the SQL Server machine?
What are the resources of the SQL Server machine when running the draw?
What is the version of SQL Server and machine specs?
Did rebuilding the spatial index make any difference?
I can say that if you are drawing all the contours at full scale, that is a ton of vertices to draw and request.
Based on your comment about timing out, I would look at the SQL machine to see what it is doing when you are trying to draw the contours. I do not think that it has anything to do with the "size on disc".
--- George T.