Thanks to everyone who tried to help me with this. I finally stumbled over what I needed. VB.Net code fragment:
Dim pLayer As ESRI.ArcGIS.Carto.ILayer
Dim pFeatureLayer As ESRI.ArcGIS.Carto.IFeatureLayer
Dim pGFL As ESRI.ArcGIS.Carto.IGeoFeatureLayer ' This was the missing piece
Dim pUniqueRenderer As ESRI.ArcGIS.Carto.IUniqueValueRenderer
Dim pSymbol as ESRI.ArcGIS.Display.ISymbol
pLayer = ...
pFeatureLayer = pLayer
pGFL = pFeatureLayer
pUniqueRenderer = pGFL.Renderer
pSymbol = pUniqueRenderer.Symbol(pUniqueRenderer.Value(n))