To remove all legend swatches you can set Legend.LegendItemTemplate to null (in XAML LegendItemTemplate="{x:Null}").
If you want to remove the swatches for the clustered graphics layers and keep them for others layers, you have to do it by code. On event Legend.Refreshed, remove the LegendItems for the clustered layers.