So I discovered a workaround to this problem.
Set your primary color.
Click the advanced color settings and locate your primary (labeled as Primary - 500).
Hover over the shade above it (Primary - 400), and make note of the hex code.
Now change your primary color to this new color and it should match better:
