In ArcMap 10.4.1, point symbols frequently (but not always) appear as text characters. See images for examples. The standard Default symbols overlay a text character in the Arial font instead of a simple circle, square, or other shape.
@nixleytp looks like you either haven't installed the ESRI fonts or your machine has lost/corrupted ESRI fonts. Try reinstalling them.
https://support.esri.com/en/technical-article/000001920
What David said. You need to have the Fonts installed.
When it was installed they should have been installed automatically if you selected Express and all options.
