I've noticed that when you've georeferenced an image and saved it; when you Close Georeference in the menu, the raster disappears from the map frame. If you re-open the Georeference menu from the ribbon, it reappears. Have tried both with PNG and TIFF.
The only way to make it reappear is to remove from TOC and readd, but at least it's still georeferenced.
Anyone else getting this too, before I report a BUG?