Hey there Min,
Thank you for your question. I researched this for a bit with my colleagues and found that Image Server does infact benefit from GPU's, but not for processing and rendering mosaic datasets. At 10.8.1 it seems that Deep learning workflows take advantage of GPU's see here: Configure ArcGIS Image Server for deep learning raster analytics
"Support is available for input raster from data store, and raster analysis service for single node multiple GPU parallel processing for Deep Learning."
I am continuing my research into this, and will reply an update if I find anything further.
Keep on keeping on!