Supporting multiple GPUs on your raster analytics node within your raster analytics site is not fully supported for all models yet. `Utilization of multiple GPUs per server node is applicable to some deep learning model configurations predefined in ArcGIS. They include Tensorflow (ObjectDetectionAPI and DeepLab), Keras (MaskRCNN), and PyTorch.`
For supported models, yes the job can be distributed to multiple GPUs on your RA node if there are multiple GPUs available and the Max instances per machine for RA services are set to be more than 1.