"Unable to initialize python raster function with scalar arguments" on an NVIDIA machine

[AsafEvenPaz]

I am trying to run a notebook given to me by Esri Support staff (case #02587809) and when I try to run the arcpy.ia.DetectObjectsUsingDeepLearning function I recieve the following error message:

Line 1:     arcpy.ia.DetectObjectsUsingDeepLearning(imagery, r"C:\Users\Administrator\Documents\ArcGIS\Projects\Haiti_AI\Haiti_AI.gdb\bldg_detected_jupyterGPU_10July2020", r"D:\TrainingData\MaskRCNN_chips\models\Building_footprint_40epochsGPU\Building_footprint_40epochsGPU.emd", "padding 64;batch_size 4;threshold 0.9;return_bboxes False", "NO_NMS", "Confidence", "Class", 0, "PROCESS_AS_MOSAICKED_IMAGE")  File c:\program files\arcgis\pro\Resources\arcpy\arcpy\ia\Functions.py, in DetectObjectsUsingDeepLearning: Line 2888:  processing_mode)  File c:\program files\arcgis\pro\Resources\arcpy\arcpy\sa\Utils.py, in swapper: Line 53:    result = wrapper(*args, **kwargs)  File c:\program files\arcgis\pro\Resources\arcpy\arcpy\ia\Functions.py, in Wrapper: Line 2877:  processing_mode)  File c:\program files\arcgis\pro\Resources\arcpy\arcpy\geoprocessing\_base.py, in <lambda>: Line 511:   return lambda *args: val(*gp_fixargs(args, True)) ExecuteError: ERROR 999999: Something unexpected caused the tool to fail. Contact Esri Technical Support (http://esriurl.com/support) to Report a Bug, and refer to the error help for potential solutions or workarounds. Unable to initialize python raster function with scalar arguments.  Traceback (most recent call last):   File "D:\TrainingData\MaskRCNN_chips\models\Building_footprint_40epochsGPU\ArcGISInstanceDetector.py", line 169, in initialize     self.child_instance_detector.initialize(model, model_as_file)   File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\learn\models\_inferencing\_maskrcnn_inferencing.py", line 158, in initialize     self.mask_rcnn = MaskRCNN.from_model(emd_path=model)   File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\learn\models\_maskrcnn.py", line 215, in from_model     data = _EmptyData(path=emd_path.parent.parent, loss_func=None, c=len(class_mapping) + 1, chip_size=emd['ImageHeight'])   File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\learn\models\_arcgis_model.py", line 60, in __init__     if getattr(arcgis.env, "_processorType", "") == "GPU" and torch.cuda.is_available(): NameError: name 'torch' is not defined Unable to initialize python raster function with scalar arguments. Failed to execute (DetectObjectsUsingDeepLearning).

I have tried various workarounds from similar threads but nothing so far. I am using a g3s.xlarge machine and remote desktop (windows machine) with GPU enabled.
I have verified that the GPU is being used by running:

cudaDevice=torch.cuda.current_device()
print ("GPU-Device: "+str(cudaDevice))
print ("GPU-Device-Info: "+str(torch.cuda.device(cudaDevice)))
print ("GPU-Device-Count: "+str(torch.cuda.device_count()))
print ("GPU-Device-Name: "+str(torch.cuda.get_device_name(cudaDevice)))
print ("GPU-Device-Availability: "+str(torch.cuda.is_available()))
#now some more info
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print('Using device:', device)
print()
#Additional Info when using cuda
if device.type == 'cuda':
 print(torch.cuda.get_device_name(0))
 print('Memory Usage:')
 print('Allocated:', round(torch.cuda.memory_allocated(0)/1024**3,1), 'GB')
 print('Cached: ', round(torch.cuda.memory_cached(0)/1024**3,1), 'GB')

# use this to test: Run code, dont close,
# go to OS and run nvidia-smi to see process runs on GPU
print (torch.rand(3,3).cuda())
os.system('nvidia-smi.exe>c:/temp/log.txt')
with open(r'c:/temp/log.txt', 'r') as f:
 print(f.read())

Resulting in:

GPU-Device: 0 GPU-Device-Info: <torch.cuda.device object at 0x000002B5B5DCD390> GPU-Device-Count: 1 GPU-Device-Name: Tesla M60 GPU-Device-Availability: True Using device: cuda  Tesla M60 Memory Usage: Allocated: 0.0 GB Cached:    0.0 GB tensor([[0.3166, 0.1292, 0.2083],         [0.2514, 0.0501, 0.4692],         [0.8568, 0.1868, 0.6476]], device='cuda:0')

Any thoughts?

[Anonymous User]

Hi,

The error is coming from C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\learn\models\_arcgis_model.py

Here 'arcgispro-py3' is the default environment, you need to setup your  environment to use this tool.

I am pasting the current installation commands From the documentation page here. 

1. You need to first clone the default environment and activate the cloned environment, help page for the same is here.
2. You can then run the following command in python command prompt to install ArcGIS Python API with deep learning dependencies

   conda install -c esri -c fastai -c pytorch arcgis=1.8.2 scikit-image=0.15.0 pillow=6.2.2 libtiff=4.0.10 fastai=1.0.60 pytorch=1.4.0 torchvision=0.5.0 --no-pin

Thanks,
Sandeep