topic Re: GPU not working in ArcGIS Image Analyst Questions

GPU not working

AdrienMichez — Tue, 14 Feb 2023 10:38:02 GMT

Hi,

I'm trying to use GPU capabilities to train a deep learning model in arcgis pro using the 'train deep learning model' tool. I used the msi installation to install deep learning capabilities. It's working fine but is so slooooooow!

From other posts, I checked that Cuda is properly installed and that the python command torch.cuda.is_available() returns True.

I've activated GPU in environment/processor type but looking at task manager, my GPU does not seem to work hard, neither the CPU's.

I also check with the nvidia-smi monitoring tool which confirms that no running processes are found:

I'm using a GeForce GTX 770. Maybe this model is not compatible? It's not brand new nvidia

Best regards

Adrien

Re: GPU not working

DanPatterson — Tue, 14 Feb 2023 11:36:37 GMT

You checked the faq?

Deep learning frequently asked questions—ArcGIS Pro | Documentation

Re: GPU not working

AdrienMichez — Thu, 16 Feb 2023 10:50:10 GMT

Yes I did

I think this is probably due to my old GPU but all the tests/checks found online came out positive so I don't have the confirmation that it is actually this issue

Re: GPU not working

PavanYadav — Fri, 17 Feb 2023 19:46:38 GMT

@DanPatterson thank you for sharing the FAQs link.

I'm copying/pasting info from the link above so in case other users refer to this thread:

The recommended VRAM for running training and inferencing deep learning tools in ArcGIS Pro is 8GB. If you are only performing inferencing (detection or classification with a pretrained model), 4GB is the minimum required VRAM, but 8GB is recommended.

If you do not have the required 4–8GB VRAM, you can run the tools on the CPU, though the processing time will be longer.

@AdrienMichez looks like geforce-gtx-770 is 2GB memory size. (https://www.techpowerup.com/gpu-specs/geforce-gtx-770.c1856)

Re: GPU not working

PavanYadav — Tue, 21 Mar 2023 20:00:26 GMT

@AdrienMichez I am curious if you were setting GPU in the Processor Type

You may also want to try to check:
>>> import torch
>>> torch.cuda.is_available()

True

>>> torch.cuda.device_count()

>>> torch.cuda.current_device()

>>> torch.cuda.device(0)

<torch.cuda.device at 0x7efce0b03be0>

>>> torch.cuda.get_device_name(0)

'GeForce GTX 950M'

Even though with a 2GB GPU, you might not see much performance improvements. I just wanted to share the above, and you should be able to use it.

Re: GPU not working

AdrienMichez — Fri, 28 Apr 2023 07:35:55 GMT

Thanks

@PavanYadav: yes I was setting environment just like that

I choose to buy a new GPU NVIDIA GeForce RTX 4090

That helped a lot 🙂

But I'm facing other problems... If you want to have a look here :

https://community.esri.com/t5/arcgis-image-analyst-questions/rare-habitat-mapping-with-deeplearning-nodata/m-p/1283713/highlight/true#M455

Re: GPU not working

PavanYadav — Mon, 15 May 2023 16:05:21 GMT

@AdrienMichez thanks for sharing. For the other issue, I discussed with my coworkers. I see one of them have already responded to you the thread. Hope, what she shared can help resolve issue.

Re: GPU not working

AlejandraFigueroaMartínez — Tue, 16 Jul 2024 23:01:30 GMT

Hi @PavanYadav I have the same problem, I have an NVIDIA RTX 5000 graphics card, but when I perform object detection using deep learning, the card doesn't seem to be working properly. It does eventually complete the task, but it's incredibly slow.

Re: GPU not working

PavanYadav — Fri, 23 Aug 2024 18:19:02 GMT

@AlejandraFigueroaMartínez

Possible Reasons for GPU Underutilization:

Small Batch Size: If the batch size is too small for the available GPU vRAM, the GPU may not be fully utilized. Try increasing the batch size to see if this improves performance.
CPU-GPU Synchronization: The GPU might be waiting for the CPU to send data, leading to fluctuating GPU usage. Monitor CPU and GPU usage to identify any synchronization issues.
Software Bug: as of today, we are not aware of any bugs related to GPU for this tool.
Small Input Rasters: If the input rasters are very small (e.g., 1024x1024), GPU/CPU will give you about the same performance.

Additional Tests for GPU Verification:

CPU vs. GPU Performance Comparison: A significant difference in processing time (3-4x to 20x) between CPU and GPU should be observed for large enough input rasters. Compare the processing times to determine if the GPU is working as expected.
Large Batch Size Test: Setting a very large batch size (e.g., 1024) should result in a "CUDA out of memory" error if the GPU is functioning correctly. If you see a "CPU out of memory" error, try a smaller batch size like 512.

Re: GPU not working

LeonelRivero_79 — Thu, 12 Sep 2024 12:07:20 GMT

@PavanYadav

I’m using ArcGIS Notebooks in ArcGIS Online with GPU support to train a deep learning model (MaskRCNN) and publish the package (.dlpk). The training and publishing work well, but object detection with DetectObjectsUsingDeepLearning in ArcPy takes about 15 hours, even with GPU.

Questions:

Web Applications: Is it possible to use the deep learning package in web applications for real-time object detection?
Optimization: How can I reduce the detection time? Is it normal for it to take this long even with GPU?
Recommended Workflow: What is the most efficient workflow for object detection with ArcGIS Online?

Thank you for your assistance.

Best regards,

Re: GPU not working

PavanYadav — Fri, 13 Sep 2024 16:59:07 GMT

@LeonelRivero_79

About #1, are you considering using the deep learning tools to detect objects as a GP service. While I'm not a web application expert, I'll consult with my colleagues for more insights.

About #2, we need to assess whether 15 hours is a reasonable processing time given the amount of data, GPU vRAM, and batch size you're using. The tips I shared in my previous post can help narrow down this. Are you seeing an optimal level of GPU vRAM utilization (greater than 80%)? If not, try increasing the batch size. If vRAM utilization is already close to 100%, consider increasing the GPU's vRAM. You can also use multiple GPUs on the same machine or scale up processing using ArcGIS Enterprise if this is something you would to often. (thinking about return on investment). If you're using ArcPy outside of ArcGIS, ensure that arcpy.env.processorType is set to 'GPU'. (by default the tool should use available supported GPU, but just in case you can try with and without if it is an issue)

About #3, please make sure the batch size is not too small. if you were using 4, maybe try 16. Please test different numbers and see for which number you get faster output. I understand when the batch size is too big, the tool might return an out of memory error.

I hope this helps.

Cheers!