Feature layer attachment download speed

Question

I'm in the process of building some python scripts to download images from my feature layer using the arcgis python package.
```

(..attachments.download(oid=oid_to_download, save_path=self.download_folder)

```

The download speed for each image is really slow - for a 1.7MB jpeg its around 11 seconds to download. When we need to download around 200 images this is pretty heavy task.

The feature layer does contain thousands of images.

I've toyed with parallelising this with more threads/processes but I'm unsure if the arcgis api supports multiple connections from the same account as I find the pictures tend to not download in parallel.

Has anyone else seen anything similar or any suggestions on other ways around this download speed?

EarlMedina · Answer

I've used concurrent.futures and asyncio with decent results. Both are significantly faster than doing it one at a time. The basic requirements are that you just get a list of all the attachments you need to download, feed them into your download function, and you're good to go. The only thing you have to be careful about is the token as that willl eventually time out - simply refresh the token before the expirtation time.

ArcDevelopment · Answer

Threading does seem to lower the overall time slightly, 2 threads seems to drop the time to about 90% of a serial implementation so theres clearly some sort of concurrency occurring but not as much as I'd like.

StaticK · Answer

There are a lot of considerations that come into play with trying to increase download speed. Is this layer in your Portal, or pointing to standalone server?

What does your threading process look like?