Is there an example for how to use the returned Future object?

simoxu

In ArcGIS Python API 1.8.1, it allows you to make an asynchronous call of delete_features function by setting the future parameter to True.

The API document is here:

arcgis.features module — arcgis 1.8.2 documentation 

Returns:  Dict if future=False (default), else a concurrent.Future class.

I'd like to check the Future.result() to see if the operation has completed or failed. (there are nearly 1 million records...) :  my understanding is it should only return when it is completed or it is failed, but actually it returns the following dictionary:

{'submissionTime': 1604031554270, 'lastUpdatedTime': 1604031554270, 'status': 'Pending'}

So, what's the use of returning a Future object which seems returned immediately and never gets updated? is there an example to show how I am supposed to use this return?  I'll ask the same question for calculate function .

Am I alone in this?

SamSzotkowski

Hello from the future.  You are not alone, I was curious as well, and I haven't been able to find any examples online.

I took a look at the source code of layer.py (defines arcgis.features.FeatureLayer objects, you might find it in "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\arcgis\features").  Let's take the delete_features() method as an example, line 2532 (at least in the current version) looks like:

 

        if future is False:
            return self._con.post(path=delete_url, postdata=params)
        else:
            params["async"] = True
            import concurrent.futures

            executor = concurrent.futures.ThreadPoolExecutor(1)
            res = self._con.post(path=delete_url, postdata=params)
            time.sleep(2)
            future = executor.submit(
                self._status_via_url,
                *(self._con, res["statusUrl"], {"f": "json"}),
            )
            executor.shutdown(False)

            return future

 

 So, indeed we're getting back a conccurent.futures.Future object.  We could then use the concurrent.futures.as_completed() function to iterate over a list of Future objects' results as they finish up.

The following snippet shows how you could send off a few different operations on FeatureLayer objects and summarize the results as they come in:

 

import arcgis
from concurrent.futures import as_completed

username = ""
password = ""
gis = arcgis.gis.GIS("https://arcgis.com", username, password)

f_lyr_urls = [
    "https://services6.arcgis.com/blahblahblah/1",
    "https://services6.arcgis.com/blahblahblah/2",
    "https://services6.arcgis.com/blahblahblah/3"
]

futures = []
for url in f_lyr_urls:
    f_lyr = arcgis.features.FeatureLayer(url, gis)
    future = f_lyr.delete_features(where="1=1", future=True)
    futures.append(future)

for future in as_completed(futures):
    result_dict = future.result()

    layer_name = result_dict.get("layerName")
    status = result_dict.get("status")
    n_deletes = result_dict.get("recordCount")

    message = f"{layer_name}: {status}"
    if n_deletes:
        message += f": {n_deletes} records deleted"

    print(message)

 

I do not claim that this is a good example, however, because I'm pretty sure the way this works is that each call to delete_features() spins up its own ThreadPoolExecutor, but if I understand the documentation correctly (it says "All threads enqueued to ThreadPoolExecutor will be joined before the interpreter can exit") then each thread pool is blocking the rest of the program until it finishes.

So, no matter what using future=True is going to be slower than just running it synchronously, since each thread pool only sends one request and is going to block everything else anyway, so you might as well not create the new thread.

Unless I'm misunderstanding something, this is a completely useless implementation by ESRI.  They should instead return to us all the junk they're passing to executor.submit() so that we can lump together all the POST requests we want to send into the same thread pool.