AddMessage not working in worker threads

2402
4
02-01-2019 09:59 AM
PlinioSantos
New Contributor III

I developed a script for ArcGIS Pro which performs 4 heavy downloads. To speed-up the process, I delegated the download process to 4 worker threads and wait for them complete in order to continue the geoprocess operations. During the download process, I use the arcpy.AddMessage() function to give some feedback.

Everything works as expected when I execute de code as a python standalone script, including the feedback messages. However some messeges did not show when running the script in ArcGIS Pro.

I suspect that the worker threads are writing the messages in somewhere else. To test my thoughs, I wrote the following script:

import concurrent.futures
import threading
import arcpy


def add_message_wrapper(message):
    arcpy.AddMessage('Thread {0} says: {1}'.format(threading.current_thread().getName(), message))


if __name__ == '__main__':
    add_message_wrapper('Hello addMessage')
    with concurrent.futures.ThreadPoolExecutor() as executor:
        futures = [executor.submit(add_message_wrapper,
                   'Hello addMessage from a worker thread') for i in range(6)]
        for future in concurrent.futures.as_completed(futures):
            pass
        add_message_wrapper('Back to main...')

The output as a standalone python script:

Thread MainThread says: Hello addMessage
Thread ThreadPoolExecutor-0_0 says: Hello addMessage from a worker thread
Thread ThreadPoolExecutor-0_0 says: Hello addMessage from a worker thread
Thread ThreadPoolExecutor-0_0 says: Hello addMessage from a worker thread
Thread ThreadPoolExecutor-0_1 says: Hello addMessage from a worker thread
Thread ThreadPoolExecutor-0_0 says: Hello addMessage from a worker thread
Thread ThreadPoolExecutor-0_2 says: Hello addMessage from a worker thread
Thread MainThread says: Back to main...

The  output as a script in ArcGIS Pro:

Messages
Start Time: sexta-feira, 1 de fevereiro de 2019 11:49:40
Running script testAddMessage...
Thread Dummy-1 says: Hello addMessage
Thread Dummy-1 says: Back to main...
Completed script testAddMessage...
Succeeded at sexta-feira, 1 de fevereiro de 2019 11:49:45 (Elapsed Time: 4,80 seconds)

So, how can I get the messages from worker threads sent to ArcGIS Pro output?

0 Kudos
4 Replies
DanPatterson_Retired
MVP Esteemed Contributor

probably just printing... build your message string and 'tweet' it

def tweet(msg):
    """Print a message for both arcpy and python.

    msg - a text message
    """
    m = "\n{}\n".format(msg)
    arcpy.AddMessage(m)
    print(m)
PlinioSantos
New Contributor III

Dan, your solution duplicate the messages in python console (stand alone python script) and yet did not makes ArcGIS Pro output to show messages sent by a worker thread.

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

wouldn't your worker scripts would need to return the messages to the main script then? or are your workers isolated once they are called?

0 Kudos
PlinioSantos
New Contributor III

All code are in the same python module, i.e., same script, just as the snippet in the main post. The concurrent.futures module starts worker threads to execute a funcion (heavy network operations in my real case) and the concurrent.futures.as_completed yelds the results as soon as the worker thread execution ends. My problem is that only messages sent from the starter thread are visible on ArcGIS Pro interface.

0 Kudos