Run external Python script from ArcMap .pyt Toolbox and wait for completion

[Manu]

I need to get a certain functionality implemented in a Python Toolbox for ArcMap 10.8.

The idea is the following:

1. The user selects some layers and an NDVI threshold.
2. NDVI is calculated and a mask (based on the threshold) is added to the map.
3. Now, a Tkinter pop-up window with a slider appears. The user can inspect the mask and decide whether to try another threshold value.
4. The user adjusts the value and a new mask is added to the map until the user is satisfied with the result.

I tried to implement this from within the Python toolbox; however, running the tool in background crashes the Tkinter window while running in foreground locks the ArcMap main window (so the user cannot inspect the new mask layer).

Now, I tried to run the pop-up window as a separate process. To this end, I have multiple scripts.

These are the contents of the tk_popup.py script that is called from within the Toolbox:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Mon Feb 17 17:56:34 2022
@author: Manuel
run in terminal: python /path/to/tk_popup.py threshold, veg_index,
    idxmin, idxmax
"""
__author__ = "Manuel R. Popp"

# Import modules
import argparse
import tkinter as tk
from tkinter import ttk

# Get arguments
def parseArguments():
    parser = argparse.ArgumentParser()
    # Positional mandatory arguments
    parser.add_argument("vIndex", help = "Vegetation index.",
                        type = str)
    parser.add_argument("min", help = "Index minimum value.",\
                        type = float)
    parser.add_argument("max", help = "Index maximum value.",\
                        type = float)
    parser.add_argument("th", help = "Index threshold.",\
                        type = float)
    args = parser.parse_args()
    return args

if __name__ == "__main__":
    args = parseArguments()

vegindex = args.vIndex
indmin = args.min
indmax = args.max
thresh = args.th

### Pop-up window to adjust threshold
# root window
global application_window
application_window = tk.Tk()
application_window.geometry("300x70")
application_window.resizable(False, False)
application_window.title("Adjust threshold")
application_window.columnconfigure(0, weight = 1)
application_window.columnconfigure(1, weight = 3)

# slider current value
current_value = tk.DoubleVar()
current_value.set(thresh)

def get_current_value(cv):
    return "{: .2f}".format(cv.get())

def slider_changed(event):
    value_label.configure(text = get_current_value(current_value))

# label for the slider
slider_label = ttk.Label(
    application_window,
    text = "Adjust {0} threshold:".format(vegindex)
)

slider_label.grid(
    column = 0,
    row = 0,
    sticky = "w"
)

# slider
slider = ttk.Scale(
    application_window,
    from_ = indmin,
    to = indmax,
    orient = "horizontal",
    command = slider_changed,
    variable = current_value
)

slider.grid(
    column = 1,
    row = 0,
    sticky = "we"
)

# value label
value_label = ttk.Label(
    application_window,
    text = get_current_value(current_value)
)

value_label.grid(
    row = 1,
    column = 1,
    sticky = "n"
)

def update_th(cv = current_value):
    thresh = get_current_value(cv)
    print(thresh)
    global application_window
    application_window.destroy()

def accept():
    print("OK")
    global application_window
    application_window.destroy()
    
ttk.Button(application_window, text = "Update threshold",
            command = update_th).grid(
                                    row = 3,
                                    column = 0,
                                    columnspan = 1,
                                    sticky = "n"
)

ttk.Button(application_window, text = "Apply threshold",
            command = accept).grid(
                                                        row = 3,
                                                        column = 1,
                                                        columnspan = 2,
                                                        sticky = "n"
)

application_window.mainloop()
### End pop-up window

I included the following lines of code in my ArcMap Python toolbox in order to find the Python executable and to run the tk_popup.py script:

import os
def external(name):
    path = "O:/Studenten/man55768/COGNAC/fls/py3"
    script_path = os.path.join(path, name)
    return script_path

def find_python():
    path = os.__file__
    py_dir = None
    ap.AddMessage("Trying to locate Python.exe")
    while py_dir is None:
        try:
            path = os.path.dirname(path)
            files = [f for f in os.listdir(path) if os.path.isfile(
                        os.path.join(path, f))]
            for f in files:
                if f.lower() == "python.exe":
                    py_dir = os.path.join(path, f)
        except:
            ap.AddError("Failed to locate Python.exe")
            break
    return py_dir

def tk_popup(threshold = 0.2, veg_idx = "NDVI", idx_min = -1.0, idx_max = 1.0, python_exe = find_python()):
    tk_script = external("tk_popup.py")
    stream = os.popen(" ".join([python_exe, tk_script, str(veg_idx),
	                    str(idx_min), str(idx_max), str(threshold)]))
    output = stream.read()
    return output.rstrip(" \r\n")

Now, the problem is that the tool does not wait for user input, i.e., for the subprocess to finish. Instead, it destroys the Tk pop-up window right away and loops endlessly.

Hopefully, there is an error in my script that I can change easily to make ArcMap wait for the subprocess to finish?

EDIT: The Tk Pop-up window magically works when I do not call a Python session directly but instead, call a CMD Prompt that starts the Python script. This is strange behaviour, though, and I would prefer if there was a way to avoid the CMD Prompt in between.

Apparently, it also works when writing the command line code as text but not when the text is created as a variable and passed to os.popen()

[Manu]

This is getting stranger by the minute. I found that I can do this:

[...]
def execute(self, params):
    while True:
        command = "C:\\Python27\\ArcGIS10.8\\python.exe O:\\Files\\tk_popup.py NDVI -1.0 1.0 0.2"
        subproc = os.popen(command)
        user_input = subproc.read()
[...]

while the same toolbox engages in the weird behaviour of constantly closing the Tk Pop-up window when I change the command by splitting it and combining it with a plus like this:

command = "C:\\Python27\\ArcGIS10.8\\python.exe" + " O:\\Files\\tk_popup.py NDVI -1.0 1.0 0.2"

Any ideas what is going on here?

[Manu · Accepted solution]

I think I found a working solution. However, I am still puzzled by the weird ArcMap Python Toolbox behaviour:

The external script is executed correctly, if I define the command like this:

def tk_cmd(threshold, vIndex, min, max, python_exe = find_python()):
    tk_script = external("tk_popup.py")
    command = " ".join([python_exe, tk_script, vIndex, str(min),
                        str(max), str(threshold)])
    return command
[...]
def execute(...)
    [...]
    exec("command = tk_cmd(threshold, vIndex, min, max)")
    subproc = os.popen(command)
    [...]
    return

In other words: I need to use exec() to define the command line input that I hand to os.popen() because apparently, a = "some string" is not the same as a = "some " + "string" during execution of the script...