Debugging ArcGIS Python Script Tools

ZacharyUhlmann1 — Mon, 04 Mar 2024 18:06:00 GMT

I've made a handful of Python Script Tools successfully. My current project is a bit trickier. In the past, I'd get the below red exclamation point and traceback my steps manually and find my error in the .pyt file. However, I can't track it down this time and there are no Tracebacks, or obvious ways in the GUI to determine the source of the error. I should note, this script tool was working prior to my recent attempted update, and still works when I manually comment out the new edits. I'm happy to post the script, but I'm primarily in need of a debugging tool or workflow. I should mention, I'm developing in PyCharm which has built-in syntax highlighting, so these errors are not Python-related necessarily. I think they are related to my validation and logic within the toolbox framework, so I need debugging from a tool, log, etc. in Pro.

Help is MUCH appreciated.

Zach

Pro 3.0.2

Re: Debugging ArcGIS Python Script Tools

AlfredBaldenweck — Mon, 04 Mar 2024 18:09:41 GMT

Generally you get that from a Syntax error. Give that a shot?

Especially if it goes away when you comment out the edits, that sounds like a syntax error.

Check syntax, make the edits needed, check syntax again, refresh the PYT

Re: Debugging ArcGIS Python Script Tools

AlfredBaldenweck — Mon, 04 Mar 2024 18:13:26 GMT

Aaaand I read the rest of your post now lol.

I'd still check for the syntax stuff to be safe.

Generally if it's a problem with validation or whatever, the tool can still be opened, but the parameters themselves will have errors next to them.

Re: Debugging ArcGIS Python Script Tools

ZacharyUhlmann1 — Mon, 04 Mar 2024 18:19:59 GMT

Didn't the right-click. Did it! Thanks for the subsequent post too. I've been doing it all wrong. Never knew about they "check syntax". I did have an issue that my eyes weren't trained to (a string with extra quotation mark). Now time to discover all the validation logic I messed up.

Re: Debugging ArcGIS Python Script Tools

HaydenWelch — Thu, 11 Jul 2024 02:37:34 GMT

I'm a bit late to this one, but you can lazy load tools into a toolbox if you have them in separate files and do module imports.

My setup is a try except block for tool loading that writes the traceback to the tool description (it initializes an empty tool class on a failed import).

This means a single failing tool won't break the whole toolbox and will give you as a developer a way to pinpoint exactly which tools are failing.

Re: Debugging ArcGIS Python Script Tools

AlfredBaldenweck — Thu, 11 Jul 2024 13:04:51 GMT

Pretty fancy! Would you be able to share a sample?

Re: Debugging ArcGIS Python Script Tools

HaydenWelch — Fri, 12 Jul 2024 13:46:46 GMT

EDIT: I wasn't happy with this code and have since totally re-written it in the pinked repo. It worked so I just hadn't really looked at it in a while and tried to apply some new knowledge to it. I'll leave the old code here for posterity, but check the repo for up to date code.

I've shared it before under the pytframe/pytframe2 framework I'm trying to develop as an open standard, but here's the reloading code:

from importlib import reload, import_module from traceback import format_exc from typing import Dict from utils.tool import Tool def build_dev_error(label: str, desc: str): class Development(object): def __init__(self): """Placeholder tool for development tools""" self.category = "Tools in Development" self.label = label self.alias = self.label.replace(" ", "") self.description = desc return return Development def import_tools(tool_dict: dict[str, list[str]]) -> list[object]: imports = [] for project, tools in tool_dict.items(): for tool in tools: try: module = import_module(f"{project}.{tool}") reload(module) tool_class = getattr(module, tool) globals()[tool] = tool_class imports.append(tool) except ImportError: dev_error = build_dev_error(tool, format_exc()) globals()[tool] = dev_error imports.append(tool) return [v for k, v in globals().items() if k in imports]

Currently it uses a higher order function as a factory for the DevError tool and writes the stack trace to the description and places it in the "Tools in Development" toolset/category. The implementation in a toolbox is done like this:

# Reloading of modules preformed here to make sure a toolbox refresh also # reloads all associated modules. A refresh in ArcGIS Pro only reloads the code # in this .pyt file, not the associated modules from importlib import reload, invalidate_caches invalidate_caches() import utils.reloader reload(utils.reloader) from utils.reloader import import_tools import utils.archelp reload(utils.archelp) import utils.tool reload(utils.tool) TOOLS = \ { "tools.development": [ "DevTool", ], "tools.utility": [ "VersionControl", ], } IMPORTS = import_tools(TOOLS) globals().update({tool.__name__: tool for tool in IMPORTS}) class Toolbox(object): def __init__(self): """Define the toolbox (the name of the toolbox is the name of the .pyt file).""" self.label = "Dev Toolbox" self.alias = "DevToolbox" # List of tool classes associated with this toolbox self.tools = IMPORTS

Where the TOOLS global stores the module paths to your tool files. The reloading is done beacuse a toolbox "Refresh" in Pro only reloads the .pyt file and not its imports meaning any changes to imported tools won't be reflected until a total restart of ArcPro/clearing of the gloabal interpreter namespace. This does slow things down a bit as the __init__ functions of all loaded tools have to run on a refresh, but you can disable those if you move the toolbox out of active development. To simplify and disambiguate what you're importing from a tool file, it requires the name of the .py file the tool is in to exactly match the name of the tool class contained within so "DevTool.py" contains this:

import arcpy from utils.tool import Tool import utils.archelp as archelp import utils.models as models class DevTool(Tool): def __init__(self) -> None: # Call the super init method to inherit from the base Tool class super().__init__() # Set local tool properties self.label = "Dev Tool" self.description = "This is a development tool." self.category = "Development" return

I've also implemented a Tool base class that allows all your tools to share a single "config" so if you have certain parameters that you always need for a toolbox, you can either define them in that Tool base class or extend that baseclass with different configs and inherit from that in your <ToolName> implementation:

import arcpy import os class Tool(object): """ Base class for all tools that use python objects to build parameters """ def __init__(self) -> None: """ Tool Description """ # Tool parameters self.label = "Tool" self.description = "Base class for all tools that use python objects to build parameters" self.canRunInBackground = False self.category = "Unassigned" # Project variables self.project = arcpy.mp.ArcGISProject("CURRENT") self.project_location = self.project.homeFolder self.project_name = os.path.basename(self.project_location) # Database variables self.default_gdb = self.project.defaultGeodatabase self.databases = self.project.databases return def getParameterInfo(self) -> list: ... def updateParameters(self, parameters: list) -> None: ... def updateMessages(self, parameters: list) -> None: ... def execute(self, parameters: dict, messages: list) -> None: ... class ToolType(Tool): def __init__(self) -> None: super().__init__() self.my_var: int = 42

So if i wanted the DevTool to have access to the self.my_var parameter, all I'd need to do is:

import arcpy from utils.tool import Tool, ToolType import utils.archelp as archelp import utils.models as models class DevTool(ToolType): def __init__(self) -> None: # Call the super init method to inherit from the base Tool class super().__init__() # Set local tool properties self.label = "Dev Tool" self.description = "This is a development tool." self.category = "Development" return def execute(self, parameters: list, messages: list) -> None: arcpy.AddMessage(f"{self.my_var=}") return

This method can incur some overhead and slow down the toolbox, but the amount of time saved during development absolutely makes up for it. I've also included a VersionControl tool in the framework that allows the toolbox framework to be stored in a git repo and synced to users no matter where the actual toolfile is located as long as they have read access to the repo.

If you want to take a peek at the code it's here I still need to write documentation for everything, but it's still heavily in development as I want to clean up the hacked together code I've been using for a couple years to be more readable and efficient applying all the lessons I've learned managing toolboxes this way for the past 2 years.

topic Re: Debugging ArcGIS Python Script Tools in Python Questions

Debugging ArcGIS Python Script Tools

Re: Debugging ArcGIS Python Script Tools

Re: Debugging ArcGIS Python Script Tools

Re: Debugging ArcGIS Python Script Tools

Re: Debugging ArcGIS Python Script Tools

Re: Debugging ArcGIS Python Script Tools

Re: Debugging ArcGIS Python Script Tools