nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Python Toolbox Import is hanging ArcPro

Hayden Welch

So I've been experimenting with modularized tools (see here). So far it's working really well, but one of the biggest issues is that seemingly randomly, adding this modular toolbox to a project will cause ArcPro to freeze up. Nothing is showing up in Diagnostic Monitor and the toolbox just silently fails to load and Arc never releases control back to the user with an error unless you force close it.

When using a helper tool that runs the ImportToolbox function, it will occasionally fail with several dozen attempts at importing the toolbox, but it's inconsistent. Also once the toolbox is added to the project it works fine and even with the import reloads, it refreshes in about half a second.

One thing that I think could be causing the issue is that I'm using a wrapper class for the Tools that pre-calculates a lot of project related data into object parameters:

class Tool(object):
    """
    Base class for all tools that use python objects to build parameters
    """
    def __init__(self) -> None:
        """
        Tool Description

        @self.project: arcpy project object
        @self.project_location: path to the project
        @self.project_name: name of the project
        @self.default_gdb: path to the default gdb
        @self.design_gdb = archelp.get_databases(self.project_location, database_name="lld_design.gdb")
        @self.design_gdb: path to the design gdb
        @self.design_features: dictionary of design features
        @self.params: tool parameters (set with archelp.get_parameters())
        """
        # Tool parameters
        self.label = "Tool"
        self.description = "Base class for all tools that use python objects to build parameters"
        self.canRunInBackground = False
        self.category = "Unassigned"
        
        # Project variables
        self.project = arcpy.mp.ArcGISProject("CURRENT")
        self.project_location = self.project.homeFolder
        self.project_name = os.path.basename(self.project_location)
        
        # Database variables
        self.default_gdb = self.project.defaultGeodatabase
        self.design_gdb = archelp.get_databases(self.project_location, database_name="lld_design.gdb")
        if self.design_gdb:
            self.design_gdb = self.design_gdb[0]
            self.design_features = archelp.walk_database(self.design_gdb ,dataset="Design_Features")
        else:
            self.design_gdb = None
            self.design_features = None
        
        # Parameters
        self.params = {}
        
        return

So Every tool is implementing this __init__() with a super.__init__() call. Could this be freaking ArcPro out? as in every subtool of the toolbox is running this init function constantly for every tool? I'm making queries in it, and when I refresh the toolbox it only seems to run those queries once for the whole toolbox and again when I open the tool, but maybe the import action is hitting some sort of confusion with this?

I know I'm using Python Tools well outside their normal operating functionality, but any hints on what exactly could be causing these import problems from someone with more knowledge about how Arc handles python tools would be great.

Vind meer berichten getagd met

Accepted answers

All comments

ShaunWalbridge

Please try moving the toolbox to a local drive and see if it has any impact. When you have remote resources for things like toolboxes, you are beholden to the network latency and bandwidth for basic operations, which can get easily into a situation where the application hangs, because it is waiting on I/O operations it expects to be fast (reading some text or data from local disk) but in reality it has to wait for a high latency connection to respond. Not specific for toolboxes, but we currently don't support cloud data storage for Pro resources for similar reasons.

If the hangs are reproducible with local files, then you've eliminated one class of issues. If it reproduces with local on disk tools, I would log a support request and we can dig into it further through a support case. Thanks!

Shaun

RogerDunnGIS

I'm pretty sure ArcGIS Pro is just using the credentials of who is logged into Windows. You can always check Task Manager to see what user the process is running under. I remember in the ArcMap era, there were times when ArcMap would try to start, and then it would stop. I thought it was the Normal.mxt template but it turned out to be file permissions. Somehow, a user would lose permissions on C:\Python27\blahblahblah and ArcMap would either exit or hang, I can't remember. So permissions is definitely something to look at.

Hayden Welch

We are accessing the toolbox files from a shared network drive managed through Active Directory, so maybe there's a permissions problem with whatever user Arc uses to import the tool files. I'll look into that, just weird that we haven't had permission issues for any other files we use that are located there

RogerDunnGIS

Maybe the problem is outside of the Python/GIS realm. One area you might look at is virus protection. Perhaps you've got a virus scanner that's holding things up, putting files into quarantine, and such. I had an situation where a virus scanner at a previous job was occasionally putting MrSID files in quarantine. If the MrSID files were recopied to the machine, that machine would never have trouble again. Otherwise, it would effect some and not others, and cause the GIS software that was trying to load the MrSID files to hang.

Hayden Welch

I'm saying that the import hangs only sometimes, other times it imports just fine. But about 50% of the time it just refuses to import. It might eventually succeed, but I'm not patient enough to wait more than 30 minutes to see. That's why I'm so confused. Hypothetically this should be consistent, but it isn't. The only thing I can think of is that the namespace of a running arc instance is causing issues with other imported tools, or Arc is doing some sort of license check that's failing on the backend that only runs into snags occasionally.

And yes all the ArcPros are on the same (latest) version. It also varies from project to project, if an APRX project file has successfully imported the toolbox it never has issues again, even if you remove and re-add it. I wish I could give more info, but getting any readouts from Arc on what's happening that's hanging the import is difficult.

Thanks for helping by the way, this is definitely a headscratcher.

RogerDunnGIS

Your original post's question is, essentially this, right?:

one of the biggest issues is that seemingly randomly adding this modular toolbox to a project will cause ArcPro to freeze up. Nothing is showing up in Diagnostic Monitor and the toolbox just silently fails to load and Arc never releases control back to the user with an error unless you force close it.

I'm sure you're not randomly adding the tools to a project. That can't be what you mean. Are you saying that it freezes some ArcGIS Pro's and doesn't freeze others? Like, the freezing is what's random? These ArcGIS Pro's can see all these other utilities, have the same Python environment (original, cloned, modded, etc.), they're on more or less the same major version as Pro, etc.?

Hayden Welch

You are correct, the archelp module is a custom module that I use to put all my generalized solutions in to allow for more efficient code reuse across a project. I don't think the chicken and egg scenario is happening as projects that already have the toolbox loaded don't have any issues. And nothing is really happening in those initialization scripts that would require anything more than the project location which is already determined when the toolbox is being imported from the open project file.

Those archelp functions are running ListFeatures functions to generate a list of the base features that I need for a project. Which I don't think would be causing any issues.

Moving the super.__init__() to the getParameterInfo() function is a good idea though. That was gonna be my next course of action if I couldn't get a good answer on exactly why the toolbox would cause an infinite loop in the import action (assuming that initialization is preformed on import and not activation).

My last possible idea is that the imports in the tool modules are overwriting the imports in the toolbox or something and causing the Toolbox to continually import modules as they're being reloaded. Though I only reload archelp once and each toolbox once and don't import tool modules into other tool modules.

RogerDunnGIS

Those are good guesses, and I'm sure one of them is right. It's unknown to me exactly when tools are loaded, even though I've got a logger. But you could be running into a catch-22. The project is loading your tools, while each of them are loading the project. Chicken before the egg, you know?

There may be better locations for some of these self variable to get initialized, like in getParameterInfo or updateParameters. Or you could put these initializations in some other method not defined by Esri, and you call those from the child classes. Obviously your goal is going to be to initialize the variables before they're needed, and I'm just assuming you need those before the user fills out parameters, rather than at execution time.

I'm also assuming you wrote the archelp module, as I can find no documentation for that. Perhaps archelp also has methods that are causing ArcGIS Pro to halt.

Doe mee aan het gesprek

Aangemelde leden kunnen berichten plaatsen, updates volgen en meer. Nieuw hier? Registreer een gratis account.

Inloggen
Registreren

Snelle Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.