topic Re: Anyone using the new Python Toolboxes at 10.1? in Geoprocessing Questions

Anyone using the new Python Toolboxes at 10.1?

KevinGooss — Wed, 05 Sep 2012 12:48:48 GMT

Just wondering if anyone has used the new Python toolboxes at 10.1?
The .pyt
I am experimenting with one and could use some tips.
fyi, if you want to edit that pyt file in your favorite python editor you may have to go into the options of your editor and set it up to explicitly recognize pyt files. Mine (Wing, pyScripter) did not do that by default and there was no coloring or intellisense until i modified the program under options to read that pyt.

I want to use pyt because at 10.0 we had such a hassle with deploying script tools and toolboxes because you always need arcatalog to setup the dang params. And they seem to get out of whack on their own over time.

I'm hoping these python-based toolboxes don't work like that.
One drawback is that you only get the one pyt file - so if you want to cram 20 custom tools in there you are looking at a huge file.

Re: Anyone using the new Python Toolboxes at 10.1?

DavidWynne — Wed, 05 Sep 2012 16:08:58 GMT

Hi Kevin,
If you have a few minutes, perhaps check out the training seminar on Python toolboxes that we did last week: it will walk through some of these questions you have:
http://training.esri.com/gateway/index.cfm?fa=catalog.webCourseDetail&courseID=2523

If you're looking for additional samples just to see, check out http://esriurl.com/pyt1, http://esriurl.com/pyt2, http://esriurl.com/pyt3

-Dave

Re: Anyone using the new Python Toolboxes at 10.1?

KevinGooss — Wed, 05 Sep 2012 16:55:43 GMT

Thanks for those tips. I will look into the seminar and samples.
My greatest concern now is that these examples are designed with one tool in one toolbox.
I have over 30 tools that i would like to present as services for my applications.

Running each one of those in ArcMap and publishing the results as a service is just not an option.
It is too much work.
I know the code works because i have been using the tools at 10.0
I just want a painless way to expose many py scripts as services.

Re: Anyone using the new Python Toolboxes at 10.1?

Luke_Pinner — Sat, 11 Dec 2021 10:55:56 GMT

I'm still using 10.0 so can't actually test this, but have you tried using the standard python package structure?

Something like:

#  \--SomeDir
#     |  toolbox.pyt
#     \--toolpackage
#        |  __init__.py
#        |  script_a.py
#        |  script_b.py


#----------------------------
#The .pyt file
#----------------------------

import arcpy
from toolpackage.script_a import Tool1
from toolpackage.script_a import Tool2
from toolpackage.script_b import Tool3

class Toolbox(object):
    def __init__(self):
        """Define the toolbox (the name of the toolbox is the name of the
        .pyt file)."""
        self.label = "MultiTool Toolbox"
        self.alias = "mtt"

        # List of tool classes associated with this toolbox
        self.tools = [Tool1, Tool2, Tool3]

Re: Anyone using the new Python Toolboxes at 10.1?

KevinGooss — Thu, 06 Sep 2012 12:32:41 GMT

I think i get what you are saying.
In my master pyt python toolbox file i just have the definitions of the tools and the params for each.
Each tool has an execute method and in that method i will just place a link to the actual py script file that i already have from pre-10.1 days. I think that will work, but i may have to add a path to the environ variable so python knows where to look for my scripts.

Hopefully this will let me keep my 10.0 py scripts intact and i can better manage the params using the 10.1 pyt structure.

One thing about 10.1 - we are spending alot of time figuring out where ags puts things during a publish and then trying to rig things to work better than that.

thanks for the tip.

Re: Anyone using the new Python Toolboxes at 10.1?

Luke_Pinner — Sat, 11 Dec 2021 10:55:58 GMT

If you didn't want to modify your existing script tools to convert them to classes, then you could do that (calling your scripts from the execute method of each Tool class). I was thinking more of just converting the scripts to classes though, especially if you're writing stubs for those classes anyway.

How you could do it depends on how you've written your scripts. If your existing 10.0 scripts are basic scripts where all the code is in the top-level scope, then in the execute method of the new Tool class you could use the subprocess module to call python and pass it the path to the 10.0 script. If your 10.0 scripts are organised into functions, you might be able to do something like:

#----------------
# 10.0 script tool
# script_10_0.py
#----------------
import arcpy
def do_some_stuff(arg1,arg2):
    print arg1,arg2

if __name__ == '__main__':
    arg1=arcpy.GetParameterAsText(0)
    arg2=arcpy.GetParameterAsText(1)
    do_some_stuff(arg1,arg2)

#----------------------
# 10.1 Python Toolbox
#----------------------
import arcpy
import script_10_0 #import the old 10.0 script tool so you can call its functions.

class Toolbox(object):
    def __init__(self):
        """Define the toolbox (the name of the toolbox is the name of the
        .pyt file)."""
        self.label = "10_0_Tool_Toolbox"
        self.alias = "Toolbox with 10.0 Tools"

        # List of tool classes associated with this toolbox
        self.tools = [Tool1]

class Tool1(object):
    #...
    def execute(self, parameters, messages):
        arg1=parameters[0].valueAsText
        arg2=parameters[1].valueAsText
        script_10_0.do_some_stuff(arg1,arg2)

Re: Anyone using the new Python Toolboxes at 10.1?

KevinGooss — Fri, 07 Sep 2012 13:47:37 GMT

that is pretty much what i did. For each individual py script i had i changed the signature and made a main def.
then in the pyt i import each of those, then call the script by name.main.
i like this because each of my scripts was pulling in a params json file. now i can pull that params file into the pyt once and dish it off to each side script.

All that remains is for me to find a way around esri wanting me to run each script in arcmap and publish the results.
if i could find a way to just publish the pyt once then throw the extra scripts in there and have them recognized that would be the bomb

Re: Anyone using the new Python Toolboxes at 10.1?

Luke_Pinner — Sat, 11 Dec 2021 10:56:01 GMT

if i could find a way to just publish the pyt once then throw the extra scripts in there and have them recognized that would be the bomb

Something like the following might work:

#  \--SomeDir
#     |  toolbox.pyt
#     |  some_script.py
#     |  another_script.py
#     |  ...
#     |  last_script.py
#
# Each *.py contains a class called Tool that's just a stub for your "main" def.

#----------------------------
#The .pyt file (all Tools are dynamically imported not hardcoded)
#----------------------------

import arcpy
import os,sys,glob

class Toolbox(object):
    def __init__(self):
        """Define the toolbox (the name of the toolbox is the name of the .pyt file)."""
        self.label = "MultiTool_Toolbox"
        self.alias = "Toolbox with Multiple Dynamically Imported Tools"

        # Dynamic list of tool classes associated with this toolbox
        path=os.path.dirname(__file__)
        self.tools=[]
        for py in glob.glob(os.path.join(path,'*.py')):
            if not __file__ in py:
                module=os.path.basename(py)[:-3]
                #Note: "Tool" class must have same 
                #name as the script "Tool" classes
                self.tools.append(__import__(module).Tool)

#----------------------------
#The .py script files
#----------------------------

#"Tool" class can be called anything but must be 
# the same in each script and in the calling toolbox

class Tool(object):

    # tool/parameter setup methods etc...

    def execute(self,*args,**kwargs):
        main(*args,**kwargs)

def main(*args,**kwargs):
    #do stuff
    return

if __name__=='__main__':
    #Run script
    main(sys.argv[1:])

Re: Anyone using the new Python Toolboxes at 10.1?

KevinGooss — Tue, 18 Sep 2012 12:37:03 GMT

Still not where i want to be on this one unfortunately.
I can get all the py and pyt files in order but that still doesn't get me around having to use ArcMap to create the sd file.
And when that sd file gets published all of the sudden there are lots of new files and folders and my code is being modified by esri hand-holding.

When i publish the sd esri's convoluted directory structure makes this:

-GPServiceName.GPServer
-extracted
.GPServiceName.sd

then in the extracted folder we have:
-esriinfo
-v101
.manifest.xml
.serviceconfiguration.json
tilingservice.xml

under esriinfo we have more:
-metadata
-thumbnail
.iteminfo.xml

thumbnail is just an image, the metadata folder has a file named metadata.xml

under v101 we have:
afoldernamedafterwhereyouputyourscripts
GPServiceName.tbx
GPServiceName.rlt

the tbx and rlt are binary
under the folder that esri creates based on your script folder you have:
all of your py files, pyc versions for each, a param text file that i was using and esri dragged in, along with any 'helper' py files
the pyt file for your original python toolbox
a pyt.xml for the toolbox
a Tool.pyt.xml for the toolbox
pyt.xml files for each Tool in the toolbox

Python toolboxes provide the opportunity to take advantage of your Python skills and create tools entirely and easily out of Python

perhaps. but getting them published is a different story.

Say i want to add a Tool to my currently published GPService? Not fun at all. I have to open my pyt, edit it, open it in ArcMap, run EVERY OTHER TOOL including my new tool and then publish?

Certainly i'm missing some very important in this workflow.

At ArcGIS 9.0, the script tool framework was first introduced and is geared towards creating Python-based tools for new users, but it is an inefficient process for more experienced users. In the script tool framework:
You define parameters through the wizard
You create validation code that lives in the toolbox
Plus you create and maintain the source script separately
All of these parts are segregated and more difficult to manage collectively.

That workflow is starting to look a lot better in hindsight.

Re: Anyone using the new Python Toolboxes at 10.1?

curtvprice — Sun, 27 Sep 2015 19:54:14 GMT

This is somewhat similar to how the Spatial Analyst Supplemental tools area set up. This structure has the advantage of the .py files could be run either from the .pyt or added as pyx script tools.

I am finally getting into this and really appreciate the discussion and Luke's examples.

Another tip on tbx's that is helping me is to run the Jason Scheirer's tbx2pyt tool, and then stealing the results to copy and paste into the getParameterInfo() code, which seems to be the most time-consuming part of this.

One thing about this that is kind of unnerving is there really isn't a very standard way to do it (yet). That will take time I guess, as people do different things for example

Re: Anyone using the new Python Toolboxes at 10.1?

StevenGonzalez1 — Sat, 11 Dec 2021 10:56:04 GMT

I'm quite late, but I haven't been able to get this method to work. The syntax on the .pyt is OK (i.e.: I'm able to view the toolbox within arcmap without it displaying the red X), but when I click the drop-down for the toolbox, it displays no tools.

My directory is set up as such:

Tools/

- bhaTools.pyt

- publishing.py

-findingpath.pyc

The findingpath.pyc:

def findpath():
  local_path = os.path.abspath(os.path.dirname(__file__))
  return local_path
‍‍‍‍‍‍‍

The bhaTools.pyt has the following:

import contextlib
import os
import sys
import arcpy
import glob

class Toolbox(object):
    def __init__(self):
        self.label = u'bhaTools'
        self.alias = 'bhaTools'
        import findingpath
        path = findingpath.findpath()
        self.tools=[]
        for py in glob.glob(os.path.join(path,'*.py')):
            if not __file__ in py:
                module=os.path.basename(py)[:-3]
                #Note: "Tool" class must have same
                #name as the script "Tool" classes
                self.tools.append(__import__(module).Tool)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The publishing.py:

class Tool(object):
    def __init__(self):
        self.label = u'Publish Parcel Fabric Parcels'
        self.canRunInBackground = False
        self.category = ["Parcel_Fabric_Migration_Tools"]

    def getParameterInfo(self):
        param_1 = arcpy.Parameter()
        param_1.name = u'inputParcelFabricParcels'
        param_1.displayName = u'Input Parcel Fabric Parcels'
        param_1.parameterType = 'Required'
        param_1.direction = 'Input'
        param_1.datatype = u'DEFeatureClass'


        #### .... more parameters .... #### 

        return [param_1,param_2,param_3,param_4,param_5]

    def isLicensed(self):
        return True

    def updateParameters(self, parameters):
        validator = getattr(self, 'ToolValidator', None)
        if validator:
            return validator(parameters).updateParameters()

    def updateMessages(self, parameters):
        validator = getattr(self, 'ToolValidator', None)
        if validator:
            return validator(parameters).updateMessages()


    # tool/parameter setup methods etc...
      
    def execute(self,*args,**kwargs):
        main(*args,**kwargs)

def main(*args,**kwargs):
    try:
        # do stuff with params
    except arcpy.ExecuteError:
        arcpy.AddError(arcpy.GetMessages())


if __name__=='__main__':
    #Run script
    main(sys.argv[1:])‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍