ArcGIS Pro bug?: Can't import custom Python module in "Calculate Value" tool of ModelBuilder

[MarcoBoeringa]

NOTE: This post is NOT about pure Python toolboxes, but ordinary ModelBuilder toolboxes enhanced with Python scripts.

Can ESRI confirm this? I am running into an issue where it seems Pro won't import custom Python modules / scripts in a Calculate Value tool of ModelBuilder. I can successfully import default ArcGIS Python modules like io etc, just not my own modules written as scripts. Pro bails out with a "ModuleNotFoundError".

To avoid confusion, with "import", I mean using a Python module import statement in the Python code textbox of the Calculate Value tools, so like:

import os

import MyModule

This doesn't happen in ArcMap at all. Even if the module is not imported in the toolbox, ArcMap will find it if it is located in the same folder as the toolbox. Pro fails both with a script in the same folder as the toolbox, as well as with imported scripts.

See the attached screenshot from Pro 2.1.2.

[DanPatterson_Retired]

Marco... perhaps models are set up differently.  I import my own modules (aka scripts) into my scripts in conventional toolboxes all the time, where they are found if they are located in the same location as the toolbox's script.

Export your model to a script to see if the paths are hardcoded or not

[MarcoBoeringa]

Dan,

This is not about importing scripts / modules into another script / module part of a conventional ModelBuilder toolbox. Like you say, that works both in ArcMap and Pro.

The problem is specifically with custom scripts / modules imported with the Python import statement inside the code block of a Calculate Value "ModelBuilder-only-tool". That works in ArcMap, but fails in Pro if the imported module is a custom script I wrote myself, so not part of the default ArcGIS Python install.

[DanPatterson_Retired]

Ok... so it doesn't work the same way.  there is too much % this % that stuff in the help topic to figure out and there is no mention of imports.... sorry

[DanPatterson_Retired]

curtvprice‌ you use modelbuilder.  in Pro?

[curtvprice]

I made a little test model tool to look at this. I created it in ArcMap and created it in Pro. Comparing the results, I found something very interesting, that you can see from the Python prompts.  Pro's sys.path does not have "" as the first element as we normally see in most Python implementations!

Note, my experience is that os.curdir (which is what the blank entry means in the sys.path) is not the folder with the tbx, but the current ArcMap home folder (where the mxd is saved) or the Pro project folder.  Would like to know a good way to do your use case (find the current folder where the model's tbx lives) no matter where the tbx is relative to the map document or project.

ArcMap 10.5's Python window:

>>> import sys
>>> sys.path[0]
''‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

From Pro's Python window:

import sys
sys.path[0]
'c:\\program files\\arcgis\\pro\\Resources\\arcpy'‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I think you can get your script to work by adding the current folder to the sys.path. Maybe.

# Calculate value expression:  f()
# Code block:
import sys
sys.path.insert(0, "")
import ### your module ###
def f():
  # do stuff
  return ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

My guess is this is a feature not a bug. Shaun Walbridge‌ do you have any enlightenment for us?

[ShaunWalbridge]

 Curtis Price‌ Thanks for digging into this. I'm doing some research on my end, and will post here when that's done. For now, I recommend doing:

import sys
sys.path.insert(0, '')

In the code block, as you mentioned in your second example. This is the exact behavior of a typical Python process, and will prepend the working directory of the current Pro project to the search path for Python code. There are valid reasons to block this behavior in an embedded use, but I'm checking if they apply to Pro specifically.

Cheers,

Shaun

[curtvprice]

Can ESRI confirm this? 

Just a note: if you want to report a bug, you need to open a support incident with support.esri.com -- this is a community site which is useful, but not the place to report bugs.

[MarcoBoeringa]

Thanks Curtis for all the detective work. Yes, I know about the support stuff, but as I wasn't exactly sure if this was a "bug" or, as you stated, a possible "feature", so I considered posting here more appropriate for now.

In addition, since this seemed to be a kind of trivial issue related to path interpretation, I hoped someone in the community like you, would have ideas for a solution (as you did).

Unfortunately, despite implementing your solution in the top part of the code block of the Calculate Value tool, it still fails to locate the module in Pro, and errors out with the ModuleNotFoundError, as I showed in the screenshot of the first post.

As I also wrote there, the same model runs fine in ArcMap.

[curtvprice]

I'm thinking that in Pro the the home folder may not where you think it is. In your model, try setting up a Calculate Value tool that shows this path will help you check, below.

For your import to work the returned path will have to be the folder where the module you are trying to import lives.

# Calculate Value expression: test()
# Code block:
import os
def test():
    return os.path.realpath(os.curdir)‍‍‍‍‍‍‍‍‍‍