Python in ArcGIS Pro 3.3 FAQ

3 Kudos

Frequently Asked Questions

Python Distribution

Q: What version of Python and/or <insert package name> are currently installed with the default Python environment in ArcGIS Pro (arcgispro-py3)?

See Available Python libraries—ArcGIS Pro | Documentation

Python Script Tools

Q: My custom ATBX script tool that uses dunder file (__file__) no longer works after I upgraded from an earlier version ArcGIS Pro to 3.2 or greater, why?

At Pro 3.2, the value in the __file__ variable was changed for embedded scripts (for both validate and execute) inside .atbx files.

Normally this __file__ variable is used to reference external resources that are deployed in a folder relative to the .atbx.

At 3.1 the value in __file__ was:

During validate : C:\Path\MyToolbox.atbx#MyTool_MyToolbox.UpdateParameters.py
During execute : C:\Path\MyToolbox.atbx#MyTool_MyToolbox.py

At 3.2 the value in __file__ is:

During validate : C:\Path\MyToolbox.atbx\MyTool.tool\tool.script.validate.py
During execute : C:\Path\MyToolbox.atbx\MyTool.tool\tool.script.execute.py

Note: this only affects validate and execute scripts inside a .atbx file. Validate and execute scripts stored outside the .atbx (as a .py file on disk) are unaffected. Additionally, embedded scripts in a .tbx file are unaffected.

Updating the affected code:

The affected code we typically see is os.path.dirname(__file__).

The proposed replacement code is os.path.dirname(__file__.split(".atbx")[0]).

This proposed replacement code will work in embedded validate as well as embedded execute scripts. This code will produce desired and consistent result with the __file__ in Pro 3.2 as well as earlier releases.

import os

# Show how various mock __file__ values from 3.1 & 3.2 are processed 
# using the proposed method:
for __file__ in [
    r"C:\Path\MyToolbox.atbx#MyTool_MyToolbox.py", # execute Pro 3.1
    r"C:\Path\MyToolbox.atbx#MyTool_MyToolbox.UpdateParameters.py", # validate Pro 3.1
    r"C:\Path\MyToolbox.atbx\MyTool.tool\tool.script.execute.py", # execute Pro 3.2
    r"C:\Path\MyToolbox.atbx\MyTool.tool\tool.script.validate.py", # validate Pro 3.2
]:

    print(os.path.dirname(__file__.split(".atbx")[0]))

Notebooks in ArcGIS Pro

Q: How does the experience of running an ArcGIS Notebook in ArcGIS Pro compare with running a Jupyter Notebook outside of Pro?

A: With notebooks in ArcGIS Pro, you can open a map view and a notebook view side-by-side, allowing you to visualize and directly interact with the data.

Layers and other contents of your map can be accessed in your notebook by name. To access data that is not currently in your table of contents, use the full path to the data. You can drag and drop an item from the contents pane or the catalog directly into the notebook.
If you have selected data in the map view, geoprocessing tools running from the notebook honor the selections.
Global geoprocessing environment settings are respected and can be overridden by environment settings set from the notebook.
The output of geoprocessing tools is added to the active map by default.
When a notebook view is active, the Notebook ribbon will appear. From here, you can create a new notebook, save the current notebook, export the current notebook to a different format, or interrupt the kernel.
The result of running a geoprocessing tool from the notebook will be logged in the geoprocessing history.
Geoprocessing workflows can occur during an ongoing editing session. If a geoprocessing tool modifies a feature being edited in an edit session (it does not save the result of geoprocessing as another feature class), you can save or discard the modifications from the Edit ribbon. Note that the Map view must be active in order for the Edit ribbon to be accessible.

Q: Can I use the standard Jupyter Notebook keyboard shortcuts?

A: Keyboard shortcuts work just like they do in Jupyter Notebook.

Starting at 3.0 we intentionally disabled the print, browse forward, and browse backward shortcuts, and enabled the zoom and find shortcuts.

Q: Can I use magic ( % ) and shell ( ! ) commands in my notebook?

A: Yes! Many magic and shell commands are supported since ArcGIS Pro 2.6. Magic and shell commands are a convenience that enhance data analysis and offer a bridge for executing shell commands without leaving your notebook. Certain magic and shell commands are unsupported, including but not limited to %history, !ipconfig, and !ping.

For more information regarding magic and shell commands, see magic commands and shell commands.

Q: Why am I getting the error "The system cannot find the path specified." when running conda magic or shell commands?

A: This error may appear if you have a separate installation of conda on your machine, and you have a PATH environment variable pointing to that installation of conda. In this case, you will need to remove the environment variable from PATH. To confirm:

Pess the windows key and search "Edit the system environment variables".
On the System Properties dialog box, under the Advanced tab, click the Environment Variables button.
Locate the entry called Path in the user variables and the system variables lists.
Open (double click) Path from the lists, and in the Edit environment variable dialog box make sure no paths point to a folder containing conda.exe (highlight the entry, then click the Delete button), with the exception of "C:\Program Files\ArcGIS\Pro\bin\Python\Scripts".
NOTE: Make sure to note down any paths before deleting in case you need to restore them later.

Q: Why are figures from matplotlib not showing up in the cell output of my notebook?

A: Inline visualizations from matplotlib are supported since ArcGIS Pro 2.6. Starting at ArcGIS Pro 3.1, you no longer need to call matplotlib.pyplot.show to display the plot inline. However, it is still recommended that you use the show method to avoid displaying the object representation string of your plot, which may be unnecessary or distracting. For example:

%matplotlib inline

# Imports
import matplotlib.pyplot as plt
import pandas as pd
import arcgis

# Create spatially enabled data frame (SEDF)
sedf = pd.DataFrame.spatial.from_featureclass("USA_States_Generalized")

# Create plot
plt.bar(sedf.STATE_NAME, sedf.POP2010)

# Update plot elements
plt.suptitle("Population of New England States", fontsize=16)
plt.ylabel("Population (in millions)")
plt.xticks(rotation=45)

# Draw the plot
plt.show()

Q: Why does a new instance of ArcGIS Pro launch when I run %pip from a notebook cell?

A: This is a known issue with embedded Python interpreters (such as what is running in ArcGIS Pro). Magic commands use the output of sys.executable to determine the host process, which in this case launches ArcGISPro.exe and not pythone.exe. You can avoid this by prefixing the pip command with ! instead of %. Note that we recommend using conda over pip.

Q: Where can I find the option for restarting the kernel?

A: You currently cannot restart the kernel unless you close and reopen ArcGIS Pro. You can, however, interrupt the kernel using the Interrupt Kernel button on the Notebook ribbon. While not a complete workaround, you can also use the %reset -f magic command to clear all variables in the namespace.

Q: Can I use R or C# in my notebook instead of Python?

A: You can only open a Python kernel in notebooks running in ArcGIS Pro, other languages are not supported.

Q: I'd like to export my notebook (.ipynb) to a Python script (.py), is that possible?

A: Yes, you can export a notebook to a Python (.py) or HTML (.html) file from the Notebook ribbon’s Export button.

You can also use the Jupyter command nbconvert from the command prompt, or you can run the command directly from your notebook by preceding the command with an exclamation mark (!).

!jupyter nbconvert --to script "Path\To\Your_Notebook.ipynb"

The resulting Your_Notebook.py file will be in the same directory as your notebook, Your_Notebook.ipynb.

Q: Can I open a Jupyter notebook in ArcGIS Pro that I have previously created outside of Pro?

A: Yes, ArcGIS Notebooks are simply Jupyter Notebooks and any notebook can be run in ArcGIS Pro. However, it is your responsibility to ensure any libraries used in these other environments are available in the active environment of ArcGIS Pro.

Q: Can I use my ArcGIS Notebook created in Pro across other platforms?

A: ArcGIS Notebooks are simply Jupyter Notebooks (.ipynb) and can be run in any Jupyter environment or as hosted ArcGIS Notebooks in ArcGIS Enterprise or ArcGIS Online. However, it is your responsibility to make sure that any libraries used in the notebook are available in these other environments.

Using Notebooks hosted on Enterprise Portal in ArcGIS Pro: There are several options for moving a hosted notebook to your local machine.

Signed into a portal within ArcGIS Pro. From the Catalog pane in ArcGIS Pro, go to Portal > My Content, right-click a notebook item, and select Download.
Signed into Portal on the web browser. From the Notebook Editor in Portal, on notebook menu ribbon, click File> Download as > notebook (.ipynb).
Signed into Portal on the web browser. From the notebook's Item Details page (accessible from the Content tab by clicking on a notebook item), click Download to download the notebook in .ipynb format.

Using Notebooks created in Pro on my Enterprise Portal:
- You can upload any notebook file (.ipynb) to your portal. Sign into your organization's Portal from the browser, go to the Content tab of the contents page, click Add Item > From your computer and choose From my computer. From the Add an item from my computer dialog, click Choose File and browse to a notebook file on your computer, provide a title and tags, and then click Add Item.

Q: The Cell menu is missing from my notebook... why?

A: This can occur when Jupyter Notebook runs out of allocated memory. Try clearing all cell outputs (you will need to use the Command Palette since the Cell menu is missing):

Enter command mode (selected notebook cell is blue).
Press the p key to bring up the Command Palette.
Type "clear all cells output" and press enter.

You may need to reload the notebook.

To prevent this from happening, try to minimize large cell outputs. For example, if a DataFrame is very large and you don't need to view all records in the table, use df.head(x) to display only the first x records.

You can also try increasing the memory limit.

Generate the Jupyter configuration file jupyter_notebook_config.py using the Python Command Prompt.
jupyter notebook --generate-config
Open the jupyter_notebook_config.py (it should be in %userprofile%\.jupyter), and edit the following property (replace <your limit> with your desired memory limit):
NotebookApp.max_buffer_size = <your limit>

Q: When opening a notebook in ArcGIS Pro, why did I receive a "Failed to load notebook" error?

A: There are various possible causes for this. Known causes and solutions:

Bad Jupyter configuration (jupyter_notebook_config.py): Modifying the c.NotebookApp.ip or c.NotebookApp.port configuration options will prevent ArcGIS Pro from opening notebooks. To resolve, comment out or reset these values.
Using a proxy server: Using the http_proxy and https_proxy environment variables prevent ArcGIS Pro from opening notebooks. To resolve, unset these environment variables.

Q: Can I open a notebook in Pro from a UNC path?

A: UNC paths are supported as of ArcGIS Pro 3.0 - just be careful about collisions as this opens up the possibility of multiple users editing the same file. If you are still on an earlier version of ArcGIS Pro and unable to upgrade, the best workaround is mapping the UNC path to a drive or alternatively creating a symbolic link.

Q: I keep clobbering variables when I have multiple notebooks open in Pro. Why are variable names being shared across multiple notebooks?

A: ArcGIS Pro runs a single Python instance. This execution context is shared by each notebook opened in a single instance of ArcGIS Pro. Depending on the contents of concurrently open notebooks, this could potentially result in variable name collisions. To avoid potential issues you may consider:

Using unique variable names across all concurrently running notebooks
Running the %reset -f magic command as needed to delete all the variables in the namespace

Q: Why are the environment variables I set in Notebooks (or Python Window) not being honored by Geoprocessing tools I run from the Geoprocessing pane?

A: This behavior is due to the environment levels hierarchy.

When setting an environment variable using arcpy.env, you are setting it at the Script level. Since this is the lowest level in the hierarchy, the setting will not be honored at any of the above levels. In order for your environment variables to be honored by both Script and Geoprocessing tools, you must set the environment variables using the Application settings from your project.