Python in ArcGIS Pro 3.6 FAQ
Q: Why does a new instance of ArcGIS Pro launch when I run %pip from a notebook cell?
A: This is a known issue with embedded Python interpreters (such as what is running in ArcGIS Pro). Magic commands use the output of sys.executable to determine the host process, which in this case launches ArcGISPro.exe and not pythone.exe. You can avoid this by prefixing the pip command with ! instead of %. Note that we recommend using conda over pip.
Q: Where can I find the option for restarting the kernel?
A: You currently cannot restart the kernel unless you close and reopen ArcGIS Pro. You can, however, interrupt the kernel using the Interrupt Kernel button on the Notebook ribbon. While not a complete workaround, you can also use the %reset -f magic command to clear all variables in the namespace.
Q: Can I use R or C# in my notebook instead of Python?
A: You can only open a Python kernel in notebooks running in ArcGIS Pro, other languages are not supported.
Q: I'd like to export my notebook (.ipynb) to a Python script (.py), is that possible?
A: Yes, you can export a notebook to a Python (.py) or HTML (.html) file from the Notebook ribbon’s Export button.
You can also use the Jupyter command nbconvert from the command prompt, or you can run the command directly from your notebook by preceding the command with an exclamation mark (!).
!jupyter nbconvert --to script "Path\To\Your_Notebook.ipynb"
The resulting Your_Notebook.py file will be in the same directory as your notebook, Your_Notebook.ipynb.
Additionally, %%writefile filename.py written on the first line of any cell will write only that cell to filename.py.
Q: Can I open a Jupyter notebook in ArcGIS Pro that I have previously created outside of Pro?
A: Yes, notebooks in ArcGIS Pro are simply Jupyter Notebooks (.ipynb) and any notebook created elsewhere (such as stand-alone in the browser, in some external IDEs, and hosted notebooks in ArcGIS Enterprise or ArcGIS Online) can be run in ArcGIS Pro. However, it is your responsibility to ensure any libraries used in these other environments are available in the active environment of ArcGIS Pro.
Q: Can I use my notebook created in ArcGIS Pro across other platforms?
A: Notebooks in ArcGIS Pro are simply Jupyter Notebooks (.ipynb) and can be run by any applications that support them, including stand-alone in the browser, in some external IDEs, and hosted notebooks in ArcGIS Enterprise or ArcGIS Online. However, it is your responsibility to make sure that any libraries used in the notebook are available in these other environments.
- Using notebooks hosted on Enterprise Portal in ArcGIS Pro: There are several options for moving a hosted notebook to your local machine.
- igned into a portal within ArcGIS Pro. From the Catalog pane in ArcGIS Pro, go to Portal > My Content, right-click a notebook item, and select Download and Open.
- Signed into Portal on the web browser. From the Notebook Editor in Portal, on notebook menu ribbon, click File> Download as > notebook (.ipynb).
- Signed into Portal on the web browser. From the notebook's Item Details page (accessible from the Content tab by clicking on a notebook item), click Download to download the notebook in .ipynb format.
- Using Notebooks created in Pro on my Enterprise Portal:
- You can upload any notebook file (.ipynb) to your portal. Sign into your organization's Portal from the browser, go to the Content tab of the contents page, click Add Item > From your computer and choose From my computer. From the Add an item from my computer dialog, click Choose File and browse to a notebook file on your computer, provide a title and tags, and then click Add Item.
- You can upload any notebook file (.ipynb) to your portal. Sign into your organization's Portal from the browser, go to the Content tab of the contents page, click Add Item > From your computer and choose From my computer. From the Add an item from my computer dialog, click Choose File and browse to a notebook file on your computer, provide a title and tags, and then click Add Item.
Q: When opening a notebook in ArcGIS Pro, why did I receive a "Failed to load notebook" error?
A: Notebooks in Pro leverage a webserver run as localhost on port 8778. If it cannot be reached or is otherwise interfered with, notebooks will fail to load. Known causes and solutions:
- Bad Jupyter configuration (jupyter_notebook_config.py): Modifying the c.NotebookApp.ip or c.NotebookApp.port configuration options will prevent ArcGIS Pro from opening notebooks. To resolve, comment out or reset these values.
- Using a proxy server: If using a proxy server, set an exclusion on your proxy settings to prevent localhost:8778 (or 127.0.0.1:8778) from going through the proxy. You'll need to work with your IT group to add this exclusion, depending on the proxy you use and how it is implemented.
- Paths with special characters may cause issues. Avoid special characters, specifically the apostrophe (') is known to cause issues when opening and saving notebooks in ArcGIS Pro.
Q: Can I open a notebook in Pro from a UNC path?
A: UNC paths are supported as of ArcGIS Pro 3.0 - just be careful about collisions as this opens up the possibility of multiple users editing the same file. If you are still on an earlier version of ArcGIS Pro and unable to upgrade, the best workaround is mapping the UNC path to a drive or alternatively creating a symbolic link.
Q: I keep clobbering variables when I have multiple notebooks open in Pro. Why are variable names being shared across multiple notebooks?
A: ArcGIS Pro runs a single Python instance. This execution context is shared by each notebook opened in a single instance of ArcGIS Pro. Depending on the contents of concurrently open notebooks, this could potentially result in variable name collisions. To avoid potential issues you may consider:
- Using unique variable names across all concurrently running notebooks
- Running the %reset -f magic command as needed to delete all the variables in the namespace
Q: Why are the environment variables I set in Notebooks (or Python Window) not being honored by Geoprocessing tools I run from the Geoprocessing pane?
A: This behavior is due to the environment levels hierarchy.
When setting an environment variable using arcpy.env, you are setting it at the Script level. Since this is the lowest level in the hierarchy, the setting will not be honored at any of the above levels. In order for your environment variables to be honored by both Script and Geoprocessing tools, you must set the environment variables using the Application settings from your project.
Other
Q: Can I use Powershell 7 as my terminal session for Python?
A: ArcGIS Pro's Python Command Prompt (based off Microsoft's Command Prompt) is the standard way to run ArcGIS Pro Python scripts from a terminal session.
With ArcGIS Pro 3.6 it now is possible to use PowerShell 7 as an alternative to the Python Command Prompt. PowerShell 7 is Microsoft's latest command-line and scripting language.
Here are the one-time steps to set up to use PowerShell 7 with ArcGIS Pro's python environment
- Install PowerShell 7 from the Microsoft Store or from Windows PowerShell run winget --id Microsoft.PowerShell --source winget
- From the Windows start menu launch the Python Command Prompt and run conda init powershell
- This step requires admin credentials
Your machine should now be setup.
From the Windows start menu you can now launch PowerShell 7 and then type activate arcgispro-py3 to activate the ArcGIS Pro python environment. Then:
- Run python -c "import arcpy;print(arcpy.version)" to confirm you're setup correctly (should print ArcGIS Pro's version)
- Run python to launch an interactive REPL session
- Run python c:\myproject\myscript.py to run a script
Server 12.0:
PowerShell 7 can also be used with Server 12.0 deployment, simply use Python 3 Command Prompt instead of Python Command Prompt in the instructions above.
Known issue and limitations:
At this time the following commands (.bat files) commonly used with ArcGIS Pro are not supported in PowerShell: proenv, propy, proswap.
Hi @HannesZiegler ,
I unfortunately have identified two other issues in Pro 3.6:
1) - The Get Count geoprocessing tool now fails with a non(!) enterprise geodatabase enabled ordinary PostgreSQL enterprise database feature class. This used to work in Pro 3.5 and below. According to the "What's new in Pro 3.6", the Get Count tool had changes applied for Pro 3.6, that seems to break this option. See the images entirely below.
2) - Using a Python "concurrent.futures.ThreadPoolExecutor" seems to make the geoprocessing in Pro 3.6 hang at the end of the thread pool processing if the processing is run in the Pro Foreground Thread. I do not see these hangs if Pro 3.6 is set to run in the background Geoprocessing Thread, which is the default.
@JeffBarrette and @HannesZiegler ,
I have been digging a bit deeper:
The example code I showed you regaring the CIM issue was a slightly modifed version of the original. The original is below:
# Change the label class "label priority ranking" 'priority' value
for cimLabelClass in cimLyr.labelClasses:
# Double check we got a label class
if type(cimLabelClass).__name__ == "CIMLabelClass":
try:
if missingLabelPriority == False:
arcpy.AddWarning("labelPriorityByLayerDict: {}".format(labelPriorityByLayerDict))
cimLabelClass.priority = labelPriorityByLayerDict[cimLabelClass.name]
arcpy.AddWarning("CIM LabelClass priority: {}".format(cimLabelClass.priority))
else:
maxLabelPriority += 1
cimLabelClass.priority = maxLabelPriority
arcpy.AddWarning("CIM LabelClass priority: {}".format(cimLabelClass.priority))
except:
arcpy.AddWarning("*** WARNING ***: Failed to set the 'Label Priority Ranking' Maplex label engine setting of layer '{}'!".format(lyrToAdd.name))
passNow, obviously, I am getting an exception on the Foreground Thread. But besides the CIM edit, there is also a dictionary lookup going on ('labelPriorityByLayerDict'). Now technically and programatically, that dictionary is guaranteed to contain the key being requested, so shouldn't error out, as it is in fact based on reading the exact same set of layer files being modified here.
However, to get further insight, I first re-tested my previous observations, confirming what I saw before:
This again shows the Foreground Thread throwing an exception during the CIM edit, but not the Geoprocessing Thread.
I then decided to also test the contents of the dictionary:
As you can see from the last two images, the dictionary is SUCCESSFULLY created in BOTH cases and contains the value 1 to set as label priority. This now unequivocally tells me the issue is with the actual CIM edit. I see no other way based on these results:
cimLabelClass.priority = labelPriorityByLayerDict[cimLabelClass.name]
@JeffBarrette and @HannesZiegler ,
I found the culprit. It is not the setting of the priority, but getting the CIMLabelClass.name property that fails on the Foreground Thread, since that is used as lookup key in the dictionary, it raises a KeyError:
I now ran the workflow with ArcGIS Pro attached to the Visual Studio Code debugger using the ESRI debugger extension. I don't think it turns up much new information, but as you can see, it is not just the label class name that got reset to 'Class 1', but all label class properties seem to have been reset to defaults in the CIM object.
Please note I do appreciate your efforts to try to reproduce it. I just think it needs a bit more digging and effort from the ESRI side.
As I already explained to @HannesZiegler , this particular piece of code is part of larger toolbox with some 20k lines of Python code, with a few dozen Python modules. I honestly cannot transfer the whole thing.
And that is what I mean with "there is a *lot* more going on".
However, I am almost 100% sure we can safely ignore that fact in relation to this is issue.
Why?: What it basically does is implement a sophisticated generalization workflow that generates a 150+ feature classes in PostgreSQL database. At the end of that processing, so when all the true geoprocessing work is done, the script from which I extracted the snippets, is called. It loads layer files from a custom style I developed (just an ordinary folder with *.lyrx files managed by the code, nothing special), and uses the symbology of those style layers in combination with the
symbolizedLayer = arcpy.ApplySymbologyFromLayer_management(featureLayer,symbologyLayerFile,inputSymbologyFields,update_symbology="MAINTAIN")[0]
tool to update the symbology of Query Layers that were created during the previous generalization geoprocessing workflow and that point to the feature classes in the PostgreSQL database. It also transfers any label classes from the style layer to the Query Layers.
Granted, the transfer of all of theses attributes / settings of one layer to another I think has become easier in Pro 3.6 with new options if I remember it well from the release notes, but my current code is based on what was possible in the past.
That said, as you may now understand, by far the majority of "there is a *lot* more going on", is really unrelated to anything to do with actual feature layer objects in the TOC. It is database work and processing. Only a fraction of the code touches upon the parts that might be relevant to this issue.
By the way, if by now you are wondering what the output of this whole workflow is, it is the below: topographic maps based on OpenStreetMap data...
