Latest Contributions by GhislainPrince

Frequently Asked Questions   ArcPy Q: After migrating to 3.7, why is my existing arcpy code throwing "I/O operation on closed file" when using arcpy.da Cursors? A: Context managers (special python objects created with the with statements) are expected to release and clean up resources when they exit. Prior to 3.7, arcpy.da cursors exhibited unintended and undocumented behavior where the cursor after the context manager exited which violated context manager expectations. # Pre-3.7 behavior with arcpy.da.UpdateCursor(...) as cursor: for row in cursor: cursor.updateRow([...]) cursor.reset() # Should fail, but did not for row in cursor: pass # Should fail, but did not Starting at Pro 3.7 the cursor object is invalidated when the context manager exists and trying to use it out of the context will now raise "ValueError: I/O operation on closed file" If you need to use a cursor make a second iteration through the table you'll want to create a fresh cursor object. Q: Is there an Object Model Diagram or Cheatsheet for ArcPy? A: While we have decided against producing an Object Model Diagram for reasons outlined here, we do offer an ArcPy Cheatsheet.   Python Distribution Q: What version of Python and/or <insert package name> are currently installed with the default Python environment in ArcGIS Pro (arcgispro-py3)? A: See Available Python libraries—ArcGIS Pro | Documentation topic in our online documentation. Additionally the ArcGIS Pro Python distribution by release.pdf contains the package set for recent releases and is available to download from github.com/Esri/arcpy  Q: The repair and/or upgrade buttons on the environment manager fail to repair/upgrade my custom environment. A: One possible cause of this, is due to how the environment that is failing to repair/upgrade was originally created. When cloning arcgispro-py3, be sure to use the package manager to create the clone. If you are making a clone using the Python Command Prompt, be sure to include the --pinned flag with the conda create --clone command. Use this flag to ensure that the integrity of the cloned environment is maintained when upgrading or installing packages. Example: conda create --clone arcgispro-py3 --prefix %localappdata%\esri\conda\envs\arcgispro-py3-clone --pinned The --pinned flag, introduced by Esri, carries over the pinned file from the source environment to the cloned environment.  Q: My default arcgispro-py3 environment is broken and now ArcGIS Pro has issues. A: The default arcgispro-py3 environment is not meant to be customized as making changes to it can cause issues in ArcGIS Pro as it relies on specific versions of packages. In the event that you need to repair the default python environment back to it's original state, you can delete the environment folder located at C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 or if you are on a per user install it will be located at %localappdata%\Programs\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 and then using the ArcGIS Pro setup files run a repair installation. If you are still experiencing issues, you can delete the entire Python folder and run the repair installation to regenerate all the locally cached packages. If you delete the entire folder, be sure to preserve any cloned environments you may have made in this folder first. Q: Why is my NumPy code broken after I upgraded ArcGIS Pro to 3.6? A: At ArcGIS Pro 3.6, the NumPy version was upgraded from 1.26.4 to 2.2.0 - and with that major version change, there are some breaking changes. Please refer to NumPy's official migration guide for more information.   Python Scripts & Script Tools Q: My custom ATBX script tool that uses dunder file (__file__) no longer works after I upgraded from an earlier version ArcGIS Pro to 3.2 or greater, why? At Pro 3.2, the value in the __file__ variable was changed for embedded scripts (for both validate and execute) inside .atbx files. Normally this __file__ variable is used to reference external resources that are deployed in a folder relative to the .atbx. At 3.1 the value in __file__ was: During validate : C:\Path\MyToolbox.atbx#MyTool_MyToolbox.UpdateParameters.py During execute : C:\Path\MyToolbox.atbx#MyTool_MyToolbox.py At 3.2 the value in __file__ is: During validate : C:\Path\MyToolbox.atbx\MyTool.tool\tool.script.validate.py During execute : C:\Path\MyToolbox.atbx\MyTool.tool\tool.script.execute.py Note: this only affects validate and execute scripts inside a .atbx file. Validate and execute scripts stored outside the .atbx (as a .py file on disk) are unaffected. Additionally, embedded scripts in a .tbx file are unaffected. Updating the affected code: The affected code we typically see is os.path.dirname(__file__). The proposed replacement code is os.path.dirname(__file__.split(".atbx")[0]). This proposed replacement code will work in embedded validate as well as embedded execute scripts. This code will produce desired and consistent result with the __file__ in Pro 3.2 as well as earlier releases. import os # Show how various mock __file__ values from 3.1 & 3.2 are processed # using the proposed method: for __file__ in [ r"C:\Path\MyToolbox.atbx#MyTool_MyToolbox.py", # execute Pro 3.1 r"C:\Path\MyToolbox.atbx#MyTool_MyToolbox.UpdateParameters.py", # validate Pro 3.1 r"C:\Path\MyToolbox.atbx\MyTool.tool\tool.script.execute.py", # execute Pro 3.2 r"C:\Path\MyToolbox.atbx\MyTool.tool\tool.script.validate.py", # validate Pro 3.2 ]: print(os.path.dirname(__file__.split(".atbx")[0])) Q: I would like to schedule my script to run often (every day, every week, etc.), but after 15 days my scheduled script fails because ArcPy fails to import, why? Python scripts and script tools can be scheduled to run automatically, see Schedule geoprocessing tools—ArcGIS Pro | Documentation. When using the Named User license to authenticate ArcGIS Pro, the license token expires every 15 days, which causes import arcpy to fail once the token expires. There are two workarounds to avoid this: A. If possible, switch to a Single Use license.  B. If using a Named User license, your administrator may allow checking out the license offline for a maximum of 365 days, see details here. Please note that in both cases, the license can be used to run ArcGIS Pro only on the single machine on which it was licensed/checked out. Q: In 'ArcGIS Pro Debugger Extension' for Visual Studio Code why are breakpoints in the 'if__name__ == "__main__" block are not hit? A: This is a known limitation. Please put breakpoint in location other than if __name__ == "__main__":  Notebooks in ArcGIS Pro Since their integration into ArcGIS Pro 2.5, Notebooks in ArcGIS Pro have been based on the open-source IPython Jupyter Notebook. The classic IPython Jupyter Notebook has since been deprecated by the community in favor of a significant redesign with JupyterLab. This major redesign breaks backwards compatibility with extensions and many classic notebook features and customizations. At the ArcGIS Pro 3.4 release, notebooks have been upgraded and are now backed by JupyterLab 4.2.4 and Notebooks 7.2.1. This transition ensures continued updates with latest features, maintenance, and security patches. Q: How does the experience of running a notebook in ArcGIS Pro compare with running a Jupyter Notebook outside of ArcGIS Pro? A: With notebooks in ArcGIS Pro, you can open a map view and a notebook view side-by-side, allowing you to visualize and directly interact with the data. Layers and other contents of your map can be accessed in your notebook by name. To access data that is not currently in your table of contents, use the full path to the data. You can drag and drop an item from the contents pane or the catalog directly into the notebook. If you have selected data in the map view, geoprocessing tools running from the notebook honor the selections. Global geoprocessing environment settings are respected and can be overridden by environment settings set from the notebook. The output of geoprocessing tools is added to the active map by default. When a notebook view is active, the Notebook ribbon will appear. From here, you can create a new notebook, save the current notebook, export the current notebook to a different format, or interrupt the kernel. The result of running a geoprocessing tool from the notebook will be logged in the geoprocessing history. Geoprocessing workflows can occur during an ongoing editing session. If a geoprocessing tool modifies a feature being edited in an edit session (it does not save the result of geoprocessing as another feature class), you can save or discard the modifications from the Edit ribbon. Note that the Map view must be active in order for the Edit ribbon to be accessible. Q: Can I use the standard Jupyter Notebook keyboard shortcuts? A: Keyboard shortcuts work just like they do in Jupyter Notebook. Starting at 3.0 we intentionally disabled the print, browse forward, and browse backward shortcuts, and enabled the zoom and find shortcuts. At 3.4 we have also disabled the Command Palette shortcut CTRL+SHIFT+C, but will assess bringing it back in a later release. Q: Why are ipywidget interactive widgets not animating in the cell output of my notebook? A: With ArcGIS Pro 3.5, you no longer need to use the %matplotlib inline magic command. However, if using %matplotlib inline, it is still recommended that you use the matplotlib.pyplot.show method to ensure the plot is refreshed. For example: %matplotlib inline import ipywidgets as widget import matplotlib.pyplot as plt import numpy as np colors=['red', 'green', 'blue'] powers={'one':1, 'two':2, 'three':3, 'four':4, 'five':5} def f(x, power, color): plt.plot(np.arange(0,11), x*np.power(np.arange(0,11), power), c= color) plt.ylim(-100,100) plt.xlim(0, 10) plt.title("Power Law: $x={}y^{}$".format(x, power)) plt.show() #<-- Needed to make sure the plot draws (updates) each time a value changes widget.interact(f, x=(-5, 5, 0.1), power=powers, color = colors) This isn't needed if %matplotlib inline is omitted. Q: Why does the matplotlib.pyplot.show method no longer display a figure pop-up with an interactive chart? A: At ArcGIS Pro 3.5 we switched the default backend away from the TkAgg backend. We made this change as a trade-off for stability, as the TkAgg backend was causing many crashes when running in the context of ArcGIS Pro. The behavior outside of ArcGIS Pro remains unchanged. To display a figure pop-up with an interactive chart from ArcGIS Pro, run matplotlib.use("TkAgg") prior to displaying the plot. Please be aware that this may result in a crash in some circumstances. Q: Why don't local images display in markdown cells A: This is being investigated as a bug, currently only web-hosted (or embedded) images display in markdown cells. Q: Why does a new instance of ArcGIS Pro launch when I run %pip from a notebook cell? A: This is a known issue with embedded Python interpreters (such as what is running in ArcGIS Pro). Magic commands use the output of sys.executable to determine the host process, which in this case launches ArcGISPro.exe and not pythone.exe. You can avoid this by prefixing the pip command with ! instead of %. Note that we recommend using conda over pip. Q: Where can I find the option for restarting the kernel? A: You currently cannot restart the kernel unless you close and reopen ArcGIS Pro. You can, however, interrupt the kernel using the Interrupt Kernel button on the Notebook ribbon. While not a complete workaround, you can also use the %reset -f magic command to clear all variables in the namespace.  Q: Can I use R or C# in my notebook instead of Python? A: You can only open a Python kernel in notebooks running in ArcGIS Pro, other languages are not supported. Q: I'd like to export my notebook (.ipynb) to a Python script (.py), is that possible? A: Yes, you can export a notebook to a Python (.py) or HTML (.html) file from the Notebook ribbon’s Export button. You can also use the Jupyter command nbconvert from the command prompt, or you can run the command directly from your notebook by preceding the command with an exclamation mark (!). !jupyter nbconvert --to script "Path\To\Your_Notebook.ipynb" The resulting Your_Notebook.py file will be in the same directory as your notebook, Your_Notebook.ipynb. Additionally, %%writefile filename.py written on the first line of any cell will write only that cell to filename.py. Q: Can I open a Jupyter notebook in ArcGIS Pro that I have previously created outside of Pro? A: Yes, notebooks in ArcGIS Pro are simply Jupyter Notebooks (.ipynb) and any notebook created elsewhere (such as stand-alone in the browser, in some external IDEs, and hosted notebooks in ArcGIS Enterprise or ArcGIS Online) can be run in ArcGIS Pro. However, it is your responsibility to ensure any libraries used in these other environments are available in the active environment of ArcGIS Pro. Q: Can I use my notebook created in ArcGIS Pro across other platforms? A: Notebooks in ArcGIS Pro are simply Jupyter Notebooks (.ipynb) and can be run by any applications that support them, including stand-alone in the browser, in some external IDEs, and hosted notebooks in ArcGIS Enterprise or ArcGIS Online. However, it is your responsibility to make sure that any libraries used in the notebook are available in these other environments. Using notebooks hosted on Enterprise Portal in ArcGIS Pro: There are several options for moving a hosted notebook to your local machine. igned into a portal within ArcGIS Pro. From the Catalog pane in ArcGIS Pro, go to Portal > My Content, right-click a notebook item, and select Download and Open. Signed into Portal on the web browser. From the Notebook Editor in Portal, on notebook menu ribbon, click File> Download as > notebook (.ipynb).   Signed into Portal on the web browser. From the notebook's Item Details page (accessible from the Content tab by clicking on a notebook item), click Download to download the notebook in .ipynb format.    Using Notebooks created in Pro on my Enterprise Portal: You can upload any notebook file (.ipynb) to your portal. Sign into your organization's Portal from the browser, go to the Content tab of the contents page, click Add Item > From your computer and choose From my computer. From the Add an item from my computer dialog, click Choose File and browse to a notebook file on your computer, provide a title and tags, and then click Add Item. Q: When opening a notebook in ArcGIS Pro, why did I receive a "Failed to load notebook" error? A: Notebooks in Pro leverage a webserver run as localhost on port 8778. If it cannot be reached or is otherwise interfered with, notebooks will fail to load. Known causes and solutions: Bad Jupyter configuration (jupyter_notebook_config.py): Modifying the c.NotebookApp.ip or c.NotebookApp.port configuration options will prevent ArcGIS Pro from opening notebooks. To resolve, comment out or reset these values. Using a proxy server: If using a proxy server, set an exclusion on your proxy settings to prevent localhost:8778 (or 127.0.0.1:8778) from going through the proxy. You'll need to work with your IT group to add this exclusion, depending on the proxy you use and how it is implemented. Paths with special characters may cause issues. Avoid special characters, specifically the apostrophe (') is known to cause issues when opening and saving notebooks in ArcGIS Pro. Q: Can I open a notebook in Pro from a UNC path? A: UNC paths are supported as of ArcGIS Pro 3.0 - just be careful about collisions as this opens up the possibility of multiple users editing the same file. If you are still on an earlier version of ArcGIS Pro and unable to upgrade, the best workaround is mapping the UNC path to a drive or alternatively creating a symbolic link.  Q:  I keep clobbering variables when I have multiple notebooks open in Pro. Why are variable names being shared across multiple notebooks? A: ArcGIS Pro runs a single Python instance. This execution context is shared by each notebook opened in a single instance of ArcGIS Pro. Depending on the contents of concurrently open notebooks, this could potentially result in variable name collisions. To avoid potential issues you may consider: Using unique variable names across all concurrently running notebooks Running the %reset -f magic command as needed to delete all the variables in the namespace Q: Why are the environment variables I set in Notebooks (or Python Window) not being honored by Geoprocessing tools I run from the Geoprocessing pane? A: This behavior is due to the environment levels hierarchy. When setting an environment variable using arcpy.env, you are setting it at the Script level. Since this is the lowest level in the hierarchy, the setting will not be honored at any of the above levels. In order for your environment variables to be honored by both Script and Geoprocessing tools, you must set the environment variables using the Application settings from your project. Powershell Q: Can I use Powershell 7 as my terminal session for Python? A: ArcGIS Pro's Python Command Prompt (based off Microsoft's Command Prompt) is the standard way to run ArcGIS Pro Python scripts from a terminal session. With ArcGIS Pro 3.6 it now is possible to use PowerShell 7 as an alternative to the Python Command Prompt. PowerShell 7 is Microsoft's latest command-line and scripting language. Here are the one-time steps to set up to use PowerShell 7 with ArcGIS Pro's python environment Install PowerShell 7 from the Microsoft Store or from Windows PowerShell run winget --id Microsoft.PowerShell --source winget From the Windows start menu launch the Python Command Prompt and run conda init powershell This step requires admin credentials Your machine should now be setup. From the Windows start menu you can now launch PowerShell 7 and then type activate arcgispro-py3 to activate the ArcGIS Pro python environment. Then: Run python -c "import arcpy;print(arcpy.version)" to confirm you're setup correctly (should print ArcGIS Pro's version) Run python to launch an interactive REPL session Run python c:\myproject\myscript.py to run a script Server 12.0: PowerShell 7 can also be used with Server 12.0 deployment, simply use Python 3 Command Prompt instead of Python Command Prompt in the instructions above. Known issue and limitations: At this time the following commands (.bat files) commonly used with ArcGIS Pro are not supported in PowerShell: proenv, propy, proswap.   Happy coding.  -The Python team 🐍 ... View more