Embedded toolbox script losing all content

[RichardHowe]

I'm posting this here in the hope that somebody may have experienced the same issue and/or have a solution. I reported it to ESRI as a bug, but I cannot reliably recreate it, so couldn't live demo it to them.

The issue generally occurs within an aprx, where I am making active edits to a python script which is embedded in a toolbox (which lives on a shared server), using the right-click and "edit" option in the Pro catalog pane.

Generally, having made the edits I will save the changes and close the idle window and I will want to run the tool immediately afterwards to check if my edits have been successful. If the tool subsequently fails, and I got back to edit it again, often (but not always) the entire contents of the script have been replaced by a single line of text referencing a path to a temporary directory on my pc which also doesn't exist.

This is really concerning and frustrating behaviour, as it totally overwrites a saved script thereby deleting a lot of work. I appreciate I could save the script as a standalone py file and point the tool at it, but I thought that embedding was supposed to make things more efficient in terms of sharing (i.e. the python always lived with the tool, so if the toolbox was moved or shared then the meat of the script would move with it).

[TonyAlmeida]

Pro can be quite clunky when it comes to toolbox scripts and model builder models. I've encountered issues where, after making changes to a model, it would no longer run. The only solution was to create a new model that was exactly the same, which would then run without any problems.

In my workflow, I use the Pro scheduler. I create a toolbox script, attach the Python script to it, then build a model, add the script to the model and set up the scheduler. However, as I've mentioned, I've often had to recreate the script and model when they fail to run. This behavior is both weird and frustrating.

[BobBooth1]

I wonder if the problem is associated with the toolbox being reached over the network.

[HaydenWelch]

See my post reply below, it has to do with the "open in external editor" button generating a new tempfile wherever the properties window is reopened. So if you close that window and then reopen it the file being referenced by your editor is detached. You can avoid this confusion by immediately closing your editor and only making edits while the property window is open.

Otherwise if you re-open the properties window the old temp file is deleted and the memory buffer from your editor will be saved to a new file with the old temp name that Pro isn't looking at.

ESRI could get around this by persisting the temp file for a bit longer and naming it something that isn't random (e.g. <toolbox>_<tool>.py) and checking to see if there's already an open editor for that tool when the properties dialog is opened.

[HaydenWelch]

This is a pretty common issue and why I don't like using embedded scripts. Arc by default will open the embedded script in a temp file and if for any reason that tempfile is deleted or updated while you're making edits to the script, the changes won't be saved.

In VScode this is made very obvious because the file tab will be marked red when the temp file is dereferenced and you know then that all your changes will be overwritten.

To get around this, do all your development in a linked script file and then only embed the scripts when you want to package the toolbox for distribution.

I think ESRI could get around this odd behavior by having the file be stored in the common data folder in your active project instead of putting it in the default Windows temp directory.

For now though, just avoid using embedded scripts until the last second or use an editor (VScode) that makes it clear when the file you're editing has been dereferenced or deleted so you can say least save your changes somewhere else before they're lost.

Bonus to this workflow is that you can have your scripts live in a git repository and actually do proper version control on them. Then packaging/embedding them you'll know exactly which version of the script you're putting into the toolbox.

There's a screenshot of what I'm talking about. This bug happens if you close the properties window and then re-open it in Pro. The tempfile it generates is different every time and it loses context from the last time you opened the script in the external editor (see the markthrough in VS Code up top). So if you did want to develop this way, you'd need to make sure you *Never* close the properties dialog while editing your code, or the next time you open it it will delete the current file you're working on and create a new one from the last saved state.

[RichardHowe]

@HaydenWelch - This is an excellent reply. Thanks for the details and explaining what is going on. It at least gives me a workaround!

@BobBooth1 - Please could you take Hayden's report/findings to the ESRI dev team. I think his description of what is happening and why is perfect and his suggestion of how to fix this should be easily achievable and would stop (what is) a very frustrating issue. I'm going to reopne my bug report and point it to this post as well.

[BobBooth1]

Done!