Task Scheduler for GIS Tasks - Skipping Python Scripts Called from Bat File

[MichaelVolz]

I have been running scheduled tasks using Windows Task Scheduler for many years with the ArcMap version of python.  Now that my org is migrating to ArcGIS Pro, I am finding that the same bat file is now randomly skipping the ArcGIS Pro python scripts.  The bat file runs fine outside of Windows Task Scheduler, so both the python scripts and bat file itself are fine.  Sometimes even the first python script in the bat file gets skipped with Windows Task Scheduler, so associating each python script with a bat file will not guarantee success with that setup.

Has anyone encountered an issue like this with a similar setup where you call multiple python scripts sequentially so once one python script has finished running the next python script is then executed?

Does anyone use a different job schedule software other than Windows Task Scheduler to run GIS jobs?

Any help or feedback is greatly appreciated.

[MichaelVolz]

Wrapping in try/except did not capture any faults, as that has already been tested.

Do you have any reasons for not using ESRI's interface to clone your python environment?

[jcarlson]

Familiarity, for one. I was using python and conda prior to doing any kind of GIS python, so I already knew how to get what I needed.

I also find the CLI to be a lot faster than any conda GUI, especially using mamba.

Third, we've got some python scripts that are totally separate from our Esri / GIS workflows, and there's simply no need to manage the envs for those scripts in Pro. I like managing everything in one place if I can.

[anonymous_geographer]

"I have already changed the bat file to use propy.bat, but the python scripts still get randomly skipped."

This has been an issue for a long time with ArcGIS Pro's python.exe and propy.bat, at least for me. Call the pythonw.exe file instead and you should be okay. It's stable.

python.exe and propy.bat will fail to launch my scripts 60-80 percent of the time. I have had zero issues with calling pythonw.exe though. It's been my workaround for a few years now.

Edit: Just know that pythonw.exe will hide the Command Prompt window. So if you need user interaction with that window or will rely on output messages within it, you'll have problems with this approach. More info here.

[GavinMcDade]

We've tried pythonw.exe, and no dice (I'm Michael Volz's co-conspirator). The forums are replete w/ people complaining about the undue time needed to import the arcpy module (most of it due to needing to make the call to the license server to authenticate the license outside of having Pro opened). The issue is squarely from the Windows Task Scheduler context... that is, something in the way that Task Scheduler executes (even as it's technically going through a .bat file) seems to be evaluating the wait time for the arcpy module to import, and just kills the script if that time value exceeds some threshold. If you manually run import arcpy in a Python window, you'll see varying delay times for it to complete. This would seem to explain why the Task runs sometimes, and skips other times. 

I don't know of any setting inside of Task Scheduler that would force it to wait for a "long" process to finish. 

[anonymous_geographer]

@GavinMcDade- What's your environment setup like? We primarily execute on virtual environments using Windows Server 2016 and 2022. Finally decommissioned Windows Server 2012 because too many oddities were occurring once support got dropped.

For troubleshooting, have you already incorporated PAUSE into any of the .bat files (temporarily) to see if any Command Prompt specific messages are being generated for you before the window closes? Sometimes that has helped me diagnose if those .bat files are even being reached during a workflow.