topic Re: Python script, when run from Task Scheduler, not returning FC list in Python Questions

Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Tue, 31 May 2022 15:37:25 GMT

I have a pretty simply python script that steps through all of the feature classes in an enterprise GDB having a certain name. When I run this script from the command line using a .bat file to trigger it, it runs great and completes successfully. When I embed the same .bat file that runs the script in Windows (10) Task Scheduler, it fails to return a list of FC names. Please see code and images below:

import arcpy import datetime f = open ('C:\Scripts\CartegraphIntegration\COJNPDESIntegrationDBLog.txt', 'a+') f.write("**********Program Started :"+str(datetime.datetime.now())+"**********") f.write('\n') # Connection file for ArcPy CGIntegrationsde = r"z:\Not\A\RealPath\connection.sde" arcpy.env.workspace = CGIntegrationsde f.write("Workspace set successfully: " + CGIntegrationsde) f.write('\n') #all of the feature classes we want to udpate all start with the word "Spatial" #but since the names are fully qualified we have to account for other leading characters featClasses = arcpy.ListFeatureClasses('*Spatial*','','') f.write("Feature Class List acquired successfully") f.write('\n') if not featClasses: f.write("The list is empty.") f.write('\n') else: for fc in featClasses: f.write("Updating fields to use appropriate case in FC: " + fc + " :"+str(datetime.datetime.now())) f.write('\n') #Script does something ArcPy like here "edited for privacy f.write("Updates complete in FC: " + fc + " :"+str(datetime.datetime.now())) f.write('\n') f.write("**********Completed: "+str(datetime.datetime.now())+"**********") f.write('\n') f.close()

When I run the script from the command line using a .bat file:

"C:\Users\?????\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\python.exe" "C:\Scripts\ProjectName\ScriptName.py"

I get what I want... The below image shows the log file I write in the above code:

When I run it in the Task Scheduler I get information telling me that the list returned from the arcyp.ListFeatureClasses command is empty:

Another weird thing is that the task always says "Running" in the scheduler even though it has reached the end of the script.

Any thoughts would be appreciated. This is really a hassle that it works when I test it and I think I'm just an "Adding it to the scheduler" step away from being done and then that's the part the bails!

Re: Python script, when run from Task Scheduler, not returning FC list

BlakeTerhune — Tue, 31 May 2022 15:54:11 GMT

Could you confirm what you have in the "Action" tab of the task? Also, is the task running as the same user you're testing with?

Here's a reference to check regarding how to set up the task in Windows Task Scheduler.
Schedule a Python script or model to run at a prescribed time: 2019 update (esri.com)

Re: Python script, when run from Task Scheduler, not returning FC list

JoshuaBixby — Tue, 31 May 2022 15:53:48 GMT

Are you connecting to the EGDB as an OS user or a database user? If the former, what credentials is the task running as?

Re: Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Tue, 31 May 2022 15:56:37 GMT

ArcGIS Pro

Re: Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Tue, 31 May 2022 16:03:29 GMT

DB Authentication connecting as Data Owner.

I'm running the task in the Scheduler using my Domain login on my desktop.

Re: Python script, when run from Task Scheduler, not returning FC list

BlakeTerhune — Tue, 31 May 2022 16:38:13 GMT

Try configuring the task action where the program/script is your Python env exe

"C:\Users\?????\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\python.exe"

and the optional argument is the path to your Python script file.

"C:\Scripts\ProjectName\ScriptName.py"

Re: Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Tue, 31 May 2022 17:13:45 GMT

Sorry. I should have mentioned that that was the first way I tried configuring it....

it doesn't seem to have a problem with "import arcpy" or setting the workspace.... just getting the FC list.

Re: Python script, when run from Task Scheduler, not returning FC list

DonMorrison1 — Tue, 31 May 2022 21:02:34 GMT

I'm not sure if this will help but maybe worth a try. I call my .bat files a bit differently from the task scheduler. In the "Edit Action" panel:

Action: Start a program

Program/script: cmd

Add Arguments: /c <my bat file name>

Start in (optional): <path to my bat file>

I assume you are running the scheduled task on the same PC as where you run the bat file and your are sure you are running under the same Windows user. Is there any thing non-trivial in your bat file? Maybe you could share that with us too?

Re: Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Tue, 31 May 2022 23:37:51 GMT

I don't have a connection to the office right now, but i will share in the morning (EST). Basically the bat file contains the exe call followed by the bat file name argument as shown above with an ECHO OFF before that line and PAUSE after it, but I really like your idea to call cmd and pass the bat as an argument. I can run the bat from the cmd manually and it works fine. I'll give it a go tomorrow morning and keep you posted.

Re: Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Wed, 01 Jun 2022 13:33:43 GMT

It runs the script but same result - ListFeatureClasses is still empty.... It must have something to do with ArcPy running in scheduler and the complete lack of error messaging in scheduler (that I know of)... maybe its erroring out setting the workspace I don't even know it.

Re: Python script, when run from Task Scheduler, not returning FC list

DonMorrison1 — Wed, 01 Jun 2022 16:34:20 GMT

Hmmm - very strange. I run a lot of programs with arcpy from the task scheduler and have not seen anything link this. I would try to access one of your feature classes directly (instead of via ListFeatureClasses) - maybe you will get an error message that will be more enlightening that the empty list. You could also do a Describe on the workspace and dump out the results:

f.write(str(arcpy.da.Describe(CGIntegrationsde)))

Re: Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Wed, 01 Jun 2022 18:00:36 GMT

I added the code to describe the workspace and another line to show the path to the python .exe being used when the program is run (on the theory that maybe Scheduler was using a different python.exe than the one intended).

The results are below. The path to the python.exe returned by the script is the same whether it is run from the command line or the scheduler

C:\Users\GarbadeK\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone1\python.exe

So that doesn't seem to be the problem. But the describe of the workspace shows data when run from the command line and shows nothing when run from scheduler. So I think the issue is ArcPy is not being import... even though import ArcPy is the first statement in the script (after # -*- coding: utf-8 -*-).

I wonder if its because one of the folders in the python.exe path is hidden?.... I'm just grasping at straws at this point.

I've run Task Scheduler as Admin, run it both as has to be logged in and as not logged in, I have "Use highest permissions" checked.... So close, but yet so far.

Re: Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Thu, 02 Jun 2022 15:17:15 GMT

I cracked the case... I'm nothing if not persistent. After writing try/except conditions and examining the description of my workspace (established using a connection file) I determined that Arcpy was indeed loading correctly and that the problem was Arcpy was not establishing a connection to my database using my connection file when my script was run from the scheduler. So, just on a lark, I moved the "connectionfilename.sde" from its current location on a network drive to a location on the C: drive (the same folder the "scriptname.py" file was in).... and Voila it worked. That was at least a day out of my life.

Just for the record I tried:

Alternating "Run only when user is logged in" and "Run whether user is logged in or not"
Alternating "Run with highest privileges"
Running Task Scheduler as "Admin"
Running the python script from a batch file
Running "path/python.exe" as the "Program/script" with "python path/file" as the argument
Running "cmd" as the program and the batch file as the argument
Alternating the "Start in" path....
All for naught

I should have thought about the easy solution first, but your keys are always in the last place you look.

Re: Python script, when run from Task Scheduler, not returning FC list

BlakeTerhune — Thu, 02 Jun 2022 15:16:35 GMT

Glad you got it figured out. See if it works on the network drive using a UNC path rather than a drive letter.

Re: Python script, when run from Task Scheduler, not returning FC list

KimberlyGarbade — Thu, 02 Jun 2022 15:32:21 GMT

Yup. UNC paths worked to...

Re: Python script, when run from Task Scheduler, not returning FC list

PDodd — Thu, 02 Jun 2022 18:36:36 GMT

The issue is related to the Environment used by the Windows Scheduler. When Windows starts up, there is a System environment (resources, environment variable, and properties) loaded and used by Windows. Any application, like the Scheduler, that is started by Windows will use this environment unless it is overridden by details from another Windows Account. Modifying the PATH for the Windows system will require a restart of Windows in order for these applications to receive the update. Modifying the PATH for a specific user only requires them to log off and back in.

When starting a Task, if you specify an account other than system, the task will use the user environment details from that account. Similar to environment variables, a user's mounted drives like your "M:", will not be available to the scheduled task unless you include provisions to mount it. The BAT script can mount a network drive by using 'net use M \\<server>\<folder>' before it launches the Python script.

Setting the ArcPy workspace to a folder does not mean that it will be accessible unless the path really exists.

You have to think about a Scheduled Task as a separate login session, not the same session you created it from.

I hope this helps...