Add a "Quiet Time" to the Notebook Tasks scheduler

1826
5
04-20-2021 06:39 AM
Status: Open
EulessNick
New Contributor III

I have a notebook that I have automated with a task that updates the data in some tables after the source data tables are updated. This is a continuous thing, and I update the data every 15 minutes. However, there is no data entry being done between 6 pm one day and 7 am the next - but my task scheduler will run the notebook every 15 minutes for 13 hours when there will be no change to the data. No big deal UNLESS I'm using an advanced notebook that's burning credits each time it runs. That's 52 runs that I pay for when I know that the data won't change. It would be nice if the Task scheduler had an optional "Quite Time", ie. 'Don't Run Between 00:00 and 00:00'. It would be hard to bridge the passing of midnight, but it could be two entries:

Don't Run Between 18:00 and 23:59

Don't Run Between 00:00 and 07:00

I might even want to shut it down at lunch or another time during the day when I do planned maintenance. There are lots of reasons to want to skip a time each day. If I have 10 Advanced Notebooks running on a schedule that are perpetual this could save me money!

 

(FYI - this would also be nice if it were called the Task Scheduler instead of just Tasks so as not to be confused with Tasks in ArcGIS Pro)

5 Comments
TexasEGRT

The Notebook Task Scheduler is wonderful and this additional functionality is greatly needed - we're a non-profit and can't waste money on running Notebooks during off times.

curtvprice

For right now, you could use the time functions in python check the time (with an if statement) and exit the notebook before doing anything for off hour runs.

JackSettelmaier

Interesting idea.  I had first thought in reading this, that you were referring to the Windows OS-based Task Scheduler, where I often set the run times to kick off various tasks.

 

Though I have not yet explored Esri's variant of a Task Scheduler you must be talking about,  it seems another option could be to use the OS-based Scheduler to limit the time a process runs?

 

That said, good request for inserting more options into the Esri Notebooks Scheduler.  

FarmersBranchBrian

@curtvprice  After talking with Esri, it appears that any notebook that has something about it that consumes credits is enough.  Scheduled Notebooks, according to their blog post, consume credits based on the amount of execution time.  Additional tasks that consume credits could in theory be gated by if/else logic but simply running the Scheduled Notebook consumes credits with every run.  Section 5 of the below linked blog post states, in part:

The ability to execute notebooks in “Scheduled Mode” can add value to organizations in terms of time, resources, and efficiency.  As such, the credit model is slightly different than notebooks in “Interactive Mode”.  All scheduled notebooks consume credits, and this consumption is based on the amount of time it takes to complete a task.  For a review of the pricing for both “Interactive Mode” and “Scheduled Mode”, please refer to the Understand credits topic in the ArcGIS Online documentation.    

blog post linked here 

As such, a quiet time could potentially be a very valuable addition for more time aware workflows (shutting down every night) or credit sensitive tasks and organizations.

curtvprice
I would assume that a notebook that quit at the beginning would use a negligible amount of credits. Credits really get burned up by running tools that consume CPU or disk.