ArcGIS Online - Notebook Task - Set Task Schedule Details

[ChristopherMask]

I would like to suggest a 2-minute interval option in the dropdown. I only request this to aid with wildfire situational awareness and rapid deployment of resources. The notebook code we developed copies NWS hotspot and potentially dangerous wildfire notification feature into our initial attack feature layer used in field maps by our resources. Currently, there is a 15-minute and 30-minute, but that time span is not sufficient. Thanks, Chris Mask - GIS Manager Oklahoma Forestry Services.

[AlfredBaldenweck]

Why not a "type in your number of minutes" option? 2 minutes is great, but being able to specify an exact number would be great.

Otherwise, how about like: 1,2,3,4,5,10,15,20,25,30,45,60,90....

[BobBooth1]

You could embed your update code in a timed loop inside the notebook, have the notebook run every 15  minutes, and then have the timed loop within the notebook run every two minutes (for a counted number of runs or a timed duration), then quit.

I've done something similar with time.sleep() in a loop.

[ChristopherMask]

Do what @BobBooth1  , smh!?!   I can't tell you thanks enough for the idea, much appreciated! Added this code snippet at the end of the code and worked like a charm.

# Timed loop to execute the function 14 times, every 1 minute
for _ in range(14):
logging.info("Starting hotspot feature synchronization...")
copy_new_hotspot_features_to_event_layer(NWS_HotSpotPoint_layer, EventPoint_layer, CriticalFireArea_layer)
logging.info("Synchronization complete. Waiting for the next interval...")
time.sleep(60)

[BobBooth1]

Awesome!  Glad it worked for you!

[ChristopherMask]

So, the task will start but consistently receives this error message. Then, the task is set to failed and will not resume. I have to manually resume the task, which is not sufficient. Thoughts @BobBooth1 ? I have tested and reduced the iterations, but it appears to happen after 5 or 6 attempts. 

KeyboardInterrupt                         Traceback (most recent call last)
Cell In[1], line 149
    147     copy_new_hotspot_features_to_event_layer(NWS_HotSpotPoint_layer, EventPoint_layer, CriticalFireArea_layer)
    148     logging.info("Synchronization complete. Waiting for the next interval...")
--> 149     time.sleep(60)  # Wait for 1 minutes before the next attempt
    150 except Exception as e:
    151     logging.error(f"Error during attempt {attempt + 1}: {e}")

KeyboardInterrupt: 

[BobBooth1]

@ChristopherMask - When there is data to update, how long does the process take (within a given loop cycle?) I wonder if maybe the notebook is being run again by the task scheduler while it is still busy doing an update?

Could manage for that case by including a check of current time when the notebook starts, then within each loop check the time, calculate duration since start, and cause it to exit if there is less time left in the current notebook task's run cycle than it would typically take to make an update?

[ChristopherMask]

Well, it there is definitely a pattern. In all the failed task, it appears there is a 4-minute mark the task is stopped and fails. If you run the script from the menu, it works like it should. Bummed 

[Ian-Horn]

I agree that it would be nice to have the option of shorter intervals, or potentially the ability to input.  The interval.   

With python, I created three different tasks using the minimum 15-minute interval.  I spaced them to start five minutes apart, 2am 2:05am and 2:10am.  This way, the data would get updated every 5 minutes.  I look at my tasks records and all tasks run within one minute of each other.  

I read yesterday that you can't run a time loop in AGOL notebook environment I'm going to try the loop suggested above with a 14 minute time out to see if that works.

[Ian-Horn]

I mentioned created 3 tasks at 15 minute intervals to ensure data is updated every five minutes.  I forgot to mention that it's a colossal waste of credits.  I've estimated it would burn ~11,000 credits/year.  That's a lot of credits 3 tasks that is not fulling the need to update every 5 minutes.

[BobBooth1]

To avoid spending credits, you can run ArcGIS Python API code using a Windows Scheduled task on some machine you control.  I have one that runs on my laptop, and a bunch of others that run in a VM. You have to pay for the hardware or hosting, but it can be essentially free, if you are piggybacking on equipment/hosting you have already.

One pattern for that is presented in this tutorial:

https://learn.arcgis.com/en/projects/schedule-automated-near-real-time-data-updates/