idea ArcGIS Online - Notebook Task - Set Task Schedule Details in ArcGIS Notebooks Ideas

ArcGIS Online - Notebook Task - Set Task Schedule Details

ChristopherMask — Mon, 13 Jan 2025 21:25:38 GMT

I would like to suggest a 2-minute interval option in the dropdown. I only request this to aid with wildfire situational awareness and rapid deployment of resources. The notebook code we developed copies NWS hotspot and potentially dangerous wildfire notification feature into our initial attack feature layer used in field maps by our resources. Currently, there is a 15-minute and 30-minute, but that time span is not sufficient. Thanks, Chris Mask - GIS Manager Oklahoma Forestry Services.

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

AlfredBaldenweck — Mon, 13 Jan 2025 21:32:36 GMT

Why not a "type in your number of minutes" option? 2 minutes is great, but being able to specify an exact number would be great.

Otherwise, how about like: 1,2,3,4,5,10,15,20,25,30,45,60,90....

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

BobBooth1 — Tue, 14 Jan 2025 16:36:32 GMT

You could embed your update code in a timed loop inside the notebook, have the notebook run every 15 minutes, and then have the timed loop within the notebook run every two minutes (for a counted number of runs or a timed duration), then quit.

I've done something similar with time.sleep() in a loop.

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

ChristopherMask — Wed, 15 Jan 2025 15:28:33 GMT

Do what @BobBooth1 , smh!?! I can't tell you thanks enough for the idea, much appreciated! Added this code snippet at the end of the code and worked like a charm.

# Timed loop to execute the function 14 times, every 1 minute
for _ in range(14):
logging.info("Starting hotspot feature synchronization...")
copy_new_hotspot_features_to_event_layer(NWS_HotSpotPoint_layer, EventPoint_layer, CriticalFireArea_layer)
logging.info("Synchronization complete. Waiting for the next interval...")
time.sleep(60)

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

BobBooth1 — Wed, 15 Jan 2025 15:37:29 GMT

Awesome! Glad it worked for you!

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

ChristopherMask — Thu, 16 Jan 2025 17:07:13 GMT

So, the task will start but consistently receives this error message. Then, the task is set to failed and will not resume. I have to manually resume the task, which is not sufficient. Thoughts @BobBooth1 ? I have tested and reduced the iterations, but it appears to happen after 5 or 6 attempts.

KeyboardInterrupt                         Traceback (most recent call last)
Cell In[1], line 149
    147     copy_new_hotspot_features_to_event_layer(NWS_HotSpotPoint_layer, EventPoint_layer, CriticalFireArea_layer)
    148     logging.info("Synchronization complete. Waiting for the next interval...")
--> 149     time.sleep(60)  # Wait for 1 minutes before the next attempt
    150 except Exception as e:
    151     logging.error(f"Error during attempt {attempt + 1}: {e}")

KeyboardInterrupt:

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

BobBooth1 — Thu, 16 Jan 2025 17:47:18 GMT

@ChristopherMask - When there is data to update, how long does the process take (within a given loop cycle?) I wonder if maybe the notebook is being run again by the task scheduler while it is still busy doing an update?

Could manage for that case by including a check of current time when the notebook starts, then within each loop check the time, calculate duration since start, and cause it to exit if there is less time left in the current notebook task's run cycle than it would typically take to make an update?

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

ChristopherMask — Thu, 16 Jan 2025 19:21:41 GMT

Well, it there is definitely a pattern. In all the failed task, it appears there is a 4-minute mark the task is stopped and fails. If you run the script from the menu, it works like it should. Bummed

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

Ian-Horn — Thu, 21 Aug 2025 15:10:09 GMT

I agree that it would be nice to have the option of shorter intervals, or potentially the ability to input. The interval.

With python, I created three different tasks using the minimum 15-minute interval. I spaced them to start five minutes apart, 2am 2:05am and 2:10am. This way, the data would get updated every 5 minutes. I look at my tasks records and all tasks run within one minute of each other.

I read yesterday that you can't run a time loop in AGOL notebook environment I'm going to try the loop suggested above with a 14 minute time out to see if that works.

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

Ian-Horn — Thu, 21 Aug 2025 15:16:08 GMT

I mentioned created 3 tasks at 15 minute intervals to ensure data is updated every five minutes. I forgot to mention that it's a colossal waste of credits. I've estimated it would burn ~11,000 credits/year. That's a lot of credits 3 tasks that is not fulling the need to update every 5 minutes.

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

BobBooth1 — Thu, 21 Aug 2025 16:03:45 GMT

To avoid spending credits, you can run ArcGIS Python API code using a Windows Scheduled task on some machine you control. I have one that runs on my laptop, and a bunch of others that run in a VM. You have to pay for the hardware or hosting, but it can be essentially free, if you are piggybacking on equipment/hosting you have already.

One pattern for that is presented in this tutorial:

https://learn.arcgis.com/en/projects/schedule-automated-near-real-time-data-updates/

Re: ArcGIS Online - Notebook Task - Set Task Schedule Details

Ian-Horn — Thu, 21 Aug 2025 16:24:01 GMT

@BobBooth1 Thank you for sending that link. I'm glad to see the overwriteFS package is still in use and has been updated.

We already run a scheduled task every five minutes with task scheduler on our server. We are migrating all our vector data from SDE to AGOL for consumption. The reason we want to control the task in AGOL (I'm quasi open to an AWS solution) is because we have different login domains for our production servers and scheduled tasks require password input. If the password expires and we haven't open and closed the task for the password prompt, the task breaks and then we get locked out of that domain after three failed attempts. Getting locked out of the production server domain just looks bad.