Create Text File or Other Log on ArcGIS Online Notebooks

pmckinney

Does anyone know if you are able to create an empty text file (.txt), and then write content to it, using ArcGIS Online Notebooks?

Based upon the item type reference for the content.add method, it does not appear to be supported.  But sometimes you all know things not mentioned in the help reference.

The context is I'm considering using the scheduled task feature of Notebooks.  I have many scripts that run as scheduled task on my organization's system.  My practice is to write information and/or error messages to text files, so that I can review what happened each time the task runs.

For running Notebooks as tasks on ArcGIS Online, I'd like to be able to review success/errors of the script.  So, if someone knows an alternative way to do this other than writing to a text file, I am interested in that solution as well.

I can perform my actions running a script on my system, but as the script is just doing things in ArcGIS Online, it would be preferable to have the script run in the ArcGIS Online system as well, if possible.

jcarlson

We use a series of standalone hosted tables for our scheduled Notebook tasks. I would not attempt to use a text file. Though it's possible to use files inside of a Notebook separate from your portal's Content, you want to make sure something important like a log or auxiliary text is kept somewhere that's accessible to the Notebook, but also to other users who may need to access it. A hosted table does that.

Here are some examples:

 

Document ID Tables

We synchronize a workflow table with documents from the County Recorder's website every night. We can't directly access their database, so we scrape the document details from the website by iterating over the document ID. The table stores the ID of the last successfully-scraped document, so that future runs of the script can pick up where the previous day left off.

Additionally, there are times when a document's details are finalized on a future date, and may be missed initially (the HTTP request returns a 400). To handle these, we have a second table of "skipped" IDs, to be re-queried in future runs of the script, so as not to miss documents added out of sequence.

 

Layer Update Timestamps

We have a few hosted layers that update from SQL queries of a non-public database. Due to the size of the layers being updated, a truncate / append or overwrite process would be time-consuming and unnecessary, due to the relatively few edits that happen on a given day.

Using a hosted table, we store timestamps of the last successful update for the hosted layers. A scheduled notebook is then able to read that table, then adjust its SQL query to only return features that were edited since that timestamp.

 

Notebook Logs

Finally, we have another hosted table that stores custom log messages from our scheduled notebook runs. Much like a standard log file, it stores a message, log source, timestamp, and message severity code. These pipe into a Dashboard List element, allowing me to quickly and easily look through recent logs and see errors and warnings right away.

The notebooks simply access the log table as any other layer, and use edit_features(adds=[...]) to add messages.