I have created a script that updates a feature service through the Python API in Jupyter Notebooks. I was wondering how I can run this as a scheduled task?
https://github.com/hildermesmedeiros/run-ipynb
I`ve been using that toolbox, for months I did it while ago, you all feel free to use it. But it would not hurt to be a default toolbox, right?
But to do so, it would need more things, it is working for me, so...
ps: My Github project is now public....I`m sorry for that!
Hello Shawn
Could you provide an example of your script?
I don't know about running a Jupyter notebook. I would recommend making it a .py script and run it at your command line. Assuming this is Windows (since you said "scheduled task" and not "cron"), make sure you can call it from the DOS prompt. I recommend calling it as "<python executable> <script> <parameters>".
Get it working that way, and then you should be able to schedule it no problem. In the "actions" tab of a Windows scheduled task, I specify the full path to my Python executable for the "Program/script" textbox, e.g.:
"C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe"
Note that this is in double-quotes because it contains a space. Then, in the "Add arguments (optional)" text box, I specify the full path to the Python script, again putting it in double quotes if it contains a space in it.
On the "General" tab, I've found that checking "Run with highest privileges" is necessary in some cases. Full disclosure - I don't really understand that, but it resolved some issues I had once, so I check it.
I've had great success using PyInstaller to compile the .py script and all its necessary components into a single executable that I can then run in Windows Task Scheduler. This frees you from worrying how to call a particular virtual environment. This also allows you to share the executable with different systems without setting up that same Python environment.
I have found a significant resulting .exe size if the API was installed with or without dependencies. If you install the arcgis package using the default command (e.g. conda install -c esri arcgis or pip install arcgis) the resulting .exe file size can be > 200 MB. Installing without dependencies (e.g. conda install -c esri arcgis --no-deps or pip install arcgis --no-deps) the resulting .exe file size can be < 40 MB. It may not be possible to install without dependencies in your case but I would recommend doing so if you can.
conda install -c esri arcgis
pip install arcgis
conda install -c esri arcgis --no-deps
pip install arcgis --no-deps
PyInstaller Compiling Snippet:
import subprocess
env = <Enter the path to your virtual environment.>fn = <Enter the desired output file name.>py = <Enter the path to your .py script.>subprocess.check_call(f"{env}/Scripts/pyinstaller -n {fn} -F {py}")
Hi Shawn, where does the feature reside, and is this on the same system as the notebook?
Hi Rohit, it's noteworthy to add that cron ENV settings are often different from ENV at the CLI when the user is developing the automation script.
Equally important to digest alongside the actual question is the following: whether it can be done and how is 100% system-dependent and context-dependent. As an example of context, environment settings can be different at a command prompt (i.e., via manual entry at CLI) from settings in a scheduled task (especially if one is not necessarily 'logged in' when the task occurs). As classic example of this is env differences on Linux systems when run under someone's crontab vs when run manually at a terminal.
I know this is off-topic, but I do this professionally and teach it at a university, and ~90% of folks simply don't remember this small fact which, eventually can inhibit the process of tracking down the cause of a problem when the 'correct answer' is not working.
Respectfully folks, please be careful when you are discussing the term environment in such a context. A given user has multiple environment contexts in an OS. As a direct comment to the above statement, conda does not know everything about the user's environment; FURTHERMORE, the environment settings that a user is given at a CLI can be quite different from when run as a task.
I am going to wait until ESRI fixes this issue with a firm release, as I do not want to hack around in the python environment. Is the location you are specifying (Users) folder, the location where the cloned python environment is placed by default?
Have you cloned a Pro 2.2.1 environment yet?
One reason why we don't do the clone recommendation if you want to install packages... you have to explain why you have to put double/single quotes around a path to the python executable as well as the script name. Then the students say.... "but windows allows spaces? don't they?" then I say "sometimes... but not when you need them too" Teaching is an arduous calling and the root of the expression "just do it"
Dan:
What file is supposed to exist at the location you are referencing? All I see is subfolders at the location you specify and no python.exe file to execute the py script.
We call the Users folder... Users is for Losers because someone wasn't thinking Unicode
p <SPAN class="operator token">=</SPAN> <SPAN class="string token">"C:\Users\dan_p\AppData\Local\ESRI\ArcGISPro"</SPAN> File <SPAN class="string token">"<ipython-input-1-08bc5c8518ea>"</SPAN><SPAN class="punctuation token">,</SPAN> line <SPAN class="number token">1</SPAN> p <SPAN class="operator token">=</SPAN> <SPAN class="string token">"C:\Users\dan_p\AppData\Local\ESRI\ArcGISPro"</SPAN> <SPAN class="operator token">^</SPAN> SyntaxError<SPAN class="punctuation token">:</SPAN> <SPAN class="punctuation token">(</SPAN>unicode error<SPAN class="punctuation token">)</SPAN> <SPAN class="string token">'unicodeescape'</SPAN> codec can't decode bytes <SPAN class="keyword token">in</SPAN> position <SPAN class="number token">2</SPAN><SPAN class="operator token">-</SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">:</SPAN> truncated \UXXXXXXXX escape <SPAN class="comment token"># ---- raw encoding or forward slashes your way out</SPAN> p <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"C:\Users\dan_p\AppData\Local\ESRI\ArcGISPro"</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
I have found the only way to download a hosted feature service table with attachments using Python is with the ArcGIS API. This will only work as a .ipynb and not as a .py. I'm aware that I can run .py files on task scheduler but have not found a solution to run a .ipynb on some sort of scheduler.
There's no obvious benefit to run notebook as a backstage task. you can copy all the code into a python script file (.py) then run it in the task scheduler.
Did you find a solution to this? I'm looking for a way to run Jupyter .ipynb file on a daily basis.
How would I schedule my .ipynb to run daily? Do you have an example of how I can achieve this?
Hi Shawn Hibbs - Open a command prompt and type:
conda info <SPAN class="operator token">-</SPAN>e<SPAN class="line-numbers-rows"><SPAN></SPAN></SPAN>
This will output a list of all your environments and the path to where they are stored.
You can also search for the arcgis package in a specific environment with the following command:
conda list -n myenv arcgis<SPAN class="line-numbers-rows"><SPAN></SPAN></SPAN>
How do I find the environment where the API is installed?
The script can be run as a genera scheduled task using Task Scheduler on Windows and cron on Linux.
You could follow the blog post at https://www.esri.com/arcgis-blog/products/product/analytics/scheduling-a-python-script-or-model-to-run-at-a-prescribed-t… for tips on how to do this except that you'd use the Python.exe from the environment that has ArcGIS API for Python installed in it. Refer to Schedule [Virtualenv Dependent] Python Script with Windows Task Scheduler - Super User for how to do this with conda.
I see nothing in the docs.... https://esri.github.io/arcgis-python-api/apidoc/html/index.html
nor in the sample notebooks https://developers.arcgis.com/python/sample-notebooks/
can the script be run as a general scheduled task?
Membros conectados podem postar, seguir atualizações e mais. Novo aqui? Registre uma conta gratuita.
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.
