topic Re: Automate Publishing of Survey123 Project in ArcGIS Survey123 Questions

Automate Publishing of Survey123 Project

jaykapalczynski — Mon, 30 Aug 2021 18:49:39 GMT

I have an existing project that has a few media files. I have frequent updates to these media files as well as occasional form updates. I need a python script that I can run via scheduler that will publish this project nightly. I have read a bunch of posts but cant find anything clearly stated that shows how to do this. Is there anyone out there that can help provide a solution?

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Mon, 30 Aug 2021 19:07:00 GMT

I found this https://www.arcgis.com/home/item.html?id=521168944ace40d0827991eeeba03689

I am just trying to figure out what is happening....I dont even know if I can still use this as its from 2017 !!

What Zip file?
Do I need to zip up the whole project or just the media files?

I'm really confused as to what is going on....I assume that I can use this to update my media files daily if need be.

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Mon, 30 Aug 2021 20:17:52 GMT

What is the ago.AGOLHelper

I know I have the correct UserName and Password but I am getting a NULL for the agol_helper.token

agol_helper.token = NULL

agol_helper = ago.AGOLHelper(portal_url) print ("...Authenticating") # login agol_helper.login(credentials['userName'], credentials['passWord']) print ("...Querying") # get the name of the zip file serviceInfo = getServiceDefinition(agol_helper.token, itemID)

Re: Automate Publishing of Survey123 Project

DougBrowning — Mon, 30 Aug 2021 20:56:27 GMT

There is a new way now using Linked Content. But it may have some issues it looks like. See this post https://community.esri.com/t5/arcgis-survey123-questions/auto-update-select-one-from-file-list-and-the/m-p/1045109

The other new way to to link to a feature service but that would require being online. It is in the early adopter only for now.

Hope that helps

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Mon, 30 Aug 2021 22:35:24 GMT

There are other things my lists do as in prepopulate etc....

I am looking for an automated process that I can replace the Media Files at will and then automatically republish the project nightly. Preferable from Python script.

Re: Automate Publishing of Survey123 Project

DougBrowning — Mon, 30 Aug 2021 22:58:25 GMT

You can link all your lists to a stand alone CSV that you store in AGOL. Works just like the media folder but no need to republish. Then you use Python to update that item. The form then sees this change and updates. Check out select_one_from_file.

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Tue, 31 Aug 2021 01:15:09 GMT

So right now I have 4 media files. The excel file points to them.

Do I simply add these CSV files to my AGOL folder?
How do I update the excel file to point to these files in AGOL?
I would then republish and it should now point to the files in AGOL???
How?

Are there any examples of how to update these tables via python?

Re: Automate Publishing of Survey123 Project

DougBrowning — Tue, 31 Aug 2021 13:21:25 GMT

It is all in the link above or this one https://community.esri.com/t5/arcgis-survey123-questions/can-a-linked-csv-file-on-agol-be-automatically/td-p/1006119

It is actually really easy to link. Then the post I linked above had the python code to update the item in AGOL.

Re: Automate Publishing of Survey123 Project

JamesKelly4 — Tue, 31 Aug 2021 19:55:03 GMT

@jaykapalczynski I still regularly use @Anonymous User 's script provided in this blog. You have to save the python2.7 script in the "download_folder" and you won't get the zip message error anymore.

Re: Automate Publishing of Survey123 Project

Anonymous User — Wed, 01 Sep 2021 04:55:10 GMT

Hi @jaykapalczynski,

You can use Linked Content to update choice lists from a hosted CSV file everytime the CSV file is updated and the survey info page is accessed. This avoids having to re-publish the survey just to update the choice list.

Please refer to external choice list section in documentation, with explanation about Linked Conent:

https://doc.arcgis.com/en/survey123/desktop/create-surveys/xlsformessentials.htm#ESRI_SECTION1_34A4D88BA6A64C48AA0E113DC61099DD

Regards,

Phil.

Re: Automate Publishing of Survey123 Project

Anonymous User — Wed, 01 Sep 2021 04:59:45 GMT

Hi @JamesKelly4 @jaykapalczynski @DougBrowning ,

Note that @ZacharySutherby has provided an updated copy of the script using Python 3 and some other improvements. See this comment here: https://community.esri.com/t5/arcgis-survey123-questions/updating-csvs-in-agol-for-select-one-from-file-and/m-p/1016256/highlight/true#M31917

Regards,

Phil.

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Thu, 02 Sep 2021 14:14:51 GMT

What zip file? I have to create a Zip file everytime?

Re: Automate Publishing of Survey123 Project

Anonymous User — Thu, 02 Sep 2021 23:59:26 GMT

Hi @jaykapalczynski,

If you are using either of the python scripts that are linked to the above comments to update CSV files or other media files, it downloads the Form item (which is a zip file), extracts it, you then update the items, and the script then re-creates the zip file again, and uploads its.

Note that this process does not update the XLSForm itself, the only way to do that is via publishing the survey again via Connect, as the xlsx file has to be converted to xml and other form item information updated. The script can only be used to update media files without publishing.

Regards,

Phil.

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Tue, 07 Sep 2021 17:10:59 GMT

Im confused....I know I have the correct Credentials but keep getting coming up with a NULL for the token I am trying to get via the script.

This returns NULL...any thoughts

print "token: " + json.dumps(agol_helper.token)

NOTE: I have ArcMap 10.8.1 installed. I have python 2.7 and python 3.4 on my machine. Could this be causing the issue? If so is there a way that I can specify to use python 2.7?

It appears I am using 2.7.18

print ("...Starting") # initialize the portal helper class # ago.py is part of the 10.3 python install agol_helper = ago.AGOLHelper(portal_url) print portal_url print ("...Authenticating") # login print credentials['userName'] print credentials['passWord'] agol_helper.login(credentials['userName'], credentials['passWord']) print "token: " + json.dumps(agol_helper.token) print ("...Querying") # get the name of the zip file serviceInfo = getServiceDefinition(agol_helper.token, itemID)

Noting that I updated the "portal_url" parameter for my AGOL online account....line 11

# User Parameters # ItemID of the Form item you want to update itemID = "00da9adCr45f4f15b3443ecadd81vfgh" # Local folder where the Form data will be downloaded (include a / at end of path) download_folder = "C:/Users/tjv34534/Desktop/CWD_Project/Update_Media_Folder/" # Updated file name (ie MyList.csv) updated_file = "CheckQuery.csv" # Folder path of the updated file (include a / at end of path) source_loc = "C:/Users/tjv34534/Desktop/CWD_Project/Update_Media_Folder/UpdatedMediaFiles/" # Address of your ArcGIS portal portal_url = r"https://xxxxxx.maps.arcgis.com/" # ArcGIS user credentials to authenicate against the portal (case sensitive) credentials = {'userName' : 'xxxxx', 'passWord' : 'xxxxx'} # ArcGIS portal security type (LDAP, NTLM, OAuth, Portal, PKI) security_type = "Portal"

Re: Automate Publishing of Survey123 Project

Anonymous User — Wed, 08 Sep 2021 02:26:00 GMT

Hi @jaykapalczynski,

As @ZacharySutherby points out in this post here and I mentioned in one of my comments above, the original update media folder script is now outdated and contains modules that have been superseded by the ArcGIS API for Python.

We have updated this script to use the ArcGIS API for python and the updated script is present in our Survey123-tools github repo.

Regards,

Phil.

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Wed, 08 Sep 2021 13:51:25 GMT

OK I think I have this working from ArcGIS Pro on my Laptop....it appears to be updating and uploading this to AGOL.

Most Important

How do I automate it to run when I want it to. I am new to using Pro and it appears to be baked into an ArcGIS Pro project?????

Re: Automate Publishing of Survey123 Project

DougBrowning — Wed, 08 Sep 2021 17:26:50 GMT

Just stick this line in there and it will grab the credentials from Pro for you.

gis = GIS('pro')

Pro does not even have to be open. Pro retains the token for 2 weeks.

But again as stated you should really switch gears to the new way and let this old way go. Way easier.

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Wed, 08 Sep 2021 17:28:56 GMT

Thanks Doug

So if I have the code baked into an ArcGIS Pro project in a "New Notebook" tab how do I automate it to run nightly.

It appears I can no longer use a .py file and use Task Scheduler to run it nightly.

Re: Automate Publishing of Survey123 Project

jaykapalczynski — Wed, 08 Sep 2021 17:31:13 GMT

So I cannot use .py files and schedule any longer???
I have to use ArcPro and Jupyter ???
How do I automate the script running?
Do I need to install ArcGIS Notebook Server to automate this?
Whats the difference between Jupyer in Pro and ArcGIS Notebook Server?

Re: Automate Publishing of Survey123 Project

DougBrowning — Wed, 08 Sep 2021 17:51:56 GMT

No you can schedule. I run this as a standalone script on schedule no problem. It just goes and grabs the Pro credentials for you.