Updating csvs in AGOL for select_one_from_file and pulldata in Survey

[LeeIrminger1]

I am following the method by @CuartaOportunidad_Local_IVC posted on the Updating the media folder of a Survey123 Form item blog.

 see direct link here

Updating the media folder of a Survey123 Form item - Page 2 - GeoNet, The Esri Community

I am wanting to update the media folder for three CSVs that are accessed with select_one_from_file and also worked with a pulldata() calculation. 

The method works and changes are picked up in the Survey123 Field App on the desktop but not in the browser. The browser is only shows updates published from Survey123 Connect. Should I do something to have it update the browser? I cleared cache and ensured that it is updating with the same name as the original name of the survey. I also downloaded the survey as a zip and verified the csv reflected the change I made.

I would rather use this vs the method which uses the older ArcREST.

shutil.make_archive(r'temp' + 'filename', 'zip', r'surveyfolder')

path = r'zippedsurvey'

surveyitemID = "itemid"

surveytoupdate = my_gis.content.get(surveyitemID)

surveytoupdate.update(data = path, item_properties={'type' : 'Form', 'title': 'filename', 'tags' : []})

thank you!

others have this issue

Survey123 Update Through Python - GeoNet, The Esri Community

Updating a Form using python - GeoNet, The Esri Community

[ZacharySutherby]

Hello @LeeIrminger1, 

We are in the process more formally presenting python scripts and will be updating that Esri Community blog post shortly. 

In the meantime we have updated the Update Media Folder script to use the ArcGIS API for Python. 

I have attached a zip file that contains a Jupyter notebook and .py file that use the updated workflow. 

I have just tested the same workflow using the script in the zip file attached and I was not able to reproduce the same behavior. 

If you test updating the CSV using the script in the zip file is the same behavior observed? 

Thank you, 

Zach

[LeeIrminger1]

Hi @ZacharySutherby 

Thank you for the response and sharing the script! The script works like a charm except I only see the updated data in the Survey123 Desktop app and not in the browser. I ultimately need this in a dashboard so the browser functionality is important to me. 

My XLSForm has 12 questions, four are select_one_from_file and there are three .csvs that those select from. Those three csvs are being updated using this new script and I see the changes when I download the zip from AGOL. The Desktop app sees and can download the update. It's like the browser version does not see that the files have been updated. I've cleared cache and all browsing data in two browsers, Edge and Chrome. Is there a way to trigger the browser-based Survey123 to look for updated media files?

Thank you!

[LeeIrminger1]

Hi @ZacharySutherby 

I replied about an hour ago but the post has disappeared. Thank you for the response and sharing the script! The script works well, but I am seeing the same issue. The ArcGIS Survey123 Desktop application can download and see the updated data but the web-based Survey123 form does not. Is that expected? 

Thank you!

[ZacharySutherby]

Hello @LeeIrminger1, 

To confirm was the CSV present in the Media folder upon initial publishing, or is the CSV hosted in ArcGIS Online and linked to the survey using the workflow outlined in this EAC documentation? 

If the CSV is only present in the Media folder and not linked to the survey I would not expect this  behavior as I did not see the behavior when I had tested on my end. If the CSV is linked to the survey there may be something different going on. 

If the CSV is only present in the Media folder the Survey123 website should be making a request for item and pulling from the response of that request. 

If you navigate to: 

https://www.arcgis.com/sharing/rest/content/items/<FormID>/info/media/<CSVName.csv?token=<accessToken>

Does this download the most recent version of the CSV? 

If you capture the web traffic when loading the web form you will also see that same request in the web traffic. Is the response of that request the updated choices or the old choices? 

Thank you, 

Zach

[LeeIrminger1]

Hi @ZacharySutherby 

The CSV was present in the Media folder upon initial publishing.

When I navigated to the URL it did not show the most recent version, it shows the original version. Interesting!

The request in the web traffic also leads to the old choices in the csv file.

Thank you for looking into this!

[LeeIrminger1]

Hi @ZacharySutherby,

If the original list is what pulls up from the rest url and when the web form is loaded but the field app can see and download the updated survey with the new csv data, does that mean there is something incorrect with the formatting CSVs that is causing this? I have the CSVs exported from a pandas dataframe using python. That is how the original CSVs were created and placed in the media folder as well.

Thank you!

[ZacharySutherby]

Hello @LeeIrminger1, 

Apologies for the delay in reaching out! I wouldn't expect it to be something with the CSV formatting but it could very well be a formatting issue. To remedy that  resaving the CSV's using UTF-8 may help if it is a formatting issue. 

I know you had mentioned that the CSV was present in the media folder upon publishing, just to confirm the CSV isn't also hosted in your organization and linked to the survey correct? 

If I could obtain a copy of the XLSForm and CSV's I can do some testing on my end and see if I can reproduce the behavior. Feel free to send them to ZSutherby@esri.com if they are not able to be passed through the thread.  

Thank you, 

Zach

[MichaelBrown4]

I'm experiencing the same issue as the original poster.  I have the exact same setup, using select_one_from_file and pulldata() that reference a linked content CSV and the web form does not update to reflect the latest updates to the CSV file on ArcGIS Enterprise.  When I inspect the form loading in the web, I can see it's grabbing the CSV from the media folder, which is old and out of date, and not referencing the linked content CSV.

Here is a screenshot of how the web form is still trying to pull from the Media folder even though there are no files there and I am using Linked Content.

[ZacharySutherby]

Hello @MichaelBrown4, 

What we determined happened was when the survey was originally published the linked CSV was included in the media folder.

What was happening was the update media folder script was working with a separate CSV than the linked one and when the script extracted the ArcGIS Online Form Items media folder it added the current timestamp the script was running to the file modify time.

Since there was an older version of the linked CSV in the ArcGIS Online Form Items media folder the field app thought it was working with the most recent version due to the time stamp, although it was using an older version.

The suggested workflow was to removed the linked CSV from the media folder in Connect and republish the survey. That way the field app can handle resolving the timestamp differences with the linked CSV, and the timestamps for the content in the Form Item's  Media folder would  be independent from the linked CSV. 

Thank you, 

Zach