Really excited about the latest Python API update. Would like to be able to export survey responses as reports with Python. I'm currently stuck at this part. I can see that the reports get generated and I can download them (image below from Survey123 website), but I receive an error in the code: KeyError: 'results' (see attachment). I'm also curious if you can access the generated reports directly with Python after making them.
Update: I put in a support ticket and there is an existing bug:
BUG-000119057 : The Python API 1.5.2 generate_report method of the arcgis.apps.survey123 module, generates the following error: KeyError: 'results'
I was playing with a workaround, where you basically just search for the generated reports in your organization and download them. I also delete them after downloading them so future runs wouldn't re-download them.
@rachelm
I'm not sure about that one actually. Maybe check the API reference? One of those arguments might do the trick!https://developers.arcgis.com/python/api-reference/arcgis.apps.survey123.html
@BHeist that did work, thanks!
Any idea how I can output the name of the saved item in AGOL to be similar to the default you can generate from the Survey123 page itself (e.g. FormName_OID_Date)? I'm stuck with nonsense names with part of the item id.
rachelm,
Try using the following code snippet. It will look through the content of the user and download anything that matched the surveyID of the report that was just generated. You'll need to define the variables 'surveyID' and 'output_folder' to match your environment.
# Search for generated survey123 reports owned by the logged-in usermy_content = gis.content.search(query="owner:" + gis.users.me.username, item_type="Code Sample", max_items=15)
for x in my_content:if surveyID in x.description:# Get ID of each Word doc AGOL itemid = x.id# Get item with matching IDdata_item = gis.content.get(id)# Download each Word doc to specified locationdata_item.download(output_folder)# Delete item from My Content after download as it's no longer neededdata_item.delete()
When I run a where statement that matches a single feature, I can access and download the resulting Word doc.
But I have two issues:
It seems other people ARE able to download the zip file, or at least the multiple reports?
Chris,
I had not seen that before since I've only ever worked with 4 positional arguments at one time. The only thing that was holding mine up previously was the structure of the SQL statement for the 'where_filter'. Once I was able to get that sorted out then it worked fine for me.
What did help get that sorted out was going to the 'Service URL' for the service layer I was querying for the feature report generation and using the 'Validate SQL' operation against that service. That helped me make sure my SQL statement was valid and also gave me a clue as to why it might not be working.
So maybe start there? It's always good to rule things out one at a time so you know what's holding you up.
Hope that helps!
Here's a code sample from my working script that I ran through a Notebook in Pro
from arcgis.gis import GISfrom IPython.display import displaygis = GIS("pro")
from arcgis.apps.survey123._survey import SurveyManager, Surveysurvey_mgr = SurveyManager(gis)
## Set / change all variables here
output_folder = "C:\\Users\\5018beh\\Downloads" # location on your local machine
survey = survey_mgr.get("insert ID here") # Creating survey object using survey ID
surveyID = ("insert ID here") # surveyID
template = survey.report_templates[2] # Feature report template from Portal
utc_offset = '+0400' # UTC offset for central time
report_title = "NCLF Daily Reports" # Title that will show in S123 recent task list
where_filter = "last_edited_date >= CURRENT_TIMESTAMP - INTERVAL '7' DAY" # Filter for survey submissions in the last 7 days
# Generate Survey Feature Report in Survey123survey.generate_report(template, where_filter, utc_offset, report_title)
Yeah that is what I tried. here is what i have.
Notice I also try passing in additional properties that are noted in the api documentation yet I get an error. I have also noticed that when only passing in the where parameter and the report name, I get a zip file with docx files but they only have the default name which is not helpful. have you seen this before?
Not sure if this would make a difference or not but have you tried passing your where_filter variable into the survey.generate_report function as a parameter? Instead of using the SQL statement as the parameter.
I would also double check your use of the "created_date" field and make sure it's the correct field both in it being a date/time object and that the name is correct.
like so:
where_filter = "created_date >= CURRENT_TIMESTAMP - INTERVAL '100' DAY"
survey.generate_report(surveytemplate, where_filter)
Also, maybe try a different day interval and see if it's the use of 100 days that's throwing it off? Maybe try 7 and see if that works?
Hope this helps!
I have been trying to get this to work for awhile, and sometimes I notice that the reports get generated and are in the temp folder on my C drive within a zip when I use "1=1", however when I try to use a where statement "created_date >= CURRENT_TIMESTAMP - INTERVAL '100' DAY", it completes but does not create any reports. any ideas?
from arcgis.gis import GISimport osfrom zipfile import ZipFilefrom arcgis.apps.survey123._survey import SurveyManager, Survey
gis = GIS("portal url","un","pw") #fill in 3 strings heresurvey_mgr = SurveyManager(gis)survey = survey_mgr.get("32a7416f7af14d8db2b20f19ca95be5b") #fill in 1 string hereprint(survey.report_templates) #see all available print templates for this surveysurveytemplate = survey.report_templates[0] print(surveytemplate)
reportTitle = "str=${lineid}_TEFIS_${tefis}_StrNo_${strno}"whereQuery = "created_date >= CURRENT_TIMESTAMP - INTERVAL '100' DAY"output_format = '{"output_format":"str=pdf"}'print(whereQuery)try: print('Trying to generate...') survey.generate_report(surveytemplate, "created_date >= CURRENT_TIMESTAMP - INTERVAL '100' DAY") print(survey.reports) print('Finished')except Exception as e: print(e)
Drew,
I was able to get the 'Where' clause to work using this syntax:
"last_edited_date" was the field in my layer that I wanted to query by. This generated a report from my survey for the last 7 days successfully
where_filter = "last_edited_date >= CURRENT_TIMESTAMP - INTERVAL '7' DAY"
survey.generate_report(template, where_filter, "+0400", "NCLF Daily Flare Inspection")
Thank you for this script. It helped me fight through the process of automating download of feature reports from Survey123. However, I have a question. I have an Excel list of about 500 feature reports that need to be downloaded. The list includes a lot of identifying information from the records in the survey's feature service. For example, the list includes a field called "1. WMIS Number:", which is a unique identifier. I need to query this field for the where_filter, and after reading your script, I tried the following as well as some other queries:
where_clause = '{"where":"1. WMIS Number: = \'100615\'"}'where_clause = '{"where":"1. WMIS Number:=\'100615\'"}'
Neither of these worked. I attached a screenshot of the field in Survey123. Do you have any suggestions?
Hi Kassandra, I think this could be related to your version of the docx library?
For Python 3.x, you need the python-docx library rather than docx.
These links have more info: https://pypi.org/project/python-docx/
https://stackoverflow.com/questions/22765313/when-import-docx-in-python3-3-i-have-error-importerror-no-module-named-excepti
Try uninstalling the old one, install python-docx, then see how that goes.
Hi
This is really helpful thank you for sharing I'm trying to hopefully use this for our workflow but I keep running into this error maybe someone can help. I made sure to download the other python libraries but looking at the docx.py I'm not sure what's going on. Any help is much appreciated
Hello, the try/except/finally block is built into Python as a way of handling exceptions.
In this instance we expect the KeyError exception bug (try/except) and tell the script to continue on after it occurs (finally).
Cheers.
Hi! I really love your work, but I have a question
Could you please explain to us how works the block try/except/finally to omit the KeyError?
Thanks
Hi Chelsea,
Firstly thank you for the code sample, this led me in the right direction for automating report creation with Python.
I took your code and added more functionality to it, which now generates reports for submissions from the last 24hrs, downloads the docx files, retrieves email addresses collected via survey, and sends a copy of the relevant docx file to the recipient as an email attachment. It also has a workaround try/except block for the known bug.
I'm sharing it here as others may find it useful:
https://github.com/nzjs/Automated-Survey123-Reports
Cheers,
John
I think I figured your issue out. By default with 1=1 (aka always true), I'm generating 3 reports because there are 3 completed surveys. The error I posted above shows that we're calling _survey.py to run generate_report(). In that .py, it says if where == "1=1": where = {"where":"1=1"} then it uses it like that. So when I put in my code a where statement that was valid and returned only 1 record when using Query under Supported Operations on the Service URL (link on the item's page in AGO), like this in my code: letslearn.generate_report(choice, {"where":"brandname='Fan'"}), only 1 report was generated. Still got my KeyError: 'results', but on the Survey123 website under Report>Recent Tasks>only 1 record in the last task. Hope that helps.
Hi, no I'm not able to access the output in Python. For my workflow I'm OK with having the resulting reports be available in AGOL, but I would like to be able only run it for a subset of the features, not all of them.
Glad to hear someone else is trying to do this. I haven't found a solution yet. Are you saying the output was fine from the Survey123 website? You aren't able to access the output in Python, are you?
Chelsea, I wondered if you or anyone else had made any further progress with this. I encountered the same error (although the outputted Word documents seem OK), and I'm also having a hard time figuring out the syntax to use in place of '1=1' if I'm trying to only generate records for a subset of the features in the feature layer. Any pointers / examples appreciated!
Aangemelde leden kunnen berichten plaatsen, updates volgen en meer. Nieuw hier? Registreer een gratis account.
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.
