topic Re: Feature report download for multiple records in ArcGIS Survey123 Questions

Feature report download for multiple records

SimonGee — Mon, 20 Jul 2020 15:26:20 GMT

I have a Python script that checks a survey every hour, exports feature reports for any features added in the hour and downloads them to a local folder. It was working fine until last week, regardless of how many records there were, often downloading multiple .docx files each time it ran. Now it works as expected for single surveys, but if there is more than one survey matching the filter query it produces a zip file on the website (you can see it in recent activity, and manually download it) but doesn't download the file to the local folder. I know that this is how it works if you export multiple reports directly on the S123 website, but the scripted download was previously returning individual files. Has something changed on the back end that has caused this behaviour, and is it expected?

Re: Feature report download for multiple records

JamesTedrick — Mon, 20 Jul 2020 16:23:21 GMT

Hi Simon,

Just to check, this is being done with the ArcGIS API for Python? Can you provide a snippet of the code you're using to generate & download?

Re: Feature report download for multiple records

SimonGee — Sun, 12 Dec 2021 09:33:58 GMT

Hi James

Yes it is. I am using a script originally posted by John Stowell in 2019, which I have amended to loop through a set of queries and output to different local folders.

The relevant part of the code is below. As I said, it works fine of there is one record returned by the "where_filter". If more than one is returned, the reports are generated into a zip file, but not downloaded. Having just pasted the code below, I am wondering whether it is because the zip file is not found by survey.reports? However, until recently the whole thing was working, and was downloading doc files regardless of how many there were.

Thanks

Simon

interval = today-datetime.timedelta(seconds = 3600)
reportCount = 0
     # Try/except/finally block to workaround the KeyError: 'results' bug in the generate_report method
     # SJG: Think this bug has been fixed, but have left try/except in just in case
     try:
        print('Generating report(s) for submissions from last hour', file=open(log, "a"))
        print('')
        survey.generate_report(template, where_filter, utc_offset, report_title) 
     except Exception as e:
        print('>> ERROR: KeyError: ',e,' (related to ESRI BUG-000119057)', file=open(log, "a"))
        print('>> Continuing...', file=open(log, "a"))
        print('')
        pass
     finally:
        print('Downloading relevant report(s) to: ',output_folder, file=open(log, "a"))
        print('')
        for x in survey.reports: 
           # Find the creation date (Unix epoch) and convert to local time
           creationdate = datetime.datetime.fromtimestamp(x.created / 1e3)

           # Only find and download AGOL reports created in the last half hour (specified in "interval" variable)
           if (creationdate > interval): 
              reportCount += 1
              id = x.id # Get ID of each Word doc AGOL item
              print('Report number: ', reportCount, file=open(log, "a"))
              data_item = agol_login.content.get(id) 
              data_item.download(save_path = output_folder) # Download each Word doc to specified location
              time.sleep(15)
              data_item.delete() # Delete the item in AGOL folder after download
        # Finally block end‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Feature report download for multiple records

SimonGee — Mon, 17 Aug 2020 14:28:56 GMT

Just bumping this. Still having the same problem.

Simon

Re: Feature report download for multiple records

Anonymous User — Sat, 19 Sep 2020 18:37:59 GMT

I'm experiencing the same issue. I'm using ArcGIS API for Python with code similar to what Simon posted (I modified the code posted by John Stowell). It works great with the exception of the Zip file when there are multiple reports run. I would like it to continue to place individual word docx files in my AGOL folder and not create the zip file. After the download of the docx file I have a secondary script that renames the docx file and saves it as a PDF.

Re: Feature report download for multiple records

JamesFaron — Mon, 12 Oct 2020 16:28:40 GMT

It seems that this issue could be solved at least for some use cases if the survey123 module for python could accommodate the selection to 'Merge' multiple reports in a single file. In other words, in the survey.generate_report parameters one would be able to specify merge. I don't suppose it could be easily 'hacked' in _survey.py, but if so, it would help some folks to know.