why won't my Survey123 results send? No error, just stuck on attachments

NicolasPfeffer-Taggart · ‎08-25-2017

First, my setup:

-ArcGIS Enterprise 10.5 (Workgroup Advanced with 5 portal users) Base-Installation (portal, server, datastore) hosted on a single Azure

-Survey123 App for Android v2.3.29 (though previous versions did this as well) on ASUS MeMo Pad 7 and Lenovo Yoga Tab 3 8" devices (as well as a handful of iPhones, and various moto and samsung phones, depends on what my crew has on hand any given day)

-publishing results to the Hosted Feature Service that was created when I first published this survey from Survey123 Connect

What happens:

My users spend 1-3 days offline collecting data on a form that includes up to 2x places for photos. When they return, they connect to WiFi and initiate sending from their tablet. Without failure, at least one (typically many) of the 12 tablets simply gets stuck at "adding attachment 1 of..." (1 or 2, its not consistent). When I say stuck, I mean hours can go by, days even (weekend), without an error message or a change in state of the app or dialog. If I tap the back button, I'm returned to the form's main page, there may be a handful of items that made it through, and there are still a number of unsent items in the outbox. Here's the kicker... when I open the survey that was stuck, and locate the questions with the image(s), there is no thumbnail, but there is a filename for the associated image. If I simply delete the image (tap the little x button) and resend, that form will upload, and maybe 50% of the time the very next record gets stuck just the same.

What I've tried:

Initially, I thought maybe the internet connection and subsequent bottleneck of 5 to 8 devices uploading might be the issues. I've instructed and maintained that crew should only upload one tablet at a time... no difference... I've even taken tablets to a known high-speed WiFi connection at home (75mbps down / 25mbps up) and tried one-by-one to no avail. Sometimes they stick, sometimes they go....

Then I caught wind that some of my crew believed they were able to take multiple photos per question (not the case, there are 2x separate questions for photos without any repeat group). They indicated they would take one photo, then click the camera button again and take another. I've tried to recreate this, but of course it works fine when I do it, I simply end up submitting the last photo I took successfully...

I'm lost at this point... has anyone else experienced this? Am I missing something entirely? Thanks everyone for your help!

CarlHolt1 · ‎08-25-2017

I have users experiencing something similar, but only because they rename the pictures in the Survey123 app. So they will common names (eg: Overview, Dumpster, Catch Basin, etc.)

NicolasPfeffer-Taggart · ‎08-25-2017

Hmm, my users aren't obviously renaming things as the filenames still appear to follow the auto-generated scheme (eg: [QuestionName]_yyyymmddhhmm...) but I'll look closely to see if they are accidentally messing up a character or two... I wonder if there is any use-case where manual manipulation of the filename is actually necessary? Could the text field for filename simply be made read-only, forcing users to take the filename that is given when selecting/taking a photo? This would seem to avoid the problem...

JamesTedrick · ‎08-25-2017

Hi Nicholas,

Have you set the surveys to take the image at full resolution? Have you been able to try the recovery workflow outlined in Troubleshoot—Survey123 for ArcGIS | ArcGIS . Additionally, if you could log a transaction (see the troubleshooting page), it would help us isolate what's going on.

We will have an improved submission workflow for ArcGIS Online with 2.4, but unfortunately Portals can't take advantage of it. We are also working on a utility to assist recovery of data by copying over the database.

NicolasPfeffer-Taggart · ‎08-25-2017

I've set images to upload at "Large" resolution (from the publishing standpoint, in Survey123 Connect). This hasn't changed since original publication. Should I try lifting image size restrictions? will "Unrestricted" allow images to be captured at the full resolution of the device camera? or will Survey123 still put some reasonable limit on them under the hood?

As for the recovery method, I get the same result via the PC after copying and altering the .sqlite file onto my PC, copying the "My Survey Attachments" contents (by this point the image file in question is nowhere to be found in the folder on the device), and running "Fix database" from within the PC app. Same symptoms on the PC that were experienced on the mobile app, filename is there, but no image/thumbnail, causing certain records stick without throwing an error.

I'm headed off for the weekend, but I'll log errors and upload the results next week. Thanks for your help!

NicolasPfeffer-Taggart · ‎08-30-2017

I've retried the upload with one of the 'stuck' records with logging enabled and I've attached the output to the original post. Here's approximately what occurred during the logging from my perspective:

I opened the outbox
I tapped "Send" (app was already logged in)
The progress screen indicated "getting service information..." then stuck on "Adding attachment 1 of 1" for several minutes.
I then backed out, opened the stuck record and found the question with missing thumbnail but apparently-intact filename
When I proceed to click the folder button, navigate to "My Survey Attachments" on the device and scroll toward where the photo should be in the list, the app crashes before I can reach the picture in question
When I check the "My Survey Attachments" folder from a USB attached windows PC, the JPG doesn't appear to be there

So ultimately the file seems to be getting lost somewhere along the way... Either during upload, or after the offline-user saves the results for upload at a later time... Not sure how best to proceed troubleshooting, as I've been unable to recreate this issue myself and I'm relying on users reporting having collected images as usual.

Jean-YvesLandry1 · ‎08-31-2017

I've seen this happen when the app crashes (I tell them to save as a draft as much as possible especially on long forms) and for some reason the photo doesn't save properly and is lost.

I've had the crashing while in the photos as well with some type of tablets, maybe processor?

JamesTedrick · ‎08-31-2017

Hi Nicolas,

We have made improvements in the 2.4 release (available via the Early Adopter Community) to enable the submission of a form even when a photo that was taken with the form is missing - previously, this would prevent submission. You may want to try getting the new version and see if that resolves the submission.

Jean-YvesLandry1 · ‎08-28-2017

I keep seeing this in some of my forms, I've noticed it happens when the surveyor mistakenly changes the name of the picture and adds a character that is not accepted or removes .JPG,. This causes the link to the photo to break. I found the only way to fix this was by either removing the picture completely or going in the attachment folder, find the picture and either rename it if there's an illegal character or missing .JPG. works every time for me.

Good luck

Jean-YvesLandry1 · ‎08-30-2017

E.G. The form upload blocks on attachment 29; just keeps turning...

stop the upload reopen the survey and count to the 29th attachment (in my case photos)

Note how the picture is not showing up and the photo id is different. The surveyor somehow presses the photo name and changes it to an incompatible name and the photo link is broken. Press the folder icon find the photo with the help of the ID.

You'll now see the photo and although the ID changed it has no effect on the upload. Resend your survey and no more issue, well unless you have another photo id problem.