Surveys stuck in outbox

2326
9
02-02-2018 01:20 PM
Highlighted
New Contributor III

We are having a reoccurring issue over multiple iPads (using v2.5) and different surveys where surveys get stuck in the Outbox folder and can not be opened to edit or resubmitted.  Nothing happens when the survey icon is clicked within the outbox and when the auditor submits the survey, no dialog box appears - the screen returns to the Outbox folder with the survey still there.  The surveys that are being submitted have multiple (80+) photographs.  Interestingly, the survey responses are submitted and can be viewed online in Survey123, but none of the photographs are present.  

I have reviewed the log files for some of these failed surveys (logs recorded during submittal attempts) and it appears that the error being reported is "Code 1000 - Cannot insert duplicate key row in object...".  I believe that because so many photographs are being submitted, the process is timing out and the photographs fail to upload.  Because the survey responses have been recorded, on resubmittal attempts, it is trying to overwrite attachments (even though they aren't technically present). 

Question 1 - should we be able to click on the survey in the Outbox and "edit and resend" - I believe that was historically an option and seemed to work when we ran into this issue before v2.5. 

Question 2 - How can I submit these surveys manually?  I've asked the auditors to send me the sqlite file and attachment database.  When I load that database into the desktop client and fix the database, I experience the same issue (in this case using v2.6) where after clicking submit, you are returned to the Outbox and it fails to the submit the survey.  Can I make edits to the sqlite to make this a unique submittal?

Reply
0 Kudos
9 Replies
Highlighted
Esri Regular Contributor

Hi Christopher

We are changing the way that we submit attachments in 2.7 and I believe that the new method may help avoid your error. We now upload attachments first and then submit the survey.

Are you a member of the early adopter program? If so I will share the latest build with you for testing (if you are happy to). Alternatively - please share your database with me and your survey form (so I can recreate the service) and I will try.

Thanks

John

Highlighted
New Contributor III

I will sign up for the early adopter program to try that.  In the meantime, any recommendations on the submittal for large surveys?  Should we always submit when connected to WiFi and keep the iPad on?  Any other tips for hopefully avoiding the timeout issue?

Reply
0 Kudos
Highlighted
New Contributor III

We are experiencing something similar with our surveys (v3.1.158). The user submits the survey, the many photos are attached and the survey submitted but the survey still becomes stuck in the outbox. I'm wondering if somethings is timing out and a response isn't getting back to the device to notify it of the successful submission?
We end up getting multiple duplicates in the database because the users attempt to send the survey again and again as they believe it has failed.

Has anyone else come across a similar issue?

Ryan

Reply
0 Kudos
Highlighted
Esri Frequent Contributor

Hi Ryan,

Were users able to successfully submit on the same device, with same survey and with same version of the app previously? Have the survey been re-published lately or any changes made to the feature service, or permissions and sharing changed?

Have you tried the Recover data from a mobile device troubleshooting steps in our doc as highlighted by John below: https://community.esri.com/message/841951-re-surveys-stuck-in-outbox?commentID=841951#comment-748364 

Are you able to upgrade to a newer version of the app on these devices? The reason being the way surveys are submitted has been improved in version 3.2, which will stop the duplicates being created in feature layer if the attachments fail. This may help if you can upgrade.

Phil.

Reply
0 Kudos
Highlighted
New Contributor III

Hey Phil,

The fault has always been present since we released these specific forms. One thing that seems to be a constant is that it happens more often with submissions with more attachments (photos).

We have tweaked and republished these surveys and services a number of times to try and alleviate the issue, to no avail so far.

The thing to note is that the survey does actually get correctly submitted. It's just the app does not get notified and "hangs" at sending. When the user backs out, the sent survey shows up in the outbox again.

We are using a custom deployment of Survey123 ( at v3.1.158) but have tried this on Survey123 as well and the same issue occurs.

Also, we are not using hosted feature services. We collect to our own published services. Wondering if there is some consideration when capturing this way? We have dozens of other surveys that work fine, it just seems to be the ones that have a large number of photos attached which are causing issues.

Thanks!

Reply
0 Kudos
Highlighted
Occasional Contributor II

Hi Christopher

In 2017 we had a client that had a lot of Survey123 records stuck on their devices. I ended up copying the databases off those devices and submitting them one database at a time with the Survey123 Desktop Client application. In some cases I had to use the specific Excel design file on each device to be able to submit successfully and recover the data that way.

It takes a little bit of know-how to do this but once you get it figured out it might be worth your while to figure it out - especially if the data is costly to recapture.

Basically what I did was:

- Copy each device's database to a laptop

- Open each database up with DB Browser for SQL Lite

- I then fixed some device specific paths such as the path to the image folder etc

- Once fixed I copied the fixed database over the Survey123 database (a084e65cd9403d813e8ec667ca329a63.sqlite)

- Once copied I would open the Survey123 desktop app and attempt to sync the records

In some cases I had to "recreate" the survey based on the Excel design file and retry to sync.

If your data is mission critical or expensive to recapture you could try this process. I didnt document the process in detail so I hope this rough workflow guides you to recovering your data.

Regards

Deon

Highlighted
New Contributor III

Thanks Deon - I think that is the approach I am going to have to take in this case.  I've had similar issues with the desktop app (not letting my sync).  Can you explain further what you mean by "recreate" the survey?  Do you mean creating another survey by using the existing answers and inputting the photographs that are present in the attachments survey for the same survey?  While not ideal, I think that's the fastest solution currently. 

Reply
0 Kudos
Highlighted
Occasional Contributor II

Hi Christopher

So for some reason some of our client's devices had different versions of the actual survey loaded  - even though they were obviously supposed to have been all the same. So I had to copy the Excel design files from each device and if the sync did not work the first time on the Windows Survey123 App I would use the Excel file to create a new Survey in Connect and have it publish to a new Feature Service. I would then download the new survey onto the Desktop Survey123 client and retry synching the records. I managed to recover probably about 95% of the data after merging the datasets into one.

Writing this from the top of my head makes me wish I documented my steps! Hope you could follow the logic!

Reply
0 Kudos
Highlighted
Esri Regular Contributor

Hi Christoper

Note that "edit and resend"  is accessed through the Sent box.

For reference: please see Recover data from a mobile device Troubleshoot—Survey123 for ArcGIS | ArcGIS for steps on recovery of data. When submitting, you could attempt to submit to a new version of the service (with the same schema, but no records) to see if you can get around  "Code 1000 - Cannot insert duplicate key row in object...".  At least this would get the surveys off the device.

In terms of best practice, the issue you are experiencing is likely to be due to the large number of photos being submitted. Whether you submit on collection, or when you are back in the office, should not matter. 

The important thing to ensure is that you are using a good WIFI connection when doing so.

As mentioned before, please feel free to share your survey and database with me (privately) and I will take a look.

Thanks

John

Reply
0 Kudos