topic Re: Duplicate Surveys Issues in ArcGIS Survey123 Questions

Duplicate Surveys Issues

RonKnepper — Mon, 27 Feb 2017 20:11:16 GMT

Ok so I have used Survey123 since its original Beta and there have been huge improvements. This past week was the first time I have used it in production in a larger scale, collecting about 250 surveys. I am doing some post processing of our data and I am noticing several duplicates. Some are complete duplicates of all data including the exact same times where the data is the same but some of the attachments are missing.

This is much better than loosing data (though I do have 1 survey that never seems to have shown up but could be user error). These all have at least 5 attachements and limited to Large size in the survey. The user who uploaded did say each day he would upload on wifi at home and of the 50 surveys each day a few would fail and he would resend them. I am guessing these are those.

Re: Duplicate Surveys Issues

RonKnepper — Tue, 28 Feb 2017 15:23:17 GMT

Ok digging into my data, I even have 1 location that ended up with 9 points. They are all full data but for the attachments they should have 5 photos but ended up with just 1 or 2 except for 1 with all 5

Re: Duplicate Surveys Issues

RonKnepper — Wed, 26 Apr 2017 00:38:28 GMT

Have been busy with other project work but we are back on this project this week. Any idea on these problems? I haven't checked our data for this round yet. I am pretty sure this issue is due to upload failures, I thought this was fixed prior?

Re: Duplicate Surveys Issues

JamesTedrick — Wed, 26 Apr 2017 17:47:40 GMT

Hi Ron,

I'm not completely sure what is going on- the only thing I could think of is that your users did multiple submissions (perhaps part of the attachment upload failed and a resubmission was attempted?)

Re: Duplicate Surveys Issues

RonKnepper — Thu, 27 Apr 2017 12:08:59 GMT

No it is definately failed uploads, I had it happen to me and to have that many isn't likely user (I was with him when submission). It is definitely failure during uploads of photos. Yesterday he lost service during upload and had to cancel the app which then caused duplicates.

Re: Duplicate Surveys Issues

RonKnepper — Mon, 01 May 2017 18:26:12 GMT

Starting to go through the data from last week, again a lot of duplicates. It is definitely when the uploads fail, one duplicate will have all data and all photos and the other will have all data and only 1 or 2 of the photos. I would guess so far about 20% or more surveys have been duplicated

Re: Duplicate Surveys Issues

JamesTedrick — Mon, 01 May 2017 19:03:28 GMT

Hi Ron,

Thanks for the update. Based on your description, I think I can adequately describe the problem as a bug to address. To clarify a little bit more, can you tell me if there is a GlobalID and/or a rowid field in the feature layer's fields (If you haven't done this before, go to the My Content page in ArcGIS Online, open the feature layer item details (it'll be in a folder names survey-<NAME>, and click on the 'Service URL' link. The list of fields will be displayed on the next page).

Re: Duplicate Surveys Issues

RonKnepper — Mon, 01 May 2017 19:45:42 GMT

This was created a long time ago and updated several times since

objectid (type: esriFieldTypeOID, alias: ObjectID, SQL Type: sqlTypeOther, length: 0, nullable: false, editable: false)
globalid (type: esriFieldTypeGlobalID, alias: GlobalID, SQL Type: sqlTypeGUID, length: 38, nullable: false, editable: false)

Re: Duplicate Surveys Issues

RonKnepper — Mon, 22 May 2017 18:05:09 GMT

Any updates on this? We are hoping to push out our work to the entire state which would mean multiple crews and a much larger amount of data coming in down the road which means a lot more duplicates to try and fix

Re: Duplicate Surveys Issues

JamesTedrick — Mon, 05 Jun 2017 21:05:32 GMT

Hi Ron

I've categorized this as a failure to detect when surveys have been been submitted but the attachments fail to submit in part or entirely. At this time, we don't have any particular timeline to address it. A couple of suggestions:

- What resolution photos are submitting? If you are forced to submit in low connectivity areas, the settings should be small or medium to minimize picture size.

- Encourage the workflow to delay submitting of data until in a known good location, like the office (or coffee shop) with wifi.

Re: Duplicate Surveys Issues

Anonymous User — Wed, 07 Jun 2017 00:14:06 GMT

You can also have the field crews download the Okoola speed app to test the connection strength before syncing. We had to do this for our data collection last year and we are still continuing this practice. We set a thresh hold of 8 down and at least 5 up...