topic Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE in ArcGIS Survey123 Questions

sent survey shows 11 records in repeat, but only 4 are saved in SDE

LynnBerni — Fri, 12 Jul 2019 23:20:33 GMT

When I reopen the survey from the Sent Box (on ipad, v 3.5.164), I can see all 11 sample records in the repeat. But, only the first 4 samples collected actually made it through to the related table in SDE.

I have only one potential clue, but it's probably a long shot. The "Sample Notes" field in the 5th record contains gibberish (butt dial maybe??). Could that have somehow prevented this record, and the following 6, from being sent through to the database?

I haven't yet tried to resend the survey.

What happened???

Thx, Lynn

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

LynnBerni — Mon, 15 Jul 2019 16:48:15 GMT

We tried deleting the gibberish text in the Sample Notes field and then resending the survey. Now it's stuck in the outbox and we got the following send error:

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

JamesTedrick — Mon, 15 Jul 2019 18:32:42 GMT

Hi Lynn,

By re-sending the survey, did you open from the Sent box to edit the entry? If the entry is no longer in the database, that would cause the issue you are seeing. You should be able to resend by creating a copy of the record and re-submitting.

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

LynnBerni — Mon, 15 Jul 2019 18:45:47 GMT

Makes sense. Thanks. Can you help us figure out why only 4 of the 11 records in the repeat were saved in the database. Was it the emojis in the notes field?

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

JordanMiller4 — Tue, 16 Jul 2019 11:15:04 GMT

What happens if you add this parameter {query allowUpdates=true allowAdds=true} for each repeat in your survey?

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

LynnBerni — Tue, 16 Jul 2019 16:16:56 GMT

Hi James,

To answer your question....yes, we opened the survey from the Sent box. We didn't actually make any edits, just tried to resend. Prior to that, we had not removed anything from the database. Those 4 sample records (the first 4 out of the 11 collected in the repeat) were still in the SDE. We have since copied and resubmitted the entire survey as a new survey (sans emojis!) and all 11 samples went through successfully.

What exactly happened here? I would very much like to avoid this problem in the future. We could have easily missed the fact that some of the samples were missing in the SDE. And, if the sent survey had been cleared from the ipad, there would be no way to recreate and resubmit the data.

Also, would Jordan Miller suggestion to add the {query allowUpdates=true allowAdds=true} parameter to the repeat have prevented this problem?

Thanks,

Lynn

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

UhullGoGoGoGo — Fri, 26 Jul 2019 12:57:22 GMT

Hi James,

Survey123 has a data loss problem regarding decimal fields.
This is a very important issue and I don't see that
people in general are aware of or understand the problem.
Once I didn't find another way to call your attention
(as someone who knows deeply how Survey123 works), I'm making this
post at this unrelated topic.

https://community.esri.com/ideas/17003-survey123-data-loss-problem-on-decimal-fields

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

JamesTedrick — Tue, 06 Aug 2019 23:06:40 GMT

Hi Lynn,

Without looking into the database/submission logs for the entire operation (or being able to fully duplicate locally), I can only guess as to what happened- it seems most likely that there was a problem with either submission/storage or deletion of records. When you opened the entry from the Sent box to edit & resubmit, the record in the database (matched by GlobalId) wasn't present, causing the submission to fail.

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

JamesTedrick — Tue, 06 Aug 2019 23:08:20 GMT

Hi Guilherme,

Thanks for providing the information the problem you are facing regarding the use of decimal separators. What type of device are you using to input the numbers?

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

UhullGoGoGoGo — Wed, 07 Aug 2019 12:54:14 GMT

We have a broad base of users. Many use Survey123 on iPad Pro, some on Android smartphones and others on PCs running Windows.

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

UhullGoGoGoGo — Mon, 16 Sep 2019 12:25:59 GMT

Hi James,

I updated and simplified my post about the decimal separators.
It wasn't very clear...

Survey123 Data Loss Problem on Decimal Fields

I'd like to know if you recognize this behaviour of Survey123 field app as problem?

Or should the users be more careful when inserting a decimal separator?

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

JamesTedrick — Thu, 19 Sep 2019 21:26:46 GMT

Hi Guilherme,

Over the course of this conversation, there has been work done to revise numeric input - could either look at a 3.6 beta available at the Early Adopter Community?

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

UhullGoGoGoGo — Thu, 26 Sep 2019 12:57:44 GMT

Hi James,

I've installed Survey123 Connect and Field app 3.6.136.
The behaviour regarding decimal fields remains the same.

It still allows the user to insert both the decimal and thousands separator.

Should this behaviour have changed? Do I need to configure something?

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

JamesTedrick — Fri, 27 Sep 2019 18:13:44 GMT

Hi Guilherme,

My apologies, for some reason I confused this with thousands separators when I gave my earlier response. The improvements in 3.6 relate to the display of integer questions (not decimal questions). We do have work remaining to comprehensively turn off locale formatting; this is filed as ENH-000120747 in the Esri Support system; if you want, you can contact Esri support to be associated with this request and automatically notified when completed.

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

UhullGoGoGoGo — Mon, 07 Oct 2019 16:29:49 GMT

Hi James,

No problem, I've checked the ENH-000120747 entry with my Esri support.
But they were not sure it meant what we wish for.

Does "Support the ability for user to disable Regional format for a thousands separator.".
mean the users won't be allowed to type the thousands separator?

Also, looking at the docs of the early adopter website, I discovered
I can program custom javascript functions. That's cool.

Using it, there's a workaround for sort of disabling the thousands separator.
Not very convenient, but it seems to work...

Here's the XLSForm I've created, with 2 fields:

//Auxiliary field, for user input

type: text
name: decimal_text
bind::esriFieldType: null
constraint: regex(., '^[0-9]+[.,]?[0-9]*$')

//Field for actually storing the value

type: calculate

name: decimal_number
bind::esri:fieldType: esriFieldTypeDouble
bind::type: decimal
calculation: pulldata("@javascript", "to_decimal.js", "to_decimal", ${decimal_text})

Javascript file, to_decimal.js:
function to_decimal(x){
return parseFloat(x.replace(",","."))
}

Is there an easier or better way to do this?

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

JamesTedrick — Thu, 10 Oct 2019 00:13:34 GMT

Hi Guilherme,

Yes, the ENH is to disallow (ignore) the thousand separator in data entry. At this point, your JS function would be an appropriate workaround.

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

UhullGoGoGoGo — Thu, 10 Oct 2019 18:03:21 GMT

Ok, thanks for the info!

Re: sent survey shows 11 records in repeat, but only 4 are saved in SDE

UhullGoGoGoGo — Tue, 10 Dec 2019 18:47:54 GMT

Hi James,

Survey123 version 3.7 now has a new appearance type: thousands-sep.
Thinking about it, would it be possible to create an appearance type to disallow the
user to type the thousands separator on decimal fields (something like "thousands-sep-off")?
Check the locale, and then only allow to type the decimal separator.

Two weeks ago, one of our field workers, again, lost data due to the way Survey123
handles decimal separators plus an unfortunate behaviour/bug from a Samsung keyboard.
He used a Samsung Galaxy Tab S3 Android 9.0 to answer a form with decimal
questions to store measurements about river springs.

The issue occured because the Samsung keyboard was only allowing him to insert periods
in the field, but Survey123 expected a comma as decimal separator.

He thougth he was writing down precise measurements in the form. Later on we found out
there were only integer numbers in the geodatabase.

If there were a way to forbid the thousands separator, he'd have noticed something weird
and then stored the measurements somewhere else =/