topic Re: Bug with missing data in nests in ArcGIS Survey123 Questions

Bug with missing data in nests

ClaireWood — Fri, 24 Aug 2018 08:12:41 GMT

Hi,

I have been made aware of a bug where relevant questions are not displayed in the first record of a repeat, if accessed from the Drafts, Sent or Inbox. This means data recorded in the first nest of repeats is potentially going missing unless it is submitted straight away. Could you let me know when a fix will be released for this as it is a big issue for us.

Many thanks,
Claire

Re: Bug with missing data in nests

JamesTedrick — Fri, 24 Aug 2018 16:03:25 GMT

Hi Claire,

We plan on addressing this as quickly as we are able - we hope to have it addressed in the next release of Survey123

Re: Bug with missing data in nests

ClaireWood — Mon, 17 Sep 2018 09:02:19 GMT

Hello James,

We are still experiencing this bug using Build 3.0.149, in which the bug was said to be fixed. We are losing data from the first repeat of a survey response after reopening from the inbox, drafts, or sent box. It is not just the case that it is not displaying - the data is being lost, which is a pretty major issue.

Best wishes,

Claire

Re: Bug with missing data in nests

JamesTedrick — Mon, 17 Sep 2018 16:32:53 GMT

Hi Claire,

Thanks for raising this to our attention. Could you share your XLS file? There might be a difference in the form design compared to the other cases that were reported.

Re: Bug with missing data in nests

ClaireWood — Tue, 18 Sep 2018 10:32:28 GMT

Hi James,

I have attached the xlsx file. The error occurs on page 3. The repeats are subject to relevant clauses i.e. on page 1, the plot type should be ‘X’ and plot found should be ‘Found’ or ‘New plot xx’.

Best wishes,

Claire

Re: Bug with missing data in nests

Anonymous User — Wed, 19 Sep 2018 06:49:57 GMT

Hi Claire,

Thanks for attaching the survey, I am taking a look at the issue. A few questions, was this survey and the repeat values when loaded from Inbox working correctly before 3.0 update? Is the issue you see now only since 3.0? If you have more than one repeat record and load from Inbox, do you see 1 of 3 for example in the repeat, or 1 of 1? ie do all 3 records load but data missing in first, or none of the records load?

Another thing to note is that I see you are using default values for select one questions that are not in the choice list. This causes the value to be striked out, and then when a new value is selected from list the strike out remains. We have an open issue for this but just thought I would point it out.

Phil.

Re: Bug with missing data in nests

ClaireWood — Wed, 19 Sep 2018 09:37:10 GMT

Hello Phil,

I have just tested version 2.7.98, and do get the error. I find the error is not consistent - the first survey I fill in seems to be OK, then the error seems to occur in a second.

If, for example, there are 3 records in the repeat, all 3 load, but data is missing from the first.

Thank you for pointing out the striked out values - we may rethink that, although it does not appear to affect the data.

Many thanks,

Claire

Re: Bug with missing data in nests

LynnBerni — Wed, 19 Sep 2018 16:17:01 GMT

There are multiple threads on this same topic. I'm having the same problem. Data in first repeat is lost when the survey is reopened from Drafts or Outbox. It's only data in relevant fields, and all are referenced within the repeat. The repeat also includes a date/time field, with default set to now(). That gets reset too, to the current date/time (ugh).

I just saw the blog post Minor Update to Survey123 Available (August 30, 2018), which states that this problem has been resolved. So I:

confirmed that we are on the latest version of the ios app (3.0.149)
installed the latest version of Connect (3.0.142)
republished surveys
downloaded "updated" surveys to ipad

Nada. It's still deleting/resetting relevant fields in the first repeat.

I don't know if this was happening prior to 3.0.

Re: Bug with missing data in nests

Anonymous User — Wed, 19 Sep 2018 22:40:01 GMT

Hi Lynn,

Would you be able to share your survey, out of interest, do you have a repeat_count in your survey for the repeats?

In addition to my comments to Claire, does your survey contain relevant statements inside repeats that reference questions outside of the repeat?

Phil.

Re: Bug with missing data in nests

Anonymous User — Thu, 20 Sep 2018 00:33:44 GMT

Hi Claire,

Thanks for the extra info. I have just tested with 2.7 and 2.8 and the survey appears to be working correctly and all repeat values are loaded and displayed when using Drafts, Sent etc. If you are having issues with Inbox with 2.7 version then it is likely because there is no query clause on the repeats, which is used to load the repeats from Inbox, this works differently compared to Drafts and Sent which loads everything as it is already local on the device.

You need to add the following query to the begin_repeat questions in the bind::esri:parameters column to be able to update and add new records to repeats: query allowUpdates=true allowAdds=true

More info about repeats can be found here: Prepare for editing existing survey data—Survey123 for ArcGIS | ArcGIS

The above doesn't explain why when open from Drafts or Sent box you do not see values in the first repeat record, using the latest version of the app, which I am still looking into.

Another thing to note, I see you are using minimal, compact minimal and horizontal appearances on groups. Note that only compact and field-list appearance is supported on groups as per the documentation.

And I assume you know that the null esri field type being used on select_multiple BRYO or other means that no field is created in feature layer and therefore no values is ever submitted and stored for this question. Therefore when you open from Inbox it will always be null.

Phil.

Re: Bug with missing data in nests

Anonymous User — Thu, 20 Sep 2018 05:08:38 GMT

Hi Claire,

Further to my comments above, I have been able to narrow down the issue to be a bug with relevant statements inside a repeat referencing questions outside of the repeat. This causes the first record repeat values for those questions with the relevance to not be displayed when loaded from Inbox, Drafts, Sent box etc.

We will work on this issue and aim to release a fix for it in our next release. In the meantime the only way to overcome this issue is to remove the relevants on the questions inside the repeat that reference questions outside of the repeat. Applying relevants on the begin_repeat question will still work as expected though.

Another issue I noted is that your choice lists contain commas and spaces, as per the Survey123 documentation special characters are not allowed in the name field of choice lists (as well as name field of question name): Quick reference—Survey123 for ArcGIS | ArcGIS

Regards,

Phil.

Re: Bug with missing data in nests

ClaireWood — Thu, 20 Sep 2018 09:18:14 GMT

Hi Phil,

Many thanks, I will work on the things you suggest and look forward to the fix.

Best wishes, Claire

Re: Bug with missing data in nests

LynnBerni — Thu, 20 Sep 2018 15:10:34 GMT

Hi Phil,

I am not using a repeat count and the relevant statements are referencing questions *inside* the repeat only.

Here's a link to my XLS form.

Thanks!
Lynn

Re: Bug with missing data in nests

Anonymous User — Thu, 20 Sep 2018 17:37:38 GMT

Just commenting to for updates on when this will be fixed, we are experiencing the same issue.

Re: Bug with missing data in nests

Anonymous User — Fri, 21 Sep 2018 06:01:08 GMT

Hi Lynn,

Thanks for providing your survey. The issue you are experiencing is similar to the issue that Claire has, however is being triggered by a different configuration. The relevant statements you are using inside the repeat are based on a note question type inside the repeat, and based on the current implementation for note question types, they do not re-trigger relevant statements on the attached questions. I have logged this as a bug and we will aim to fix it in a future release.

In the meantime, if you change the note field to a read only text field, I believe it will work as you are expecting. I tested this with latest version of the app and beta release and the values were present when opening from Drafts or Inbox.

I found an issue with your survey which you need to fix, on the begin repeat question, you do not have a "query" clause, this means no records are ever queried and downloaded from the Inbox. See above comments above regarding this: https://community.esri.com/message/800823-re-bug-with-missing-data-in-nests?commentID=800823&et=watches.email.thread#com…

Also I see you are using null esri field types on the MILEAGE, TC_concat, ECOLI_concat fields, in this case the values are never stored, so will never been seen in the feature service or when opening survey from Inbox, they will always be null.

Hope this helps.

Phil.

Re: Bug with missing data in nests

LynnBerni — Mon, 24 Sep 2018 19:10:01 GMT

Hi Phil,

Thanks for the feedback. Per your suggestions, I changed the SITEID note field to a read-only text field (in fact, I changed all my calculated notes!). In the bind::esri:parameters field for the begin repeat question, I had allowUpdates=true. I have now changed that to query allowUpdates=true allowAdds=true.

I republished and retested my survey, but it did not solve the problem. I also tried changing the syntax to include a "=" after "query": query=allowUpdates=true allowAdds=true.

That didn't work either.

Btw, it would be helpful if Prepare for editing existing survey data—Survey123 for ArcGIS | ArcGIS was a bit more specific about the correct bind::esri:parameters syntax for repeats. Based on what I had read there, I thought that allowUpdates=true was correct.

Please advise, thanks!

Lynn

Re: Bug with missing data in nests

JamesTedrick — Wed, 03 Oct 2018 18:53:58 GMT

Hi Lynn,

allowUpdates=true is the correct syntax, however the query= is incorrect; you can have either query by itself (to indicate the default of 1=1) or query=<a DB query to select which records are displayed>.

Re: Bug with missing data in nests

ClaireWood — Tue, 13 Nov 2018 18:47:53 GMT

Hello,

I am still experiencing problems with losing data from repeats with questions with relevant clauses set. I have been testing the St Kilda Penguin release, and still have issues.

The issue I had above was related to questions inside repeats having a relevant clause set, based on a value outside the repeat. I have worked around this by removing the relevant clause from questions inside the repeat, which appears to work. However, I have a similar survey, which has questions with a relevant clause based on a value from inside the repeat - and again am losing data from the first repeat when loaded from the Drafts/Outbox.

I have attached the xlsx form, the issue occurs in the repeats on page 4.

Best wishes, Claire

Re: Bug with missing data in nests

BrandonArmstrong — Tue, 13 Nov 2018 20:54:07 GMT

Hi Claire,

The behavior you are encountering, with the data not appearing in repeats after the first, when there is a relevant clause applied to questions within the same repeat, has been raised in our internal backlog. We are currently investigating the cause. I will take a look at your XLSForm and confirm whether or not it looks to be the same behavior that you are encountering and will update this thread with more information if it appears to be something different. In the meantime, can you please confirm that the actual data is present in the feature service, and that this is a display issue as is the case with the previous issues listed?

Thanks,

Brandon

Re: Bug with missing data in nests

ClaireWood — Wed, 14 Nov 2018 10:50:28 GMT

Hi Brandon,

Each time I experience this problem with repeats, the actual data is being lost completely - it is not just a display issue. In this latest case, the repeat questions are required, therefore if you reload the survey from the Outbox or Drafts, then try and send, lots of errors for empty fields appear, for fields that were originally populated.

I have experimented with taking off 'required' for the fields in question then:

1. Save to drafts (or outbox)

2. Reload from drafts or outbox

3. Noted data is missing in survey from first repeats.

4. Submitted survey

5. Data is missing when viewed as feature service

6. Data is missing from exported geodatabase

Best wishes,
Claire