What is new in Survey123 (July 3, 2020)

9695
63
07-02-2020 07:47 PM
Esri Frequent Contributor
11 63 9,695

We are happy to announce a new update to Survey123. This update only touches on the web components of Survey123 (Survey123 website, report services and the web app) and it is available in anticipation to our official 3.10 release, which will be available later in July or early August, including updates to Survey123 Connect, the field app and the documentation.

[Added August 6,2020]: Survey123 3.10 for the field app and Connect is available. Check https://community.esri.com/groups/survey123/blog/2020/08/07/whats-new-in-survey123-august-2020 for details.

The main driver for this early update is to bring to you a number of important fixes into the Survey123 web app, but since sometimes it is hard to decouple fixes from new features, you will find in this release some extra juice. One important note to make is that new features in this update are not officially documented. The documentation will come in just three weeks in late July, along with updates to other components of Survey123 like Connect and the field app.

Enough preamble, here are the highlights:

Survey123 Reports (Price reduction and increased capabilities)

Survey123 includes out of the box functionality to help you generate high quality documents from records submitted from the Survey123 web or field apps. This is extremely useful when you need to present your Survey123 data following strict formatting guidelines, for example when mimicking legacy paper forms or to comply with legal documents. If you are not familiar with Survey123 reports, check the https://community.esri.com/groups/survey123/blog/2019/07/23/understanding-survey123-feature-reports blog post and our Feature report templates—Survey123 for ArcGIS | Documentation help topic.

With this update, the price of this ArcGIS Online premium service drops from 2.5 credits to 0.5 credits per report. We are aware that this was an important aspect for many of you and we want to thank to all of you that took the time to discuss with Esri staff your use cases and reasoning behind your request. We also want to thank all your comments and community discussion at the Reduce Excessive Credit Cost of Survey123 Feature Reports ArcGIS idea thread.

Additionally, you can now try new expanded report template syntax, allowing you to include in your reports summary and statistics sections. This is a significant shift for the report capabilities and here is why: So far, the Survey123 report syntax allowed you to create 'feature reports', that is, a report representing a single survey record. Say for example you use Survey123 to perform routine asset inspections. When using feature reports, your reports include information about a single inspection: The date, status, comments, photos, a map of the location of the inspection and potentially data from related records such as all the incidents found. In feature reports, one report represents a single survey record.

Using the new summary section syntax in report templates, you can now create reports that include many survey records. For example, you could create a single report that includes all inspections ever performed, or all inspections in the last week. In a summary report, one report can include many survey records. Summary sections are generally shown as tables. In the following animation you can see a typical workflow where a summary section is used in a report to present a collection of survey records. The report is done against all water hydrants inspected in the past 18 days and it breaks down inspections in color coded tables (summary sections), one for each condition assessed.

Survey123 Summary Reports

Below is a still screenshot of the report. In it you will be able to appreciate that a small section is included at the beginning with some statistics: Number of hydrants inspected and hydrants that need maintenance. That is, of course, a statistics section, which is also part of the new report template syntax that this update includes.

Water Hydrant Summary Table

The new statistics and summary sections bring quite a bit of flexibility to your reports. Best of all, you can combine them all in a single report with a feature report section too. 

Back to the early discussion about the price, it will be worth clarifying how summary and statistics account for the price of your reports. Essentially, a report in Survey123, regardless of how many summary and statistic sections are included in it, will always cost 0.5 credits. In the event that you include additional feature report sections, the cost will increase at a rate of 0.5 credits per survey record included. For example:

Report includingArcGIS Online credits
1 feature report (one survey record)0.5 credits
1 summary section (as many survey records as you want)0.5 credits
4 summary sections (as shown above in the example)0.5 credits
1 feature report + n summary sections + n statistics sections0.5 credits
5 feature reports (5 survey records)0.5 * 5 = 2.5 credits

Photos and maps, can only be included within a feature report section.

To help you get started with the new report template syntax, we have added the capability to auto-generate sample report templates with summary and statistics sections as shown below. We will also be releasing a dedicated blog with details on this new syntax in the next few days.

Enhancements to Survey123 designer

Survey123 designer lets you visually create your own smart forms. With this update we are introducing the new Ranking question type. The Ranking question allows users to compare items within a list and sort them in order of preference. Based on the order selected, each item gets a numeric score.

Ranking Survey123

The main use cases for this type of functionality originate in online surveys. For this reason, this question type will not be initially supported in the Survey123 field app.

To avoid bias in responses, the ranking question can be configured to arrange the choices in the list randomly. This can easily be done right from within Survey123 designer.

Management view for ArcGIS administrators

If your account has administrative privileges in your ArcGIS organization, you will find a new 'Organization' tab in the Survey123 website. This new view, will let you easily find and manage any survey published in your organization. You will be able to update the surveys, change their sharing, etc.

Other fixes and enhancements

Last but not least comes a list of various fixes and other enhancements included in this update. It is a long list, and it includes work in very critical areas. As I described earlier in this blog, this is really the main driver for this update, to bring to you as quickly as possible the many fixes and enhancements listed here:

  • BUG-000120594 - Support for nested repeats in the Survey123 web app
  • Web Accessibility: Support keyboard navigation in the Survey123 web app (except for ranking question type).
  • ENH-000123510 Send related (repeats) record information to webhook providers.
  • BUG-000131124 A user within a Portal for ArcGIS Organization with a username of 20 or more characters long of only numbers for example,12345678901234567890, is prompted with the following error message, "Unable to publish Details: For input string: "12345678901234567890" when publishing a survey in the Survey123 Web Designer".

  • BUG-000126461 Unable to generate a stakeholder view layer when the source of the survey is a hosted feature layer view.

  • BUG-000126792 Attributes in a field created in Survey123 Connect for ArcGIS 3.6.1.137 using the select_one question type that reference a list_name with True and False choices do not populate in Feature Reports exported from Survey123 for ArcGIS.

  • BUG-000127161 The Basemap Gallery does not honor the custom sorting settings configured in the Basemap Gallery settings of the organization when a survey is accessed from the Survey123 for ArcGIS website. ([Design, web app] Refine the map list for geo questions to respect org settings)

  • BUG-000129766 Public-facing survey's service URL exposes the survey's data, even if the survey's sharing settings are set to private. (Restrict permissions on public surveys)

  • BUG-000131590 Pasting an image in the note's description of Survey123 Web Designer fails to publish the survey, returns the error message, "Invalid protocol: data"

  • BUG-000130588 Survey123 for ArcGIS website fails to show the note in preview mode when a note includes '%' and returns the following error message, "Error Initializing Form - URIError URI malformed." (Percentage sign % in group description or note question which is inside group and page cannot be Previewed)

  • ENH-000123861 Create options for summary of data in report for survey123 ([Report] Support summary report or summary section inside a report)

  • BUG-000127719 [Data]Edit in Individual Response with calculation may change original value after loading

  • BUG-000122381 Unable to view the hosted feature layer of a survey in ArcMap, published to ArcGIS Enterprise from Survey123 website.

  • BUG-000130716 The owner is unable to view any data and receives the message, "An error occurred while loading the page. Some features may not work" in the Data tab in the Survey123 for ArcGIS website when choosing 'Only add new records' in the 'What can submitters do' section. ([Data] Error message "An error occurred while loading the page. Some features may not work." raised during loading the page due to lack of Query capability in the survey layer)

  • BUG-000130065 Individual response: map cannot display correctly when the survey has a geopoint question has a default map which the spatial reference is neither geographic (4326) nor web mercator (3857)

  • BUG-000108603 When adding a geopoint question to a survey using the Survey123 for ArcGIS web designer, it is possible to select a non-web mercator basemap for the question, even if the basemap is not compatible with the survey.
  • BUG-000105581 A Survey123 for ArcGIS survey created on survey123.arcgis.com web hub which includes geopoint question with a user defined home location does not zoom to the correct location in the Survey123 field app; instead it zooms to the 0,0 origin point. (On publish, write Survey123 App - compatible survey settings)

  • BUG-000109078 BUG-000110137 Default map of Geo Point question from website will not work on Survey123 app

  • BUG-000130365 Select_One question records the first choice clicked in a repeat, instead of the final choice selected (select_one in a repeat cannot reflect the selected status in UI as expected)

  • BUG-000130473 If a page does not have a label, any Single Choice question in it cannot work

  • BUG-000127417 With Integrated Windows Authentication (IWA) portals, the geopoint question in Survey123 for ArcGIS gets stuck in Loading state on Google Chrome, Mozilla Firefox and Internet Explorer browsers.

  • BUG-000125138 Signature attachments submitted via webform are not visible in AGOL

  • BUG-000131123 A user within a Portal for ArcGIS Organization with a username that is 20 or more characters long of only numbers (ex. 12345678901234567890) is prompted with the error message, "You cannot submit records to the survey since you are not Submitter or the Owner" when accessing a Survey123 form in the web.

  • BUG-000130565 When taking a survey in the web browser created in Survey123 Connect for ArcGIS and a question has the 'minimal' appearance chosen and 'readonly' set to yes, the user is still able to select an answer for the survey (Read only minimal questions can still take value)

  • BUG-000130090 Geopoint question with the 'read-only' parameter enabled does not capture the latitude or longitude when filling the survey through the Survey123 for ArcGIS web app. (Read-only behavior of locate in Geopoint question)

  • BUG-000127924 When a 'View' URL is configured from a Survey123 web form which has pages, it does not show the questions.

  • BUG-000126498 In Operations Dashboard for ArcGIS, an embedded web form fails to reflect edits performed on a geopoint structured using the relevant fields and calculations in an Excel form.

  • BUG-000125101 The error message, "Error Initializing form.... Failed to load data from" is returned when opening a survey in the Survey123 for ArcGIS website that has been configured with a JavaScript extension in Survey123 Connect to pull data from an existing feature service.

  • BUG-000124744 HTML markup codes appear at the start of the published survey in Survey123 for ArcGIS.

  • BUG-000128852 Survey123 feature reports fail to generate when feature layer name contains accented character (example: é)

  • BUG-000125498 The bit.ly link that is generated by Survey123 will generate a 404 error response when navigating through the browser intermittently.

  • BUG-000131376 Adding a new signature to a survey via web using the edit URL does not save the changes if using a mobile device.

  • BUG-000130958 In the Survey123 for ArcGIS web app, the today() function in the calculation column of an XLSX survey file does not return a value if it is a hidden field with an 'esri:field:type' set to DATE.

  • BUG-000131141 The Analyze tab shows incorrect field in the ArcGIS Survey123 website when using a survey published from Survey123 Connect.
  • BUG-000132326 The error "Error occurred when querying data from the feature layer. Pagination is not allowed unless ''supportsPagination'' is true." is returned when running ArcGIS Survey123 feature reports. (Pagination error occurs when generating a report that has attachments for a survey in ArcGIS Enterprise 10.8).
  • BUG-000132135 Some publicly shared ArcGIS Survey123 surveys prompt a sign-in window when opened through the website (Regression 3.10: Public surveys that use one of 6 old basemaps from Survey123 Connect prompt for sign in if there is also no webmap in the org's basemap group with the same name).

Next steps

Officially, our 3.10 update is scheduled for July 23, 2020. On this date we will update Survey123 Connect, the field app and also include further work in the Survey123 website, web app and report services. We are furiously working towards having everything ready for you on that date. If you want to know what is coming, I encourage you to join our Survey123 Early Adopter Community. You will be able to download the Beta version of the software and provide feedback to the team. Additionally, we will share more information about the 3.10 release at the Esri User Conference.

63 Comments
Occasional Contributor

Hi Ismael Chivite,

Thanks for another great update, the summary reports should come in very handy.

Unfortunately since the update I'm finding that I can't generate any reports. The processing window shows they go through the Creating stage but then get stuck on Uploading (see below). I've tried clearing my browser cache and re-uploading my template but neither have fixed the issue. Is this something that could be looked into please?

Thanks

Anthony

New Contributor

Hi Ismael Chivite,

Did this update happen to include placing limits on the number of repeats a feature report will output?

As of yesterday, each sample feature report that I generate will only display the first 3 repeats of every survey. Just curious if this is another change that was made since it wasn't explicitly mentioned. 

Thanks,

Berg

Esri Frequent Contributor

Hi Berg Ellemers`‌  Thanks for your comment. I was able to reproduce your issue. This is not intentional. Definitively something worth reporting via Esri Technical Support as a bug. I added a note in our development repo so the team is aware. Stay tuned as we should have news soon.  I misunderstood your statement. Correction to my previous response: Yes, this is intentional. The Preview sample report functionality is limited to 3 survey records and 3 related records.

Esri Frequent Contributor

Hi Anthony Jones‌  I sent an email to you so we can get on a call and diagnose this more closely. This is not something I have seen before. Thanks!

New Contributor III

Hi Ismael - actually the feature report is now broken since the change was implemented last week. 

This is a huge issue that needs to be fixed ASAP - field crews are no longer receiving reports and we're about to get swamped with calls.

Beginning on Thursday, 7/2/20 our Feature Reports are no longer working either. I'm assuming this is because of the changes that were made to the Survey123 report features by ESRI. I don't know what's different or incompatible, I get the same Internal Server Side error message. I tried creating a new sample template from a submitted survey and that did not work either, so it's not something that's specific to my customization - the ESRI generated sample report does not work and returns the same error. This is a huge problem that needs to be fixed ASAP.

Internal Server Side Error

New Contributor III

It appears the issue is somehow related to having multiple sections that have repeats. See the examples I posted in this thread:https://community.esri.com/thread/256065-internal-server-side-error 

Occasional Contributor

Thanks Ismael. Just a further update on this. I've found that I can export individual reports but anything in bulk just hangs and times out. I've tried this on a couple of separate surveys now (albeit with very similar structure) and single exports in Word or as pdf work fine.

Esri Regular Contributor

Hi EnelGreenPowerNA,

Thanks for the feedback and the useful screenshot, we are looking into the issue.

Thanks,

Zhifang

New Contributor III

Hi Ismael and team, I have a customer who has raised an issue with the latest release of Survey123. They are receiving the following error when creating a Feature Report

The report has been working fine until the latest release. The field this error is referencing is a hidden field on the left hand side of the table. It appears that it errors on all of the fields. Here is a screenshot of what the table looks like with the hidden columns visible

Could someone please take a look and see what the issue may be? 

Regards

Jamie

Esri Regular Contributor

Hi Jamie Till‌,

Would you mind letting the user use a double quotation instead of a single quotation in the template? For example,

${if signage_place_com_type=="comment"}

We stopped supporting the single quotation in a report template since 3.4 release, it would be a little strange if the template could work before the 7/2 release of the website. Otherwise, can you please share the XLSForm and the template so we can look into the issue?

Thanks,

Zhifang

Esri Frequent Contributor

Anthony Jones‌  As discussed today, we will issue a fix on Thursday to address the 'upload' issue described.

New Contributor II

Hi Zhifang,

That was the problem - I have bulk replaced all single quotes with double and the reports are happy now

Thank You for you prompt diagnosis.

Steve

New Contributor III

Hello Ismael, since this recent update, our team on ArcGIS Portal Enterprise can't get more than 1 picture in Feature Reports. It used to print all feature attachments available, and now it prints only 1 and usually the last one... Is there any different way of referring to attachments in Feature Reports? Previously we've been using this to get all feature attachments printed: ${#$attachment}${$file | size:475:0}${$/attachment}

Esri Regular Contributor

Hi EnelGreenPowerNA,

We updated the beta website https://survey123beta.arcgis.com with a fix to this issue, would you mind trying on the beta website to see if it works for you? The fix will be applied to the Survey123 website https://survey123.arcgis.com late this week.

Thanks,

Zhifang

New Contributor III

Please, if you could look at this issue ASAP! Our fieldworkers also getting anxious about not being able to print feature reports as usual... Thanks in advance! /Aurelija

Esri Regular Contributor

Hi Aurelija Ruta Viluckyte‌,

Can you please share which version of ArcGIS Enterprise you are using?

New Contributor III

Yes - it's now working correctly on the beta site. Need to get this pushed up to the production site ASAP. Thank you.

New Contributor

Hello,

Since this release I noticed that the signature box in my survey is causing a "Cannot read property 'type' of undefined" when I try to submit it. If I remove the signature box it submits correctly. I tried it on the beta site as well and it still errors out.

Thanks,
Rich

Occasional Contributor

Hi, since this release a previously published public survey with browser direct access now includes a prompt to open in ArcGIS Online.  How do we disabled this logon prompt?  Users are able to click cancel and submit, but we prefer to not show prompt.  The survey was originally published using Connect

Esri Regular Contributor

Hi Richard Hammond‌,

Would you mind sharing the XLSForm with us so we can look into the cause of the issue?

Esri Regular Contributor

Hi Deena Bowman‌,

Thanks for the feedback.

We've noticed this issue and looking into a hotfix to include the fix. Before that, you can turn on the version lock in Settings->Version of the Survey123 website to set the survey use the previous web app other than 3.10 as a workaround.

Esri Regular Contributor

Hi Aurelija Ruta Viluckyte‌,

We have identified the issue and are working on the fix, will let you know once the fix is available to test.

Sorry for the inconvenience. 

New Contributor III

Hello Zhifang, I will ask our customer, because I don't know exactly. 

New Contributor III

Will be looking forward to the fix! FYI > The version of the ArcGIS Enterprise used for that project is 10.7.1 

New Contributor III

However, I reproduced the same issue with my company ArcGIS Enterprise, and it uses 10.6 version. 

Occasional Contributor

Hi Zhifang Wang

Thanks for the reply, this works!

Esri Regular Contributor

Thanks for the additional information, we just figured out that the bug applies to feature service that does not support /queryAttachment (ArcGIS Enterprise < 10.8)

New Contributor III

Hmmmz will this be fixed for earlier versions as well? I mean it was working 1 week ago.  

Esri Regular Contributor

Hi Anthony Jones‌, 

The fix for your feature report issue is on Survey123 website https://survey123.arcgis.com now.

Thanks again for your feedback.

Esri Regular Contributor

Hi EnelGreenPowerNA,

The fix for feature report issue is on Survey123 website https://survey123.arcgis.com now.

Thanks again for your feedback.

Occasional Contributor

Thanks for your efforts on this Zhifang, it's much appreciated.

New Contributor

Gusmilan Pratiantono assisted me with this issue in another post. It turns out page labels that were blank were causing this issue. They were blank since I created this survey a few months ago, so I'm guessing this was something caused with the recent update. Thanks for the help, Gusmilan! 

New Contributor III

Hi, I've been running into an issue when generating a feature report with data from a repeat since this update. 

More details here:Error occurred when querying data from the feature layer (in  a Feature Report)

Any ideas?

Thanks,

Josh

Occasional Contributor

Hi,

I am facing the same. I checked the supportPagination and found it 'true'!

Any solution? 

Esri Regular Contributor

Hi Aurelija Ruta Viluckyte‌,

The fix to the issue is available on our Beta website https://survey123beta.arcgis.com now, would you mind trying to see if this works for you? If it works well, we plan to deploy the fix to the Production website later this week.

New Contributor III

Yes Zhifang, your fix seems to be successful. Now this function returns all attachments into the feature report just as before. Great work! Let me know when it is deployed in the main website  

New Contributor

Hi,

What about statistics, summary and repeats (related tables). Is it possible to use stats on related tables ? In our case, using forms to collect data, most of the information is stored in the related tables (repeats) ??

However, we like all of the (coming soon) new reporting and other new features in survey123 connect and fieldapp 

New Contributor III

Hi Zhifang Wang‌ do you have any information regarding the issue I mentioned above?  It looks like something that may have been introduced with this update.

Thanks!

New Contributor

Yes. We did some investigation and were able to do. Although in this way it's not respecting the relation.

But according to the documentation the syntax for statistics/summary is

${layerName | stats:"outStatistics":"groupByFieldsForStatistics":"havingClause"}

For LayerName you can use each layer within the hosted layer. Creating and publishing a survey with repeats will create a hosted layer with mulitple layers. Each repeat will be a layer (a table) within the hosted layer.

For example. The hydrants has a repeat for the inspections.

In the report you can have:

${Hydrants | stats:"count,OJBECT_ID"} (the number of hydrants)

${Inspections | stats:"count,OJBECT_ID"} (the total number of inspections)

 

IMPORTANT: the relation between e.g. the hydrants and inspections is not taken into account. You can do it yourselves using the WHERE clausule in de syntax. In our repeats we always have a number of fields from the parentlayer to be able to report indepently from the parentlayer. We use this fields for grouping and selecting.

Esri Contributor

Hi Richard,

Thanks for bringing it up, and you are right about adding the page label to work around it. We have fixed the issue and it will be available for the coming release at the end of next week. Would you please have a try then?

Thanks,

Jody Zheng Liu

Esri Contributor

Hi Deena,

This is fixed on survey123.arcgis.com.

Thanks,

Jody Zheng Liu

Esri Contributor

Hi Aurelija Ruta Viluckyte‌,

The fix is on https://survey123.arcgis.com, would you please have a try?

Thanks,

Jody Zheng Liu

Esri Contributor

Hi Joshua Ittenbach‌,

This should be fixed on survey123.arcgis.com, would you please have a try?

Thanks,
Jody Zheng Liu

Esri Contributor

Hi Hussam Rabea‌,

This should be fixed on survey123.arcgis.com, would you please have a try?

 

Thanks,
Jody Zheng Liu

Occasional Contributor

Hi, the Report is generating successfully. Thanks

the photos did not appear in the report. I am using portal 10.8

New Contributor III

Hello, I have informed my team about this, will let you know if the fix is working for us next week (i am personally

on holiday, so i cant test it). Thank you! 

New Contributor II

Hi,

On behalf of Aurelija,

I can confirm that its working as it should now.

Thanks for this quick fix.

Best,

Lars

Esri Esteemed Contributor

Hi Hussam,

Are you using the 

${#$attachment} 
${$file}
${/}

workaround to display images?  Otherwise, images are not supported with ArcGIS Enterprise.

Esri Regular Contributor

Hi GISAdmin GKB‌,

Thanks for the sharing, glad to hear the new summary/statistics capabilities work for you (to some extent).

By design, if you do statistics or iteration for records in both the survey main layer and its related layers (repeat in Survey123) in a summary section, you need to use the '| where:""' filter to control what records to be included in the stats/iteration. This is because there is no "default context" or you can consider the "default context" is all records in each layer ( '| where:"1=1"').

On the contrary, if you do statistics or iteration of records in a related layer/repeat in an individual section/individual report, there is a "default context": data range for related layer/repeat will be restricted to records that only related to the current record in the main layer.

For example, in your individual report template, if the current record in the main layer is hydrantID=15, then the following:

  • statistics: ${inspections | stats:"count,objectID"}
  • iteration: ${#inspections}...${/}

will only use inspection records that related to the hydrantID=15.

  • Furthermore, if you need to do a stats for other records that are not related to hydrantID=15 in the current individual report, you can use the !important keyword to overwrite the current "context". For example: While the hydrantID=15 has ${inspections | stats:"count,objectID"} inspections, there are ${inspections | where:"hydrantID=16 !important" |  stats:"count,objectID"} inspections for the hydrantID=16.

Hope this may help and please let us know if you have any question.

Thanks,

Zhifang