What's new in ArcGIS Survey123 (February 2021)

[IsmaelChivite]

We are happy to announce our first update to ArcGIS Survey123 for 2021. This release includes fixes and new features, including a new gallery of survey templates in the web designer, much faster lists in the field app, greatly improved support for custom JavaScript functions in Survey123 Connect and a few other enhancements. Let’s get into the details:

Build numbers, versions, and other housekeeping items

• The Survey123 field app has been updated across all supported platforms. The app version is now is 3.12. You can update the app from the Google, Apple, and Microsoft app stores as usual. If you upgrade the Survey123 field app from a previous version in Android, you will note that a short process will run when you first start the app. This process should take only a couple of minutes and will tidy up the app files stored on your device. This process is necessary due to recent changes made to the Android operating system to improve security. We spent quite a bit of time making sure this transition is smooth for you. As with any update of the app, you should be up and running with the latest version smoothly.
• Survey123 Connect gets bumped to version 3.12 as well. For Windows users, just a reminder that Survey123 Connect is now available in the Microsoft Store. It is a good idea to install Connect from the Microsoft Store as the upgrade process does not require you to uninstall the previous version and you get automatic notifications for new updates. If for whatever reason you cannot install from the Microsoft Store, you can still use the traditional .exe installer files. If your current version of Connect was not downloaded from the store, you will need to uninstall it first, before you can apply version 3.12 but don't worry, you will not lose any of your work.
• Survey123 website and report services: The update to these components is now live. Nothing extra you need to do!
• Survey123 web app: As you may already know, by default we apply version locking to every online form you publish. What this means is that enhancements and fixes in the web app will not take effect in your existing online surveys unless you specifically upgrade your forms to the latest version. We do this on purpose, so you can control when a new version should be used with a production survey. To learn more about version locking and how to safely upgrade your online surveys, see this article. If you publish a new survey, it will automatically be published using the latest version available.
• Survey123 connector for Integromat and Microsoft Power Automate: We are just a few days behind, but we expect to share with you a new update to the Microsoft Power Automate connector soon. Stay tuned.

Survey template gallery in the web designer

With the Survey123 web designer you can easily create your own smart forms, visually and from your web browser. The Survey123 web designer is a great starting point for anyone who wants to quickly put together an online form. This update introduces a new gallery of survey templates to help you get started with some pre-configured designs. You can use these templates as they come or modify them to suit your needs.

The Esri templates are categorized by industry. We will progressively add new templates to this gallery.

Allow others to update your Survey123 web designer forms

For smart forms authored from the web designer, you can now grant other members of your organization permissions to update your survey. In this way, you no longer need to transfer ownership of your survey for others to work on it or maintain it. To grant others permission to update your survey, you simply need to share it with what is known as a shared update group.

Conceptually, a shared update group is a group where its members can modify items in the group, even if they do not own them. Only the owner of the item can delete the item and change its sharing, but other aspects of the items can be modified by anyone in that group.

Here is a list of a few important things members of the shared group can do:

• Change the survey design with the Survey123 designer. This includes things like adding, removing or updating questions, re-organize questions in the survey with groups and pages, changing the survey theme and the Thank You screen. In short, anything you can do with the Survey123 web designer.
• Create, change and add new webhooks. This includes automations from Integromat and Microsoft Power Automate as well as custom web hooks.
• Upgrade a survey to the latest version.
• Modify existing report templates.
• On the Collaborate tab, close the survey or schedule the closing date as well as modify the closing window, change access privileges to the survey or survey results.

Here are the things that members of the shared update group cannot do, unless they own the survey:

• Delete or move the survey
• Delete the survey
• Add new report templates.
• Change the scope of sharing. For example, adding or removing groups from sharing.

To use this capability, you must first ask your administrator to create a shared update group and invite people to it. Once you are granted privileges to share to a shared update group, open the Collaborate tab of your survey and activate the Update Survey section as shown below.

The list of shared update groups will show in the right so you can share your survey accordingly. If the list is empty, that means that you have not been granted privileges to share items with a shared update group. You will need to contact the  administrator at that point.

I want to emphasize that this capability is only available for surveys authored in designer. Survey123 Connect does not yet support shared survey ownership.  Additionally, please be aware that it is not a good idea to have two persons editing a survey at the same time. This is because the survey design is not locked to a single user and there is no conflict resolution logic. If two people edit a survey at the same time, the last one saving edits wins! That is just like saying that the changes from the first editor will be lost. This feature is not about collaborative design of surveys. This is more about allowing different people in a group to perform maintenance and occasional changes to a survey as needed.

Bigger, faster, better lists in the Survey123 field app

Lists are an important aspect of smart form design. They help us categorize observations, measure levels of agreement, intensity and preference. Lists also allow us to create datasets that can be analyzed more easily. Often, the challenge with lists is their size: How do we effectively work with a list including hundreds, thousands or tens of thousands of choices? How do we make these big lists work fast even in low-spec devices?

This update introduces significant changes around how lists are displayed and filtered in the Survey123 field app. It also incorporates new ways to help you work with big lists. Starting with 3.12 you will be able to handle bigger lists than ever before!

To start, the field app can now handle lists more efficiently. It uses less memory and is much faster. This is accomplished in part through special choice indexes that the app will build when the form is initially downloaded.  You may see a small "Optimizing survey" dialog appear when you load a form for the first time. That's when the indexes are being built! They get built once, and the rest is all joy.

To speed up very long lists, it helps if you use the select_one_from_file and select_multiple_from_file question types in your XLSForm design. You may remember we introduced these in our previous release. I would not recommend using these question types with small lists (up to 200 choices); but with anything larger, these question types will make a noticeable difference. The following table compares how long it takes to load a list in an iPhone when using a select_x versus a select_x_from_file.

As a reminder, select_one_from_file and select_multiple_from_file are equivalent in functionality to select_one and select_multiple, but they get the list from a .csv file rather than from the choices worksheet of the XLSForm. The .csv file must include at least the name and label columns, but you can also add other XLSForm columns to support multiple languages for your labels, choice images and choice filters. Learn more about this from our doc.

Our work with lists continues as we speak. If you check our Survey123 Early Adopter Community website, you will find documentation describing how we plan to allow you to populate a list dynamically from a feature layer query, and how to reference CSV lists stored as ArcGIS items. Your feedback is welcomed to help us bring these features to the released software.

Custom JavaScript function utilities in Survey123 Connect

In our previous update we introduced the notion of using custom JavaScript functions to complement XLSForm expressions. JavaScript functions offer flexibility for handling complex logic that is otherwise difficult to implement in XLSForm

With the 3.12 release, you will now find a new Scripts tab in the bottom bar of Survey123 Connect.

Writing the actual code does not get any easier, but there are a handful of nice features in the Scripts tab to make your work simpler. First, you no longer need to create the scripts folder manually: By adding a new JS function on the Scripts tab, Connect will place your code where it belongs. You can test your code right from the Scripts tab and also copy the corresponding pulldata() function call you need to insert in your XLSForm. You can also launch your script in your favorite developer IDE.

Learn more about the Scripts tab in our help.

Better XLSForm support

XLSForm is the industry standard language for smart forms. In every release we aim to improve our support for and extending XLSForm support.

Just as a reminder, in our previous update we enhanced our support for multilingual surveys using XLSForm notes. You can check the details here.

In this update, we now support the max-pixels parameter, which will help you limit the maximum size of submitted photos. The value represents the maximum size of the longest edge of a photo, in pixels.  Here is the link to the help.  

You can still control the size of images using the image size setting on the Options tab in Survey123 Connect, but this XLSForm parameter gives you a chance to overwrite the survey-level setting and apply custom image reduction rules to specific questions in your form.

Another nice addition is the compact-n appearance. With it, you can define exactly how many columns you want to use when arranging elements in a list. For example, the following will arrange the countries list in two columns. You could use compact-4 if you want 4 columns, and so on. The maximum number of columns you can use is 10. We hope this is plenty!

We also added support for indexed-repeat() and position() functions. They both help you work with data within repeats.

As shown in the list at the bottom of this post, we addressed quite a few issues related to XLSForm expressions. As good as these fixes are, they may break compatibility with form designs you may already have.  I want to highlight a situation where this could happen:

• Handling empty values: As described in this help topic, a mathematical operation will result in an empty value if any of the input values is empty. For example ${a} + ${b} will output an empty value if either a or b are empty. Some mathematical functions such as int() and number() did not quite follow this rule and converted an empty value into zero.  For consistency, these functions now will return an empty value.  This will break a function like this int(${a} + ${b}) if either a or b are empty. In the past, the operation would succeed because the empty value would be converted to 0 by the int() function. Now, it will result in an empty value.  If you really want your empty value to become a 0, then you would do something like this coalesce(int(${a}),0) + coalesce(int(${b}),0).  Even better, it you use the bind:type column to declare both the a and b questions as of type int, you could reduce your expression to coalesce(${a},0) + coalesce(${b},0).  Overall, the expression is more verbose, but all the mathematical operations are more consistent.

Other note-worthy enhancements

• Starting with this release, we will support Survey123 over-the-top installations. This means that you will no longer need to uninstall the previous version to upgrade Connect, or the field app.
• The report syntax now lets you better control number formatting. For example, assuming number1= 314159265358.975

${number1 | locale:"de"} will output 3141.5926.5358,975
${number1 | locale:"de" | useGrouping:false} will output 314159265358,975
${number1 | locale:"de" | toFixed:5} will output 3141.5926.5358,97500

Fixes Connect/field app:

• BUG-000099210 pulldata() will not work if values in the CSV file contain commas and double quotations.
• BUG-000105386 In Survey123 for ArcGIS, survey questions answered by using external-select cascading disappears on changing the language from default to any other.
• BUG-000135921 ArcGIS Survey123 Connect crashes if the survey contains a repeat_count calculation that uses two date questions with default values.
• BUG-000133831 Issues when scanning a Colombian ID using the PDF417 barcode type in ArcGIS Survey123.
• BUG-000134790 The select_one_external question does not display the choice list if list_name in the external sheet is the same name as an external CSV used in a pulldata calculation.
• BUG-000134556 In a multiple language survey form, if the default answer is specified for a select_one question (with minimal appearance), the survey will only show the default language's label in Survey123 Connect and the Survey123 app.
• BUG-000133776 ArcGIS Survey123 crashes when taking a picture with a complex survey with the older iOS devices.
• BUG-000135728 NMEA file returned empty when recording with integrated location provider using Survey123 Field App Version 3.11.
• BUG-000134370 In ArcGIS Survey123, inconsistencies in the EXIF metadata GPSImgDirection in images on Android 9.x.
• BUG-000124643 When using an Integer or String field with the choices tab, the numbers are shown with decimals.
• BUG-000133715 (Data-specific) Surveys with List questions that filter the available responses in other List questions have a large performance issue in Survey123 field app 3.10 when compared to Survey123 field app 3.6.
• BUG-000131344 The horizontal and vertical accuracies do not update in the ArcGIS Survey123 field app when connected via bluetooth with a Bad Elf GNSS Surveyor on an iOS device.
• BUG-000131446 ArcGIS Survey123 (iOS) crashes when the data entered for a survey contains a Date type question populated via external select.
• BUG-000121481 Switching from the default language to a different language after filling fields in does not change labels to the specified language in the Survey123 for ArcGIS field app.
• BUG-000129834 In Survey123 for ArcGIS (iOS) 3.7.57, a form using cascading selects crashes the application if a populate calculation is present in the same row as a choice filter referencing more than four fields.
• ENH-000133079 Provide functionality for indexed-repeats to extract information from repeats based on position within repeats.
• ENH-000119959 Add capability to populate a survey's choice list from a dynamic location, such as another feature layer or a database table.
• ENH-000109503 Request to have longer audio type questions in Survey123 for ArcGIS app.
• ENH-000100936 Provide the ability to overwrite existing installations for Survey123 for ArcGIS app.
• ENH-000100909 Provide the ability to overwrite existing installations for Survey123 Connect for ArcGIS.

Fixes website/web app:

• BUG-000136997 Generating a feature report from ArcGIS Survey123 web app fails and returns the following error message, "An error occurred while generating reports. Render expression is: ${undefined} Please see details in the recent tasks panel."
• BUG-000136239 ArcGIS Survey123 sends a poorly constructed query that is logically correct but appears as a potential SQL injection attack to firewalls.
• BUG-000136358 Users with a Viewer role receive an incorrect warning tip that they are unable to generate reports when multiple survey submissions are selected.
• BUG-000132481 The calculate question returns the message: "Cannot get answers for this question. Details: unable to find the field in related feature service" in the Analyze tab when bind::esri:fieldType is set to null.
• BUG-000135196 Generating reports fails for a survey with more than one geopoint type question in ArcGIS Survey123.
• BUG-000136085 When there are duplicate field names in the survey, the pop-up error message in ArcGIS Survey123 web designer does not go away until the survey is saved or the browser is refreshed.
• BUG-000136082 ArcGIS Survey123 creates duplicate question or field names on a survey which has a 'Single Choice Grid question' when the total length of the question label and initial common characters of 'questions (rows)' is longer than 31 characters.
• BUG-000133493 Geopoint question results in empty or null geometry values if the latitude and longitude values are cleared despite being a required question when the form is completed in a web browser.
• BUG-000135411 Creating a new survey on ArcGIS Survey123 connected to Portal for ArcGIS 10.6.1 returns the error message, "Error 500: UNKNOWN_ERROR."
• BUG-000132895 The Survey123 website Design tab loads infinitely when the organization shared theme is used for the Survey123 form appearance.
• BUG-000131870 Surveys with a calculation on a geopoint question do not honor the default map location set in the map settings when viewing the form in the Survey123 for ArcGIS web app.
• BUG-000131137 The number symbols for numerical questions in a survey configured with multiple languages change position when changing the language to English in the ArcGIS Survey123 website.
• ENH-000121409 Allow for owners of surveys to have the ability to appoint co-owners in Survey123 for ArcGIS Website.
• ENH-000130831 Allow for Survey Design Collaboration without needing to change ownership constantly.
• ENH-000129947 Provide a functionality to keep the map default ‘units’ to honor the organizational setting if a survey form is shared/accessed publicly.
• ENH-000132317 The ability to control the quality of an image in a Survey 123 for ArcGIS Feature Report by specifying the dpi (dots per inch) of an image.
• ENH-000131829 Modify the resolution of a photo attachment in a template for a Feature Report in Survey123.