ArcGIS Survey123 Blog

cancel
Showing results for 
Search instead for 
Did you mean: 

Other Boards in This Place

Latest Activity

(189 Posts)
Esri Frequent Contributor

With each Survey123 release, we assess the platforms we support and adjust these based on customer needs and technology trends. The purpose of this article is to provide as much advance notice as possible regarding these changes.

Read more...

more
3 1 97
Esri Contributor

Setting boundaries for survey submission using JavaScript in Survey123

Read more...

more
4 0 134
Esri Frequent Contributor

New ranking question, enhanced choice filters, hidden appearance, improved report template syntax and much, much more in this release.

Read more...

more
9 11 3,488
Esri Frequent Contributor

The best way to tackle a big problem is to break it down into smaller ones. Lists are no different. Choice filters let you leverage user-entered values in your form to dynamically control which elements of a list are shown or hidden. The most common use for a choice filter is to construct what is known in XLSForm jargon as a cascading select, where a selection in a list (for example countries), controls the contents of a second list (for example, States, or regions within that country).  However, as we will see in this article, choice filters can be used well beyond cascading selects.

This blog introduces choice filters in Survey123 and some common techniques to work with them effectively. If you are already familiar with choice filters and cascading selects, you may still want to read this article as it describes a few new interesting things added in version 3.11 (October 2020).

If you are not familiar with choice filters, XLSForms or Survey123 Connect, you may find this content a bit steep. I suggest you watch this 2 minute video with an introduction to XLSForm select questions to warm up. This is not rocket science, but basic familiarity with Survey123 Connect and XLSForms is assumed.

The basics (Cascading selects)

In XLSForm jargon, choice filters are often associated with the notion of 'cascading selects'. The idea is that a selection in one list, determines which values will be presented in the following list. Say we have a list of countries, and a second list of regions. As you select a country in the first list, you want the second list to just show regions for that country.

Below is what the countries and regions cascading select example could look like:

Survey123 Choice Filters

Building your own cascading select is pretty easy. Here is a brief step by step guide using a hyper-simplified countries and regions example:

  • Open Survey123 Connect and create a new survey using the Advanced template
  • Switch to the choices worksheet and add the countries and regions lists as shown below:

  • Now switch to the survey worksheet and add two select_one questions.

  • Save your spreadsheet and preview your form in Survey123 Connect. All the regions will be shown, which is not good. We need a choice filter for that!
  • Switch back to the choices worksheet and add a new country column to the worksheet. This new column can be called whatever you like, but in our case we will call it country.
  • For each of the rows in the regions list, populate the country column accordingly. The values in this column must match the name values from the countries list.

  • Now that the choices worksheet specifies the country for each of the regions, we can define a choice filter in the survey worksheet.
  • Switch to the survey worksheet and look for the choice_filter column. Apply the following expression to the region question: selected(${country},country)

  • Save your spreadsheet and preview your changes in the Survey123 Connect preview. Now your cascading select should work as expected: As you select a country, the list of regions changes accordingly.

As you can see the concept is simple: The expression in the choice_filter column of a select question determines which elements of the list will be displayed. In the choice_filter expression you can reference questions from your survey and  custom columns from the choices worksheet. In our scenario, the expression was selected(${country},country). The choice_filter expression is evaluated against every choice in the region list.  If the expression evaluates to true, then the choice is kept. If the expression evaluates to false, the choice is dropped.

If you have been using choice filters for a while with Survey123, I bet your choice_filter expression would have looked like this: country=${country} instead of selected(${country},country).  Both expressions are correct. We will talk more about the differences between these two expressions, and when to best use one or the other, later in this blog in the 'Working with very large lists'.

Chaining cascading selects

In the previous example we constructed what is known as a cascading select. You can cascade, or chain, as many selects as you like. For example, you can add a cities list next:

In the choices worksheet, note that I added a new region column for the cities list.

In the survey worksheet, I added a new select_one question for the cities and its corresponding choice_filter.

Choice filters on checklists (select_multiple)

You can apply choice filters to select_one and select_multiple questions. In the example below, note that I converted the regions and cities questions into checklists. Say for example, you visited Spain. Then I want users to select all the regions they visited in Spain, and also all the cities visited within these regions.

To do the above, I simply changed the type of questions from select_one to select_multiple.  Since the selected() function works for all select questions (select_one and select_multiple), you do not need to change the choice_filter.

Support for cascading selects on select_multiple questions was added in version 3.11 (October 2020)

More on choice_filter expressions

Given that the choice_filter column can evaluate XLSForm expressions, you can get creative in the way you apply filters to lists.  The key concept to understand is the following:

A choice_filter expression is evaluated against every choice in the list. If the choice_filter expression evaluates to true, then the record is kept in the list. If it evaluates to false, the record is hidden from the list.

With the above in mind, here are some ideas:

  • Filters on numeric values: If the choice list custom column contains numeric values instead of text, the > (greater than), < (less than), = (equal), etc. comparison operators will be useful. Below is an example using price<=${dollars} as a choice filter.  The range question at the beginning sets a numeric value (the dollar amount), which is compared against the price custom column added in the items choice list.

  • Functions on strings: Other than the selected() function and = operator, which we already described, there are other functions you can use to work with strings. Let's pretend we need some work done by a contractor. We want to use a first list to define the work that needs to be done. The list of contractors needs to be filtered to show only the contractors with the necessary skills.

Unlike the countries and regions example we explored earlier, in this case elements from the filtered list can appear multiple times. For example, Patricia is an electrician, gardener and welder. As shown in the screenshot below I added a skills column to the contractors list. This column includes a comma separated list of the skills of each contractor.

To find if a particular skill is present within the skills column, we can use a regular expression such as regex(skills, ${skill}). Alternatively, we can also use contains(skills, ${skill}).  Other string functions such as starts-with() or ends-with() can be handy as well.

  • Working with custom JavaScript functions: In some cases, to model complex logic, you may want to resort to the use of custom JavaScript functions with the pulldata() function.  The key to use your own custom JavaScript functions is simple: if your function returns true, the choice will be added. Otherwise it will be hidden. This option is powerful but it can, depending on the complexity of your function and the length of the list, add an overhead to your app.

As usual, using the survey samples that come with Connect is often the best. In Survey123 Connect, create a new survey and select the Cascading Select sample. You will find in there a few additional great examples illustrating the different patterns you can follow to create filter expressions.

Working with very large lists

Very long lists can slow down your survey.  The definition of a 'long list' varies. It varies depending on the compute power of your device as well as the size of your survey. Generally speaking, you should not worry too much about performance when working with lists under 500 records, but as you get to that threshold, you may want to consider a few tips to optimize lists in your survey.

There are different aspects that require your consideration when working with long lists:

  • How you display lists: Appearance
  • How you store the lists: External choice lists
  • How you filter lists: Choice Filter expressions

Appearance

Survey123 supports multiple ways to display lists. By default, choices in a list are directly presented in the form. In this way, end users can see right away all the choices and tap on them quickly.  This default appearance, is great for very small lists (under 10 elements). For lists that grow larger, this default appearance is not adequate because it forces users to scroll the entire form in order to see elements at the bottom.  If the list grows even larger, the initial loading time of your form will be negatively affected because the default appearance forces Survey123 to render every choice in the list within the form.

Using the XLSForm minimal appearance , you collapse list choices into a dropdown control.  Everyone wins: end users do not need to scroll up and down the entire form, and Survey123 does not need to render the entire list until the user chooses to open the dropdown.   When working with long lists, the minimal appearance reduces the initial survey loading time.

You can also use the autocomplete the appearance. With autocomplete, the dropdown list will not show any choices until the user starts typing. This also saves time because only elements in the list that match the user input need to be rendered. For very long lists, autocomplete is often the best option for both the end user as well as to speed up your survey behavior.

Ultimately, you will need to choose what appearance works best for you. The performance differences of these different options will vary depending on your device too. As I wrote this article, I measured how much time a list with 1,500 elements took to load using the different appearances. Autocomplete was the winner. It reduced loading time 400% when compared to the default appearance and it was also slightly faster than the minimal appearance.

External choice lists

The most common way to define a list in XLSForms is through the choices worksheet. Using the choices worksheet is convenient because you can easily visualize and change your lists when authoring your XLSForm. All it takes is to switch from the survey to the choices worksheet.  Having said this, it is best practice to keep very long lists as separate CSV files.  External CSV files reduce the initial loading time of your survey and accelerate choice filters on your lists. 

Starting with version 3.11, Survey123 supports variations of select_one and select_multiple question types that allow you to reference lists in external files: select_one_from_file and select_multiple_from_file

The Survey123 Connect XLSForm templates do not include the select_one_from_file and select_multiple_from_file question types in the drop-down, but you can add them manually to your XLSForm.

The XLSForm below shows how these new types could be used. The number of municipalities in the Philippines is roughly 1,500. Already a big list, so we are using a select_one_from_file. Note that the list name referenced in this question is referencing a csv file called municipalities.csv. Municipalities are broken down into barangays. As of 2020, there are more than 40,000 barangays in the Philippines. So again, we use a _from_file select to speed up loading time and filtering.  Note that in this case we choose a select_multiple_from_file and also applied a choice filter.

typenamelabelchoice_filter
select_one_from_file municipalities.csvmunicipalityMunicipalities
select_multiple_from_file barangays.csvbarangaysBarangaysmunicipality=${municipality}

The contents of the csv files are not much different from what is included in the standard XLSForm choices worksheet. The list_name column is not needed, as the name of the list is defined by the csv file name (no spaces allowed in the name). The name and label columns for choices in your list are mandatory and any other column (language columns, media, image, etc) are optional.

The csv file names do not accept spaces and the file must be stored within the media folder of your survey.

I want to emphasize here that using external csv files for your lists is adequate for very long lists (over 500 choices), but it is not going to necessarily add much, other than overhead to you, for smaller lists. Keeping lists in the choices worksheet of your XLSForm is a very valid and convenient approach.  For most of your forms, you will want to keep your list definitions in there.  It is only when you work with monster lists that _from_file selects will help you.

The use of _from_file selects supports both select_one questions (select_one_from_file) and select_multiple (select_multiple_from_file).  You can also apply choice filters, calculations, constraints and any other XLSForm tricks you already.  Other than where the list is stored (external file versus the choices worksheet in your XLSForm), your XLSForm authoring experience should remain the same.

The XLSForm specification supports a variation of external choice lists called external selects. I will not be covering in this blog how to work with external selects, but if you are interested you can have a look at them in the Cascading and external selects—ArcGIS Survey123 | Documentation help topic.  I personally like to use external choice lists instead of external selects.

Choice filter expressions

How you write your choice filter expressions can impact performance as well. The fastest choice filters are those using simple comparison operators. For example:

  • country=${country}
  • dollars>${price}
  • code!=${code}
  • priority>=${priority_reported}

Remember that comparison operators like the above work only against select_one questions. For select_multiple questions you need to go with functions like selected(), regex(), contains() and others, which can't be optimized by Survey123 as efficiently as simple comparison operators.

You should not be afraid of using selected(), regex() and other functions as performance will generally be pretty good, but if you want to squeeze the best of your device, simple operators will help.

Be cautious when using custom JavaScript functions with the pulldata() function in choice filters. Since your JavaScript function needs to be evaluated once for every record in the list, performance can be greatly affected. 

Learning more

As you can see, there is a lot around choice filters in Survey123. If you would like to see more examples of expressions you can apply, check the Cascading Selects sample included in Connect. 

more
9 10 1,439
Esri Frequent Contributor

The Survey123 web app supports a variety of URL parameters that can be used to initialize the contents, look and feel, and behavior of your online surveys. This article describes how you can make the use of URL parameters more secure, by encoding them.

In the following example, URL parameters are used to prepopulate and hide the submittedBy question:

https://survey123.arcgis.com/share/1cb28b212b5542acbbdbaa35feba0765?field:submittedBy=Fernando%20Par...

Since the URL parameters are in clear text, a smart user could look at the web browser's navigation bar, figure out that a question is being hidden, and manipulate the URL parameters to show and change the question.

Here is the same URL, with encoded parameters:

https://survey123.arcgis.com/share/1cb28b212b5542acbbdbaa35feba0765?code=ShFmaWVsZDpzdWJtaXR0ZWRCeXI...

Once the URL parameters are encoded, only the Survey123 web app can read them, making the URL much more secure.

Tip: If you are not familiar with Survey123 web app URL parameters, check these two blog posts:

Introduction to URL parameter encoding in Survey123

URL parameter encoding is a technique that allows you to obscure the contents of URL parameters. When parameters are encoded, it is not possible for end users to manipulate existing or add new parameters to the URL. This functionality is only available with Survey123 online forms published with version 3.11 or newer.

Tip: If you wish to upgrade your online forms to newer versions, see:

Encoding parameters manually (&encodeUrlParams=true)

Let's pretend you want to add a link to your survey in a website and Facebook. Using URL parameters, you prepopulate a hidden question so you can tell from which source (website or Facebook) the survey was sent. In this case, since you only need to create and encode two links, you can simply use your web browser. Here is how:

  • Create your own online survey URL and add parameters to it as usual.
  • Add &encodeUrlParams=true at the very end of the URL and reload your page.
  • The encoded URL will appear in your web browser's navigation bar.
  • Copy the URL and use it somewhere else.

This animation shows the process step by step.


Encoding parameters with the Survey123 REST API

Manually encoding your survey URLs is only practical when you have a handful of URLs to share. If you need to create many Survey123 links, or you need to create them dynamically, the best choice is to use the Survey123 REST API to programmatically encode your URL parameters.

The encodeUrlParams operation on the Survey123 REST API allows you to pass a collection of Survey123 parameters and get back the encoded URL string.

The URL endpoint of this operation is: https://survey123.arcgis.com/api/encodeUrlParams

It expects a POST request including a valid ArcGIS token and the parameters you want to encode. For example:

  {
"token": "<TOKEN>",
"params": {
        "field:submittedBy": "Fernando Paredes",
        "hide": ["field: submittedBy"]
}
"portalUrl": "<Portal URL>",  //Optional

  }

Note that it is not necessary to pass any information about your survey. All you need to pass are the URL parameters that you wish to encode and a token. The portalUrl parameter is only needed if you are working against ArcGIS Enterprise.

 

The response from the Survey123 REST API will look like the following:

{
"code": "ShBmaWVsZDpjdXNUcmFja05vchRmaWVsZDpjdXNUcmFja05vPTEyMw==",
    "success": true
}

The value of the code property represents your URL parameters, encoded.

Encoding parameters in bulk with the Survey123 REST API and Python

Say you wish to send an email to a very large group of people. You want the email to include a personalized Survey123 link, prepopulating a few questions in the survey based on the recipient's information you already have. You need to create a CSV file that you can import into an email marketing tool like MailChimp, Drip or MailerLite, including the email of the recipients and their corresponding Survey123 link. Of course, you want the Survey123 links to have the URL parameters encoded.

You can use the encodeUrlParams operation in the Survey123 REST API from any scripting language able to send a POST request. The example below highlights how links can be created from an ArcGIS Notebook. The code first performs a query on a feature service and generates a unique link for each record in the returned set. The script prints out the email and its corresponding Survey123 link with encoded parameters. Click here to preview the notebook.

Survey123 Notebook

As you can see, the code is straight-forward. A single POST request lets you specify the URL parameters you want to encode. The output of the Survey123 REST API can then be added to your Survey123 link.

Encoding parameters with the Survey123 REST API and Microsoft Power Automate

 

If you want to create a collection of links in bulk from some information you already have, the Python approach described above is a good fit. In other cases, you may need to create links dynamically.

For example, say you publish a survey for people to self-report damage to their property after a natural disaster. You would like respondents to automatically receive an email with a link that would allow them to update their own information. As you already know, using URL parameters it is possible to create a survey link to open a record in edit mode. You can also dynamically create such a link and email it right after a survey is submitted using Microsoft Power Automate and Integromat, but is it possible to also encode the URL parameters? Yes!

Overall, the Microsoft Power Automate flow could look like the following:

Survey123 Automate

Note that the trigger for the flow is the Survey123 connector. Every time a survey is submitted, the flow is executed. The payload of the trigger includes the globalId of the record just created, which is made available in Microsoft Power Automate as dynamic content.

The HTTP module is used to encode the URL parameters.

The HTTP method is set to POST. The URI parameter targets the encodeUrlParams operation in the Survey123 REST API. In the queries group, the token parameter is populated dynamically with the token value provided in the Survey123 webhook payload. The params parameter includes the URL parameters we want encoded. In this case, we pass edit for the mode and the globalId of the feature we want to edit. The globalId is coming as dynamic content from the Survey123 webhook payload as well.

 

Next the Parse JSON module is used, so that later we can easily extract the encoded values from the output of the Survey123 REST API.

Finally, the Send Email module is used to dynamically create a Survey123 link within the subject of the email. Note that the encoded parameters are dynamically added to the Survey123 link.

Summary

The Survey123 web app supports several URL parameters, allowing you to preload responses to questions, and change the look and feel and behavior of your online surveys. Using the encodeUrlParams operation in the Survey123 REST API, you can obscure these parameters to avoid manipulation by end users. 

You can generate Survey123 links with encoded parameters manually right from your browser, programmatically through scripting languages like Python, and dynamically through workflow automation in Microsoft Power Automate and Integromat.

more
2 8 852
Esri Frequent Contributor

The ranking question lets people sort items in a list in order of preference. Here is an example where we ask respondents to rank a series of topics for an upcoming Survey123 workshop. Note how the ranking of a topic in the list can be changed by simply dragging it to the desired position. Users promote their favorite topics to the top of the list.

As the user submits the survey, choices in the ranking question are scored accordingly to their position in the list: The higher in the list, the higher the score. Survey123 lets you analyze survey responses to understand the average and variance of the scores for each item. The chart below shows preferences showed by respondents. The "Survey123 design best practices" ranked first with an average score of 6.53, followed by the "Survey123 reports" topic.

The Ranking question is a great resource anytime you want to measure people's preferences over a list of well defined choices.

This blog post describes how you can create your own Ranking questions with Survey123. It first presents how you can add this type of question using Survey123 designer. It then shows how you can explore results from the Survey123 website and ultimately, how you can use XLSForms to use this question type in Survey123 Connect.

Adding a Ranking question in the Survey123 designer

The Survey123 web designer lets you visually build your own surveys from a web browser. To add a Ranking question to your survey, drag it into the design preview. You can interactively add choices to the list, or add them in bulk as shown in the animation below.

Adding too many choices to a Ranking question will be overwhelming to users. It is recommended that you do not add more than 6 choices.

Choice randomization

It's human nature that people tend to either favor whatever choices are presented first (primacy effect) or last (recency effect). To avoid biased responses, you may want to randomize how choices are initially presented to the user. In Survey123, you can choose to randomize choices in all types of lists (dropdown, choice lists, checklists...). Randomization is particularly useful for the ranking question.  In the Survey123 web designer, you will find a 'Show choices in random order' option. It will be enabled by default when you add a new Ranking question, but you can choose to disable it.

Analyzing rank results

As the user submits the survey, choices in the Ranking question are scored. Here is how scoring works: If you have 5 choices in your list, the choice at the top gets a score of 5, the choice after that a 4, etc. If you have a list with 7 choices, the one at the top gets a score of 7... You get the point.

When you look at the results of your survey in the Survey123 website, you will be presented with a chart and a table. The chart sorts your choices using the average score. It also shows you the number of responses, and the number of times the question has been skipped (if appropriate). Here is an example. This chart will give you insight as to who are the winners, overall. 

The table is useful to understand the variance of the rankings. The table indicates what the average score is, and how many times a particular choice was ranked at each position.  For example, in the table below we see that the 'Survey123 design best practices' topic was selected as the top choice by 83 of the respondents (almost 30%).  We also see that while 'XLSForm advanced techniques' ranked third position, it was chosen as the first choice by a larger group than 'Survey123 reports', which ranks second.  This tells us that on average, the Survey123 reports topic is more popular than 'Advanced Techniques', but proponents of 'Advanced Techniques' care proportionally more about the topic.

If you were to look at the raw data stored in your GIS records, the output of a rank question is a comma separated list of values.  The first value in the list is the top choice for that record, followed in descending order by the rest.

Ranking questions in Survey123 Connect

If you work with Survey123 Connect, you can also add ranking questions. Use the rank question type and assign it a list from your choices worksheet. The syntax is similar to that of select_one and select_multiple questions, but you use rank instead.  If you want to randomize the choices, set the parameters value to randomize=true. It could look something like this:

typenamelabelparameters
rank topicstopics_rankedFavorite topicsrandomize=true

Ranked choices are stored as a comma separated list of choice names.

banana, apple, orange, kiwi

The comma-separated list of values honors the order of the user selection. In the example above, banana was placed at the top, followed by apple, orange and kiwi.

Keeping this in mind, you can do a few clever things with XLSForms. A common request is to be able to store the score of each choice as an attribute. Here is what I mean. Note how the scores for the fruits automatically calculate as the choices are ordered:

Once you know how to get the score of an item, you can use these values to implement your own custom validation or skip logic.

Getting the score values can be accomplished with the help of a simple JavaScript function and pulldata. If you are not familiar with using custom JavaScript functions with XLSForms, seehttps://community.esri.com/groups/survey123/blog/2020/08/07/extending-survey123-smart-forms-with-cus... 

Here is what the XLSForm could look like:

typenamelabelcalculationparameters
rank fruitsfruits_rkndFavorite fruitsrandomize=true
calculatebananaBananapulldata("@javascript","myJS.js","getScore","banana",${fruits_rnkd})
calculateappleApplepulldata("@javascript","myJS.js","getScore","apple",${fruits_rnkd})
calculatekiwiKiwipulldata("@javascript","myJS.js","getScore","kiwi",${fruits_rnkd})

And here is a JavaScript function to back it up:

function getScore (fruit, ranked_fruits)

{

         var fruits = ranked_fruits.split(',');
         return fruits.indexOf(fruit) + 1;

}

more
1 0 450
Esri Frequent Contributor

The original release of the Survey123 report capabilities centered around the notion of producing reports for individual records.  With each Survey123 update, we progressively expand the report template syntax to help you build better reports. In this blog, I want to explore in detail summary sections, which were added in our July 2020 update. The content of the blog is not for beginners. I assume that you are already familiar with creating your own report templates. If not, I suggest you learn the basics from the following resources:

Summary sections are generally used to create a single report for a collection of survey records. This would commonly be referred to as a 'summary report'.  Below is an example of a Hydrant Inspection Report that illustrates this concept. Note that tables are used to categorize and report all the hydrants that have been inspected within a certain period. You can click on the image to enlarge it.

Water Hydrant Summary Table

Other examples where you may want to use reports like this include include:

  • Code violations by district report
  • Monthly hydrant inspections report
  • Property damage assessments by severity report
  • County household survey summary

A great way to get started with summary reports is to generate a Summary sample template on the Survey123 website.
The sample template demonstrates some useful summary syntax. You can then combine what you learn from the template with the Survey123 documentation to adapt the sample into your own report template.

In this blog, I will show you how to build a template completely from scratch. I want to do it that way, because I want to describe some aspects of the report template design process that will not be apparent when using the sample template.

The best way to follow this blog is hands-on. Set aside 90 minutes or so and let's build the hydrant report together.

Setting up the hydrant inspection survey sample

Before you can create a report template you need a survey with some data. We will use the Hydrant Inspections sample template that comes with Survey123 Connect:

  • Open Survey123 Connect and sign in using an account with publishing permissions.
  • Click on New Survey, and look for the Hydrant Inspection form in the Community group.

  • Make sure you set your own title for the form. In my case, I will call it Redlands Hydrant Inspections.
  • Before you publish your survey, go to the survey's Options dialog and enable the Inbox and disable the Sent folder.

  • Publish your survey and add three or four of records using the Survey123 field app.

Now that we have a survey and some data in it, let's proceed with the creation of our report template.

Getting started with the report template

We are going to progressively build up our hydrant report template. To get started, we will add a header, footer, title and a table to display a survey record selection. Below is a screenshot of our initial goal. From this humble beginning, we will add more and more contents one step at a time.

Headers and Footers

Adding a custom header and a footer to your template is generally a good practice. A descriptive header and footer make your report more presentable and provide useful information. Headers and footers cannot include Survey123 report placeholders, but you can insert Microsoft Word autotext elements and static information that will be displayed across all pages in your output report.  In my case I added an Esri logo and a fictitious address in the header. In the footer, I inserted autotext to automatically display the date when the report is created and also the page number.

You can add other Microsoft Word design elements, such as page borders, watermarks and backgrounds. Make sure you use these features wisely as they can help give your reports a professional touch.

Title and Table (Basic setup)

Next, we will add a title and a table into our report, as shown below. The table will display a handful of attributes from the survey records. For now, simply add a title and a table as shown below.

Understanding summary sections

Our template is still incomplete, but let's upload it to see how it works.

  • Sign in into survey123.arcgis.com.
  • Open the Data tab for your hydrant inspection survey. You should see the records you added earlier.
  • Open the Feature Report panel and click on the Manage templates link.

  • Click on Upload new template.
  • Browse for your report template and upload it.
  • Next, select three of the hydrant records you created and generate a preview sample report:
    • Select 3 records in the table
    • In the side report panel, make sure the report is executed against the selected records only.
    • In File options, select the option to merge into a single continuous document
    • Select your template
    • Click on the Preview sample report link

Your sample preview should look like this:

I bet the output is far from what you expected:

  • Why the table is empty? The table shows no records because we did not add instructions in the report template to fetch data from your survey. We just added a title and an empty table in it.
  • Why are the table and title repeated three times? The table is repeated because when you ask Survey123 to create a report for a selection of records, your report template is processed once for every record in your selection. Unless you enclose your template contents within a Survey123 summary section, this output is expected.

OK. So now let's add a summary section. A summary section is opened with $<$summary> and closed with $</>. Edit your template as shown below. Note that I highlighted the summary section tags in green. The formatting of the tags does not make any difference to the Survey123 report engine, but it can make your template more readable.

Note that the title is inside the summary section. As we just learned, contents outside of the summary section will be processed once for each selected record. We do not want multiple titles in our case, or multiple tables, so of these elements need to be placed inside the summary section. 

Finally, note that I added the $<$summary> opening tag in the same line as my title.  I did this on purpose, because I do not want to introduce an extra line in my output report. When the $<$summary> tag is on its own line, the line it occupies will not be removed in the output report. Sometimes you want to have $<$summary> in its own line, and sometimes you don't. It depends on what you want to do.

As we make changes in the report template, it is important to regularly upload it to the Survey123 website so we can check the syntax. Otherwise it can be challenging to figure out where exactly syntax errors happen.

  • From the report panel on the right side, click the Manage templates link.
  • Use the Edit icon in your template to upload your new version.

  • With your 3 records selected, click on Preview sample report.

At this moment, since you have no contents in your report template outside the summary section, the output of your report request is a single report, in a single file, with a single title and table.

Populating our table

We need to add instructions within our table to fetch information from the selected set of records. To do this, we need to reference in our template the layer and fields from which we want to retrieve the data. Since we may not remember the exact name of layers and fields, we will use the reference table from the Upload new template dialog.

  • From the report panel on the right side, click the Manage templates link.
  • Use the Edit icon in your template.
  • The Upload new template dialog shows the reference table at the bottom, as shown in the screenshot below.

We will use this reference table regularly, so keep this dialog open.

First, we will create what is known as repeat section. When a repeat section is encountered by the Survey123 report engine, all contents within the section are processed for every record referenced in the repeat. A repeat section is opened with ${#LayerName} and is closed with ${/}. As you can see in the screenshot below, the layer name needs to be replaced accordingly. You can get the layer name for your survey from the reference table in the Upload new template dialog.  In the context of Survey123 report template syntax, the # sign in the opening tag is an indication that the section is a repeat.

If you were to upload and run a report with the template above, new rows will be added to the table: one row per survey record. To complete the table, we need to make sure we have instructions to pull values from the corresponding fields.  The table for our survey will look like this:

It is important to note that out of all the Survey123 report template syntax we have added in our table, some parts will not be included at all in the output, and others will be replaced with data. 

  • The repeat section opening and closing tags will not be included at all. This means that you can use whatever formatting you want in your design to make it more legible.
  • The syntax to retrieve data from fields will be replaced with values from the records, so the formatting you apply to your report syntax matters. For example, if you insert an instruction like ${asset_id} setting the font color to blue, then the Asset ID will be rendered blue in the output report.

To be more exact, to define the formatting of your field values in the output report, you do not need to change the format for the entire Survey123 report template syntax. The only character that matters is the initial $ sign. This is important because sometimes the syntax can be long and interfere with your design. A good example is the Inspection Date in our sample. It uses three lines in the cell, affecting the height of the whole table in the design view.  To avoid this, use a smaller font size for the expression and apply the font style you want for the Inspection Date to the $ sign only. Here is an example of a sample preview report with various formatting styles applied:

At the end of the table, you can see an empty white row. If you do not like that, go into you report template design and remove it from the table.

So far, we have explored how to insert a table into our report to display all the input records as rows.  This is a basic, yet powerful technique that will help you model many reports you may be asked to build. Next, we will cover how to include multiple tables in a report and control the contents of these tables using filters.

Filtering and sorting report records

When defining your repeat section against the survey layer, you can use parameters to filter and sort records in your selection. For example:

${#H_Inspection | where: "last_status = 'needs_repair' " | orderByFields:"last_inspection_date DESC"}${/}

The where parameter filters the input selection from the report request. The orderByFields parameter is used to present rows sorted by inspection date in descending order.  Using filters in this way, you can insert into your report multiple tables, each for the different conditions found in your data.

Filters in your report template apply to only the collection of records selected for the report. For example, if you use the Survey123 website to select 100 records out of 2000 and then request a your summary report, the filters within the template only apply to the 100 selected records.

Statistics

A summary report might also include statistics. For example, in our hydrants example we may want to present a total count of all hydrants inspected, or a table showing a count per status. The following expression returns the total count of records from the Redlands_Hydrants layer that have been submitted to the report.

${Redlands_Hydrants |  stats:"count,objectid"}

This expression returns the total number of hydrants that need repair.

${Redlands_Hydrants | where:"last_status = 'needs_repair' " | stats:"count,objectid"}

To get the statistic, you need to pass the statistic you want (sum, max, min, count, avg) and also the field on which you want to calculate the statistic. Records with null values will be ignored.

Back to our template, we could do something like this:

And here is what I get in the final report with a selection of 3 records. The report is now looking a lot more like the summary we set out to create!

!important

The expressions we have used so far to process records within the survey layer, or to extract statistics out of it, are limited in scope to the survey records selected for the report.  By using the !important parameter, you can make the report engine work against all records in the layer you are referencing. 

For example:

Retrieve the count of all hydrants in the Redlands_Hydrants layer that need repair:

${Redlands_Hydrants | where:"last_status = 'needs_repair' !important " | stats:"count,objectid"}

Retrieve the count of all the selected hydrants in the Redlands_Hydrants layer that need repair:

${Redlands_Hydrants | where:"last_status = 'needs_repair' " | stats:"count,objectid"}

Create a repeat block to process all hydrants in the Redlands_Hydrants layer that need repair:

${#Redlands_Hydrants| where:"last_status = 'needs_repair' !important "} ${/}

Limit the repeat block to selected hydrants that need repair:

${#Redlands_Hydrants | where:"last_status = 'needs_repair' "} ${/}

The !important modifier must be used in the context of a where statement. For example, if you want to get the total number of records in a layer, you cannot do this:

${Redlands_Hydrants | !important | stats:"count,objectid"}

You have to do this:

${Redlands_Hydrants | where:"1=1 !important " | stats:"count,objectid"}

Additionally, this is incorrect, because the !important parameter is left out of the where statement.

${Redlands_Hydrants | where:"1=1" !important | stats:"count,objectid"}

This is correct, because !important is enclosed by the closing quote of the where statement

${Redlands_Hydrants | where:"1=1 !important " | stats:"count,objectid"}

An important (no pun intended) note when using !important in a repeat section is that the number of records retrieved will be limited by the maxRecordCount property in your target feature layer. Say, for example, you have 3,000 hydrants and you use !important to create a table with all 3,000 records. You will find that only the first 1,000 will be shown in the table, because the default maxRecordCount is generally set to 1,000.  If you want to alter the maxRecordCount property refer to How To: Update the maximum record count for feature services in ArcGIS Online 

The maxRecordCount threshold does not affect statistics. For example, if you use !important to do a count of all hydrants, you will get the correct number.

Group By Summary Tables

Sometimes, you may want to include in your report a table that groups a statistic for your records. In our hydrant report sample, we may want to include a table to show how many inspections (count statistic) were performed by each inspector (grouped by inspector).  In SQL jargon, this would be possible with a GROUP BY query. 

This can be accomplished by adding a repeat section, with a filter that includes a GROUP BY. Here is what the syntax would look like for the hydrant report:

InspectorInspections performed
${#H_Inspection | stats:"count,objectid,inspectionCount":"Inspector": "" | where:"1=1" | orderByFields:"Inspector DESC" }${Inspector}${inspectionCount}${/}

 

A screenshot of the resulting report is below. The blue table at the bottom shows the inspections by inspector.  The complete reference for these types of filters can be found in Feature report queries—ArcGIS Survey123 | Documentation 

Other considerations and tips

No images or maps within a summary section

Summary sections have some limitations. You cannot include a map or a photo within a summary section. If you want to include maps or images in your report, just make sure you reference them from outside the summary section.

'Straighten' your quotes

A mix of straight (" ") and cursive (“ ”) quotes in your Survey123 report template syntax can generate syntax validation and/or execution errors. For consistency, I personally like to use always straight quotes: (") for double quotes and (') for single quotes.  Microsoft Word tends to replace straight quotes with cursive (also known as smart quotes), but you can change this behavior as described at Smart quotes in Word - Word 

Embed Fonts in Microsoft Word report template (if needed)

If the output PDF files generated by your template do not show fonts the way you expect them, embed fonts in your template as described at Embed fonts in documents or presentations - Office Support 

Pricing

If your survey is hosted in ArcGIS Enterprise, you can create as many reports as you like for free. If your survey is hosted in ArcGIS Online, your reports will consume credits. Here is how credits are calculated:

  • If your survey includes all Survey123 syntax within a summary section, the cost is 0.5 credits for the entire report. It does not matter how many records are included in your summary section. 
  • If your report does not have a summary section, or you use Survey123 syntax outside a summary section, the cost of the report in credits is the result of multiplying the number of selected records you are including in the report by 0.5.

Get Creative (Cover pages and more)

The focus focus of this blog has been to show how summary sections can be used to embed tables within your report. You can, however, use summary section in more creative ways. As we saw earlier on, the contents of a summary section are only processed once during a report request. For this reason, they also make a good candidate for you to create a cover letter for a report where you want to include multiple merged individual reports.

The Damage Assessment Summary report below also shows a creative use of a summary section. The report shows a list of addresses, which has been obscured on purpose in the screenshot. The grid uses check-marks to indicate the type of property and the severity of damage.  This report template uses conditional statements extensively to evaluate if a particular cell must be checked or not.  At the very bottom of the report you can also see totals and category totals. These values are all extracted using statistical functions described in this blog and the Survey123 documentation.

more
3 5 1,942
Esri Contributor

Some organizations have a requirement to work in completely disconnected environments, where access to the internet is unreliable or not available at all. For these types of environments, Esri has released the Survey123 website installer via the My Esri site. 

The Survey123 website installer lets you host your own copy of the Survey123 website and the Survey123 REST API, which is used by Survey123 Connect and the Survey123 web application.

The Survey123 website installer is designed to be configured against your ArcGIS Enterprise deployment, enabling you to work with Survey123 within the boundaries of your own network infrastructure.

Caution:

You should only install the Survey123 website if you have a specific requirement to host the Survey123 website or Survey123 REST API on your own infrastructure. There is no additional benefit to using the installed Survey123 website, but there are some disadvantages and limitations. For example, the setup and maintenance overhead, and the absence of the feature report functionality. In most cases survey123.arcgis.com should be the preference.

You can expect the near full functionality of the Survey123 system, including:

 

  • Access to your own hosted copy of the Survey123 website, allowing you to publish new surveys using the Survey123 designer, manage surveys and access the Data and Analyze survey tabs.
  • Publish surveys from Survey123 Connect to your ArcGIS Enterprise deployment.
  • Submit surveys from both the Survey123 field app and web app.

 

 A notable exception is the Survey123 feature report service, which cannot be installed locally.

Configuring a local copy of the Survey123 website is a process reserved for ArcGIS administrators with a good understanding of ArcGIS Enterprise and all of its components. Survey123 Connect can also be configured during this process to access survey templates and samples from ArcGIS Enterprise. For more details check out the Documentation and the following Knowledge Base articles:

Other Considerations

  • The Survey123 website can be installed alongside your ArcGIS Enterprise deployment or on a stand-alone web server. If installing on a stand-alone web server, the same Microsoft Windows operating system requirements and hardware requirements as ArcGIS Enterprise 10.7 apply.
  • The Survey123 website installer will support ArcGIS Enterprise 10.7 or later (versions that are in the General Availability phase of the product life cycle at the time of release). The Survey123 website installer is not linked to a specific version of ArcGIS Enterprise, so upgrades can be done independently.
  • A new version of the Survey123 website installer will be released approximately two weeks after each release of the Survey123 website, to ensure any hotfixes to the Survey123 website are included. Existing installations of the Survey123 website can be upgraded 'in place' by running the new version of the installer, which maintains all configuration settings.
  • The Survey123 website installer is only available for Windows.
  • The setup and post installation is in English only. Once installed, the website itself is available in all 38 languages that are currently supported by survey123.arcgis.com
  • The installed website contains a packaged version of the latest help documentation that can be accessed while offline, however external links used in the documentation will not work while offline. 

more
5 1 496
Esri Frequent Contributor

This article describes how you can configure a file location provider in the Survey123 field app. This allows you to feed the Survey123 app with locations from a file where you can store, for example, a track.  This technique is useful for testing and for demonstrations.

This functionality is only available in Survey123 version 3.10 or newer

Use a File Location Provider to simulate locations in Survey123

As you may already know, the Survey123 field app can be configured with a custom location provider. This is typically used to connect your app to external GNSS receivers via Bluetooth or a network connection. These connections feed the Survey123 field app with NMEA messages, carrying location information from the GNSS receiver. 

You can also configure a location provider from a file, which is exactly what you need to do to simulate locations in Survey123. The file format Survey123 expects is (no surprise!) a NMEA log file. Most GNSS receivers can export NMEA logs, so if you get out in the field and want to replay your locations in Survey123, you can easily use those files as a location provider. If you do not have a GNSS receiver, there are a handful of utilities that you can use to create a NMEA file.

How to create a NMEA file using the Survey123 field app

You create a NMEA file, as you walk or travel, from Survey123. It is extremely simple:

  1. Tap on the satellite icon located in the top banner of the app. This opens the Location status dialog which describes the incoming data from your location provider.
  2. Tap on the red 'record' button at the bottom of the screen. This will immediately start recording your location in a NMEA file. You can now start moving if you like.
  3. Tap on the stop button to stop recording.

The NMEA file will get stored in the Survey123 directory automatically. You can now use them to replay them at any time. Lets see how next.

Configuring the file location provider in the Survey123 field app

Survey123 NMEA File

If you already have your NMEA file loaded into your device, all that's left to do now is target your NMEA file from the Survey123 app.

Here is how:

  1. While in the survey gallery, tap on your profile icon.
  2. Go to Settings
  3. Open the Location section
  4. Tap on Add location provider
  5. Select File
  6. Browse for your NMEA log file
  7. Tap on Add (IMPORTANT!)
  8. Return to the survey gallery and open your survey

Once the file location provider has been set, your location will loop through the coordinates in your NMEA file. Speed, direction of travel, fix time and other properties will be honored.

Tip: Use the loop at the end of file and update rate settings in the file provider about page to better control how your NMEA file is played.

While creating NMEA files right from within the Survey123 field app is very easy, there may be times when you can't really walk or drive the track you want to replay. Lets explore next other practical options to create NMEA files and move them into your device so you can use them from the Survey123 field app.

How to create a custom NMEA file using nmeagen.org

One particular tool I like to use to create NMEA files is https://nmeagen.org/. Here is a video showing how easy it is:

Creating a NMEA file with nmea.org

If you have a shapefile with a line representing a path, you can use the Feature Vertices To Points—Data Management toolbox | Documentation tool in ArcGIS Pro to create a point layer, then run the Add Geometry Attributes—Data Management toolbox | Documentation tool and export the coordinates to a CSV file. The CSV file can then be transformed into a NMEA file using nmeagen.org.

Moving your NMEA file into your device

If you plan to run Survey123 on Windows or macOS, keep your file in a known folder so you can browse to it in the field app. If you plan to use an Android or iOS mobile device, then you need to copy your file onto the device.

In iOS, you can move the NMEA file into your device using iTunes as shown in the animation below. With this workflow, you will need to connect your phone with your computer through a cable.

Copying data into your Survey123 iPhone folder with iTunes

Alternatively, in iOS, you can email the NMEA file to yourself as an attachment and then save it locally on your phone or tablet.

Survey123

Here are the steps:

  1. Open your email and tap on the attachment.
  2. At the bottom of your screen, a dialog will pop. Pull up the dialog until you see the Save to Files option.
  3. Navigate to on 'On My iPhone->Survey123' and then click on the Save option in the top-right corner. This will copy the file into the Survey123 app directory.

You can also Transfer files between your computer & Android device - Android Help.

Once you have the NMEA file in your computer, you are ready to configure the file location provider as described above to use it.



more
1 0 855