Survey123 Tricks of the Trade: Ranking question and choice randomization

10-16-2020 09:50 AM
Esri Frequent Contributor
1 6 1,592

The ranking question lets people sort items in a list in order of preference. Here is an example where we ask respondents to rank a series of topics for an upcoming Survey123 workshop. Note how the ranking of a topic in the list can be changed by simply dragging it to the desired position. Users promote their favorite topics to the top of the list.

As the user submits the survey, choices in the ranking question are scored accordingly to their position in the list: The higher in the list, the higher the score. Survey123 lets you analyze survey responses to understand the average and variance of the scores for each item. The chart below shows preferences showed by respondents. The "Survey123 design best practices" ranked first with an average score of 6.53, followed by the "Survey123 reports" topic.

The Ranking question is a great resource anytime you want to measure people's preferences over a list of well defined choices.

This blog post describes how you can create your own Ranking questions with Survey123. It first presents how you can add this type of question using Survey123 designer. It then shows how you can explore results from the Survey123 website and ultimately, how you can use XLSForms to use this question type in Survey123 Connect.

Adding a Ranking question in the Survey123 designer

The Survey123 web designer lets you visually build your own surveys from a web browser. To add a Ranking question to your survey, drag it into the design preview. You can interactively add choices to the list, or add them in bulk as shown in the animation below.

Adding too many choices to a Ranking question will be overwhelming to users. It is recommended that you do not add more than 6 choices.

Choice randomization

It's human nature that people tend to either favor whatever choices are presented first (primacy effect) or last (recency effect). To avoid biased responses, you may want to randomize how choices are initially presented to the user. In Survey123, you can choose to randomize choices in all types of lists (dropdown, choice lists, checklists...). Randomization is particularly useful for the ranking question.  In the Survey123 web designer, you will find a 'Show choices in random order' option. It will be enabled by default when you add a new Ranking question, but you can choose to disable it.

Analyzing rank results

As the user submits the survey, choices in the Ranking question are scored. Here is how scoring works: If you have 5 choices in your list, the choice at the top gets a score of 5, the choice after that a 4, etc. If you have a list with 7 choices, the one at the top gets a score of 7... You get the point.

When you look at the results of your survey in the Survey123 website, you will be presented with a chart and a table. The chart sorts your choices using the average score. It also shows you the number of responses, and the number of times the question has been skipped (if appropriate). Here is an example. This chart will give you insight as to who are the winners, overall. 

The table is useful to understand the variance of the rankings. The table indicates what the average score is, and how many times a particular choice was ranked at each position.  For example, in the table below we see that the 'Survey123 design best practices' topic was selected as the top choice by 83 of the respondents (almost 30%).  We also see that while 'XLSForm advanced techniques' ranked third position, it was chosen as the first choice by a larger group than 'Survey123 reports', which ranks second.  This tells us that on average, the Survey123 reports topic is more popular than 'Advanced Techniques', but proponents of 'Advanced Techniques' care proportionally more about the topic.

If you were to look at the raw data stored in your GIS records, the output of a rank question is a comma separated list of values.  The first value in the list is the top choice for that record, followed in descending order by the rest.

Ranking questions in Survey123 Connect

If you work with Survey123 Connect, you can also add ranking questions. Use the rank question type and assign it a list from your choices worksheet. The syntax is similar to that of select_one and select_multiple questions, but you use rank instead.  If you want to randomize the choices, set the parameters value to randomize=true. It could look something like this:

rank topicstopics_rankedFavorite topicsrandomize=true

Ranked choices are stored as a comma separated list of choice names.

banana, apple, orange, kiwi

The comma-separated list of values honors the order of the user selection. In the example above, banana was placed at the top, followed by apple, orange and kiwi.

Keeping this in mind, you can do a few clever things with XLSForms. A common request is to be able to store the score of each choice as an attribute. Here is what I mean. Note how the scores for the fruits automatically calculate as the choices are ordered:

Once you know how to get the score of an item, you can use these values to implement your own custom validation or skip logic.

Getting the score values can be accomplished with the help of a simple JavaScript function and pulldata. If you are not familiar with using custom JavaScript functions with XLSForms, see 

Here is what the XLSForm could look like:

rank fruitsfruits_rkndFavorite fruitsrandomize=true

And here is a JavaScript function to back it up:

function getScore (fruit, ranked_fruits)


         var fruits = ranked_fruits.split(',');
         return fruits.indexOf(fruit) + 1;


New Contributor

Hi @IsmaelChivite , this advanced option by JavaScript seems very powerful! I'm not familiar with coding, I tried to follow your instructions and the JavaScript function you suggest here, and of course I missed something because it doesn't work.. at any position is my choice, I always get 0 as score.

Actually the function I'm trying to reach is slightly different from the one you illustrated and I think I do not need to use the JavaScript because it is a pretty basic goal.

I need just to extract the first three items from the comma separated string produced from a rank question type in three different fields. I found somewhere in the blog and posts that this result can be reached by the selected-at  function.  It works indeed well when I submit the survey by the web form but it doesn't' work at all if I submit it by the field app.

I cannot find any posts related to this anomaly and I need the user to be able to submit by both web and field app. 

Any help would be really appreciated!

Thank you


Esri Frequent Contributor

Hi @surveysmanager   I was able to reproduce the issue you describe: the selected-at function does not work against rank questions in the Survey123 field app.  I will add an internal note for the team to be aware of this but I strongly suggest you open an Esri Technical Support case so this software defect is officially documented.

I agree that for your particular case, using selected-at looks like a cleaner solution that a custom JS function. However, to get things going while this issue is addressed, you may want to get help from our Esri Tech Support team to get your JS function in place.  Thanks!

Esri Frequent Contributor

Hi Again. The selected-at function BUG will be addressed in Survey123 version 3.12 (tentatively planned for release in early 2021).

New Contributor

Hello - I'd like to provide a choice/field in the ranking question that could provide an "other" comment or where the respondent can type in their choice.  Is this possible?  TX!

Esri Frequent Contributor

@MTINot possible. Sorry!

New Contributor

Hi, I am trying to use a question from Survey123 web app in Dashboards and using Arcade to try and replicate a list of results or serial charts like appears in the Survey123 web design.  I know that the multiple-choice question uses Javascript, but is there a way to get that look in Dashboards?



However, when I look at it in Dashboards, the values are reported as a mix.


Rather than being separated out individually, they are lumped as a unique selection, when what I want is each value separate, like in the chart.  Any ideas of how this might be accomplished?