ichivite-esristaff

Survey123 Tricks of the Trade: Repeats

Blog Post created by ichivite-esristaff Employee on Apr 9, 2020

You can think of XLSForm repeats like sub-forms: A form within a form that can also be completed multiple times. Repeats are a very powerful feature in Survey123 because they can be configured in many different ways to support different business workflows. This article covers everything from the basics to the more advanced features of repeats. 

 

Some use cases

 

Let's first review some common scenarios that scream for a repeat:

 

  • In a door-to-door population survey, you will want to capture the location of the household and address, but also information about every member of the family. A repeat will help you model the form to be completed for each of the members of the household, including information about their name, age, gender, etc. The household survey will have a household member repeat in it, which will be completed for each person in the household.
  • In an asset inspection form you may want to capture information about each defect found. The defects repeat (or sub-form) will include questions to let you document each defect with its own photo, a description of what needs to be repaired and estimated cost of repair.
  • Repeats are also a great way to handle routine inspections. An asset form can include a repeat for the inspections: one form for the asset with its ID, location, etc and a sub-form (repeat) for every inspection ever made, with its date, issues found, etc. Since repeats can be nested, the inspection repeat form can also include its own repeat or sub-form for every defect found.

 

The basics

 

Repeats are supported through XLSForms via Survey123 Connect. If you are familiar with XLSForm groups, you have a lot of ground covered, because a repeat is really a type of group that lets you cycle through the questions in the group again and again.

 

To add a repeat to an XLSForm you need to enclose a set of questions within a begin repeat and end repeat set, as shown in this example:

 

typenamelabel
texthh_idHousehold ID
texthh_addressAddress
begin repeathh_membersHousehold Members
textmbr_nameName and last name
integermbr_ageAge
select_one gendermbr_genderGender
end repeat
select_one yes_noall_members_presentAll household members present?

 

In the survey above, the main body of the form is really about the household itself and is used to collect the household ID and address. The repeat block is all about household member information. We expect the enumerator to complete the Household Members repeat, or sub-form, once for each person in the household. The All household members present? question is outside the repeat, so it also belongs to the main form, but it will be presented right after the Household Members repeat.

 

Below is what the survey above would look like in the Connect preview. Note that the repeat appears as a common question group, except that at the bottom there are multiple options to navigate records within the repeat: a plus button to add a new record as well as next and back buttons (once you have more than one household member).

 

 

The user experience for repeats within the Survey123 web app is slightly different, presenting all the repeat records at once. You can navigate through them by scrolling up and down, as shown in the next animation.

 

Repeat records in the Survey123 web app are shown all at once

 

You can add one or more repeats to your survey. For example, in a school survey, you could include a repeat to document all classes in the school, and a separate repeat to gather information about teachers, etc. 

 

Understanding repeat data in your feature layer

 

Survey123 repeats are modeled in ArcGIS as related tables (or related layers if your repeat includes a geopoint, geoshape or geotrace question). In the example household survey above, the information about the household (id, address...) is kept in the main layer of a feature service, and the data from the household member repeat (name, age, gender) is modeled as a separate, but related table. The relationship between the tables/layers is kept through internally created global IDs.

 

If you would like to preview what your geodatabase schema looks like you can use the Schema Preview option in Connect, as shown in the animation below. Note that the name of the repeat in XLSForm becomes the name of the related table.

 

Survey123 Connect lets you preview the geodatabase schema of your survey

 

Using the Survey123 website, you can quickly explore data from repeats using the Data tab. The table will show records from the main layer of your survey as usual, and also extra tabs for every repeat in your form. You can propagate selections across the tabs for easier exploration of your data.

 

Use the Survey123 website to explore related records (repeats)

 

Repeat appearances (minimal and compact)

 

Through the appearance column in the XLSForm you can control how the repeat initially loads and appears in your form. By default, a repeat with no appearance value set is shown expanded, with one initial record showing. This can be good or bad. The good is that all questions within the repeat are ready for end-users to enter data right away. The bad is that if no data is entered, a new empty record will be added to your related table. I generally avoid using repeats with no appearance unless questions within the repeat are flagged as required.

 

If you want to avoid the repeat from having an initial record preloaded, use the minimal appearance. This will show your repeat group expanded, but with no records. To initialize data entry in the repeat, the end user will need to explicitly add a new record to the repeat by clicking the plus button.

 

The compact appearance simply collapses your repeat when the form initially loads. Of course, you can combine the minimal and compact appearances.

 

 

XLSForm expressions and repeats

 

XLSForm expressions allow you to bring sophisticated logic into your forms. Generally, XLSForm expressions are used to auto-calculate values (Calculations), define skip logic (Relevant), and to build data validation rules (Constraints). The XLSForm expression syntax for calculations in repeats is basically the same as with any other question in the form (more details at Formulas—Survey123 for ArcGIS | Documentation), but here are some important things to highlight.

 

When writing XLSForm expressions for a question within a repeat, you can reference other questions inside the repeat, but also questions outside! A common use is that where you want to bring data into your related table from the parent record. For example, in the example household survey you may want to store, along with every household member, their corresponding household ID. It would look something like this:

 

typenamelabelcalculation
texthh_idHousehold ID
texthh_addressAddress
begin repeathh_membersHousehold Members
hiddenparent_hh_idParent Household ID${hh_id}
textmbr_nameName and last name
integermbr_ageAge
select_one gendermbr_genderGender
end repeat
select_one yes_noall_members_presentAll household members present?

 

Note how the parent_hh_id question within the repeat is calculated to automatically fetch the value of the Household ID for that person. The Household ID is outside the repeat, but the calculation brings that value into the repeat record. This is a common use of calculations within repeats because it helps bringing more information into the repeat table that would otherwise require a join.

 

Styling

 

There are a few techniques you can use to style your repeat with custom colors and labels. This can be help users navigate your form more efficiently.

 

One technique is to dynamically set the label of your repeat to provide some context as to what information should be entered into the repeat. For example, in our household survey, you can include the address of the household in the header of the repeat. Instead of just reading 'Household Members', you can make it read 'Household Members at: <Address of the household comes here>. This is what is known as a dynamic label and it is described in more detail in the Understanding Dynamic Labels in Survey123 for ArcGIS blog post.  This is quite handy when working with long surveys, because it brings additional context for the person filling out the form. Here is what this would look like. Note how the label for the repeat is dynamically generated using a value from a question outside the repeat.

 

typenamelabel
...
texthh_addressAddress
begin repeathh_membersHousehold Members at ${hh_address}
...

 

You can also use colors to make your repeats stand out better within the form, like in the following screenshot. The repeat block uses a darker background color than the rest of the form, clearly separating questions within the repeat, from those outside.

 

The background and border colors of your repeat can be controlled through the body::esri:style column. In the example above, the vale in this column for the begin_repeat question is: backgroundColor='#cbcddb' borderColor='#494a4f'.  The color is expressed in hexadecimal format or HTML color name.  You will generally not want to use bright colors for you repeat backgrounds. Try to be subtle using slight variations of the background color of your form.

The body::esri:style color parameters are only honored by the Survey field app.

 

Aggregation functions (max,min, count and sum)

 

XLSForms support a handful of functions specially built to work with repeats. Specifically: count, sum, min, and max. They are all pretty self-explanatory. The design below shows the max and count functions in action.

 

typenamelabelcalculationbind::esri:fieldType
texthh_idHousehold ID
texthh_addressAddress
begin repeathh_membersHousehold Members
textmbr_nameName and last name
integermbr_ageAge
end repeat
hiddenmax_ageAge of oldest personmax(${mbr_age})esriFieldTypeInteger
hiddentotal_countTotal persons in housecount(${mbr_name})esriFieldTypeInteger

 

The count function can get a bit tricky as it will only consider in the count records with a value in the question it references. For this reason, it is best to use the count function against a question within the repeat that is flagged as required.

 

The sum question is also an interesting one in that it can sum not only numbers, but also text and even geopoint questions as well. If you sum text, it will concatenate all responses to your question within the repeat. If you sum on a geopoint question within the repeat, it will generate a geotrace object.

Check the Repeat Aggregation Survey123 Connect survey sample for an example using aggregation functions.

                            

 

Aggregation functions (join)

 

There is one more aggregate function that may be of interest: join. The join function is used to create a single list including all responses to a particular question within the repeat. Its syntax looks as follows:

join (",",${questioname})

The first parameter defines a separator. You can use a comma ",", a hyphen "-", or other character. The second parameter is the question within your repeat from which you want to create the list.

 

In our household survey example, we could use join to display a list of all people interviewed so far.

 

typenamelabelcalculation
texthh_idHousehold ID
texthh_addressAddress
begin repeathh_membersHousehold Members: ${interviews}
textmbr_nameName and last name
integermbr_ageAge
end repeat
hiddeninterviewsEvery person interviewedjoin(",",${mbr_name})

 

Note that the hidden question at the end uses the join function in the calculation column. Then the label in the begin repeat takes that value and displays it dynamically. You can see the effect in the next animation:

 

 

We can take join a bit further. Let's pretend we need to design a manhole inspection form. We want all six parts of the manhole to be inspected: the cover, chimney, cone, wall, bench and channel. We will use a repeat to document the inspection results of every component, and the join function to help us make sure all components have been inspected. Let's first have a look at what the form's behavior looks like:

 

 

Note that a checklist at the bottom of the form keeps track of the components inspected so far. When a new component is selected for inspection within the repeat, the value of the checklist is recalculated accordingly. This checklist at the bottom is set as read-only, so the user cannot manipulate it. A constraint in the checklist is also in place to avoid users from submitting the form unless all components are checked.

 

A simplified version of the XLSForm, highlighting the use of the join function is shown below:

 

typenamelabelcalculation
textmanhole_idManhole ID
begin repeatcomponents_groupComponents
select_one componentscomponentComponent
select_one conditionsconditionCondition
end repeat
select_multiple components
comps_inspectedComponents Inspected
join(",",${component})

 

To complete the XLSForm to mimic the behavior shown in the animation, you would also need to:

  • Set the comps_inspected question as read-only, to avoid users from manipulating the checklist manually.
  • Set the constraint of the comps_inspected question to count-selected(${components_tested})=6.

 

Repeat Count

 

If you want to control how many records must exist within a repeat, use the repeat_count XLSForm column. If you set the value to 2, then your repeat will be initialized with 2 empty records. If you set it to 5, it will contain 5, etc. You can also set the repeat_count dynamically as shown in the example below.

 

typenamelabelrequiredrepeat_count
texthh_idHousehold ID
texthh_addressAddress
integerhh_totalHow many people live in this house?
begin repeathh_membersHousehold Members${hh_total}
textmbr_nameName and last nameyes
integermbr_ageAgeyes
end repeat

 

The repeat_count column simply defines how many records are shown in the repeat. All records will be empty until the end user goes through every record to add data. Unless you have required questions within the repeat, Survey123 will allow you to submit records from the repeat with no data, and they will show as what they are: empty records. Generally speaking, you do not want to use repeat_count, unless you want to force people to submit a specific number of records in the repeat. If that is the case, you will want to add required fields within the repeat group. In the household example above, we first ask, "How many people live in this house?" We use that number to define the exact number of records in the household roster that must be completed. By making the questions within the repeat required, we guarantee that data is entered for every person in the house.

 

In cases where you want users to submit a number of records within a range, you will not want to use repeat_count. Instead, you will want to use constraints and the count function. For example, in an electric panel survey, you include a repeat to document all circuit breakers. Say that the number of circuit breakers is always between 5 and 20. This is what you would want to do:

 

typenamelabelconstraintcalculation
textcc_idCircuit Panel ID
begin repeatbreakersCircuit Breakers
select_one typebreaker_typeType
select_one statusbreaker_statusStatus
end repeat
integertotalTotal Breakers${total}> 4 and ${total}<21count(${breaker_type})

 

As described above, for the count function to work effectively, you will want to make the breaker_type question required.

 

Nested repeats

 

A nested repeat is a repeat within a repeat. For example, say you are doing a survey of a campground in a National Park. You want a survey for the campground itself (the name, the location of the entrance, etc). Within each campground you have campsites, so that gives you one repeat (location of the campsite, photo, size, etc). Then for each campsite you have equipment, which gives the second repeat (equipment type, condition, etc).

 

You can nest as many repeats as you want. That is, you can nest 3, 4 even 5 repeats if you like. Technically you could nest even more but for the sanity of your users avoid it whenever possible. 

 

Nested repeats are only supported in the Survey123 field app.

The Survey123 web app does not support nested repeats.

                          

Repeats in the Inbox

 

The Inbox is a powerful feature in the Survey123 field app, allowing you to edit existing GIS records in a feature layer. If you are not familiar with the Inbox, I strongly suggest you watch this short video tutorial.

 

 

By default, when GIS features are loaded into the Inbox, the repeat records will be empty. That is, the Survey123 field app does not fetch data from related records unless told otherwise. When updating an existing record from the Inbox in this way, you will be able to add new repeat records, but not view or update existing ones from the feature layer.

 

If you want to look at or update existing related records, you need to specify in your XLSForm what you want to download into the app, and for what purpose. This is all done through the bind::esri:parameters XLSForm column.

 

For example, let's pretend you are creating an asset inspection form and you want the complete history of inspections to be available in read-only mode. Then you would go with something like this:

 

typenamelabelbind::esri:parameterscalculation
textpanel_idElectric Panel ID
geopointlocationLocation
begin repeatcircuitsCircuit Breakersquery
datetimeinsp_dateInspection date
select_one statusstatusStatus
end repeat
integertotalTotal Breakers${total}> 4 and ${total}<21count(${breaker_type}

 

The query value in bind::esri:parameters will force the Survey123 app to download every related record, so all your inspections will show in the repeat of the asset form. The values supported in this column include:

 

  • query: Used to fetch related records and populate repeats. If query is not used, surveys from the Inbox will not display any repeat information from the existing feature. You can also control which records should be fetched, by defining a where statement for your query. For example: query="status='needs_repair'" will populate the repeat with all related records where the status is set to needs_repair. If no where statement is passed, the query parameter will fetch all related records.
  • allowUpdates: This parameter controls if the existing data in the repeat can be updated by the end user of the Survey123 field app or not. This is extremely useful because often you want end users to have read-only access to historical data (for example previous inspections of an asset) in the repeat, while allowing entering a new inspection to the repeat. allowUpdates by default is always false, but you can turn it on: allowUpdates=true.
  • allowAdds: By default this parameter is always true, allowing the end user to enter new records in the repeat. You can turn it off by passing allowAdds=false.

 

In the example below, our electric panel survey will display all existing circuit breakers ever recorded for that asset. It will also allow the end user to modify information from those previously recorded breakers because allowUpdates is set to true. The only exception will be the installation_date, which will be shown in the repeat as read-only.

 

typenamelabelbind::esri:parameterscalculationreadonly
textpanel_idElectric Panel ID
geopointlocationLocation
begin repeatbreakers Breakersquery allowUpdates=true
datetimeins_dateInstallation dateyes
datetimeinsp_dateInspection date
select_one statusstatusStatus
end repeat

Outcomes