Work with CSV data in ArcGIS Survey123

15670
12
03-24-2022 06:10 PM
KateChan
Esri Contributor
6 12 15.7K

Background

ArcGIS Survey123 authors often have existing data in a tabular format they would like to use in their surveys. For example, you could have a list of building locations or plant species. While you can create a choice list on the choices worksheet in the XLSForm, if you have a large list of hundreds - or even tens of thousands - of rows, adding a CSV file in Survey123 Connect is more efficient. Once the CSV is added to the survey, you can look up and retrieve a single value from the file or populate a choice list using a column from the CSV.

This blog post provides an overview of the methods for adding CSV data to your surveys and how to work with that data in XLSForm.

Add CSV data to your survey

In Survey123 Connect there are two ways to add a CSV file to your survey – either by copying it manually to the media folder or by linking your survey to a CSV item in your ArcGIS content. If you know your data is not going to be updated often then the manual method is fine for adding CSV files.

Add CSV.png

If your data will be updated regularly, for example, a list of personnel, then it is beneficial to link your CSV via Linked Content as this will enable you to update your CSV outside of Survey123 without the need to republish the survey. Ismael explains this in more detail in his Survey123 Tricks of The Trade: External choice lists blog. Additionally, other people can update the CSV item in your ArcGIS organization as well, if they have the necessary permissions.

Use CSV data in your survey

Once the CSV has been added, you are then able to use it in your survey. You can either derive a single value from a CSV or populate a choice list using a column in the CSV.

Use CSV-01.png

 

Pulldata()

To look up a single value, use the pulldata() function in the calculation column of the XLSForm. This is useful to populate an answer based on a previous response. For example, if you have a list of tree species, pulldata() can be used to look up and return the tree’s genus and family, or it can also be used to calculate values based on a previous selection. The pulldata() function is explained in more detail here.

There are two options to populate a choice list from a CSV file: select from file and the search() appearance. 

 

Select from file

Select from file uses either a select_one_from_file or select_multiple_from_file question type to retrieve a column from a CSV file. For example, it could be used to populate a choice list with employee IDs. Like a choice list on the choices worksheet, the CSV must contain name and label columns for the function to work. In the XLSForm, enter the name of the CSV file after the question type to populate the choice list. For example, select_one_from_file teams.csv.

You can also add a choice filter in the choice_filter column, like selected(${teams},team) to narrow down the choices based on a previous selection. This requires an additional column to be added in the CSV, which is explained further in the video below. If you are using select from file and have multiple choice lists in your survey, you will need a different CSV for each list.

 

Search() appearance

The other method to populate a choice list is the search() appearance, compatible with select_one and  select_multiple type questions. Your CSV does not need to have any specific column names and one file can be used in multiple lists. For example, it could be used to populate a list with products to restock. In the choices worksheet of the XLSForm you will have to create one choice that has the same name as the column in the CSV file you would like to return in your survey. To return the entire column as the choice list, in the appearance column, enter the name of the column after search. For example, search(‘product’).

Your choices can be filtered with further parameters. There are four parameters that are compatible with CSV files: startswith, endswith, contains, and matches. They could be used to filter by date, colour, or name, for example. Brett’s blog further describes how to use the search() appearance.

The video Work with CSV Data in Surveys goes through examples for each of the methods mentioned and may help you better understand how to add and use CSV data in Survey123.

 

 

12 Comments
NourSalam
New Contributor III

I have used select_from_file and found it extremely useful. However, when I open the data tab of the Survey123 web-based workspace, and attempt to edit a record, any submitted data coming from a select_from_file within a repeat does not show up. The field appears blank and there is no drop-down. If I submit any edits, the empty values will overwrite my existing values causing data loss. I have already submitted an esri ticket about this but it's been two weeks of investigating without a solution yet.

DougBrowning
MVP Esteemed Contributor

Is it doing this?   https://community.esri.com/t5/arcgis-survey123-questions/issues-with-new-search-appearance-domains-a...

What does the domain look like on that field?

NourSalam
New Contributor III

@DougBrowning great question. When I publish my Survey123 using Connect, I am not even enabling the option to create domains. This is due to the number of options I have in my csv and the fact that they will be updated on a yearly basis. I have however attempted to publish with the option to create a domain enabled and still get a blank within the repeats using select_from_file.

NourSalam
New Contributor III

@DougBrowning I was told by customer support that my case is now under "#BUG-000145478 Unable to edit a ‘Select One From File’ type question inside a repeat using Survey123 Web Editor if a question with the 'null' esrifieldtype bind exists anywhere in the form".

I still have the same issue even after removing any of the null esrifieldtypes from my form and I am waiting on an actual workaround. Thanks for any help.

RobertAnderson3
MVP Regular Contributor

I'm having issues with getting the search() to work with a hosted CSV as linked content. 

I have the set up in the choices tab for telling it which field to look at for the name/label as UNITID, but can't seem to get anything to show in my choice list when I put search('UNITID') or search('PPA') which is the name of the CSV. 

I had this working with the feature layer it's based off of (but too many records and can't change maxRecordCount currently) but can't seem to figure out the CSV..

DougBrowning
MVP Esteemed Contributor

Post the full code.  But you do need the matches term in there.

Looks like this

search(nameofcsv, "matches", "columnname", "value")

RobertAnderson3
MVP Regular Contributor

@DougBrowning 

Thanks Doug! Okay so the nameofcsv is pretty straightforward, "matches" is the search style, "columnname" is the column in the csv right? And then "value" is what it's searching for in the rows to pull that entry?

I want it to return ALL the values in the specified column in the CSV, not based on another response to a previous question. I wanted to use the search() appearance instead of just linking the CSV because the column names don't match the name/label format and it felt easier than getting my Python script to modify the columns when it makes the CSV.

The CSV is "ParkPointAppurtenances" the field for both the name and label is "UNITID" and in the choices tab list name is assetIDs, and the name and label columns are UNITID.

DougBrowning
MVP Esteemed Contributor

Have you tried leaving off the matches part so it is just the name of the csv?  I cant remember if that worked.   I have also just created a column that has a Y (for Yes) in it and used that as my search column.  But then you may as well change the columns names and use select from file.  So sorry not sure if you can.

RobertAnderson3
MVP Regular Contributor

Thank you for the help @DougBrowning !

That's what I was trying to do with just having search('ParkPointAppurtenances') similar to the method for the feature layer being search('url') but it doesn't seem to register. It feels like something you should be able to do so I feel like I'm missing a step.

That said, I realized there is a column in the file that is the same for all so I could just use that for my matching.

But I would appreciate if anyone knows how to make it work just CSV and column name wise returning ALL values like you can with the feature layer.

list_namenamelabel
assetIDsUNITIDUNITID

 

File name is: ParkPointAppurtenances.csv
Field name is: UNITID

The choice list is empty when I run it, I could get the feature layer search() to work but it only returned 2000 records and because of the way the layer is updated changing the maxRecordCount won't persist so I export it to a CSV instead.

Thank you for any help to catch what super obvious thing I'm missing!

EDIT: Okay I just feel silly, I had been using Python to create a CSV from a feature layer in AGOL and the tocsv command on the dataframe created an index field with a blank column header, so when I removed that my search() works fine.

WhitneyWeber
Occasional Contributor

This is a fantastic blog, thank you! I'm struggling with getting the search appearance to work with a survey question:

autocomplete search('colony','matches','county',${County_Municipality})

hard-coding the value works, but of course that is not very helpful!

autocomplete search('colony','matches','county','Charles County')

What am I missing? I even tried a concat to add single quotes:

autocomplete search('colony','matches','county',concat("'",${County_Municipality},"'")

I'm having the same problem with the choice_filter using selected with the survey question: selected(${County_Municipality},county) 

I have verified the value from the County_Municipality question ('Charles County'), via a note type, matches the value within the colony csv.

DougBrowning
MVP Esteemed Contributor

Usually this happens due to spaces or other special chars in your lists.

MihaRobar
New Contributor III

Hi,

 

i am unable to expose csv table - attributes into survey123. how can i set the proper field to display?

All i get is an 10x "label" option, witch is not on my list. I have 10 rows and 4 collumnes on csv file.

Errorsurvey123_csv.JPG