Aggregate results from multiple surveys

Question

Hi there,  I worked for a non-profit organization which uses Survey 123 to collect data for their projects around the world. Those surveys share some common questions but are not identical most of the time. Right now they would like to create a dashboard based on the data collected from those surveys.  My question is is there any efficient way we can aggregate data from multiple surveys (they are over 1200 surveys in this case and they are updating frequently).  I am new to the Survey 123 platform. Any useful information would be highly appreciated. Thanks!

lmelendez_ADMIN · Answer

hello @IsmaelChivite , can you explain what you mean by option 2?  how can one have a master layer but many surveys?  I've been trying to think about it and looked it up but did not find anything

creating a single "master survey" (option 1) seems ok but would that impact the loading time in the field?  also still I think I like the idea of option 2 better anyways but just curious how that would actually work

any help/resources you could offer would be appreciated, thanks

Vanilla2020 · Answer

Hi Ismael,

Thank you very much for your reply. I will try those options out and see how it goes.

IsmaelChivite · Answer

Hi. How data is aggregated depends largely on how the surveys were originally authored.

Option 1 (One master survey): Since surveys share some of the questions, one could have created a single master survey and use conditional visibility expressions to hide/show questions and groups as appropriate. In this case all data gets aggregated automatically, since there is one big single survey.

Option 2 (One master layer, many surveys): Having one master layer with all possible questions, one can author multiple surveys and have them all target the master layer. In this case, data gets aggregated automatically as well as there is only one layer.

Option 3: Many surveys are authored each with its own layer. In this case, you can use the Append geoprocessing tool in ArcGIS Pro to load data from one layer into another. When you run the Append tool, you specify how fields from the source and target layers match.  This GP tool can also be run using a Python script.   If you go to the item details page of a layer, you will also find an Append option which does a similar thing. If you use that tool, you need to first export the data you want to aggregate into a CSV or some other format.

If you have many surveys and data comes to them regularly, you eventually will want to consolidate surveys into a single layer (Option 2 or 1). This is something that you can do with Survey123 Connect only.

Here are some resources for Append:

* Append GP tool
* Append layer tool

There are other approaches in ArcGIS to aggregate data. One particularly flexible is ArcGIS Data Interoperability.  It may not be practical for you to consolidate all surveys into a single layer. If that is the case, Data Interoperability may be your more robust option.