ichivite-esristaff

Securing data in public surveys (Survey123 Connect)

Blog Post created by ichivite-esristaff Employee on May 11, 2020

By definition, a public survey is accessible to anyone who wants to submit data to it, but that does not mean that anyone should also be able to look at the data itself. If your public Survey123 form contains sensitive information, you should configure your survey to prevent users in the public domain from downloading, querying or changing already submitted data. Unfortunately, it is not uncommon to find public surveys where the security configuration of the survey is not set appropriately, allowing unauthorized access to the survey’s data. 

 

This article describes best practice for securing the data of public surveys published from Survey123 Connect. If you are interested in securing data for a public survey published with the Survey123 web designer, refer to Securing data in public surveys (Survey123 web designer) 

 

If you are not familiar with the basics of public surveys, refer to Getting Started with Public Surveys.

                      

A bit of context before we start

 

To properly secure your survey results it is important to understand first some basic concepts.  When you publish a survey using Survey123 Connect, a new folder is created in your ArcGIS account. This folder includes the name of your survey so you can easily find it.  Inside this folder, you will find a Form item and a Feature layer item:

 

  • Form item: The Form item contains the definition of the questionnaire presented to users: The labels of your questions, the calculations, media files and other resources needed to render your form.
  • Feature layer: The feature layer is the item where responses to your survey are stored.

 

In short, the survey folder contains one item (the form item) for the survey questions and one item (the feature layer) for the survey responses.

 

If you are working with sensitive data, you never want to share your surveys source feature layer. Instead, you will want to keep your survey feature layer private, and build feature layer views on top where you can better control the sharing and privilege properties. At the very least, you will want to create two feature layer views:

 

  • A view for the Survey123 web and field apps to use.  This view will allow the apps to add, and if appropriate to edit records in your feature layer.
  • A view for the Survey123 website to use. This view will control who can access the survey results through the Survey123 website, and with what privileges: just view, or also view and edit.

 

Additionally, you may want to create extra views to support other applications, such as ArcGIS Dashboards, Web AppBuilder apps, etc.

 

This article describes in detail how to build these feature layer views and associate them with your survey. If you are not familiar with the concept of feature layer views, I suggest read the Create hosted feature layer views—ArcGIS Online Help | Documentation help topic.

 

Do not create the views too early.

 

Let me put this upfront: as of version 3.9, Survey123 Connect does not like views: Connect does not create views, and does not handle them well when you delete or modify the survey. This is something that is going to change, rendering this whole article unnecessary, but for now bear with me.

 

As stated above, my recommendation is that you always use feature layer views when your survey is shared with people, but from a practical perspective you do not want to create the views too early. Survey design is an iterative process where you will be adding, changing and removing questions from your survey frequently. Some of these changes necessarily affect the schema of the surveys feature layer. If your survey is configured with views, Connect may not be able to change the schema of the source layer. If the schema of the layer is changed, your views will break.

 

For this reason, keep your survey without views for as long as you are working on it and configure the views when you are ready to put your survey in production, right before you share your survey with users.

 

Create a view for the Survey123 website first

 

I said before you will want to create at least two views: One for users who will look at your survey results through the Survey123 website, and another one for users to submit data through the Survey123 web and/or field apps. It is best to start with the view to control access to the survey results.

 

To build the view:

 

  • Login into the Survey123 website
  • Navigate to the Collaborate tab of your survey
  • Switch to the Viewer panel and hit Save.

 

 

 

If you return now to the Survey123 folder under Content in ArcGIS.com, you will notice that a new view has been created for your survey. This view has a 'stakeholder' suffix.

 

 

This new view will control who can use the Survey123 website to look at your survey results.  You will want to use the Viewer panel in the Survey123 website to control this. For example, say that users in a group called 'City of Cilantro' need to be able to look at the results of your survey, create reports and download data. Then you will go into the Collaborate tab, switch to the Viewer panel and share your survey results with that group. At that point, 'City of Cilantro' users can log into the Survey123 website and use the Overview, Data and Analyze tabs to do what they need. 

 

I would not recommend that you modify the sharing of this view through ArcGIS.com. For the Survey123 website to properly work, the sharing of the Form and stakeholder view items must be in sync.  The Collaborate tab in the Survey123 website takes care of that.

 

Create a view for the Survey123 web and field apps next

 

Configuring the view for the Survey123 web and field apps is a bit more involved. We need to create this view manually, then associate the view with the Survey123 Connect survey.

 

  • Log into the arcgis.com website and click on the My Content tab.
  • Click on the Form item to open its item details page, then click on Create View Layer. You can choose any title for your feature layer view.

To make your survey work against your own  feature layer view, you need to configure the submission_url and form_id XLSForm settings in your survey. This can be an error prone process at first. Once you are familiar with this I am sure you will do this with your eyes closed, but here I am going to follow a long but safe route:

 

  • In Survey123 Connect, from the survey gallery, click on New Survey and then choose the Feature Service option.
  • Look for the feature layer view you just created and give your new survey a throw-away name, such as temp or delete_me.

 

  • Open the XLSForm of your temporary new survey and switch to the settings worksheet. Then copy the values in the form_Id and submission_url cells into a text editor or a safe place, so we can paste them later into the original survey.

The submission_url value defines the feature layer (or feature layer view) that the survey is targeting. If empty, Survey123 Connect will create a new feature layer when you publish the survey. If a value is provided, the survey is published targeting the specified layer by the submission_url . The form_id value defines the sub-layer in your feature layer that drives the questions in your survey.

 

  • Back in Survey123 Connect go back to survey gallery, and open the survey that you want to make public.
  • Open the XLSForm, paste the submission_url and form_id values.
  • Save your XLSForm and publish your survey again.

 

The Publish dialog will indicate that your existing survey will be updated to use a custom feature service as specified by the submission URL, as shown in the next screenshot.

 

Now that your survey has been updated to target your own feature layer view, you can share your survey publicly with confidence. We will do that from the Survey123 website.

 

Sharing your survey publicly

 

  • Log into the Survey123 website at survey123.arcgis.com.
  • From the survey gallery, open the Collaborate tab of your survey

  • The Submitter panel controls who can submit data to your survey. While in the Submitter panel, look for the section named 'Who can submit to this survey?' and check the Everyone (Public) option to share your survey publicly. 

If the option to share your survey publicly is missing, contact your ArcGIS administrator.

                      

  • Scroll down the page and look for the 'What can submitters do?' section. Check if not already the 'Only add new records' option and click on Save at the bottom to persist all changes.

At this moment, your survey is shared publicly, allowing anyone to submit data through both the Survey123 web and field apps. You can get the link to your survey from the top of the Collaborate tab and distribute the link with your users. Since you have restricted access to 'Only add new records' in the Collaborate tab, it will not be possible to query, update, delete or download your survey data through the Survey123 web or field apps. Your survey's feature layer will also be secure, preventing any type of access (other than adding new records) from other Esri apps, third party apps or programmatic access.

 

If you go back to My Content in arcgis.com and check your survey folder, you will find that your feature layer view and the Form item are now shared publicly, while the feature layer remains shared only with you, the owner. This is the way you want it. Do not share the source feature layer if you want to keep your data safe.

 

Sharing your survey results in web applications and dashboards

 

The same technique we used to create a feature layer view for the Survey123 web and field apps can be replicated to support other apps and uses. It is not good practice to reuse the feature layer view we just created or to share the source feature layer. Build new views, restrict access to data as appropriate to the needs of the web app and share accordingly.

 

Here are a few links to learn more about feature layer views:

 

Outcomes