ichivite-esristaff

Downloading data captured with Survey123 for ArcGIS

Blog Post created by ichivite-esristaff Employee on Nov 9, 2015

In this blog post I will describe how you can download data captured with Survey123 for ArcGIS. There are three basic scenarios where downloading your survey data is useful:

 

  • Analysis in third party toolsStata, SPSS, SAS, Tableau or  Microsoft Excel are just some examples of the many tools out there that people use to analyze data. You can download data captured with Survey123 for ArcGIS in formats that these tools understand such as CSV. You can also download your data in shapefile and FileGeodatabase format in case you want to use GIS tools compatible with these formats.
  • Back-up: Download the data you capture to have an extra copy. Just in case!
  • Enterprise integration: Download your data so you can load it into your own database (Microsoft SQL Server, Oracle etc) or massage it before you bring it over into your own enterprise system.

 

I will describe a few different techniques to download your data:

 

Manually downloading your data from survey123.arcgis.com

 

The easiest way to download your data is using the http://survey123.arcgis.com web site:

  1. Log into http://survey123.arcgis.com with your ArcGIS Online credentials
  2. In the Survey gallery, click on your Survey and go to the Data tab
  3. Scroll down, select the output format you want (CSV, Excel, Shapefile or File Geodatabase) and download your data.

 

 

If you are working with Portal for ArcGIS, the technique above will be of no use, because the http://survey123.arcgis.com web site only works against ArcGIS Online (for now), but do not worry, the next tip will work with both ArcGIS Online and Portal for ArcGIS.

 

Manually downloading your data from ArcGIS Enterprise or ArcGIS Online

 

  1. Log into your ArcGIS organization (in ArcGIS Online or Portal for ArcGIS) and go to MyContent.
  2. For every survey you have published, you will find a folder and within it, a Feature Layer. Open the details page of the Feature Layer and select Export Data.

 

 

Downloading data manually as described above is fine except when you want to do it very often. Scripting is the way to go if you want to regularly download your survey data. A classic example is when you want to pull data from ArcGIS Online daily, or hourly, into your own database for further analysis or QA/QC workflows.

 

Downloading your data with Python

 

Since data captured with Survey123 for ArcGIS is stored in Feature Services we can use the standard REST API in ArcGIS to trigger the download. Javascript, Python, PHP, Scala, Java or .Net are some of the environments from which you can invoke the ArcGIS REST API.

 

My favorite Python script to programmatically download Survey123 data into a geodatabase was written by James Tedrick and is shared in this blog post.  James describes at length how to use the script in his post and there are many useful comments from people that used it.

 

Alternatively, you can also use this Python script to download data into a local shapefile, file geodatabase or csv file. To run this script:

  • Download the code into a machine with ArcGIS for Desktop 10.3
  • Edit with IDLE of a Python editor to input your own feature service id, credentials and output folder
    • Feature Service ID: In arcgis.com or your own Portal for ArcGIS, navigate to the details page of the feature service you want to download. Get the ID from the URL. It will look something like this: f4a06f02b1dd4b1f8904ed0d4e8713bd
    • Output Format: Choose between 'Shapefile', 'File Geodatabase' or 'CSV'.
    • Credentials: Typically the owner of the Survey. Simply replace the 'XXXXX' and 'YYYYYY'
    • You can define a Portal URL and the output folder where you want the data to be saved.

 

The advantage of using a Python script is that you can easily run it anytime and get the latest data from your Survey.  You can also schedule your script to run every so often.

Outcomes