Skip navigation
All Places > Survey123 for ArcGIS > Blog > 2015 > November
2015

Survey123 for ArcGIS is no longer an Esri Labs project. We have transitioned Survey123 for ArcGIS into Beta. With this move, Esri commits to fully productize Survey123, adding it into the core suite of apps included with ArcGIS. This means full localization, documentation, Esri Technical Support, formal Training from Esri Educational Services as well as a Product Life Cycle statement. It also means a much tighter integration with the rest of ArcGIS including Portal for ArcGIS, other apps, content etc  All of this will happen progressively, starting today with a Beta and with a plan to release in the first half of 2016.

 

We have still a long way to go, but we are pleased to announce to you that Survey123 for ArcGIS is here to grow and stay, with full commitment from Esri.  Thanks to all of you that during our Esri Labs journey helped us with your feedback!

 

Today, we also want to look in retrospect at what we have learned during our time in the 'lab':

 

First, we wanted to experiment with the idea that a form-centric approach for capturing data in the field made a lot of sense, even in the context of the ArcGIS platform. Over the past few months, we have been able to validate that the majority of field data capture projects do not have a map at their heart, but forms. We have learned that building forms, just like building maps, is an art: so much to learn! The experiment turned out great, proving that Smart Forms are a great way  to capture data in ArcGIS.

 

In the lab we wanted to play with the idea of reversing the traditional way of building projects in ArcGIS.  The traditional way is bottom-up, where you first think about the data model, and then the map and the the app that will support it. With Survey123 for ArcGIS, you think about the questions and the logic you want to put into your app, and leave it to the software to create the data model for you. You don't need to worry about domains, subtypes, attachments or related tables. This is essentially what Survey123 Connect for ArcGIS does: it transforms your questions and logic into a survey, connected to an ArcGIS feature service, with the full support of the ArcGIS platform at the back end.  This experiment also brought good news: a whole new group of people, with little to no GIS skills were able to setup complete data collection workflows in ArcGIS.

 

With Survey123 for ArcGIS, we wanted to try technology that would enable us to release an app simultaneously on as many platforms as possible. So far, Survey123 is supported on smartphones and tablets (iOS, Android, Windows Phone) as well as desktop platforms (Windows, OS X and Ubuntu Linux). We achieved this by building Survey123 for ArcGIS on top of AppStudio for ArcGIS: a cross-platform development environment for building GIS applications.  AppStudio for ArcGIS has its foundation in technology from a cute company called Qt.  Without AppStudio for ArcGIS and the underlying Qt framework, we would not be here today. Such an amazing technology!

 

There are other lessons we have learned from our time in the 'lab'. Since our initial launch in May 2015, we have released an update every month. Some bigger than others, but always trying to resolve the issues you have been encountering. What we learned is that updating frequently and across all platforms is not just a good thing, but it is expected. Releasing often is something that we have enjoyed and want to keep doing as we move forward.

 

The best of our Lab journey has been the fast paced, community-driven development that happened. We want to keep that spirit. We are more excited than ever about Survey123 for ArcGIS! Take Survey123 for ArcGIS into the field, use it, give us feedback and together we will build great things.

 

Next update coming around December 15: OAuth support, a first cut for offline basemap support and miscellaneous fixes.

 

The Survey123 for ArcGIS team.

In this blog post I will describe how you can download data captured with Survey123 for ArcGIS. There are three basic scenarios where downloading your survey data is useful:

 

  • Analysis in third party toolsStata, SPSS, SAS, Tableau or  Microsoft Excel are just some examples of the many tools out there that people use to analyze data. You can download data captured with Survey123 for ArcGIS in formats that these tools understand such as CSV. You can also download your data in shapefile and FileGeodatabase format in case you want to use GIS tools compatible with these formats.
  • Back-up: Download the data you capture to have an extra copy. Just in case!
  • Enterprise integration: Download your data so you can load it into your own database (Microsoft SQL Server, Oracle etc) or massage it before you bring it over into your own enterprise system.

 

I will describe a few different techniques to download your data:

 

Manually downloading your data from survey123.arcgis.com

 

The easiest way to download your data is using the http://survey123.arcgis.com web site:

  1. Log into http://survey123.arcgis.com with your ArcGIS  credentials
  2. In the Survey gallery, click on your Survey and go to the Data tab
  3. Scroll down, select the output format you want (CSV, Excel, Shapefile or File Geodatabase) and download your data.

 

 

If you are working with Portal for ArcGIS, the technique above will be of no use, because the http://survey123.arcgis.com web site only works against ArcGIS  (for now), but do not worry, the next tip will work with both ArcGIS  and Portal for ArcGIS.

 

Manually downloading your data from ArcGIS Enterprise or ArcGIS

 

  1. Log into your ArcGIS organization (in ArcGIS  or Portal for ArcGIS) and go to MyContent.
  2. For every survey you have published, you will find a folder and within it, a Feature Layer. Open the details page of the Feature Layer and select Export Data.

 

 

Downloading data manually as described above is fine except when you want to do it very often. Scripting is the way to go if you want to regularly download your survey data. A classic example is when you want to pull data from ArcGIS  daily, or hourly, into your own database for further analysis or QA/QC workflows.

 

Downloading your data with Python

 

Since data captured with Survey123 for ArcGIS is stored in Feature Services we can use the standard REST API in ArcGIS to trigger the download. Javascript, Python, PHP, Scala, Java or .Net are some of the environments from which you can invoke the ArcGIS REST API.

 

My favorite Python script to programmatically download Survey123 data into a geodatabase was written by James Tedrick and is shared in this blog post.  James describes at length how to use the script in his post and there are many useful comments from people that used it.

 

Alternatively, you can also use this Python script to download data into a local shapefile, file geodatabase or csv file. To run this script:

  • Download the code into a machine with ArcGIS for Desktop 10.3 (or newer)
  • Edit with IDLE of a Python editor to input your own feature service id, credentials and output folder
    • Feature Service ID: In arcgis.com or your own Portal for ArcGIS, navigate to the details page of the feature service you want to download. Get the ID from the URL. It will look something like this: f4a06f02b1dd4b1f8904ed0d4e8713bd
    • Output Format: Choose between 'Shapefile', 'File Geodatabase' or 'CSV'.
    • Credentials: Typically the owner of the Survey. Simply replace the 'XXXXX' and 'YYYYYY'
    • You can define a Portal URL and the output folder where you want the data to be saved.

 

The advantage of using a Python script is that you can easily run it anytime and get the latest data from your Survey.  You can also schedule your script to run every so often.