Skip navigation
All Places > Survey123 for ArcGIS > Blog > 2019 > February
2019

New Britain is the largest island in the Bismark archipelago of Papua New Guinea. William Dampier, takes credit for being the first British man to land on this island. This happened on February 27, 1700. Since then we call this island New Britain (Nova Britannia at the time). Both Wiliam Dampier and New Britain are fascinating. This island, of volcanic origin, was populated by two main indigenous groups: the Papuans who arrived tens of thousands of years ago, and the Austronesians, who joined the party around 2000 years ago. The island hosts remarkable traditional cultures. Still today, over 10 different Papuan languages  are spoken in the island and as for the Austronesians languages, we count more than 40! William Dampier is known for being the first man to circumnavigate the world three times. He grew as a pirate attacking Spanish ships in America and then became a successful navigator and explorer. Dampier documented winds, tides and currents that were critical for future explorations by James Cook and Horation Nelson. His natural history observations were also highly influential on works of Alexander Von Humboldt and Charles Darwin.

 

With our 3.3 update, we celebrate the adventures of William Dampier and his landing over 300 years ago to the island today known as New Britain.

 

 

Enhanced support for GNSS receivers in the Survey123 field app

 

Starting with version 3.3, you can establish a direct connection between the Survey123 field app and your GNSS receiver. This is useful for several reasons:

 

  • More accurate location data:  By leveraging dedicated high accuracy GNSS hardware, you can substantially improve the quality of the location data you capture with your smart forms.  There is a wide range of GNSS receivers, some of them able to produce, if used properly, sub-meter and sub-centimeter accuracy. 
  • Better understanding of the quality of the data: Having more accurate location data is not enough. It is also important to understand how accurate that data is.  Most GNSS receivers are able to provide important information that describe how the locations you are getting were produced. This includes things like the type of fix (RTK Fixed, RTK Float, GPS, Differential GPS, SBAS...), the differential age, reference station ID and many others.  With version 3.3 of Survey123 and above, you can select which of these properties are important to you and store them as GIS attributes in the feature layer columns of your choice. For a complete list of all the location metadata you can extract from your location data with Survey123 refer to the extract geopoint values help topic.
  • Custom validation rules: You can use all the GNSS metadata described above to create custom data validation rules that get triggered while the data collection is happening. This is critical, as it can save many trips back to the field! It is important to note that you can use all the power of XLSForms to build sophisticated expressions that take advantage of this metadata.

 

In this initial release, we have thoroughly tested GNSS receivers built by a number of vendors including Leica, Eos Positioning Systems, Bad Elf, SXblue, and Trimble, although it technically can work with other receivers as long as they output NMEA sentences. Our choose a receiver help topic includes more information regarding currently supported hardware.

 

To support direct connections to external GNSS receivers, you will notice we have introduced several changes in the Survey123 field app. The Settings dialog in the app now includes options to connect to external location sensors via Bluetooth and network ports. For each connection you can also set user preferences such as a custom name for your device, the height of the antenna and handy visual and audio alerts to warn users in case the connection between the GNSS receiver and your device is interrupted.

 

Our XLSForm support has also been expanded so you can include in your form expressions to help work with the GNSS metadata in real time.  This really opens a wealth of possibilities for you to use this information to design soft and hard data validation constraints in your forms.  The geopoint help topic in our doc describes in more detail how to take advantage of this.

 

Point to Point Measurements with Spike® and the Survey123 field app

 

We introduced support for the Spike laser range-finder in March 2018.  Spike is an excellent complement to Survey123, allowing you to quickly bring laser-accurate measurements into your forms. Jim Moore and Brett Stokes, who recently joined the Survey123 team, put together this video to demonstrate the basics of bringing measurements from Spike into Survey123.

 

 

In this release, we are bringing support for the Point to Point (P2P) measurement mode which was missing in our initial implementation. P2P is a very handy way to calculate distances between two objects because it does not require you to identify a 2D plane on which to measure. In P2P mode you can simply aim Spike at the first object, such as the side of a house, and take a photo. Then, aim at the second object, such as a tree or fence, and take a second photo. Spike will calculate the distance between those two objects and bring that value into your Survey123 form. You can also use Point to Point to measure the height of an object such as a tree, streetlight or building.

 

If  you are already using Spike with Survey123 and wish to try Point to  Point measurements, make sure you upgrade your Spike app from the store  with the latest version.

 

For more details on how Spike and Survey123 work together, check out theUsing Spike® with Survey123 for ArcGIS blog post.

 

Date/Time question type in web designer

 

A new Date/Time question type has been added in web designer.  This will help you capture both a date and a time in one single row, saving vertical space in your forms. Within web designer you can set a default value for this question, either selecting a static value, or the exact time when the form is open.  You can also set validation rules to constrain inputs to a time window.

 

Date-Time Question type in Survey123 web designer

 

Better field and choice naming in web designer

 

If you never liked auto-generated field names such as field_0, field_1... in your Survey123 feature layers, you will want to pay attention to this one. Let me start with a bit of context first:  When you publish a survey in web designer, we create on your behalf a feature layer field for every question in your form.  These fields are used later to store responses to your survey. Up until this release, the name of these fields followed a simple field_0, field_1 pattern. This was typically never an issue when using the Survey123 web site to inspect your data, but it was not ideal when exporting results into Excel, CSV, or when configuring web hooks.

 

Starting with this update, a more friendly name will be given to these fields based on the label of your questions. The following animation compares what an auto-generated feature layer schema looked like before, compared to what you will get now.

 

 

Field and choice names are now derived from your question and choice labels. Given that there are some constraints that apply to the names that can be given to fields in a feature layer, you will notice that your labels will not be used literally. For example, spaces will be replaced with underscores (_), very long labels will be shortened and special characters will be eliminated.

 

Using the Modify Schema dialog you can optionally fine tune the names of fields and choices to your liking, although remember that you will not be able to make adjustments in the schema once the fields have been added to the feature layer.

 

The option to Modify Schema in the Publish Survey Dialog has also been made more visible as a reminder of this important step before you publish your survey.

 

 

Delete and edit existing records for survey owners

 

As a survey owner, I am sure you always submit a couple of records to test out your survey designs, immediately checking values in the Data tab. You can now easily delete, and even edit values right from the Survey123 web site. You no longer need to use a separate tool to do that. All you need to do is to select the records in the table view, and then choose the Delete or Edit options in the right panel to make the changes you want.

 

Edit and delete records, right from within the Survey123 web site

 

At  this moment in time, the option to edit records will not be available  for surveys published from Survey123 Connect, but we are looking to  relaxing this limitation in the future.

 

 

 

Other fixes and enhancements

 

Survey123 web site:

 

  • ENH-000107831 Provide the ability to edit the survey submission "Thank you" screen seen in the browser when the survey is published from Survey123 Connect
  • Import custom themes from other surveys.
  • For text questions, in the Analyze tab, you can now choose to display a list of word counts or complete responses.
  • Much improved user experience for configuring custom report templates.
  • BUG-000119133 Field aliases in a survey in the Survey123 for ArcGIS application are not respected in single choice and drop-down lists.
  • BUG-000098360 When downloading data from a Survery123 survey that was made using Survey123 web designer, the field names are converted to {field_1}, {field_2}, ... instead of honoring the name in the data table.
  • BUG-000103150 Surveys created in Survey123 web designer using drop-down choices display 'choice1, choice2, ...' when exported to a CSV file or a shapefile.
  • BUG-000105778 Survey question choices from single choice questions do not correctly translate when exported to CSV or shapefile from surveys that were created via Survey123 web designer.
  • BUG-000114116 When using a custom template that writes out results from related and non-related attributes from a survey, the non-related records do not populate the template table.
  • BUG-000113548 When an audio attachment is included in the survey, the Survey123 for ArcGIS website fails to generate a report.

 

Survey123 field app and Connect

 

  • Allow users to zoom beyond level 18 if the tile package includes more detailed levels.
  • BUG-000119481 When creating a Survey123 for ArcGIS survey with an existing feature service, the float or single field types do not auto populate in the XLSForm created
  • BUG-000110786 When signing in to the Survey123 for ArcGIS (iOS) app configured with an IWA portal, the app uses the username and password entered at the beginning of the app session for all subsequent sign-in attempts.
  • BUG-000119847 Date questions in certain languages are not displaying the calendar correctly
  • BUG-000119899 Integer type questions with the "spinner" appearance type do not work
  • ENH-000105170 In Survey123, allow users to change the Location Provider, from the internal receiver to an external receiver.
  • ENH-000109937 Provide an indicator within the Survey123 interface to display which GPS unit the app is using.

In this blog post I will describe how you can integrate Survey123 with Spike. Spike is a laser-based measurement solution by Esri partner ikeGPS.   What makes Spike unique is that it allows you to easily measure the dimensions of remote objects, such as areas and lengths, all from a photo.

 

Say, for example, while doing a property inspection you need to calculate the height of a balcony, or the size of a garage door.  Rather than using a ladder and measuring tape, you can simply take a photo of the building, and then calculate dimensions by drawing on top of the photo.  You can similarly use Spike to calculate the size of road signs, billboards, poles, etc.   

 

You can embed Spike questions within Survey123 and automatically bring all measurements from Spike into your form: Tap on a Spike question, snap a photo, measure on it… and your measurements will get persisted in ArcGIS along with the rest of the survey data.

 

The integration of Survey123 with Spike is exciting news, because it has the potential to boost the productivity of certain data capture workflows tremendously.  This short video tutorial describes the basics of how Survey123 and Spike work together:

 

 

Lets next get into the details...

 

 

About Spike®

 

The Spike laser: The most visible aspect of Spike is the laser instrument. It is an eye safe Class-1 laser, designed to measure distances in a range between 2 and 200 meters with an accuracy of 5 centimeters. The laser is designed to be attached to your smartphone or tablet and paired using a Bluetooth connection.

 

Spike units have a retail price of $499 and are sold online and through a network of resellers.

 

The Spike app: Once the laser is paired, the Spike app uses the camera on your smartphone or tablet so you can take a photo of your object of interest. Using the distance to the photographed object captured by the laser, the Spike app can accurately calculate lengths and areas as you draw lines and polygons over the photo.

 

The Spike app is available for iOS and Android operating systems and is certified for use against a wide collection of smartphones and tablets

 

Getting familiar with Spike: Spike is straight-forward to use, but knowing a handful of tips and good practice in the field is key to get the best out of it.  The best reference to take your first steps is the Spike User Manual

 

Spike can be operated through two main modes:

 

  • Photo Measurement: When working in New Photo Measurement mode, you take a single photo, and then derive all measurements from it.  You can for example simply get the distance from your location to the object, or extract lengths and areas by drawing on your photo.
  • Point-to-Point Measurement: In this mode you can calculate the distance (horizontally and vertically) difference between two objects by taking a separate photo of each object.

       

Photo Measurement mode:In its simplest form, the Photo Measurement mode will allow you to simply snap a photo of an object, and get back the exact distance to that object form your current location.  This alone can be useful in multitude of use cases.  Say for example you need to document the width of a road: stand on one side of the road and take a photo targeting at the other end across. After taking the photo, Spike will give  you the exact distance. Not only you can get an accurate measurement, but you can do it without risking your life; safely from one side of the road. 

 

With Photo Measurement mode you can also measure distances (and areas) on objects by simply drawing on top of the photo you just took. For example, say you want to know what is the height of a building. Well, you take a photo of it, and then you extract the height by drawing on that photo.  If you want to derive measurements from a photo in this way, certain conditions need to be met.  When used correctly and in ideal conditions, you can expect a measurement with an accuracy of 1%.  For example, if you measure the height of an object to be 10 meters, you can expect the accuracy to be around 10 centimeters.

 

Let’s describe next what used correctly and ideal conditions means:

 

  • Measurements against a flat surface: You should only perform measurements on flat surfaces, such as a wall or a sign. The specific shape you want to measure should be on a rectangular plane. This is because you will need to define the plane in the photo before you can measure anything. 
  • Face your target front-on: For the most accurate measurements, you should take your photo while facing your target perpendicularly. That is, you need to be right in front of the plane on which you will do your measurements. If you need to take your photo at an angle, the expected accuracy of the measurements will be reduced progressively as you get further from being perpendicular.

 

 

  • Ideal target range: The ideal working range of the laser is between 6 and 100 meters, although technically you can push the limits and measure objects between 2 and 200 meters away.
  • Center your target: You also want to make sure that the object you want to measure is right in the center of the camera bulls eye, because the laser always calculates distances through the bulls eye.
  • Draw accurately: Once you have taken the photo, you will want to make sure you accurately draw your lines and areas for measurement. The Spike app has a handy magnifying window that will help you with that task.   If your photo is taken following all the tips above, but you do not take pride in carefully drawing the exact shapes you want to measure… the measurement will not be as accurate as it could be.

 

So far all of this may sound a little bit intimidating, but once you have the Spike device in your hand things are much simpler. Not just simpler, but also fun. Next you can see Brandon having a good time while ‘spiking’ outside Esri.

 

 

From our own experience, the Photo Measurement mode works best with man-made structures: walls, signs, billboards, doors, straight poles, streetlights etc.  Measuring the height of trees, rocks etc is much more challenging because you often do not have a good plane to reference your measurements.  If you do not have a good 2D plane to make your measurements on, then you should consider using the Point to Point measurement mode.

 

Point to Point Measurement mode: Point to Point or P2P is very useful when all you need is to measure the distance between two objects in front of you and you either cannot fit the two objects within a single photo, or when you do not have a good 2D plane on which to base your measurements. With P2P, you simply need to center your camera on your first target, say for example a tree and take a photo. Then you turn from that same location to take a photo of your second target, say for example a wall. Spike will compute the horizontal and vertical distances between the two photographed objects.

 

Like with any tool out there, it takes a bit of practice to fully understand what is the best measurement mode for the job.  Spike can greatly boost the productivity of field workers saving valuable time. It also can provide a much safer way to make measurements. For hard to reach objects, Spike can make it possible to perform measurements that could not be done otherwise.

 

Now that I have described the basics of Spike, it is time to go into the details of how Survey123 and Spike work together.

 

Integrating Survey123 with Spike (Photo Measurements)

 

Starting with Survey123 version 2.6, you can incorporate Spike functionality into your form. This allows you to streamline the use of Spike within your data collection workflows. For example,

if you have a survey to help map the presence of graffiti in the city, you will want to capture many things including the location, typology (Tag, Throw-up, Blockbuster…), surface, photo and size.  Spike can help you capture the size accurately, so you can estimate the cost of removal.

 

Adding Spike questions into a survey: You can add Spike questions to a survey using Survey123 Connect for ArcGIS. In your XLSForm, add a question of type image, and then set the appearance to either spike-full-measure, spike or spike-point-to-point:

 

  • spike: With this appearance, Survey123 will launch the Spike app so you can simply aim at your object of interest and snap a photo of it.  The spike appearance is useful when you simply want to get the distance to the photographed object and its estimated location.
  • spike-full-measure: This appearance sets the Spike app in New Photo Measurement mode as described in the previous section. When using this appearance, you will be able to get the measured lengths and areas, into your form.
  • spike-point-to-point: Launches Spike in Point to Point mode so you can calculate the horizontal and vertical distances between two photographed objects.

 

When using the spike-full-measure, spike-point-to-point and spike appearances, Survey123 will let users easily launch the Spike app from within the form.  Once the photo has been taken and measurements added, Spike will return back to Survey123, so all measurements of interest can be extracted and stored in the survey.  The measure object with Spike help topic describes step by step how Survey123 and Spike work together.

 

Spike photos will be stored as an attachment automatically.  This is a great feature because photos will become a verifiable record persisted with your data. Photos will show how accurately the measurement was made in the field, and if needed, the photo itself could be used in the back-office to make additional measurements.

The use of Spike within Survey123 is only available to logged-in users. Spike questions can be used while online or offline, but you must be logged-in.

                            

 

Extracting Spike measurements with pulldata(): Spike uses EXIF photo headers to store information such as the location from which the photo was taken, the direction, distance and measurements made in the Spike app.  In a previous blog post we explored how to extract EXIF metadata from an image in Survey123 using the pulldata() function. This same technique can be used to extract Spike metadata.

 

The Spike metadata in the EXIF of the image is split between standard EXIF tags and custom information stored in the ImageDescription tag. The following table describes standard tags populated by Spike:

 

 

EXIF tagDescription
gpsdestlongitude

The estimated X coordinate of the photographed object

gpsdestlatitude

The estimated Y coordinate of the photographed object

gpsdestcoordinate

The XY coordinate pair of the photographed object

gpstimestamp

The time when the photo was taken

gpsdatestamp

The date when the photo was taken

gpsdestdistance

The distance between the photographed object and your device

imageDescription

A JSON string including measurements captured by the Spike app

 

This expression will extract the distance in meters between the photographed object and the device:

 

pulldata("@exif", ${spike}, "gpsdestdistance")

 

In order to extract photo-measurements from Spike, your pulldata() function will need to first get the ImageDescription EXIF tag, and then parse the returning JSON string to extract the specific dimensions you are looking for.

 

For example, the following expression assumes the first measurement made in Spike was of type length, and gets its value in meters:

 

pulldata("@json",  pulldata("@exif", ${spike }, "ImageDescription"), "Spike.objects[1].dimensions")

 

The next expression assumes the first measurement made was of type area and extracts its value in square meters:

pulldata("@json",  pulldata("@exif", ${spike_full}, "ImageDescription"), "Spike.objects[1].area")

 

This expression gets the horizontal distance between two objects as measured through the Point  to Point measurement mode:

 

pulldata("@json",  pulldata("@exif", ${spike }, "ImageDescription"), "P2P.horizontal_distance")

 

 

Spike returns many other properties through the ImageDescription EXIF tag including: the pitch, roll and direction of your device when the photo was taken, the unique identifier of the specific Spike laser unit that captured the distance measurement etc. The Spike section of the Media help topic describes in detail the contents of the JSON object stored in the ImageDescription.

 

Another way to explore the contents of the ImageDescription tag is to simply add a text question to your form and have it populated with the contents of the ImageDescription tag through a calculation.  When you do this, it is important that you set the esriFieldLength column of your text question to 1000 characters. The ImageDescription JSON string can be quite big.

 

A great starting point to experiment with all the things you can do with XLSForms and Spike is our Spike sample. I strongly recommend that you have a look at it:

 

1.    Open Survey123 Connect for ArcGIS
2.    Click on Create New Survey
3.    Search for the Spike survey in the Samples category
4.    Publish it and test from your smartphone
5.    Look into the XLS file to learn all the techniques used in the sample.

 

Integrating Survey123 with Spike (Estimate Locations)

 

While the heart of what Spike excels at is helping you measuring dimensions over photos, you can also use Survey123 to roughly estimate the location of remote objects.  For example, you can map the location of a building without stepping out of your car, or the location of a streetlight across the street.

 

Before we go into the details of estimating the location of remote objects, I want to emphasize that Spike is not a high accuracy surveying device.  It can measure accurately distances to objects, but when estimating the location of remote objects there are several sources of error that will come into play outside of what Spike is designed to do.

 

To define the location of a remote object, Spike relies on the following:

 

  • The location of your own phone or tablet: This is the first source of error. A typical smartphone is not going to give you a highly accurate location for your phone. Your location accuracy may vary depending on many factors and if your location is off, the estimated location of the remote object will be off too.  To correct this problem, you can connect an external GNSS receiver to your own phone or tablet via Bluetooth. If used correctly, external GNSS receivers will improve the source location used by Spike in its calculations. In the side photo, for example, Survey123 is combined with Spike and Juniper Systems Geode Sub-meter GNSS antenna.
  • The distance to the remote object: This is going to be very accurate as long as you keep yourself within the working range of Spike (2 to 200 meters) and you aim at the remote object using the bull’s eye in the Spike app. Measuring distances to objects is what Spike is designed to do, and does it very well.
  • The azimuth to the remote object as measured from your own device location: The azimuth is another important source of error. The azimuth is obtained from the compass in your own smartphone or tablet. The compass in your smartphone must be calibrated before use by Spike. It is critical that you follow steps in this compass calibration video-tutorial before you estimate remote locations. Even with your compass calibrated, it is normal to get deviations of a few degrees, which will increase the error as the distance to the remote object increases.

 

The gpsdestlongitude and gpsdestlatitude properties will be populated with both the spike and spike-full-measure appearances.If you do not plan to capture any length or area measurements and all you want is the location of the photographed object, then the best is to use the spike appearance.

 

To extract the estimated latitude and longitude of the photographed object, you will use the pulldata() function within a calculation. For example, the following expression will extract the latitude:

 

pulldata("@exif", ${spike}, "gpsdestlatitude")

 

You can store the latitude and longitude as attributes in your survey, and then create an XY Event Layer with them in ArcGIS Pro or other tools.  

While it should be possible for you to calculate a geopoint question with the latitude and longitude values extracted from the Spike photo, there is currently a known limitation in Survey123 preventing the successful calculation when applied. This is in our backlog and we will work towards resolving the problem as soon as possible.

          

As stated before, it is best that you exercise Spike in the field in all its capabilities. To fully understand its potential and limitations you must experience it in the field in real working conditions.

 

If you need some ideas, here is an informative video by IkeGPS further describing some common use cases for Survey123 and Spike.

 

Our Developer Summit conference is approaching fast.  We at Esri always put a lot of energy into this event. It is an opportunity to meet face to face with many of you, share our work, discuss new features and road map plans. 

 

2019 will be very special for us as we will make 4 years since we announced the graduation of Survey123 from an Esri Labs proof of concept into a full fleshed ArcGIS Product. This year we will announce details on ambitious plans for the new releases to come: direct connections to external GNSS receivers in the field app, added support for polygon and line features, a release date for our Survey123 report services, enhanced integration with Spike laser measurement tools, a new extensibility framework and many more great news. Members of the Survey123 team from our offices in Beijing, Australia and Redlands will come to Palm Springs. We are really excited for the opportunity to meet you again.

 

Today we want to share with you some tips and a guide for anything Survey123 at these events.

 

The Events App

 

I highly recommend that you download the Esri Events app, so you can always be on top of last minute changes and manage your schedule.  As of the time when I write this post the detailed schedule is not yet up in the app, but it will be available shortly.  This app will be a good companion throughout the entire conference.

 

Pre-Summit Hands-On Training

 

Our Esri Pre-Summit Hands-On training sessions take place on Sunday 3 and Monday 4. These are full day sessions with a mix of lessons and practical challenges. You bring your own laptop, so you can take all homework home. Massive learning guaranteed!   Our sessions start at 8:30 sharp. You can register starting at 7:30 at the Hotel Riviera.

 

  • Mastering Smart Form Design with Survey123 and XLSForm.
    • Sunday, March 3 from 8:30am to 5:00pm
    • Royal II at Hotel Riviera Palm Springs.
  • Extending Survey123 for ArcGIS.
    • Monday, March 4 at Royal II from 8:30am to 5:00pm
    • Royal II at Hotel Riviera Palm Springs

 

Esri Showcase and GIS Solutions Expo

 

The GIS Solutions Expo area opens Monday from 2:00pm to 7:00pm at the Primrose Ballroom. This is a great place to meet many partners who work closely with Survey123.  Here are a few:

 

  • Microsoft. Diamond sponsor this year. Showcasing the Survey123 Microsoft Flow connector.
  • Bad Elf, EOS Positioning Systems, Trimble, GENEC:  Providing GNSS receivers for which Survey123 3.3 will provide direct connectivity capabilities.
  • Zebra Technologies: Manufacturing rugged devices compatible with Survey123.
  • Geo Jobe: Custom software development services.

 

During this time, you can also swing by the Esri Showcase area to visit the Survey123 team in the Apps area.

 

On Tuesday from noon to 8:30pm, Wednesday from 1:00pm to 6:30pm and Thursday from 9:00am to 4:00pm the Esri Showcase will continue open in Oasis 1 and 2. Come to the Showcase and look for the Survey123 booth. This is your chance to meet people from the Survey123 team, have a relaxed conversation, ask questions and network with other Survey123 users.

 

Meet the Teams

 

The Meet the Teams event will take place from 6:00pm to 8:30pm in Oasis 1 and 2 on Tuesday. In other words, at the Esri Showcase area. Drinks, food and GIS to spare.

 

Esri Dev Summit Technical Workshops

 

These are 60 minute sessions. All take place in the Palm Springs Convention Center. A good place to learn from the best, ask questions and meet other Survey123 users.

 

Other related Technical Workshops

 

If you plan to extend or customize Survey123, the AppStudio sessions are a must.

 

 

Survey123 Demo Theaters

 

Short and to the point sessions focusing on very specific topics. Live demos and time for questions are guaranteed. All Demo Theaters take place at the showcase area.

 

 

Survey123 Special Interest  Group (SIG)

 

This session is a great place to mingle with other Survey123  Ninjas and members of the Survey123 team. Keep your own lunch box because this SIG takes place at noon. We will break the ice with help from a special guest who will share his own work deploying Survey123. Then we will open an interactive Q&A session.

 

  • Survey123 SIG
    • Thursday, March 7 from noon to 1:00pm
    • Primrose A at the Palm Springs Convention Center

Many web applications support the notion of URL parameters; Survey123 web forms are no exception. URL parameters allow you to dynamically pass data into your survey. This data can be used to prepopulate questions, hide questions, change the language of a form and many other things with your survey form. In this blog post, I will explore the options available in detail.

 

Before I start, I want to clarify that in the past we already covered how to use the Custom URL Scheme in the Survey123 field app. The concept of a Custom URL Scheme is similar to URL parameters, but they are two different things: the Custom URL Scheme applies to the Survey123 field app and the URL parameters discussed in this post only apply to web forms. That is, to forms loaded within a web browser, as opposed to loaded into the Survey123 field app in your device.

 

With that, lets get started:

 

The Basics

 

Lets first get familiar with how surveys get loaded into the Survey123 web app. Take the following URL:

 

https://survey123.arcgis.com/share/1cb28b212b5542acbbdbaa35feba0765

 

The first part of the URL refers to the Survey123 web app: https://survey123.arcgis.com/share/ We host the Survey123 web app in our ArcGIS Online servers, and it is used to render your surveys as web forms.  The second part of the URL is the unique identifier of your survey. In this particular case, it is: 1cb28b212b5542acbbdbaa35feba0765. The complete URL will open the Survey123 web app in your web browser and immediately present the survey 1cb28b212b5542acbbdbaa35feba0765, in this case it is an Oak Tree Health survey.

 

So far, no URL parameters have been used, but now we are going to add one parameter to automatically populate one of the questions. Try the following:

 

https://survey123.arcgis.com/share/1cb28b212b5542acbbdbaa35feba0765?field:submittedBy=Fernando Paredes

 

You will notice that now the Submitted By question is automatically adding the name Fernando Paredes in the survey.  Well, that was a URL parameter!  Note that immediately after the basic URL of the web form, I added a question mark (?). The question mark indicates that what follows are URL parameters. Next, I added the name of the parameter, which in this case is field:submittedBy and a value for that parameter: Fernando Paredes.

 

You can add multiple parameters by simply separating them with an ampersand (&). For example:

 

https://survey123.arcgis.com/share/1cb28b212b5542acbbdbaa35feba0765?field:submittedBy=Fernando Paredes&field:commonName=Oak

 

Next, lets learn about the different URL parameters you can pass to Survey123 web forms.

 

Some of the URL parameters described here are supported in surveys published with 3.2 or newer versions. If your URL parameters do not seem to take effect, re-publish your survey so it gets upgraded to the latest version, and then try again.

 

Setting values on questions (field:fieldname)

 

In the previous section you saw how a URL parameter can set a value to a question in your form, but lets take a closer look at how it works. The syntax is simple:

 

field:commonName=Oak

 

The name of the parameter is made  out of two distinct parts. First, you have the field: suffix and then you have the name of the question where you want to put a value.  Figuring out the name of a question can be tricky. The question name should not be confused with the label of the question. The label is what users see in the form. The question name represents the column in your feature layer  where the response will be stored. 

 

If you author surveys in Survey123 Connect, I am sure you already know what  I am talking about: The question name is represented by the name column in the survey worksheet of your XLSForm.  If you design your forms in Survey123 web designer, then finding out the exact name of your questions deserves a more detailed explanation.

 

After adding a question into your survey in web designer, you need to set its label and other properties, but not the name. The name of your question is automatically assigned by Survey123 web designer.  Assigning a valid question name needs to follow very strict rules (the name cannot start with a number, cannot have special characters, or  spaces and must be unique within your entire form) and that is why we put that out of your way in designer. This simplifies  authoring for beginners, although in cases like this it obscures what is really going on.   In any event, you can look at the question names by using the 'Modify Schema' link during the publishing process. 

 

Modify Schema in Survey123 web designer

 

The Modify Schema dialog will let you define the exact names of your questions only when you add your questions to the form for the first time. Once you publish your survey, questions that already exist in your survey will appear as read-only in the Modify Schema dialog.   In my case, I really did not want to change anything, I simply wanted to get the names so I could use them with my URL parameters.

 

Question names are case sensitive. You must pass the exact name. For example, in the oak survey, field:commonName=Oak will work, but field:CommonName=Oak will not work.

            

 

Centering geopoint questions (center)

 

The center parameter lets you set the default location of the geopoint question in your survey.  It looks like this:

 

https://survey123.arcgis.com/share/1cb28b212b5542acbbdbaa35feba0765?center=34.1,-117

 

This parameter accepts a latitude/longitude coordinate pair where the values are separated by a comma.  Now. there are a couple of important things to remember when using this parameter:

 

  1. This URL parameter only takes effect if the geopoint question is flagged as required, AND
  2. If authoring from web designer, the option to 'Ask user's location when opening this question' must be disabled'.

 

If the geopoint question is not required or the user location is used, the values from this URL parameter will be overridden at runtime when the end-user opens the question.

 

Hiding questions and survey elements (hide)

 

The hide parameter is handy when you want to control visual aspects of your survey. You can load a survey and hide questions, the navigation bar and more.  This parameter takes as an input more or more values, separated by a comma.

 

Parameter ValueDescription
navbarHides the navigation bar. That is, the bar at the top of the survey that includes options to login/logout and to change the language. This is particularly useful when you embed your survey within a website. Example.
headerHides the survey header. The header typically includes the title of the survey or alternatively a banner image. This is particularly useful when you embed your survey within a website. Example.
descriptionThe description element is optionally included if you author your survey in web designer.
footerHides the footer element, which includes the Powered by Survey123 link, or an alternative link if you modified it. Also useful when embedding your survey within a website. Example.
themeUse the theme value to disable any custom theme applied to your survey.  Example.
field:fieldNameThis value is used to hide specific questions within your survey. If you want to hide multiple questions, simply add their names separated by a comma. Example hiding the Submitted by and Survey Date questions.

 

The hide parameter is very flexible, allowing you to hide multiple elements at once. That is, you can hide the navigation bar, the footer, description and as many questions in the survey as you like.   To hide multiple elements, you simply need to keep adding comma separated values to hide the different components. For example:

 

...?hide=field:submittedBy,field:surveyDate,theme,footer

 

Another interesting thing you can do is hide a question and at the same time set its default value through a URL parameter. For example, you can hide the Submitted by question in the Oak survey and also set a value for it through your URL parameter.

 

Redirect the Survey123 web app to your own ArcGIS Enterprise (portal)

 

While the Survey123 web app is hosted in ArcGIS Online, there is no reason why you can't load surveys from your own ArcGIS Enteprise instance.  You do this through the portal URL parameter, which takes as a value the URL of your own ArcGIS Enterprise  Web Adaptor. For example:

 

https://survey123.arcgis.com/share/acb28b212b5542acbbdbaa35feba0765?portalUrl=https://acme.com/gis

 

The above URL will load into your web browser the survey with identifier acb28b212b5542acbbdbaa35feba0765 as found in the https://acme.com/gis ArcGIS Enterprise instance.

 

Set the app that will open your survey (open)

 

You can use the open URL parameter to determine the app to use with survey. Valid values for this parameter include: web, native and menu.

 

https://survey123.arcgis.com/share/1cb28b212b5542acbbdbaa35feba0765?open=menu

 

Parameter Value Description
webOpens your survey in the Survey123 web app, as a web form. This is the default. Example.
nativeWill open your survey in the Survey123 field app. This option will only work if the Survey123 field app is already installed in the device. Example.
menuWill open web page to let the end user decide if the survey should be open in the Survey123 field app or within a web browser. Example.

 

Set the language of your survey (lang)

 

The lang parameter switches the language of your survey dynamically. This parameter only will work if you have included language translations for your survey.  For more details on mutiple-language surveys refer to Introducing Multiple Language Surveys 

 

The lang parameter accepts ISO language codes. For example: lang=zh-cn  and lang=en.

National Weatherperson's Day, also known as National Weatherman's Day, is observed on February 5 primarily in the United States. It recognizes individuals in the fields of meteorology, weather forecasting and broadcast meteorology, as well as volunteer storm spotters and observers. It is observed on the birthday of John Jeffries, one of the United States' first weather observers who took daily measurements starting in 1774. Jeffries also made the first balloon flight over London in 1784 with the purpose of gathering data for a scientific study of the air at high altitudes.

 

Our forecast for the 3.2 release is as follows: Bright like the sun, new features and enhancements in the morning, particularly in the Survey123 website and web designer. A joyful rain of fixes for the Survey123 field app at night, including massive improvements to repeats and nested repeats. 

 

Pages for your web forms

 

In this update you can organize questions in your surveys across multiple pages. Pages are particularly useful when working on very large forms, helping you logically organize your questions and giving end users a sense of progress across the entire survey.

 

 

In the example above, I added a welcome image and text on the first page to create a cover for  my survey. Note that the first page does not expect any input from the survey. It is just a welcoming page, or cover.  Next, I organized the actual questions within my form into pages, giving each page a descriptive title at the top.  As users move from page to page, a progress bar at the bottom shows how close they are to the end of the survey.

 

You can combine groups and pages to further help organize questions within your survey!

 

If you use web designer to create your surveys, you will find that we have added a Page option in the main panel. Just like you do with other question types, simply click on Page, and it will be added to your form. You will be able to drag and drop questions into your new page, move questions from one page to another and even merge and duplicate pages!

 

 

If you want to work with pages in Survey123 Connect, you need to do two things:

 

  • First, in the settings worksheet of your XLSForm, you will set the value of the style column to pages.
  • Then, in the survey worksheet, you will use the field-list appearance for any XLSForm begin_group question type that you want to turn into a page. 

 

Also note that when using the pages style, pages will be automatically created for any question outside field-list groups.

 

Whilst the implementation for pages is complete in our web form, currently in the field app constraint and required checks will not be performed when you leave a page.  These checks will only be done when you attempt to submit the survey. In some scenarios this can make it difficult for end users to identify the exact question in the form where the validation rules were broken. Make sure you define good error messages for your required and constraint rules. Other than that, pages work pretty well in the field app!

 

Embedded public web forms

 

You can now embed public web forms in your own web site. As shown below, the Collaborate tab of your survey will now include an Embed option from which you can copy the HTML code needed to embed your survey within a web site.

 

 

A handful of embed options let you quickly configure your embed HTML code to hide elements of your survey that you may want to hide, such as the header, sign-in, multiple language options, footer, etc.  You can even disable the theme colors to provide a more neutral presence of your embedded survey within the web site.

 

Below is an example showing a Survey123 form embedded within a web page.  Note that the header, footer and themes have been disabled to make the survey fit better within the web site.

 

 

Better word clouds

 

The Survey123 website includes a number of reporting capabilities to help you understand data captured through your surveys. The Analyze page, for example, aggregates responses for categorical, date and number questions and creates charts and maps to more quickly identify trends on your data. 

 

Questions with free text are always trickier to aggregate, but that is where word clouds come to help. A word cloud takes the most common words found in your data and sizes them according to popularity. The bigger the word, the more repetitions. It is a simple concept but it often gives great insight.  Unfortunately, the most common words in our language are far from interesting in this context. Think about it: this, that, of, a...  all pretty useless stuff in a word cloud, don't you think?  These words are known as stop-words, and now you can set a filter to ignore them. In English at least.

 

Here you can see how word cloud filters made a difference for our friends at City of Lakeland, in Florida:

 

In this particular scenario, the survey asked people in the city 'What are your thoughts on how can we better educate drivers to pay attention?'. The most popular words among the responses are 'the', 'and', 'to' and 'of', but if we  apply the stop word filter, much more relevant words will come out:  speed, signs and people.  With stop word filters you can more easily understand certain relevant trends in your data.

 

More on custom themes

 

We introduced Custom Themes not long ago. Definitely a good addition to give your surveys a nice final touch! If you haven't already, I highly recommend you watch this video-tutorial or read the Introducing Custom Themes blog post.

 

 

In this update 6 new theme templates have been added. 5 of them will help you quickly switch through different color combinations. You can also make tweaks in them, of course, but they provide a nice start.  The sixth template is more interesting, in that it will mirror the Shared Theme of your ArcGIS organization.  If you are not familiar with this concept of Shared Theme, have a look at Ian's blog post Introducing Shared Theme: a new app styling capability in ArcGIS Online.  The Shared Theme is set by the administrator of an ArcGIS Organization defining the basics for styling configurable apps in it: text and background color, etc.  Now, you can easily match that style!

 

Web designer: Option to automatically center map at the device's location in web forms

 

A new option has been added in web designer to automatically center the map at the user's location when a geopoint question is open within a web form.

 

 

Lets describe in more detail what is going on here, because this option only makes sense once differences between the behavior of geopoint questions across web forms and the Survey123 field app are understood. 

 

Geopoint questions in the Survey123 field app are by default centered at the user's location. In a Web Form, the behavior is very different.  When your geopoint questions are shown within a web browser, the location of the map remains as set by the author of the survey. The Survey123 web app by default will not attempt to gather the user's location. Otherwise, as soon as the survey is loaded, the web browser will automatically present a dialog to the end user asking for permission to gather the device's location.  Without proper context on why that  message is appearing, this would cause distrust in many users.

 

The user can change the location of the map by either manually panning the map or centering it at a particular address or at the user's location. By enabling this new option, the map, once in edit mode, will automatically center to the user's location.  This will trigger the web browser to require confirmation before handing over the location to the Survey123 web app, but this will happen in context.  Users will understand at that point why the survey is attempting to gather their location.

 

For the reasons described above, this new setting has no effect when the survey is loaded in the Survey123 field app.

 

Other enhancements and fixes

 

Survey123 Website

 

  • BUG-000112899 Publishing a survey on survey123.arcgis.com connected to a Portal for ArcGIS using Integrated Windows Authentication (IWA) returns the error message, "Unable to publish the survey. Unexpected token < in JSON at position 0".
  • BUG-000116898 Renaming the survey under 'Edit Survey Info' in survey123.arcgis.com will cause the feature service to change the name from _fieldworker to _stakeholder.
  • BUG-000117623 When survey123.arcgis.com is accessed via Portal and only one record is present in the service, the default extent and zoom level configured on the survey are not honored when viewing from the Data tab.
  • BUG-000116269 When printing a web map in a custom report from the Survey123 for ArcGIS website, at least one element of the web map must be public, otherwise the error message, "An error occurred while generating report(s)" is returned.
  • BUG-000117239 Survey123 does not honor survey’s viewer collaboration with a group in ArcGIS Online if the survey has been authored through Survey123 Connect using an ArcGIS Server secured feature service added to ArcGIS Online as item with Credentials stored.
  • BUG-000116800 Reports generated from the Survey123 for ArcGIS website show the Object ID as a date.
  • BUG-000116268 When printing custom reports from survey123.arcgis.com, the web map in the report fails to display all layers in the map.
  • BUG-000118284 In Survey123 for ArcGIS, the map scale setting is not honored when including a map in a report.
  • BUG-000118703 Surveys with calculate type questions that use pulldata() on a geopoint question within an if statement display the error "Error: Initialing Form... FormLogicError: pulldata with incorrect number of parameters found: pulldata('@geopoint', // , '')" and load without form logic when opened in the Survey123 Web App.
  • Custom Reports can now access all fields within the underlying feature service.
  • Custom Report syntax in Microsoft Word documents is now checked immediately after uploading the template.
  • Added support for the XLSForm function @property in Web Forms.

 

Survey123 Connect

 

  • Survey123 Connect will no longer overwrite survey settings (such as web hook configurations) updated through the Survey123 website.
  • BUG-000111392 Survey123 Connect installed on a Windows 2012 R2 machine crashes when publishing a survey to Portal for ArcGIS 10.5.1 configured with enterprise logins.
  • BUG-000119026 Delete refresh token stored in SecureStorage if auto login fails with invalid refreshtoken error.
  • BUG-000097165 - Copy images used for form background into the media folder.

 

Survey123 Field App

 

  • Various fixes affecting repeats and nested repeats (BUG-000117852, BUG-000117987, BUG-000118094, BUG-000118449 and many others).
    • BUG-000117852 : Surveys for geopoint, picture and select one questions are not displayed for the second and subsequent repeat questions/related tables in sent folder if those questions are relevant questions.
    • BUG-000117987 : Calculation fields fails to retain submitted values in Inbox if the field are part of "Repeats".
    • BUG-000118094 Relevancy questions within repeats do not save the information when the survey is saved as a draft.
    • BUG-000118449 : When including a signature field in a relevancy controlled group within a repeat in Survey123 Connect, the first signature entered is not displayed in the outbox (drafts).
  • BUG-000117493 "SSL Handshake failed (6)" error is generated in Survey123 app when using Android 7.0 or higher and using non-CA certificates for Portal.
  • BUG-000110786 - Survey123 cannot change from an Windows Authentication (IWA) portal to another.
  • BUG-000113293, BUG-000113294 - User signed out when losing network connection.
  • BUG-000098854: Survey123 App Portal cannot be added after tapping "Add Portal" in Windows Mobile.
  • BUG-000118375 Mathematical calculations from Inbox do not trigger when NaN or null values are used.
  • BUG-000106578, BUG-000112873 - When choice_filter used, the name text is saved as the data value, instead of the label text.

 

Did you know that using XLSForm you can choose to save the label of the selected item in a list instead of the name? For more details see: Survey123 Tricks of the Trade: XLSForm functions for lists 

 

  • ENH-000109073 - Include a comma (thousands separator) in decimal questions.
  • Various bug fixes for localization issues.

 

For more details on the 3.2 release, check out our What's New Help Topic, which also includes an archive of all new features added in previous releases.