Skip navigation
All Places > Survey123 for ArcGIS > Blog
1 2 3 Previous Next

Survey123 for ArcGIS

142 posts

A minor update to the Survey123 website and field app has just been made available. This new release includes a handful of critical issues:

 

  • BUG-000125792: Japanese and Chinese keyboards not working in the field app on iOS 13.
  • BUG-000125769: The Survey123 field app image gallery is unsorted on Android devices.
  • BUG-000125687: Attempting to publish or save a survey in the Survey123 website containing a multiple choice question with the choice name containing more than 32 characters fails with the error, "Errors when parse form."
  • Null geometry is submitted by the Survey123 web app if there’s no Geopoint question in the survey or the Geopoint question is left blank.
  • Corrupted characters being displayed in some languages in the field app on iOS 13.

 

The latest build number for the Survey123 field app on Android and iOS is now 3.6.157. The Windows version of Connect and then field app has not been updated. 

 

The source code of the Survey123 Enterprise Template in AppStudio for ArcGIS has also been updated to version 3.6.

 

In preparation for the 3.7 update, which is planned to be released before the end the year, we will shortly make the Beta builds of the Survey123 website and the field app available in the Survey123 Early Adopter Program.

Josh_Shelton

UAS Flight Log

Posted by Josh_Shelton Oct 8, 2019

UAS Flight Log 

Using Survey123 to perform UAS preflight checklists and track UAS flight projects

By: Josh Shelton, GIS Manager, Pend Oreille County, Washington

      Eric Roth, County Surveyor, Pend Oreille County, Washington

 

UAS Launch Logo

 

Sometimes the most elegant solutions are the simplest. This was the thinking behind using Survey123 to transform what was a pen and paper menial task into a more efficient, portable and user-friendly digital workflow.

In this article, we will discuss our motivations to use Survey123 at Pend Oreille County to support UAS (Unmanned Aerial Aircraft) operations. We will also share details on our implementation, approach to engage with stakeholders and lessons learned.

 

Understanding the need and benefits of using a UAS in Pend Oreille County

 

Pend Oreille County, Washington is a scenic rural county in the northeastern corner of Washington state nestled against the state of Idaho and Canada. Pend Oreille County encompasses approximately 1500 square miles of lush coniferous forest and is divided by the Pend Oreille River, a major tributary of the Columbia River.

 

Pend Oreille County utilizes UAS primarily for gravel pit & stockpile quantity calculations. The benefit here is that our crews spend a third less time collecting data than it would take utilizing conventional survey methods. We also use UAS for roadway pre-design topography maps, bridge inspections, flood tracking, and land slide tracking. By using UAS for these situations, our personnel can operate the equipment from a remote location keeping them safe and out of harm’s way.

UAS Launch Location

UAS launch site prior to flight

Eric Roth, Pend Oreille County

 

Unlocking the potential of Survey123 – Empowering our Users

 

Being a rural county and an organization with less than 200 employees, at the Pend Oreille County GIS group we are constantly trying to think outside the box when it comes to improving workflows while maintaining our need to be fiscally responsible for our constituents.

 

In an effort to better facilitate collaboration and information sharing, the Pend Oreille County GIS staff formed an internal GIS user group in 2009. This group discusses new technologies, showcases new projects and shares ideas empowering our users to improve the way they work with GIS.

 

This is exactly what happened when the GIS Department showcased Survey123 to the group for the first time. We discussed the power and simplicity of Survey123 and showcased how easily smart forms can be created without the need for advanced GIS or developer skills. When our county surveyor saw the technology he immediately began thinking of ways he could leverage Survey123 to improve his own workflows. This ultimately led to the creation of the UAS Flight Log.

 

FAA regulations mandate that all UAS (Unmanned Aerial System) pilots perform and document a thorough preflight check on their equipment and surroundings. UAS flight logs must also be documented.

Commonly, preflight checks and flight logs are manually recorded on paper. Completing these paper forms is cumbersome and can lead to errors and even data loss. There are several reasons why Survey123 smart forms are a natural replacement for obsolete paper forms: 

  •  Survey123 runs on the same mobile devices that everyone already has. There is no need for an additional piece of paper when the form can be completed on your own mobile phone or tablet.
  • Survey123 smart forms streamline data capture, allowing UAS pilots to focus in areas of the preflight check and flight log that require more attention.
  • Certain critical information such as location, times and photos are particularly easy to capture with Survey123 in a systematic and accurate way. That is the opposite of what happens with traditional paper forms.
  • Survey123 forms can be easily synced with our ArcGIS organization. This reduces chances for data loss, eliminates transcription errors, and facilitates immediate access to this information for visualization, reporting and sharing.

 

The nuts and bolts of our survey design

 

The survey was developed entirely using the Survey123 web designer at survey123.arcgis.com. The basic premise was to take the preflight checklists already being performed and recreate these in the survey. The added benefit here is that one could integrate post-flight observations as well as log our flight location and record the event with photos. The survey design is split into 4 logical sections:

UAS Flight Log
Form Summary

 

  • Flight Details – Here we record the following:
  1. Date of capture – This auto populates further simplifying the process
  2. Location – Uses device GPS to provide location or the user can select from the map.
  3. Purpose of flight – Comment field allowing for a customized description
  4. Photo of the UAS at the launch location – This helps to prove project location and UAS used, as well as the condition of the launch location
  5. Weather conditions – Comment field for customized descriptions
  6. Name of RPIC (Remote Pilot in Command) – Comment field for customized entry
  7. UAS pilot name, if different than the RPIC – Comment field for customized entry
  8. VO (Visual Observer(s)) names, if applicable – Comment field for customized entry
  • UAS Details – Here we record the Make and Model of the UAS.
  1. UAS Manufacturer – prepopulated dropdown containing the manufacturer of the UAS being used
  2. UAS Model – prepopulated dropdown containing the model of UAS being used
  • Pre-Flight Check List – This is a simple check list where the following items are “checked off” as they are inspected:
  1. Batteries charged and secure?
  2. Aircraft Hardware OK?
  3. Equipment & Gear OK?
  4. Remote Control OK?
  5. Propellers OK & Secured to UAS?
  6. Software & Firmware Versions Up to Date?
  7. Remote Control is Powered ON?
  8. UAS is Powered ON?
  9. UAS Compass is Calibrated?
  10. Camera is ON & Calibrated?
  11. UAS GPS is Functioning?
  12. Flight Control App is Running & Connected to UAS?
  13. Pre-Flight Notes – Here any ancillary information can be noted
  • Post-flight details – here we record flight start/stop times and note any post-flight observations
  1. Begin Flight – record the time the flight began
  2. End Flight – record the time the flight ended
  3. Post-Flight Notes – Records any additional observations

 

For those of you interested in seeing what we've done, here is the XLSForm

 

Using UAS for road and bridge inspection

Eric Roth, Pend Oreille County

 

Lessons Learned and next steps

 

From a GIS management perspective, the ability for our non-GIS personnel to develop and put into production their own data collection tools has been essential to success. Inspiring and empowering county personnel to implement GIS technology for their own needs has allowed GIS personnel to focus efforts on other tasks which help our organization improve the information, tools, and services we provide and utilize. It’s been said that necessity drives innovation and in this case the need to simplify a simple redundant workflow is proof that there is room for improvement in the seemingly most simplistic of tasks.

 

The use of Survey123 for UAS operations validated major benefits.

 

  • Ease of use – The intuitive design of the Survey123 mobile app facilitates rapid, efficient data capture with minimum training.
  • Flexibility – Using Survey123 web designer, smart forms can easily and timely be modified to match new requirements.
  • Reliability – Paper forms can easily be damaged or misplaced and handwritten notes are often illegible or difficult to decipher. Survey123 form data is directly stored in ArcGIS as soon as the mobile device as network connectivity. Using ArcGIS Online provides us with security and peace of mind that the data will be there should the need arise where we need to go back and review prior projects.
  • Portability – The Survey123 app is supported on a wide range of devices and platforms allowing us to easily make use of existing devices such as smartphones, tablets, etc.
  • Offline capability – Our operating areas are rugged and remote. Reliable network access is not always available making offline collection a necessity.
  • Longevity – Since the records are stored digitally within our ArcGIS Online organization and can be manually archived, we can guarantee access to the data into the future increasing the value and return on our investment.

 

Not only were we able to achieve our goals but we’ve helped reduce the time it takes to prepare and perform a flight. We can now get up to date imagery of our public works projects, use the imagery in conjunction with Drone2Map to calculate volumes of sand and gravel piles before road projects and in preparation of sanding roads in the winter. We’ve also helped to improve the safety of our employees by using the UAS to perform inspections on bridges that are often difficult and unsafe to survey manually. This critical return on our investment has helped to solidify the sustainable use of UAS technologies in our everyday workflows. Talks are currently underway to expand the use of UAS technologies to other departments in the hopes that it will add further improvements to our workflow processes.

 

Josh Shelton is the GIS Manager for Pend Oreille County in Washington state. Josh has been doing GIS and supporting the county for over 11 years. Pend Oreille County GIS supports all county departments and strives to help empower people to use GIS technologies to better themselves, the organization and the communities in which we live. Josh loves the outdoors and has an enthusiasm for technology, it’s this passion and drive which enables him to continually push the boundaries and think outside the box providing technologies and solutions to county personnel and their constituents.

 

Eric Roth is the County Surveyor for Pend Oreille County. Eric has been a licensed surveyor in Washington state for over 15 years and has been working for Pend Oreille County for the last 3 years. His love of surveying and technology led him to create the first UAS program in the county. Since the program’s inception, Eric has been working on ways to make the program more efficient, safe and cost effective.

According to The Hobbit and The Lord of the Rings trilogies written by J.R.R. Tolkien, Bilbo and Frodo were born during the Third Age in the years 2890 and 2968 (of the Shire calendar) respectively. Translated to our Gregorian calendar, Bilbo was born in 1290 and Frodo in 1368. Bilbo and Frodo share the same birthday on September 22 and that makes this day a very special one: Hobbit Day.

 

Go back?” he thought. “No good at all! Go sideways? Impossible! Go forward? Only thing to do! On we go!”
Bilbo Baggins, The Hobbit, Riddles in the Dark.

 

Survey123 version 3.6 is now available to all, even the Elves!

 

Critical fixes

 

This update includes over 30 critical bug fixes in the Survey123 web app and field apps. An area or particular focus has been ensuring a more correct and consistent behavior of your XLSForm expressions. For example, dealinig with relevant statements and submitted data. The general rule is that data should not be submitted from non-relevant questions. That is, the output of your calculations or default values set, will never be sent to ArcGIS for questions where the relevant expression evaluates to false. In addition, with this update relevant statements apply to hidden and calculate questions. Thus, the use of relevant expressions is reinforced not just as the means to hide questions, but also to better control what data is submitted to the target feature layer.

 

You will also find a list of device-specific fixes in this release. Barcode scanning capabilities have been greatly improved on Android devices, as well as other improvements to keyboard and input behavior. Others fixes relate to the new iOS 13 update, which introduced some changes affecting the Survey123 field app.

 

A list of critical BUGs (and relevant enhancements) addressed in this release can be found at the end of this doc.

 

 

Lines and polygons

 

You can now work with line and polygon GIS features within the Survey123 apps. This is possible through the new geotrace and geoshape XLSForm question types, which you can incorporate into your surveys through Survey123 Connect for ArcGIS.  

 

Geotrace and geoshape questions are similar to geopoint questions in that they are shown in your form as a map. The only difference is that with geotrace and geoshape you can create and update lines and polygons respectively.

 

Support for geotrace and geoshape is available in both the Survey123 field and web apps. That is, you can work with lines and polygons from a web browser and also from the native app. The website has also been updated so you can view, filter, analyze and even create feature reports with line and polygon data.

 

There is quite a lot to know when working with lines and polygons in Survey123. You can find more details in our Survey123 Tricks of the Trade: Lines and Polygons post.

 

 

 

 

Single choice grid in the field app

 

We introduced single choice grid in the Survey123 web designer and the web app at version 2.8. We are now expanding support to Survey123 Connect and the field app. Single choice grid allows you to compact series of choice list questions, reducing the need for end-users to scroll up and down the form. They can also help you make your forms more easily readable. The key to using single choice grid is a sequence of more than 2 or 3 questions sharing the same list choices. In XLSForm jargon: A sequence of select_one questions sharing the choice list. This new appearance is ideal when you plan on using forms with a tablet, although keeping labels short and choices limited can give you good results even in smart-phones.

 

In the next animation you can see a single choice grid compacting a set of 5 building condition questions. Note how the choices are labeled just once for the entire set. 

 

 

In XLSForm, you can add a single choice grid by simply enclosing your select questions within a group and setting the appearance of the group to table-list.

 

typenamelabelappearance
begin groupassessment_groupReport building conditiontable-list
select_one evaluationassess_collapseCollapse
select_one evaluationassess_leaningBuilding leaning
select_one evaluationassess_structuralStructural damage
select_one evaluationassess_fall_hazFalling hazards
select_one evaluationassess_group_movGround slope movement
end group

 

Please note that groups using the table-list appearance will take the whole width of the form when your form uses the theme-grid style.

 

Feature report enhancements

 

Survey123 feature reports allow you to create high quality printable documents from your Survey123 data. The layout, content and look and feel of your reports are defined in a Microsoft Word document template. For a basic introduction to Survey123's feature reports, refer to the Understanding Survey123 Feature Reports blog post.

 

 

This update includes a few refinements to the feature report syntax. You can now:

 

  • Control how many decimals should be shown in a number.  ${myNumber | round:2}
  • Extract the elevation of a geopoint question. ${myGeopoint | getValue:"z"}
  • Get the x/y coordinates of a geopoint in a particular spatial reference: ${myGeopoint | getValue:"x":26865}.

 

The template syntax validator has also been improved to handle some logical errors that were not handled before.  In the past, these invalid expressions would pass the syntax validation and then be ignored at run-time.  Starting with this release, you will get a validation error when you attempt to upload the template. For example, the following expression is incorrect:

${if image_field_1 | getValue:"size">0}

A parenthesis should be added as follows:

${if (image_field_1 | getValue:"size")>0}

Or alternatively, you could use:

${if image_field_1}

Another common syntax error is as follows:

{if choiceQuestion1="choice1"}

The equal sign above is used as an assignment. A double = sign should be added  for a comparison:

{if choiceQuestion1=="choice1"}

A side effect of this more strict template syntax validation is that existing templates you may have uploaded will start triggering 'parsing errors'.

        

The complete feature report syntax is described at: Feature report templates—Survey123 for ArcGIS | ArcGIS 

 

If you happen to be using Survey123 web hooks and Integromat with ArcGIS Enterprise, you will notice that you can now configure the Survey123 Integromat module to create Feature Reports with your own ArcGIS Enterprise instance.

 

Finally, be aware that coinciding with the upcoming ArcGIS Online update in October 2019, each feature report generated will consume 2.5 ArcGIS Online credits. Refer to Understanding Survey123 Feature Reports for more information.

 

Note to ArcGIS Online users: Coinciding with the upcoming ArcGIS Online update in October, Survey123 Feature Reports will consume ArcGIS Online credits.

    

 

Web form view-only mode

 

In most cases, you want to use a Survey123 form to help people capture data. Using web form URL parameters, your survey can be launched with some predetermined responses. You can also use URL parameters to load data from existing records into a survey and let people update that information. With this update, you can also open a survey in read-only mode. This can be done in two steps:

 

  • Go into the Collaborate tab in the Survey123 website and share your survey results (Viewer tab) as appropriate.

 

  • Build a URL setting the mode parameter to view, and pass the objectid of the record you want to display.

https://survey123.arcgis.com/share/1cb28b212b5542acbbdbaa35feba0765?mode=view&objectId=10

If you get an error like shown below, make sure  you share your survey results through Collaborate as described in step 1 above.

 

 

As the mode=view parameter is only introduced with 3.6, you will need to republish old surveys to take advantage of this feature with them. Alternatively, you could also add the version=latest parameter to dynamically upgrade your survey when it loads in the user's web browser. However, it is generally best to republish the survey with the latest version.

 

Extended XLSForm support in the Survey123 field app

 

With this update, we are introducing a handful of very specific enhancements to our XLSForm support in the Survey123 field app. These new features are currently not supported in the Survey123 web app. Demand for these types of capabilities is much more general among Survey123 field app workflows, so we are introducing them in that context exclusively for now.

 

 

Range question

 

The range question is designed to restrict integer or decimal user input to a well defined range. If you setup your survey on top of an existing feature layer, range questions are a perfect fit for fields with a range geodatabase domain. Range questions display in your form as a horizontal slider, ideal for touch screens, although when the control has the focus you can also change its values using the left and right arrow keys.

 

In the screenshot on the right, you can see a range question being used to help users document pH values on a water sample. Values can be easily set between the pH range of 0 (acidic) and 14 (alkaline).

 

Through the XLSForm spec, you can dictate different aspects of the slider such as the minimum and maximum accepted values and the colors of the control. Like other XLSForm questions, you can apply calculations, data validation constraints, default values, etc.

 

Check out the Range help topic for more details.

 

 

Custom height for signature, text (multiline), and geo questions

 

You can now set a custom height for multiline text, signature and geo (geopoint, geotrace, geoshape) questions. This is particularly helpful when you arrange multiple questions in a single row using the theme-grid style, allowing you to adjust the height of different controls to keep your design more symmetric. In the case of text input, expanding the default height also helps signifying to the end-user that large amounts of text can be inputted.

 

The height is set using the body::esri:style column, which has been added to the far right of the XLSForm survey sheet. You can also manually add this column to old XLSForms you may have.

 

typenamelabelappearancebody::esri:style
geopointlocationLocationheight=15
textcommentsCommentsmultilineheight=7
imagesignatureSignaturesignatureheight=20

 

The height value unit is expressed as lines by default, where a line is the height of the current input text font. Alternative units include % (height=25%), where the height is calculated as a percentage of the current window height, and pixels (height=300pixels). Remember not to add spaces between the unit and the value!

 

Pictorial lists

 

There are a few common ways to represent a list of choices: a drop-down list, horizontally, vertically... etc. Carefully choosing the best style for a list, is important. A horizontal arrangement is a good choice for short lists, reducing the need for users to scroll the form up and down. Vertical lists present options in a very clean way, but if you have many choices, a drop-down list will save users time, particularly if you enable auto-complete.

 

There are some cases, where a pictorial representation of a list is by far the best option. Brett Stokes put together a nice collection of examples, which you can see in the animation below. In all these examples, making selections over the images comes more naturally than making the selection over a traditional list of choices. It is more intuitive, can be less prone to errors and quicker for the user to make their selection.

 

 

To learn about including selectable images in your smart form, refer to the Using the image-map appearance in Survey123  post by Brett. Well worth a read!

 

Critical fixes and minor enhancements

 

Survey123 Field app (XLSForm support)

 

  • BUG-000097105 Default values are present in fields that are not relevant.
  • BUG-000124014 Non-relevant repeats are being submitted with values, if the default values or calculations are applied within the repeat.
  • BUG-000123547 Unable to calculate geopoint locations when repeating a survey in the same survey form.
  • ENH-000123296 Hidden and Calculate fields should support relevant expressions.
  • ENH-000105822 Provide ability to create surveys using geometry types other than points in Survey123 (lines, polygons).
  • BUG-000124284 Survey123 Connect for ArcGIS 3.5.x and Survey123 Field App 3.5.x do not hide questions accurately if a relevant column value is calculated on a complex pulldata expression.
  • BUG-000124360 When manually entering Lat/Long for a geopoint location on a 'repeat' question, the 'device location' preview window does not show the device's location until the second repeat and onwards.
  • BUG-000124121 If the spatial filter is enabled for a survey with a 'Query expression' in the 'Inbox Filter', records are omitted even if the records are within the map extent before clicking Refresh.

 

Survey123 Field app (device specific issues)

 

  • BUG-000125295 The Survey123 Field App freezes/crashes when attempting to attach an image from the local device on iOS 13.1 Beta.
  • BUG-000121080 Barcodes cannot be scanned in the Survey123 application on Samsung Galaxy Tab E.
  • BUG-000123872 Survey123 Field app crashes upon launch on Android 5.1.1.
  • BUG-000123228 The keyboard of Android devices with 5.2 inches or lesser screen size disrupts the Survey123 for ArcGIS app sign in page.
  • BUG-000123572 Unable to connect error in Survey123 Field App 3.5.164 on iOS 12.3.1 device when Allow Location Access setting is set to "Never" or "While Using the App".
  • BUG-000115243 The final page in Survey123 for ArcGIS App 3.0 installation wizard on Windows 10 German OS contains an action text that does not match the text indicated on the button below the text.

 

Survey123 Field app (Usability)

 

  • BUG-000105325 Survey123 for ArcGIS maps do not automatically open to the Home extent as default behavior.
  • BUG-000120565 In Survey123 for ArcGIS, Inbox icons in the Map View randomly turn orange or disappear after a point has been edited and sent, or when the Inbox is refreshed within the Map View.
  • ENH-000107453 When using the Tab key to move through text questions in Survey123 field app on the Desktop, the survey does not scroll automatically.
  • BUG-000124636 Editing the Signature field submitted using the Survey123 for ArcGIS field app creates a permanent black bar over the response.

 

Survey123 Web app

 

  • ENH-000113292 Allow the pulldata(@property, utcoffset) function to be used in Survey123 web forms.
  • BUG-000124448 Survey123 webapp version 3.5 fails to return addresses from geopoint using the pulldata() function when geopoint and address questions are in repeat.
  • BUG-000122210 Reverse geocode fails to return results in the Survey123 web app when using a custom geocoder.
  • BUG-000123616 In Japanese language, the data does not show correctly for multiple choice questions in the Survey123 for ArcGIS web app.
  • BUG-000122892 When submitting the form several times in Survey123 for ArcGIS, the sentence mentioned for submitting another survey is misleading in French.
  • BUG-000123376 The setting 'Show message to prompt the user to submit another record' is not honored when the survey is created from Survey123 Connect for ArcGIS.
  • BUG-000118801 An existing secured ArcGIS Server feature service added to ArcGIS Online with credentials saved is not be able to attach images when using the web browser to submit the survey (the data is submitted, but not the attachments).
  • ENH-000118845 Survey123 GeoPoint questions should have the option "Explore" on the Web Map, which activates zoom/pan on the Map.
  • Multiple usability enhancements added to the map control, including an option to enlarge the map when working in a desktop computer and making map navigation easier.

 

Feature reports

 

  • BUG-000111657 Photos submitted in landscape orientation in the Survey123 field app, result in poor quality when individual responses are printed or saved to PDF from the Survey123 website.
  • BUG-000125306 Conditional statements for images with the relevancy setup are not honored in a feature report in Survey123 for ArcGIS.

 

Survey123 Designer and website

 

  • BUG-000123663 The Data tab fails to zoom to the point extent initially for Survey123 for ArcGIS website.
  • BUG-000124275 Ordnance survey basemaps that are specific to the UK act fuzzy within the Survey123 website.
  • You can now manually set choice names for multiple choice questions in the the Survey123 designer.
  • BUG-000125107 The Analyze tab at survey123.arcgis.com does not show the correct Bar Chart reports and percentages for surveys with multiple choice-questions configured with the Batch Editor option when the language for ArcGIS Online Organization is set to 'Japanese - 日本語'.

 

Security

 

  • BUG-000124114 In an organization that does not allow HTTPS only, if a Survey123 for ArcGIS form is accessed via the item details page, the error message, "Request needs to be made over HTTPS" is returned.
  • BUG-000122890 Survey123 for ArcGIS sends a query that is logically correct but appears as a potential SQL injection attack to firewalls.

 

Documentation

 

  • ENH-000124633 Update Survey123 documentation to inform that 'Hints' are not supported within 'Repeats'.
  • BUG-000124657 Update Survey123 with ArcGIS Enterprise documentation to provide more details on the limitations of creating feature reports with feature services hosted in Portal.

Starting with version 3.6, Survey123 supports working with line and polygon GIS features within your smart forms. There are many common scenarios for creating new, or working with existing, line and polygon GIS features in Survey123. For example:

 

  • To support a noxious weed control program, you need a form to report areas where certain chemicals are being applied. You will want your form to include a question for users to delineate the exact area (polygon) where treatment occurred.
  • You want to create a digital form following the SCAT methodology for assessing the impact of an oil spill along a shoreline. Your form must be able to let users highlight the segment (line) of the shoreline, or path, that was assessed.
  • Let's pretend you want to expedite the completion of ATC building inspection forms against an existing collection of building GIS features. In this case, you will want users to open an inspection form for the selected building (polygon) and let them complete all other details.

 

This blog describes how you can add support for line and polygon GIS features in your Survey123 smart forms using Survey123 Connect and XLSForm.  If you are not familiar with Survey123 Connect and XLSForm, please have a look at our Survey123 video tutorials, our XLSForm essentials—Survey123 for ArcGIS | ArcGIS help topic, or the Survey Customers to Gain Marketing Insight | Learn ArcGIS Learn lesson.

 

Support for configuring map questions in the Survey123 web designer to capture line and polygons is planned for a future release.

                            

The Basics

 

The basics: geoshape and geotrace in Survey123 Connect

 

Hopefully by now you are familiar with the XLSForm geopoint question type. It allows you to bring a map question into your survey, so end users can define a point location of interest as part of your form. The geotrace and geoshape XLSForm question types will also be shown as a map within your form, allowing users to work with line and polygon shapes respectively.

 

To get started, I suggest you build a new test survey with Survey123 Connect and add the following questions:

 

typenamelabel
datetreatment_dateTreatment Date
geoshapetreatment_areaTreatment Area

 

If the geoshape question does not render correctly when previewing your survey, make sure you download and install the latest version of Survey123 Connect.  Geoshape and geotrace questions are only supported with version 3.6 and above.

                              

Next, publish your survey to test your geoshape question within the Survey123 field app or web app. If using the field app, make sure you have version 3.6 or later installed on your device. You should be able to delineate a treatment area and submit. Using the Survey123 website Data tab, you will be able to look at your newly created features.

If you attempt to use geoshape or geotrace questions in forms authored prior to 3.6, keep in mind that the geoshape and geotrace options will not show in type column choice list. If you want to make these new question types appear in your old form, have a look at the Survey123 Tricks of the Trade: Microsoft Excel blog post.

                            

The basics: geoshape and geotrace questions in the Survey123 apps

 

You can use geoshape and geotrace questions in both the Survey123 field app and web app. The user experience in the native app and web browser are very similar: A map is presented to the end user with tools to sketch or digitize the corresponding shape (line or polygon).

 

Here is what using a geoshape question looks like from a web browser (Survey123 web app):

 

 

And here is the experience for capturing a line in the Survey123 field app using a geotrace question:

 

 

You will notice that the map control shows an info bar describing the length and area of the shape digitized. The units in this info bar are set by your device locale. If your locale is set to the United States, imperial units will be used. Otherwise, metric units will be shown.

 

The basics: Looking at captured data in the Survey123 website

 

You can view and perform QA/QC operations on all Survey123 captured data using the Survey123 website. Through the Data tab you can visualize data captured through the geotrace and geoshape question types in a map and individual response views.

 

 

You can also bring your line and polygon data into your own custom Survey123 feature reports.

 

Beyond the basics

 

Adding multiple geo type questions in a survey

 

Values for geo type questions (geopoint, geoshape, geotrace) are stored in the SHAPE column of your surveys feature layer. The ArcGIS geodatabase model only allows for one single SHAPE (geometry) column per feature layer, so this restricts the number of geo questions you can add to a survey to one: either one geopoint, or one geoshape or one geotrace question. The only exception is when you add repeats to your survey: each of your repeats can also include its own geo question.

 

For example, with our SCAT survey, lets pretend you want to use a geotrace question to document the segment of shoreline surveyed and then you want to flag locations where severe oil damage has been found. You could create a survey as follows:

 

typenamelabel
datesurvey_dateSurvey Date
geotraceshoreline_surveyedShoreline segment surveyed
begin repeatsevere_damageSevere damage observations
geopointissue_locationLocation of damage
textissue_commentsComments
imageissue_photoPhoto
end repeat

 

Note that the survey includes two geo questions: one in the main body of the survey, and one inside a repeat. This is allowed because questions within the repeat will be stored in a separate (and related) layer.

 

Input Methods: vertex vs sketch

 

When using geoshape or geotrace questions, you can optionally define the input method of your map.That is, you can control which style of tools will be presented in the map to help users delineate a shape.  You can do this through the body::esri:style column in the survey XLSForm sheet.

 

The map supports to input methods:

 

  • sketch: Enables drawing tools in the map. This is meant to help users quickly draw, or sketch, a line or polygon in one single take.
  • vertex: This input method is more appropriate when you want users to digitize shapes more accurately. The vertex input method provides tools to add vertices one at at time by:
    • Tapping on the map.
    • Centering the map at a target location.
    • Getting the coordinates through the built-in or external GPS.

The vertex method also lets users switch between edit and navigation modes, update and delete existing vertices.

 

You will want to experience on your own each of these two input methods and decide what is best for your particular workflow. In some cases you want a simple and quick way for people to sketch a geometry. In others you want to give them more control, even if that implies some initial trial and error with the tools until they get familiar with them.

 

The following example shows how you can set the vertex input method using the body::esri:style column:

 

typenamelabelbody::esri:style
datetreatment_dateTreatment Date
geoshapetreatment_areaTreatment Areamethod=vertex

 

 

XLSForm functions: Calculating perimeter, area and length.

 

Along with the geotrace and geoshape question types, we have also extended XLSForm support, so you can create expressions to work with these geometries. Two common functions we anticipate you using are area and length. Back to our noxious weed abatement scenario, the area function could be used to automatically compute the area treated.

 

typenamelabelcalculationbind::esri:fieldType
datetreatment_dateTreatment Date
geoshapetreatment_areaTreatment Area
hiddentreatment_sqmCalculated Area in sqmarea(${treatment_area})esriFieldTypeDouble

 

Note that in the example above, the calculated area is stored in a hidden field. This is optional, as you could be using a decimal question type instead. You can use a hidden question to keep the output value without presenting it to the end user.  Hidden questions are mapped automatically to text questions in the geodatabase but since I want to store the area as a number, I added esriFieldTypeDouble value to the bind:esriFieldType XLSForm column.

 

The output of the distance and area functions is always in meters (or square meters), but you can easily apply conversions and rounding as needed.

 

round(area(${treatment_area}),2)Extracts the area in square meters of a geoshape and rounds the value to 2 decimals
distance(${treatment_area})Calculates the perimeter of a geoshape or the length of a geotrace. Output in meters.
distance(${shoreline_surveyed}) div 1000Gets the distance of a geotrace geomery in meters and divides by 1000 to get the distance in kilometers.
distance(${pipe_to_install}) * ${price_per_meter}Estimates the installation cost of a pipe.

 

The distance and area functions can also be used in other contexts. In the example below, we use the function to control if a particular question will be required or not. If the area treated is less than 100 square meters, then a justification text question will be shown, but it will not be compulsory. If the area is larger than 100 square meters, then a justification must be provided.

 

typenamelabelrequired
datetreatment_dateTreatment Date
geoshapetreatment_areaTreatment Area
textjustificationJustificationarea(${treatment_area})>100

 

More on XLSForm functions:  repeats, sum and count.

 

With some creative thinking, there are some more advanced XLSForm techniques you could leverage when working with geotrace and geoshape questions. In the next example, a repeat is used to capture the exact location of electric poles. For each pole, type, observations and some GPS metadata for the pole location is collected. Using the sum function, a line feature representing the electric line joining all the poles is automatically generated.

 

typenamelabelcalculation
datesurvey_dateSurvey Date
begin repeatpolesPoles
geopointpole_locationPole Location
hiddenpole_haAccuracy (m)pulldata("@geopoint", ${location},"horizontalAccuracy")
select_one typespole_typePole Type
textpole_obsObservations
end repeat
geotracepole_lineElectric Linesum(${pole_location})
hiddenpole_count# Poles in Linecount(${pole_location})

 

While not included in the example above, it would make sense to set the geotrace question as readonly, to ensure that the electric line is truly following all the pole locations within the repeat. Otherwise the end user could open the map for the pole_line question and manually alter its geometry. Depending on your workflow, the desired behavior is up to you and can be configured either way.

 

Geotrace and geoshape in the Survey123 field app Inbox

 

So far, we have explored how to use the geotrace and geoshape questions to enter new line and polygon features. In some scenarios however you may want to simply reference existing line and polygon features and help drive data capture workflows within Survey123.

 

A classic example is that of building inspections. For example if you need to score a set of buildings to assess fire risk, or to evaluate damage after a hurricane or earthquake. Your intent in this scenario is not to create or update the polygon footprints. Instead, what you want is to update attributes in these existing buildings based on your own on-site assessment.

 

In this case, you will want to build your survey on top of the existing building (polygon) layer. The geoshape question will help you drive the whole user experience in the Survey123 field app, allowing the selection a building from the Inbox map and opening the assessment form for the selected building right away. When submitting your survey, the polygon shape of the building will remain untouched, but the attributes reflecting the assessment will change.

 

The next animation shows the whole workflow: 1) Bring features (polygons) into the Inbox, 2) Select a feature from the map to open its corresponding form, and 3) Submit changes to update the feature:

 

 

If this pattern interests you, keep in mind that the Survey123 field app Inbox is meant to work with a small number of features. When enabling the Inbox in Connect, I strongly recommend you set a filter to query and filter the features based on attributes (buildings pending inspection, or buildings assigned to the logged-in user, for example). From a practical perspective, with 3.6, you will not want to load more than 200 or 300 surveys in the Inbox. You will see some new performance improvements for the Inbox coming in future releases, but as of version 3.6, you will need to keep the number of records loading in the Inbox as low as possible.

 

Performance improvements to the Inbox are coming with version 3.7. As of 3.6, keep the number of records in the Inbox under a couple of hundred by applying spatial and attribute queries to the Inbox in Survey123 Connect.

 

For a more detailed description on the Inbox, check our Prepare for editing existing survey data—Survey123 for ArcGIS | ArcGIS help topic.

 

Including lines and polygons in your Feature Reports

 

Using Survey123 Feature Reports you can create high quality printable documents for any of your Survey123 records. TheUnderstanding Survey123 Feature Reports blog post describes in more detail how to use this feature. The Feature Report syntax has been extended to allow you to dynamically include in your report the perimeter and length and area properties of a survey record. For example:

 

  • ${segment_surveyed | getValue:"length":"meters":"geodesic"}
  • ${treatment_area} | getValue:"area":"hectares":"geodesic"}

 

The length property returns the length of a line and the perimeter of a polygon. The area property only applies to polygon features. The length unit options are feet, kilometers, meters, miles, nautical-miles and yards, for length. For area: acres, hectares, square-miles, square-kilometers, square-meters, square-feet and square-yards. The last property defines if you want to get the planar or geodesic measurement. If your data is in WGS 84 Web Mercator Auxiliary Sphere, which is going to be the case with Survey123 unless you set your own custom projection on the data, you should always use geodesic. If your data uses a local projection, typically planar will be the best option.

vehicle damageStarting with Survey123 3.6, the field app now supports a new appearance of image-map. This new appearance allows Scalable Vector Graphics (SVG) files to be rendered in your survey form and provides a clickable image for both select_one and select_multiple questions. Choice list selections can now be made by simply selecting different parts of the image. The image-map appearance is currently not supported in the Survey123 web app.

 

So what is an SVG? An SVG file is an XML-based vector image format for two-dimensional graphics, that has support for interactivity and animation. SVGs also have the ability to have raster image formats such as .png and .jpg embedded within them.

 

The image-map appearance provides a powerful way to customise the UI of your survey without having to write a single line of code. This enables fast and intuitive data collection with the underlying user selections stored in the feature service. The 'Image-map Sample' survey, available in Survey123 Connect, shows a variety of use cases including Risk Assessment, Body Parts, USA States and Soil Texture.

 

The animation on the right shows a Vehicle Damage Inspection survey that combines the image-map appearance with the photo watermark feature. This enables the inspection photos to be stamped with the corresponding vehicle part for easy identification later. 

 

These examples involve relatively complex shapes and highlight what is possible, but the most simple of SVG files can also provide real improvements to survey usability. The ‘big button’ experience is easily set up with a basic SVG file. Also included in the 'Image-map Sample' survey is a ‘Level of Difficulty’ example that is essentially just an SVG file containing 4  clickable <path> elements (plus some text and symbols for ease of use).

 

 

Implementing the image-map appearance

The following steps relate to the 'Level of Difficulty' example in the 'Image-map Sample' survey. You can follow this process using the 'difficulty.svg' file attached to this blog.

 

1) Add a select_one or select_multiple question to your XLSForm (more information on this step).

 

2) Set up your survey choice list. Each entry needs a list_name, name and label. The values in the name column are the ones that link the choices to parts of your image.

 

list_namenamelabel
difficultybeginnerBeginner
difficultyintermediateIntermediate
difficultyadvancedAdvanced
difficultyexpertExpert


3) 
Create an SVG file or assess an existing file for compatibility with Survey123. The second half of this blog goes into more detail on creating SVG images, for now just open 'difficulty.svg' (attached) in a text editor to assess it's compatibility.

  • Ensure that your file has <path> elements with id values that match the name values in your choice list. The <path> elements with id values become the clickable areas of the image.
    <path
       d="M 5.4998702,5.4998718 H 129.50013 V 194.50013 H 5.4998702 Z"
       id="beginner" />

    <path
       d="M 135.48042,5.4804192 H 259.51959 V 194.51958 H 135.48042 Z"
       id="intermediate" />

    <path
       d="M 265.48041,5.4804192 H 389.51956 V 194.51958 H 265.48041 Z"
       id="advanced" />

    <path
       d="M 395.48041,5.4804192 H 519.51959 V 194.51958 H 395.48041 Z"
       id="expert" />
  • Non-clickable areas are <path> elements without id values and other element types eg <rect>, <circle>, <text> etc.
  • See below for more detailed SVG requirements and tips for using with Survey123.

 

4) Copy your SVG file into the media folder of your survey. The media folder is located in the survey folder  (~\ArcGIS\My Survey Designs\{survey name}) which can be quickly accessed using the Files button in Survey123 Connect. If the media folder does not exist, you need to create it inside the {survey_name} folder.

 

5) Add image-map to the appearance column, and the name of your SVG file (include the .svg file extension) to the media::image column of your select question.

 

typenamelabelappearancemedia::image
select_one difficultydifficultyLevel of Difficultyimage-mapdifficulty.svg

 

That's all there is to it!

 

Using SVG files in Survey123

While implementing the image-map appearance in Survey123 is simple, the contents of the SVG file itself can get confusing. SVG files can be created in a multitude of ways and each method may introduce issues when trying to use within Survey123. The remainder of this blog post will look at the most common ways of creating an SVG file and the associated tips and tricks for getting them to work in Survey123.

 

Minimum SVG requirements for Survey123:

  • Must contain a <path> element that has an id value. Non-<path> elements such as <rect>, <ellipse>, <circle>, <image> and <text> will not be clickable. Likewise, <path> elements without an id value will not be clickable.
  • No empty groups. Having empty groups ie <g> </g> within your SVG makes all <path> elements become un-clickable.
  • Can only have 1 group of <path> elements per nest level. You can have as many nested levels as you like, but if you have more than one group at a particular level (including the parent level), all of those <path> elements and their descendants become un-clickable.
  • SVG height and width (Viewport) must be defined. While your image will likely display properly in Survey123 without these, your clickable areas may become offset from the underlying image.
  • Viewbox must match Viewport (if specified). You don't need to specify a 'viewBox' parameter in your <svg> element. However, if you do specify a viewBox that differs from your height and width values, your clickable areas will be offset from the underlying image.

Other tips for SVG files:

  • Transform functions. It is possible to have SVG elements with transform functions work perfectly in Survey123. However, we must understand that transforms have a cumulative effect when applied to nested elements. It is often best to avoid transform functions if possible.
  • Number of clickable areas. Although most web browsers can render SVG files with up to 5,000 elements without  any issues, these are obviously not practical for use in Survey123. Be mindful that each clickable <path> element needs to be large enough on the screen for the user to select. You'll need to consider what platforms, devices and screen sizes you're using Survey123 on, to decide on a practical limit that suits you.

 

Downloading the perfect SVG from the web!

So you’ve found the perfect SVG online and have downloaded it for use with Survey123. While the image you found online may look perfect, it needs to include the required <path> elements to be clickable in Survey123.

 

There are numerous desktop vector graphics programs like Adobe Illustrator, Corel Draw and Inkscape as well as online SVG editors like Vectr, BoxySVG and DrawSVG. Most of these have tools that will convert different types of objects (eg <rect> or <circle>) to <path>, so perhaps your downloaded image can be easily tweaked to suit Survey123.

 

There are online resources that have SVG files for download that are designed in a way that suits Survey123, particularly those with geography content like www.simplemaps.com and www.amcharts.com. Most SVGs you'll find online have not been designed in this way so open your file in a text editor to quickly assess the contents.

 

Converting an existing raster image into an SVG

You may have the perfect .png or .jpg raster image that you would like to use with the image-map appearance in Survey123. The image-map appearance only supports SVG files, so you will need to convert your raster to SVG first. Vector graphics programs have tools that basically trace your raster and create SVG objects which can be very effective. Alternatively, online converters such as www.pngtosvg.com or www.image.online-convert.com/convert-to-svg can help you achieve this. Results can vary, but some experimentation with the settings and thresholds can yield usable SVGs in no time.

 

It is best if your raster image contains filled shapes with a limited range of colours. The 'Pipe Fittings' example was created using www.pngtosvg.com to first convert a .jpg to SVG. Then the unwanted light grey <path> elements were deleted using a text editor:

 

                 

Creating an SVG from scratch

lawn varietiesThis option gives you the greatest control over your SVG file. While it is possible to write an SVG entirely in a text editor, vector graphics programs like the ones listed above are the easiest way to create an SVG from scratch. If you're completely new to digital graphics then perhaps the online SVG editors are your best bet. Otherwise, desktop programs often have a larger toolset without limitations on importing and exporting your finished SVG. Inkscape is a free and open source vector graphics editor and was used for some of the SVGs in the sample survey.

 

The main advantage of using a vector graphics program is that you can easily embed raster images too. Embedding a photo into your SVG can be very useful, for example when the user needs to identify the selection they're making. Here is an example that includes photos of different lawn varieties:

 

TIP: Having a raster image in the background of your SVG and drawing some simple shapes over areas you would like the user to click is often the quickest and easiest way to create your SVG for use with Survey123. The 'Stranded Oil Distribution' example was created using a .png file as the background image. Then 24 x squares were drawn as <path> elements over the top of the background image. They were given no stroke or fill properties so they are invisible, but they define the clickable areas that correspond to the underlying parts of the background image.

 

oil distribution

 

More Information

The image-map appearance type is included in the Survey123 documentation here: Select one and Select multiple.

 

Detailed information on the SVG specification can be found here: Scalable Vector Graphics (SVG) 1.1 (Second Edition) and Scalable Vector Graphics (SVG) 2

If you use Survey123 Connect to author forms you are surely to become a regular of Microsoft Excel and the Survey123 XLSForm templates and samples. In this blog post, you will find some tips to help you be more productive with all of that. The first half of the post focuses on the Survey123 XLSForm templates and samples, describing how you can manipulate them to your liking. The second half compiles a handful of general productivity tips for Microsoft Excel.

 

Survey123 Connect XLSForm Templates

 

Some context first: XLSForm, .xls files and Microsoft Excel

 

Before jumping into the actual tips, I want to provide some clarification about Microsoft Excel and how it relates to XLSForm and Survey123. The XLSForm specification describes the syntax you need to follow in order to define a smart form in an .xls spreadsheet file. Survey123 adheres to the XLSForm specification. The foundation of Survey123 forms is ultimately an XLSForm document (.xls file).

 

Xls files were introduced in 2007 by Microsoft. Initially it was a proprietary format, but it was ultimately shared as an Open Specification. Technically, you can use any means to create Survey123 forms as long as you follow the XLSForm syntax and the output is stored in a proper .xls binary file. You do not need to use Microsoft Excel, other programs such as Open Office, Libre Office, Numbers can create xls files.

 

Formulas and data validation in Survey123 templates and samples

 

Survey123 Connect includes a number of samples and templates to help you get started with XLSForms. These .xls files (technically .xlsx) have been created using Microsoft Excel, and while we know they can be used and manipulated outside Microsoft Excel, we do not test them outside of this program.  On top of the compulsory XLSForm sheets (survey, choices and settings), our templates and samples include a number of formulas and data validation rules meant to facilitate the authoring of XLSForm compliant documents.  For example:

 

  • Several XLSForm columns in the survey spreadsheet include choice lists: type, appearance, bind: esriFieldType, etc.  This is to make it easier for you to pick a valid XLSForm choice.
  • A data validation rule in the name column of the survey sheet will highlight cells that have duplicated values. It also ensures that the value you enter will be accepted by ArcGIS as a field name. It checks for reserved keywords, length and special characters.
  • A yellow tip briefly describes the intent of each XLSForm column.
  • etc.

 

The formulas and data validation rules included in the Survey123 templates and samples are for your convenience, but they are not strictly necessary. If you feel these validation rules and formulas are in your way, you can manipulate and even eliminate them.

 

The next animation shows how you can access the data validation rules of a cell in Microsoft Excel.

 

 

Virtually every cell in our samples and templates has data validation. At this moment, I just want to make sure you know how to open the data validation rules for a particular cell.

 

Yellow tips (and how to hide/disable them)

 

When you select a cell in the survey sheet, you always get a yellow tip.  We added them to describe the purpose of the different columns in the survey sheet.  Below is the tip for the bind::esri:fieldAlias for example.

 

 

The yellow tips provide good info, but they also can get on your way obstructing the view of your form design. If you simply hit the Esc key, the tip goes away until you select another cell.

 

 

The yellow tip, in Microsoft Excel jargon, is an Input Message. To disable it, you need to go into the data validation dialog as shown in the next animation. 

 

 

If you feel ready for it, you can also disable all the Input Messages in one go. First, select all cells in the survey sheet using Ctrl-A, then open the Data Validation dialog and disable the Input Message for all.

 

Manipulating XLSForm lists such as appearance, type, etc.

 

As described above, some specific columns in the survey sheet limit inputs to values within a list. This is the case, for example, for the type, appearance, bind::esri:fieldType, required and readonly columns. It is important to know how to manipulate these lists.

 

Take the type and appearance lists, for example. They define, respectively, the XLSForm question types and question appearances that Survey123 supports. As support for new question types and appearances grows, lists in old surveys  you may have created could become obsolete.  For example, support for geotrace and geoshape question types came with version 3.6.  If your survey used a template downloaded with 3.5, these question types will be missing.

 

If you look at the data validation of a cell in the type column, you will see that it limits input to a list. The list is defined as a range of cells coming from the types sheet.  The types sheet is not part of the XLSForm spec.  It was spcifically added to the Survey123 Connect templates to help build data validation rules like this. It also serves as a nice quick reference for XLSForm syntax, listing the different question types, appearances, operators  and other things you can use.

 

If you want to add a missing entry to a list, you will do it in the types sheet. The next animation shows how you can add the geotrace and geoshape question types into a pre-3.6 template.

 

 

The same technique described above can be used to add new appearances. 

 

You should keep in mind two important things:

You must add new entries in the types sheet one by one. That is: You insert one new row, and you add values in it. Then you add the next row and so on and so forth. If you insert more than one row without values, your list range will not be happy.

You must enter new entries somewhere in the middle of the list. That is, never add a new entry at the very beginning (first position) or end (last position) of the list.

In the case of the lists for the required and readonly columns, you may want to delete the data validation rule completely if you want to enter an XLSForm expression in them.  The list included by default only has one value: yes. If you want to enter expressions you may want to delete the data validation rule to eliminate the drop-down list.

 

Creating your own template

 

By now you may be wondering if you can create your own template. To some extent, the answer is: yes. You can first create a new survey from the Advanced Template and make your own modifications to the Excel file. You can then use the Create a new survey option to derive new surveys out of your own template.

 

 

What I like about this approach is that not only you can create a new survey with all the modifications in your Excel file, your new survey will also inherit other properties of your template, such as the thumbnail, summary and description of your own template. Style, Inbox and Image settings in your template will also be matched, as well as custom content in your media folder if you ever add it to your template form.

 

If you publish your template from Survey123 Connect and share it with groups within your organization, other people will be able to use it  too.  Your survey, once published, will appear in the My Organization category of the New Survey dialog in Connect.

 

The real advantage here, again, goes well beyond tweaks to the data validation rules and formulas. It is other things in your template, like settings or even common lists you may have added in the choice sheet that would benefit anyone in your organization using your template.

 

Microsoft Excel Productivity Tips

 

So far, I have described specific tips to handle the Survey123  templates and samples that come with Survey123 Connect. Next, I want to compile a few general Microsoft Office tips for all of us who spend time designing Survey123 forms using XLSForms, Microsoft Excel and Connect. It is likely that you will already know many of the tips listed, but if just a single one you don't, that may make this read worthwhile.  If you have other tips you want to share, please add them as a comment below, so we can keep the list growing.

 

Inserting new rows, the quick way: Ctrl-Shift-+.

 

It is no secret that you can insert a new row by selecting a row, right-clicking and then using the insert option in the context menu. If you want to do that same thing much more quickly, select a row and then hit Ctrl Shift +.

 

 

Careful with copy paste!

 

Copy-pasting is often our friend. Nothing wrong with it... unless you copy too much. In Excel, when you do a standard copy-paste, you actually are copying way more than you think: you are copying the cell values, but also formatting style and formulas.  It is the formula bit that can get you in trouble, particularly when you copy cells from one xls file to the next.  To avoid any issues, I like to copy just the values.  This is described well in this Microsoft article.

 

Extend values and formulas

 

This tip is particularly valuable when constructing lists in the XLSForm choice sheet. Simply double-click on a cell's bottom right corner to extend that value and formula down to the end of the list. In the animation below this technique is used to apply a list name to over 250 list choices.

 

 

Keyboard shortcuts

 

In Microsoft Excel, using the mouse is often the slowest approach to get anything done. There are keyboard short-cuts for anything you can imagine. I use keyboard shortcuts for selections quite a bit: Ctrl + A to select all cells in a sheet and  Ctrl + Shift + Down Arrow to expand a selection to the bottom of a list.

 

But there are way too many shortcuts to describe and only a few that you will truly take advantage of. If you hit Alt, Microsoft Excel will describe keyboard shortcuts to open a ribbon. For example, Alt + A opens the Data ribbon but you do not have to remember that, because if you simply hit Alt, the Data ribbon will be highlighted with A as a reminder. Even better, if you hit Alt + A not only the Data ribbon opens, but also every function within it is also highlighted with its shortcut. For example, Alt + A + V opens the Data Validation menu.

 

 

Again, just a few tips. If you want to share a few more, please do so below!

There is truth in the saying a picture is worth a thousand words, but sometimes I wonder if that is enough, no pun intended, to give you the whole picture. Lets burn into that image the date and time when the photo was taken, the geographic coordinates of that location or some other critical information, and the value of that picture is now multiplied. That will make your photo be worth a few more than one thousand words, and most importantly, give you the whole picture.

 

Watermarking is an old, yet useful concept, allowing you to superimpose extra information on a photo.

 

In the Street Service Report survey example in the right side, you will appreciate that the photo taken from Survey123 automatically includes the Job ID as well as the date and time when the photo was taken.  

This information is part of the image, it is a photo watermark.

 

Watermarking is often used for two main reasons:

 

  • To add critical information necessary to understand what is depicted in the photo, even if the photo is presented out of context.
  • To show evidence that the photo was taken at a particular time and location, or by a particular person.

 

Many field data collection workflows can benefit from photo watermarking. Asset inspections, damage assessments or code violation reports are good examples.  For an asset inspection, you may want to burn into the photo the unique identifier and status of the asset. In a damage assessment or while documenting a code violation, you can use watermarks to carry the location, time and name of the inspector or team that shot the photo.

 

Starting with version 3.5, the Survey123 field app has built-in photo watermarking capabilities.  As you will learn through this article, you can create watermark expressions to add date, time, location and user information into your photos. You can even dynamically set the content of your watermarks with data previously entered in your smart form.  Survey123 watermark expressions give you a great deal of flexibility to control the placement and display properties of your watermarks.  All it takes is a bit of XLSForms magic.

 

To configure a survey with photo watermarking capabilities, you need to design your survey with Survey123 Connect. Survey123 Connect is a desktop application used to author Survey123 projects following the XLSForm specification. With XLSForms you define the questions and behavior of your smart form in Microsoft Excel (following the XLSForms syntax). XLSForm files are then published as Survey123 forms into ArcGIS with Survey123 Connect. If you are not familiar with Survey123 Connect and XLSForms, I suggest you start with our XLSForms video tutorials in YouTube, or with our step by step XLSForm guides.

 

Getting started with photo watermarking

 

Watermark definitions are applied to image questions using the bind::esri:parameters XLSForm column. For example:

 

typenamelabelbind::esri:parameters
geopointlocationGPS Location
imagephotoPhotobottomRightWatermark="Bula World"

 

The above will add a watermark with the text Bula World in the lower-right corner of your photo. Pretty much self explanatory. Don't you think? The most obscure aspect in all of this is the Bula part.

 

A photo watermark expression in Survey123 is split by an equal sign into two parts:

 

  • Watermark placement: The first half defines the placing of your watermark within the photo.
  • Watermark content: The second part is always enclosed in double quotes and defines the contents and display properties of your watermark.

 

Watermark expressions are extremely sensitive to extra spacing and casing. Tip: Do not add extra spaces between the placement and the equal sign, or in between your watermark content quoted string and the equal sign.

Watermark placing

 

The first half of your watermark expression defines its location within the image. You can choose betweeen 9 different positions:

 

  • topLeftWatermark
  • topCenterWatermark
  • topRightWatermark
  • leftCenterWatermark
  • centerWatermark
  • rightCenterWatermark
  • bottomLeftWatermark
  • bottomCenterWatermark
  • bottomRightWatermark

 

For the bottom-right corner of your photo, you can either use bottomRightWatermark, or simply watermark, for short. That is, bottomRightWatermark="Bula World" and watermark="Bula World" are equivalent.

 

You can add more than one watermark to your photos. For example, you can add one in the top-left corner, and another one in the bottom-right corner.  We will look at this later but before that, lets progressively disclose how to control the contents of your watermark.

 

Adding static text and basic text formatting

 

The content and display properties of your watermark are set in the second half of your watermark expression.

bottomLeftWatermark="Bula World&color=red"

leftCenterWatermark="Bula World&color=red&size=18"

watermak="Bula World&color=red&outlineColor=black&haloColor=white&size=12&bold=true&margin=10"

 

Note that you can add one or more formatting parameters. All you need to do is to separate them with an ampersand sign (&). The complete list of text formatting parameters can be found in the Watermark Hep Topic.  Here are the most common:

 

ParameterExampleNotes
color

color=red

color=#0000FF"

You can specify the color using a color code name or an hexadecimal value. The default color is blue. Find a list of colors at W3Schools.
outlineColoroutlineColor=blackHexadecimal color values and color names are also accepted.
haloColorhaloColor=white
fontfont=ArialArial, Helvetica, Cochin, Scheme, Verdana, Optima, Courier New...
sizesize=12Set in pixels
marginmargin=10Sets the margin in pixels from the edge of the image

 

Colors: In terms of colors, I like to set a bright color for the text (color), a dark one for the outline and then white for the halo. That ensures the watermark will stand out on top of your photo. The W3Schools list of color codes and hexadecimal values is a great place to help you define colors properly.

 

Text (and image) Size: The size parameter deserves a more detailed explanation.  Setting the right size is going to be a trial and error exercise and will largely depend on how you plan to use your watermarks.  For example, you may want to watermark photos so you can use them in a printed report, or simply show them in a screen within a dashboard or web mapping application. Depending on the use, you may want larger or smaller text.  Testing the size of your text in Survey123 Connect can be tempting, but you really want to look at your watermarks in the same environment where your users will ultimately use them. Do not go by what you see in Connect: validate your text size on paper if you intend to print, on a mobile device if you want people to preview the watermark from a device, etc

 

The size parameter is measured in pixels. For this reason, the size of the photo really comes into play.  While there are many devices out there and each may generate photos of different sizes, Survey123 allows you to define the size of photos taken with the Survey123 camera.  This is controlled through the Settings | Images dialog in Connect as shown below.

 

 

By default, Survey123 Connect always sets your image size to 640px on the longest edge.  In a 640px image, text at 12 pixels in size will show pixelated.  You could increase the size to 16 pixels for better results. If you change the image size to 1280, at 12px your text will be very small; you may want to change then the size to 24 or 26.

 

The default image size set by Connect (640px) is not ideal for watermarking. Give it a go in your own, but I typically use 1280px because otherwise there is barely any room for the watermark.

 

Breaklines: If you want to break your text into two or more lines, you can simply add \n to your text. For example:

watermak="Bula \n World"

You will likely use break lines all the time to split your watermark content.

 

Adding dynamic content: Location and Dates & Times

 

Very often you will want to add information in your watermark about the current date, time and location where the photo was taken. You may be tempted to use XLSForm variables to get this information into your watermark, but you should not. We will learn later how to add data from your form into the watermark.

 

To add the current location or time information you will want to use watermark-specific syntax. For example:

watermak="Inspected on: @[dateTime:short]"

watermak="Inspected on: @[dateTime:short] \n At @[latitude longitude]"

watermak="Inspected on: @[date:dddd] \n At @[latitude longitude] \n Accuracy: @[accuracy] meters"

watermak="Lat @[latitude] \n Lon: @[longitude]"
watermak="USNG: @[usng]"

 

Here is a list of accepted placeholders to extract location-related information. All location data is relative to the place where the photo is taken.

 

PlaceholderNotes

@[longitude:ddm]

@[longitude:dms]

@[latitude]

@[longitude]

@[latitude longitude]

Use the ddm and dms qualifiers to get the coordinates in degrees-decimal-minutes versus degrees, minutes and seconds. If you do not explicitly set a coordinate format you get DMS.

You can use @[latitude longitude] to get the coordinate pair at once, but it is useful to get the coordinates separate so you can split the data across two lines.

@[mgrs]

@[usng]

@[utm]

Reports the location where the photo has been taken in MGRS, USNG and UTM coordinate formats respectively.
@[altitude]Returns the elevation in meters where the photo was taken as reported by the device.
@[compass]Returns the compass bearing at the time the photo was taken.

@[direction]

@[speed]

Returns the direction and speed of travel when the photo was taken. If the user is not moving when capturing the photo, these values will be empty
@[accuracy]The horizontal accuracy as reported by the device when the photo was taken.

 

Lets go with time-related information, which is also relative to the instant when the photo is taken.

 

PlaceholderNotes

@[datetime]

@[datetime:short]

@[date]

@[time]

Returns the date and time when the photo was taken. Unless otherwise specified, the date and time are formatted according to the date and time formatting rules in the device.  The :short qualifier will output the date/time in a compact format, but still using the formatting rules from your device.

 

If you want to dictate date/time formatting rules independent of the device used to capture the photo, then you need  to explicitly set formatting rules in the watermark expression.

 

Here are a few examples for formatting dates and time:

 

@[datetime:d MMM yyyy HH:mm]  Example output: 7 Aug 2019 14:01

@[date:dddd]                                  Example output: Sunday

@[time:hh:mm t]                              Example output: 14:23 Pacific Daylight Time

 

To create your own formatting rules, the following tables should come handy:

 

FormatterDescription
dThe day as number without a leading zero (1 to 31)
ddThe day as number with a leading zero (01 to 31)
dddThe abbreviated localized day name (e.g. 'Mon' to 'Sun')
ddddThe long localized day name (e.g. 'Monday' to 'Sunday')
MThe month as number without a leading zero (1 to 12)
MMThe month as number with a leading zero (01 to 12)
MMMThe abbreviated localized month name (e.g. 'Jan' to 'Dec')
MMMMThe long localized month name (e.g. 'January' to 'December')
YYThe year as two digit number (00 to 99)
YYYYThe year as four digit number

 

FormatterDescription
hThe hour without a leading zero (0 to 23 or 1 to 12 if AM/PM display)
hhThe hour with a leading zero (00 to 23 or 01 to 12 if AM/PM display)
HThe hour without a leading zero (0 to 23, even with AM/PM display)
HHThe hour with a leading zero (00 to 23, even with AM/PM display)
mThe minute without a leading zero (0 to 59)
mmThe minute with a leading zero (00 to 59)
sThe second without a leading zero (0 to 59)
ssThe second with a leading zero (00 to 59)
zThe milliseconds without leading zeroes (0 to 999)
zzzThe milliseconds with leading zeroes (000 to 999)
APThe AM/PM display. AP will be replaced by either "AM" or "PM"
apThe am/pm display. ap will be replaced by either "am" or "pm".
tThe timezone

 

 

Adding previously entered data from your form

 

It is possible to add a watermark using data previously entered in your form. In the example at the very beginning of this article, we added the Job ID value to the watermark.  You could also add an address or a priority, even some comments from the person completing the form.

 

Data from your form cannot be added directly in the bind::esri:parameters column. Do not attempt to include standard XLSForm functions within your watermark expression. The right way to do this is to first put together the watermark content in a separate question, and then reference that. Here is a basic example:

 

typenamelabelbind::esri:parameterscalculation
geopointlocationLocation
textcommentsComments
calculatewatermarkWatermarkconcat("Comments", ${comments},"&size=20")
imagephotoPhotowatermark=${watermark}

 

Note how the first half of the watermark expression defining the placement of the watermark is added in the bind::esri:parameters while the second half, defining the content, is prepared in a separate calculate question and then referenced through an XLSForm variable.

 

To compose the watermark content, I am using a calculate type of question. Calculate questions are never shown in the actual form, but are useful when you want to compute some temporary values to be used somewhere else within the form. The value is computed using an expression in the calculation column.  A calculate question is very similar to a hidden question.  The only difference is that calculations are compulsory for calculate questions and not for hidden questions.

If you do not want the output of your calculate to be stored as a GIS attribute in your feature layer, set its bind::esri:fieldType column to null.

The safest way to construct complex watermark expressions is through the concat() function.  You can do pretty interesting things when working in this way. Not only you can bring data from your form, but also control the formatting of your text dynamically. In the next example, we set the color and content of the watermark text dynamically based on information entered by the user in the form.

 

typenamelabelbind::esri:parameterscalculation
geopointlocationLocation
select_one yes_nois_urgentIs urgent?
textcommentsComments
calculatetext_colorText Colorif(selected(${is_urgent},yes),'red','green')
calculatewmWatermarkconcat("Comments", ${comments},"&color=",${text_color})
imagephotoPhotowatermark=${wm}

 

Finally, note that when you reference watermark content in bind::esri:parameters as shown above, you no longer need to enclose the second half of the watermark expression with double quotes. Double quotes are needed to help the parser process the contents of bind::esri:parameters in case that you include spaces in your watermark expression. When you reference pre-computed watermark content like in this example, you can't add spaces, so no quotes are needed.

 

 

Adding images to your watermark

 

Say you want to add a logo to your photos. You can. Simply add your image in PNG or JPEG format into the media folder of your survey, and then use the image and imageSize watermark parameters:

watermark="image=MyLogo.png"

watermark="image=MyLogo.png&imageSize=35&text=All rights reserved"

In the example above I wanted to show the text after the image, so I decided to add the text parameter explicitly.

 

Adding multiple watermarks

 

Now that you know how to create your own watermark expressions, you may like to hear that you can add many to the same photo!  All you need to do is to separate your different watermark expressions with spaces within the bind::esri: parameters column.

 

For example, you can write this within bind::esri:parameters:

topLeftWatermark="Bula World" topRrightWatermark="Bye World"

 

As with many things XLSForms, you will want to start easy and increase the complexity of your watermark expressions progressively. You can use Survey123 Connect to make sure your syntax is good and do some preliminary testing, but eventually you will want to test your watermarks from the field app and if you plan your photos to be printed, print them to make sure you get them right.

 

Known limitations

 

  • Photo watermarking is only available in the Survey123 field app. All watermark expressions will be ignored by the Survey123 web app.
  • You cannot combine watermark expressions with the annotate or draw appearances of image questions.
  • On iOS, the Unrestricted image size currently doesn't support watermarks. The image will be scaled down to 1920 pixels on the longest edge.

 

Other resources

 

A great resource to see many photo watermark examples is the Watermarks XLSForm sample included in Survey123 Connect.  It includes over 20 different watermarks across multiple pages.

 

 

Our Watermarks—Survey123 for ArcGIS | ArcGIS help topic is a great reference too.

 

 

Happy watermarking!

A new update to the #Survey123 field app is now available in the app stores. The new build number is 3.5.177. This minor update is a targeted release addressing two critical issues recently raised:

 

  1. BUG-000124227: Downloading a survey will hang when using Portal 10.7 or earlier, or when trying to download and open a shared survey via the share URL link from website (shortened URL).
  2. Offline TPK files are not displaying correctly in the basemap menu on 32-bit Android devices.

 

Many, many thanks to TCPUD GIS , Eric Pescatore , Lindsay King , Kenneth Eggering , Braden Burkholder , Carol Northcott , Tatag Bagus Narendra , Jennifer Lidgett , Mason Bailie and Luke Amos-Binks for helping diagnose the issues  above and validating the fixes through our Early Adopter Program.

A Survey123 update is now available across all supported platforms. This release includes some important fixes. To download the latest version of the Survey123 field app and Connect refer to the Install Survey123 for ArcGIS—Survey123 for ArcGIS | ArcGIS help topic. Changes to the Survey123 website are available without further action on your side.

 

Here are the main issues addressed:

 

Survey123 website

 

  • BUG-000123405 Cannot open a survey published with Survey123 Connect from the form item details page.
  • BUG-000123464 Generating Feature Reports in the Survey123 for ArcGIS website fails with the error message, "Generate reports failed. You do not have enough credits to perform this operation. Contact your organization administrator for assistance" when an organization has negative credits. Thanks to Becky N who described this problem in GeoNet:Feature Reports Failed - Not Enough Credits 

 

Survey123 Connect and Field App

 

  • BUG-000123925 A survey containing hidden question type storing the data values for start and end time of the survey (using calculation) fails to load the records in the Inbox folder.
  • BUG-000123746 Constraint validation fails to execute if used against a question set as non-required if the question is followed immediately by a hidden question with a calculation. Thanks to Tarun Jaiswal for raising this.
  • BUG-000123822 Pulldata (CSV) will not work against a question of type note if the XLSForm name column is left empty.  Thanks to Florence Raimbault for finding it out.
  • Date questions are now set to use a default time of noon as opposed to mid-night. This helps avoiding unintended time-zone shifting issues when applying XLSForm expressions. Kudos go to Nicolas Roger.
  • Geopoint calculations within a repeat are now working.Thanks to Adilson Furlani for reporting issues with calculations affecting the first record of a repeat with 3.5.
  • The page counter when using the pages style in a survey now works correctly in Arabic.

 

With this update, we are also introducing support for Garmin Glo 2. To learn how to make a connection to your Garmin Glo 2 from Survey123, please check Direct GNSS external receiver support in Survey123 3.3 

 

As usual, our What's new in Survey123—Survey123 for ArcGIS | ArcGIS help topic highlights what is new in this and previous releases.

 

In addition to this update, you will also find the initial Beta builds of our 3.6 release available for testing through our Early Adopter Community | Esri Early Adopter Program  The Beta builds include support for polygons and polylines, web maps, mobile map packages and many other map enhancements. A Windows setup program to install the Survey123 website and REST API locally is also ready for testing. We are looking at September for our next update.

Survey123 includes out of the box functionality to help you generate high quality documents out of submitted surveys. This is extremely useful when you need to present your survey123 data following strict formatting guidelines, for example when mimicking legacy paper forms or to comply with legal documents. 

 

In this blog post I will describe how you can configure this feature report functionality against your own surveys, defining the content and look & feel of your own feature reports.

 

 

A bit of context: Survey123 feature reports vs summary reports

 

Before we get into the details, I want to provide some context up-front about what exactly I mean by feature reports.

Generally, the word report is associated with summary reports where data is aggregated to help identify trends. Survey123 includes basic summary report capabilities, but these are not the focus here. This discussion is about feature reports. A feature report presents data from a single survey record.

 

Say for example you use Survey123 to support asset inspections and you want to generate one document, as proof-of-work, for each inspection performed. For that you use feature reports. Say Survey123 is used to document city code violations and you want to generate and mail individual tickets for violations observed... For that you use feature reports too.  The purpose of a feature report is not to aggregate data across multiple surveys. The purpose is to represent the data a single submitted survey.

 

In a way, the Survey123 feature report capability is about bringing your survey data back into a form format, except that in this case, you want the form to present data, as opposed to using the form to collect data.

 

Creating a feature report from the Survey123 website

 

The following animation shows how you would generate a report from the Survey123 website. Note that the feature report is triggered from the Data tab in the Survey123 website.  The Data tab is designed to help you explore and drill down into individual survey records (features).  Using the map or the table views you can select a particular survey record and generate a report for it. That is, of course, assuming that you have previously configured your survey with a template. You will learn shortly about creating your own report templates.

 

 

Feature reports can be generated for one or multiple records at once. You can use filters or manually select multiple records using the shift key.  If you create a single report, you will be able to open the document right away from your browser. If you create reports in bulk (from a selection), then the outputs will be stored in your ArcGIS account.

 

The Survey123 feature report service will always create a single document for every record you select. That is, reports from multiple records will not be merged into a single document.  If you select 200 survey records and you trigger feature reports for them, then Survey123 will create one document for each record: 200 separate documents.

 

Creating a Sample Report Template

 

To get started on your own, you need first a report template to create feature reports. This is how you create one:

 

  1. In the Survey123 website, go to the Data tab of your survey.
  2. Select one record from the table or the map.
  3. Click on Feature Report in the top bar
  4. In the Feature Report panel that just open on the left side, click on the Manage Templates link.
  5. Create a new sample template as shown in the animation below and close the dialog.
  6. Generate a report.

 

 

Now that you have a template, click on the Generate Report button and open the resulting output document.  The sample template is generated automatically, and as such, will not create the most beautiful results, but it is a great starting point for you to refine it.

 

Feature report templates are stored along with your survey in your ArcGIS account. If you navigate to the directory of your survey within your ArcGIS content, you will find your feature report Microsoft Word template in there.  You can associate one or more templates with a single survey.

 

All templates you associate with your survey will be available to any user that has Viewer access to your survey. That is, anyone who has been granted Viewer access to your survey, will also be able to generate reports using the templates you define for that survey.

 

Getting started with custom feature report templates

 

Going back to the Manage Templates link you will be able to download the sample template to your computer.  Once you download the template, you will realize that it is a simple Microsoft Word document. This is what the sample template for my Water Violation v2 survey looks like:

 

 

You can see right away how templates work: A series of placeholders for questions in your form are included in the template. The most basic syntax for the placeholders looks like ${question_name}. All you need to do is to make your Microsoft Word template document look the way you want and insert the Survey123 feature report placeholders accordingly. 

 

Once you are happy with your report template, you can upload it using the Manage Templates link.  Until you get familiar with the placeholder syntax, I suggest you start easy: Start by rearranging text within the document and copying and pasting the placeholders from the sample template. Change the color and style of the text, arrange existing placeholders within tables, etc.

 

Generate a report again with your updated template to see the effect.

 

Using advanced feature report syntax

 

Survey123 feature report syntax can get a bit more sophisticated than just a generic ${question_name}. For example, note in the example below how extra parameters are used to format dates, to control the size of a photo or signature and even to define the scale and contents of a map.

 

 

This is what the final output of the template above looks like when the report has been generated.

 

When working on your template, you first need to understand how to uniquely reference questions within your survey. Downloading the sample template is the easiest way to get a complete list of placeholders, but you can also get the complete list from the feature report template dialog as shown below.  This dialog also lets you explore some more advanced syntax for your placeholders.

 

 

You can insert a | sign after your question name (aka: field code) and include some extra parameters to either format the value written to the report, or extract other information from your survey question.

 

Feature Report SyntaxDescription
${myQuestion}Inserts  in your report the value stored in the specified survey question.
${myDateQuestion | format:"DD/MM HH:mm"}Date questions are tricky, because they will not come pretty unless you format them properly. The format parameter is your friend. For a complete reference look for the Formatting Dates section in our Formulas—Survey123 for ArcGIS | ArcGIS help topic.
${myPhoto | size:300:0}When working with image questions, you can set the exact size in pixels that the image will have in the output report document. The first value defines the width and the second the height. If you set the height to 0, then the width will be applied and the height will be dynamically set so your image aspect ratio is preserved.

${myPhoto | getValue:"name"}

${myPhoto | getValue:"date"}

${myPhoto | getValue:"time"}

Also for image questions, you can add to your report the photo filename using the getValue:"name" parameter. Similarly, you can also extract the date and time when the photo was taken and have it written in your report. That, of course, assuming that the EXIF of the photo includes that info.
${myLocat | size:400:400}The size parameter is valid for map questions too. In this case, we will add in the report  map with a size of 400x400 pixels.

${myLocat | mapSettings:"6712da572":5000}

${myLocat | mapSettings:"":5000}

Use the mapSettings parameter to describe the exact webmap you want to draw and the map scale.  First you pass the webmap item id and then, separated by a colon, the map scale. If you do not know how to get the webmap item id, have a look at Lisa Berry's  Where can I find the item ID for an ArcGIS Online item? blog post. If you pass an empty webmap item id, then the report will use a default webmap.

${myChoiceQuestion | selected:"yes"} Yes

When working with choice questions you can use the selected parameter to output a checkbox. The expression on the left will output Yes if the option selected is yes.  This works well with both checklists (select_multiple) and single choice and dropdown questions (select_one).

 

Combining your mastery of the syntax above and a bit of good taste applying layouts and formatting in Microsoft Word will go a long way. You can pretty much do anything you want with your Microsoft Word document. For example, you can change the size and orientation of the page, use tables to arrange content, set the alignment and justification of text, its color, size, etc.  You can also include Survey123 placeholders in headers, footers or anywhere in the body of your document.

 

As you get started with the feature report syntax, I suggest that you start slowly, making small incremental changes against the sample template and you let your confidence grow progressively. You will need to upload your template again and again until things look they way you want. Every time you upload the template, the syntax of your report template will be validated.  You will get errors when you add an incorrect number of parameters on your placeholders, when you reference field codes that do not exist in your feature service or when you do not open or close the brackets correctly.  Also be careful making sure parameters are set with the right casing and honor the necessary spaces.

 

On top of including photos, signatures and maps in your Survey123 feature report templates, you can also handle related records (repeats) and even define conditional visibility statements. The complete reference for the feature report syntax is included in our Feature report templates—Survey123 for ArcGIS | ArcGIS help topic.

 

ArcGIS Online support and credit consumption

 

Survey123 feature reports are generated through an ArcGIS Online premium service. At this moment, the use of this service does not consume ArcGIS Online credits, but it will starting with our next update to ArcGIS Online.

The feature report service will consume ArcGIS Online credits at a rate of 2.5 credits per document generated starting with the ArcGIS Online release in October of 2019.

            

In the event that a user sees the Feature Report functionality disabled in the Survey123 website, an administrator of the ArcGIS Online organization can enable the Feature report functionality for that user as shown in the animation below.

 

 

ArcGIS Enterprise support

 

The Survey123 feature report service is at this moment hosted in ArcGIS Online and it is designed to work with surveys hosted in ArcGIS Online. However, with some limitations, this service can also be used with surveys hosted in your own ArcGIS Enterprise 10.5 or newer.

 

  • Your ArcGIS Enterprise instance must be accessible to the Survey123 ArcGIS Online Feature report service. That is, you must configure your  ArcGIS Enterprise web adaptor to expose access to your Enterprise instance from the internet. Please note that portals with Integrated Windows Authentication will not work with the report service.
  • The inclusion of any type of feature service attachments (photo, signature, annotated images...) is not supported when your data is hosted in ArcGIS Enterprise. This is a limitation listed in the backlog of ArcGIS Enterprise.

 

Bearing the limitations above, the use of the ArcGIS Online Feature Report service from your ArcGIS Enterprise instance will not incur into any ArcGIS Online credit costs.

 

Automation

 

You can automate the generation of Survey123 feature reports through Integromat and the ArcGIS Python API.

 

  • Integromat: You can the Survey123 module in Integromat to automate the generation of feature reports when a new survey is submitted. Even better, with Integromat, you can also take that feature report document and include it as an e-mail attachment, or upload it into cloud storage.  You can learn the basics of automating report generation with Integromat in this brief video-tutorial:

 

 

  • ArcGIS Python API: The ArcGIS Python API is ideal if you want to generate reports in bulk in an automated way. For example, in case that you want to automatically generate reports for all surveys submitted daily or weekly. Refer to our ArcGIS Python API developer documentation for details.

Yeap! World UFO Day is approaching so we want to use this out-of-this-world release to remind you. This is a really substantial update to Survey123, including many, many fixes, new features and enhancements. We are really excited about 3.5 and we hope you will too! 

 

Survey123 Feature Reports

 

Many of you are already familiar with the Survey123 Feature Report capability, which we have had in Beta for quite some time. With this update, the Survey123 Feature Report service is no longer Beta. It is fully supported by Esri Technical Support and it is here to stay.

 

The Survey123 Feature Report service is designed to help you generate beautiful reports of your Survey123 records. You can define the contents and look and feel of your reports using a report template in Microsoft Word format. Once your template is uploaded to ArcGIS Online, the Survey123 Feature Report service can generate individual reports for records submitted from Survey123. 

 

The Feature Report functionality can be triggered from the Data tab of the Survey123 website as shown below. You can create reports for a single selected record, or in bulk, for a larger selection set.

 

 

You can control the layout, contents and look and feel of your output reports through report templates. One or more templates can be associated with a survey, so users in your organization with view access to the survey data can generate reports on their own.  Report templates are authored in Microsoft Word. You can add headers, footers, tables and format the text  in your template as you like.   Feature Report Templates include placeholders to bring responses from your survey record into the report. This includes text, dates, numbers, choices and checklists, photos, annotated images, signatures and of course, maps!  The syntax to bring data from your survey into a template is described in our Feature report templates help topic.  A full blog post dedicated to the feature report syntax is on its way.

 

Feature Report generation is available as a premium ArcGIS Online service. Users who want to use the service will need to be granted Feature report privileges. By default, all roles and role templates included in ArcGIS Online include this capability, but if you use custom roles you will need to turn it  on.  This can only be done by an administrator of an ArcGIS Online organization by including Feature report access to the custom member role. The next animation shows how:

 

 

As of today, and until the next release of ArcGIS Online (planned for October 2019), you can generate as many reports as you want for free. Starting October 2019, feature report generation will consume 2.5 credits per report file generated.

 

One final note for ArcGIS Enterprise users: The Survey123 Feature report service is designed to work and is fully supported with surveys and data hosted in ArcGIS Online. However you can also take advantage of this service with surveys hosted in ArcGIS Enterprise 10.5 or newer, with some limitations:

 

  • Your ArcGIS Enterprise instance must be accessible to the Survey123 ArcGIS Online Feature report service. That is, you must configure your  ArcGIS Enterprise web adaptor to expose access to your Enterprise instance from the internet. Please note that portals with Integrated Windows Authentication will not work with the report service.
  • The inclusion of any type of feature service attachments (photo, signature, annotated images...) is not supported when your data is hosted in ArcGIS Enterprise. This is a limitation listed in the backlog of ArcGIS Enterprise.

 

Bearing the limitations above, the use of the ArcGIS Online Feature Report service from your ArcGIS Enterprise instance will not incur into any ArcGIS Online credit costs.

 

 

Web surveys: Survey open, survey closed! 

 

Up until this update, if you wanted to disable access to a public survey,  you would simply unshare it. In this way, people could not longer submit data to your survey, but you could still view and analyze its results. The challenge with unsharing a survey is that when attempting to open your survey users would get a generic dialog asking for a user and password, and that is the last thing someone would expect.

 

You can now configure a "Survey closed" dialog and define when to show it. In the 'Survey closed" dialog you can add your own text, images and links describing to survey takers why survey responses are not being accepted. Through the Collaborate tab, you can also define the exact time when the survey will be closed or open.  For example, you can design and publish a survey today, schedule it to start accepting responses in November 1, 2019 and stop accepting responses in December 13, 2019.

 

 

This feature is currently only supported for web surveys.  The Survey123 mobile app will not honor the "Survey closed" custom dialog until a later release.

 

Web surveys: Avoid multiple submissions

 

This new feature is also built with web surveys in mind. The idea here is to prevent users from submitting a survey multiple times.  By default, users can submit data to your web survey as many times as they like, but you can now configure your survey to limit submissions to one. This is done through the Collaborate tab as well.

 

For clarity, the way this works is not bullet proof for public surveys.  If working with a public web survey, this setting will persist a cookie in the user's web browser. If the cookie is present, the user will not be able to submit again. A smart user can delete the cookie by clearing the browser's cache or running an incognito session in the browser.  Also, for public surveys, we can't really do anything if one person submits from different devices (a phone, a desktop computer in the office, a laptop at home).

 

 

If you really want to confidently block multiple submissions from the same user, the best option is to secure your survey to force users to identify themselves.  At that point, we can more consistently block multiple submissions.

 

Working with related records (repeats) in the Survey123 website Data view

 

Through the Data view in the Survey123 website you can interactively inspect all submitted records as a map, table and a form.  In this release, the table view and map views have been enhanced so you can navigate to related records in the table view and display or hide related records in the map. You can also apply filters to the table view.

 

 

In the example above, we have a Zika Household survey example. At each household, we have one or more household members and for each of them we document all trips they did in the past 4 weeks.  The member households and travel locations are modeled in the smart form as a nested repeat, and in the feature service relational model as two nested related tables. Look carefully at the table view and you will see that each of these layers/tables is shown in an independent tab. You can switch between the tabs, look at the data, apply filters and select records.  This gives you much more flexibility to explore your survey data!

 

Survey owner? Double-click to edit (Survey123 website)

 

One more enhancement on the Survey123 website Data tab. Double click on a cell in the table view... and voila, now you can edit the value.You can even edit related records. This function is only available to survey owners.

 

 

Photo Watermarking (Connect & Field App)

 

Over time we have received a number of requests to add watermarking capabilities into Survey123. This is important in some field data collection workflows; for example when performing certain damage ***assessments, documenting work performed or inspecting assets. 

 

The basic idea is to burn some critical information into the photo while it is taken from the device. Information such as the current date and time, the compass direction or the latitude and longitude. We have extended the XLSForm syntax so you can do all of this and much more.

 

In the example on the right we have a street service report survey. This is used to document work performed.  Note how the photo is watermarked automatically to include in the bottom-left corner of the image the unique identifier of the job. This job id comes from one of the questions in the form.

 

In the bottom-right corner you will see the date and time when the photo was taken as well.

 

You can control almost anything you want in these watermarks: the position, color, font, halo effects and of course the content, which can be dynamically calculated or retrieved from the form itself.  You can even use a logo in your watermark expressions!

 

To learn more about watermarks, check out our Survey123 Tricks of the Trade: Photo Watermarks blog post and our Watermarks help topic

 

Grid Theme (Connect & Field App)

 

Grid theme has been a long awaited feature, so we are really proud of announcing it today. Using grid theme you can control the layout of your survey to arrange multiple questions in a single row. This is important when you want to make your surveys resemble more closely paper forms, and when you want to take advantage of the wider screen of a tablet or desktop computer.  Using grid themes wisely can help you create much more usable forms to accelerate data capture because you can reduce the need to scroll up and down through your form.

 

Below is an example of a Certificate of Roadworthiness. No kidding, that word actually exists, you can google it .  Note how the form includes multiple sections: A first section at the top for the Tester's License Number, where we arranged a single question all the way to the right. Then a second section for the main title, with a single note centered in the screen. In section A (vehicle description) where we place questions, notes and even a signature pad side by side.

 

 

When taking advantage of grid themes, you will want to test thoroughly in your own field devices. A form with the layout shown above, for example, will work well on a tablet, but not on a smartphone.  Combining grid themes and multiple pages in your form is a really powerful combination too.  Grid themes and pages are worth their own blog post, so wait for one coming up soon. If you cannot wait, check our Appearance help topic for details on the XLSForm syntax, or the newly added sample in Survey123 Connect.

 

As of this release, grid themes are only supported in the field app, but we are working towards bringing them into the web as well.

 

Other miscellaneous topics (Connect & Field App)

 

You are going to find many other enhancements throughout Survey123 Connect and the field app in this release.  I simply want to highlight a handful, although you can find the complete list in our What's New help topic.

 

  • In Survey123 Connect, you can now preview your surveys in different form factors.

 

 

  • Constraint and required messages now appear inline, helping you identify the exact questions in your form that need attention. This is particularly useful when you have long forms, or forms with multiple  pages.

 

  • Many visual controls in the field app and Connect have been refreshed bringing a cleaner look.
  • The Survey123 field app now sends information about the type of submission (add or edit) to webhooks. This was introduced in our previous release for the Survey123 web app, and now we close the circle with this update for the field app.

 


Critical Fixes and Enhancements

 

There are a lot of new new features in this release, but in truth our biggest commitment for this update has been quality.  If you have been testing Survey123 through our Early Adopter Program you will recognize that many of the features we make official today had been on the works for quite some time. Our 3.5 release has been for the most part, devoted to fixing issues and applying some popular enhancements.

 

We want to thank all of you who have worked with us to validate many of these fixes over the past 10 weeks.  Here is a list of the most important bugs addressed.

 

Survey123 website and web app

 

  • BUG-000118801 An existing secured ArcGIS Server feature service added to ArcGIS Online with credentials saved is not be able to attach images when using the web browser to submit the survey (the data is submitted, but not the attachments).
  • BUG-000119623 Survey123, non GlobalID/Guid primary key field gets automatically populated with a GlobalID when submitted from the web form.
  • BUG-000120589 When a survey is published from Survey123 Connect, then modified in the Web Designer to check on "Allow the respondent to add another record" the option does not appear in the Web App.
  • BUG-000121051 Pulldata calculation on select_one question does not work when accessing the survey in a browser.
  • BUG-000122048 Survey123 doesn't submit survey through mobile browsers if the survey has pages and signature fields.
  • BUG-000121719 In the Survey123 for ArcGIS website, the total number of features displayed changes when logged in using the credentials of other people in the organization apart from the owner.
  • BUG-000122052 When running a report on a Survey123 for ArcGIS form that contains the Other option as a response from a Multiple Choice question, the Other option is repeated after each of the Other selected responses.
  • BUG-000122750 The sum() aggregate function does not concatenate string inputs and records a "NaN" value when submitting responses from a web browser.
  • BUG-000123131 Exporting results to a file geodatabase fails if a survey contains more than one nested repeat from the Survey123 for ArcGIS website.
  • BUG-000114635 Photos do not appear in the Analyze tab of the survey in survey123.arcgis.com when logged in as a custom role account.
  • BUG-000121812 The custom basemap is not visible in the Survey123 for ArcGIS web form when the basemap is not shared publicly.

 

Survey123 Connect and field app

 

  • BUG-000121623: Survey123 Connect for ArcGIS 3.3.51 and Survey123 Connect Classic 2.9.3 crash when using a pulldata() function that references a CSV with 1048,576 records.
  • BUG-000100441: Survey123 Connect: Truncated "Search" string and wrongly placed search icon in search box.
  • BUG-000103155 Surveys created from the Survey123 for ArcGIS Web interface that contain geopoint questions allow field users to zoom in further on the map than is allowable in the Survey123 for ArcGIS mobile app.
  • BUG-000103942 Large surveys crash the Survey123 for ArcGIS iOS field app when specific answers are provided on an iPhone 6 without saving data.
  • BUG-000113684 Surveys with multiple repeats with the repeat counts added crash upon opening in the Survey123 for ArcGIS (iOS) field app.
  • BUG-000113734 The survey creation performance degrades when greater than 5-6 repeats are added to a single survey form.
  • BUG-000114396 The Survey123 field app successfully submits the survey even when all the required questions in the Repeat field are not answered.
  • BUG-000097367: Truncation in Date field in Survey123 and Survey123 Connect.
  • BUG-000094992  Truncation in Publish survey in Survey123 Connect.
  • BUG-000094993: Truncated string in Settings-Style in Survey123 Connect.
  • BUG-000115227 When the language of an Android device is set to the Korean language, switching the device keyboard to Korean to type Korean characters into an autocomplete field in a Survey123 for ArcGIS survey is not possible.
  • BUG-000118014 Unable to use a Japanese keyboard in Survey 123 for ArcGIS (3.1.158) on Android devices. This behavior appears to be restricted to the Survey 123 for ArcGIS (3.1.158) release. This issue is not reproducible on iOS.
  • BUG-000118208 In the Survey123 field app, when a constraint is applied to a select_one question, it is not validated until submitting the survey.
  • BUG-000118366 Unable to send surveys in the Survey123 for ArcGIS field app when Layer ID has values other than the default value.
  • BUG-000119632 Once relevancy exposes a repeat in Survey123, the information in the repeat will get submitted regardless of whether the repeat becomes non-relevant again.
  • BUG-000120023 Select_one questions with an expression in Required or Relevant fields do not validate in Survey123.
  • BUG-000121057 In the Survey123 3.3 mobile app, a select_one question in a nested repeat conditional based on a select_one question within a single repeat will not display after changing the answer to the single repeat select_one.
  • BUG-000119414 In Survey123 Connect for ArcGIS, nested repeats are not preserved when scrolling through the Form Preview.
  • BUG-000121921 Data not displaying in first record of nested repeat with relevant expression referencing a question outside of the nested repeat when open from Drafts.
  • BUG-000122112 When required fields are not answered, the survey does not honor question ordering when alerting the user of required, unanswered questions.
  • BUG-000122534 Survey123 for ArcGIS hangs indefinitely when Form Item is moved or deleted from ArcGIS Online. An error message must be thrown instead.
  • BUG-000122255 The Survey123 for ArcGIS field app does not retain the font size setting on iOS and Android devices.
  • BUG-000121156 Submitting surveys from the Survey123 for ArcGIS field app retains related records from a repeat question, even after removing the attribute values for the same related records before submitting the survey.
  • ENH-000106588 Provide the ability to expand a Group in Survey123 by clicking the Group header text, not just the arrow.
  • ENH-000115797 Survey123 should allow users to have control over color schemes for the group box on a survey form.
  • ENH-000118020 Lessen the touch precision needed for the Group expand/collapse button in the Survey123 field app.
  • ENH-000120369 Clarify the messaging when deleting a survey from Survey123 Connect to be more explicit that AGOL/Portal items will be deleted.
  • ENH-000121453 Provide additional functionality for there to be two "Text" question types be adjacent on one line in the "Form Preview" page of the Survey123 Connect for ArcGIS, instead of only one "Text" question type on top of another "Text" question type.


Next steps

 

We are now furiously working on 3.6. Join our Survey123 Early Adopter Program to test some of the new features we are working on, including:

 

  • Support for custom webmaps and mobile map packages.
  • On-premises setup for the Survey123 website and Survey123 REST API.
  • Support for polygons and polylines.
  • And more...

 

For developers customizing or branding the Survey123 field app, we will be making version 3.5 available with AppStudio for ArcGIS 4.0 Beta in the next weeks.

 

Do you believe?

Here we are once again, so close to the Esri User Conference!  Here is a short guide to help you find Survey123 related events at the conference.

 

Technical Sessions:

 

Technical Sessions are one UC's favorites. Always well attended, Tech Sessions run for about 60 minutes and cover technical aspects of Esri products.  We always try to keep a good chunk of time at the end for a live question & answer section. Tech Sessions start on Tuesday morning and run until Friday.  They all take place in the second floor of the San Diego Convention Center (also known as SDCC).

 

If you want to start planning your sessions, go to our UC 2019 Online Detailed Agenda. You can search sessions by topic and presenter and if you are logged-in you can add sessions to your agenda.

 

Here is a list of some of the Survey123-related Technical Sessions we are offering:

 

  • Survey123 for ArcGIS, an introduction: This session covers the basics of the Survey123 website, including designing smart forms with the web designer, publishing and sharing surveys, generating high quality reports for submitted data and other capabilities available through the Survey123 website. The session will end with a sneak peak of smart form capabilities going beyond what is possible through web designer.  I recommend attending this session even if you are already familiar with Survey123, because there are many new things that have been added to the Survey123 website in the past few months.  Philip Wilson, James Tedrick and Zheng Liu and will be leading this session; fun and good content guaranteed!
    • Tuesday, July 9 10:00 am - 11:00 am SDCC - Ballroom 08
    • Wednesday, July 10 2:30 pm - 3:30 pm SDCC - Ballroom 06 E
  • Survey123 for ArcGIS: Working with webhooks: Learn how to use web hooks with Survey123 to automate all sort of tasks aftere a survey is submitted. Automate e-mail notifications, instantly move Survey123 data into databases and spreadsheets, integrate Survey123 with image recognition software and much more.  Ismael Chivite, Brandon Armstrong and Calvin Jung will show you the way!
    • Wednesday, July 10 10:00 am - 11:00 am SDCC - Ballroom 09
  • Survey123 for ArcGIS: Advanced Topics: This session will introduce advanced capabilities of Survey123 including the new theme-grid layouts, editing existing features and designing surveys on top of existing feature layers, workflow automation with Survey123 and web hooks and other exciting features. There will be lots of live demos and time for live questions.  Ismael Chivite, Elvin Slavik and James Tedrick will present this session.
    • Thursday, July 11 1:00 pm - 2:00 pm SDCC - Room 10
  • Survey123 for ArcGIS: Extending the mobile app: Awaken the developer in you. Join this session to learn how you can extend the Survey123 mobile app with your own Javascript functions, the new extensibility framework or even brand and create your entirely new Survey123 mobile app. Presenters: Ismael Chivite, Elvin Slavik and Erwin Soekianto.
    • Thursday, July 11 2:30 pm - 3:30 pm SDCC - Room 31 B/C
  • ArcGIS Apps for the Field: An Introduction: In this session we will describe how all ArcGIS mobile apps work together. A great place to learn how Survey123 works along with Workforce for ArcGIS, Navigator for ArcGIS, Collector and even Operations Dashboard for ArcGIS.  Scheduled presenters include Scott Ball and James Tedrick. 
    • Tuesday, July 9 4:00pm to 5:00pm SDCC - Room 2
    • Wednesday, July 10 8:30am to 9:30am SDCC - Room 2

 

Survey123 User Group Meeting

 

We are happy to announce that this year we will have our own Survey123 User Group Meeting. It runs on Tuesday at lunch time from 11:30 AM to 12:30 PM, but no lunch is provided. This is for the ones with a strong Survey123 spirit!  Our User Group will be a great opportunity to meet other Survey123 Ninjas. This is fundamentally about connecting with other people and the team.  We will have a couple of 7-minute lighting talks to warm-up and then the rest of the time is for group discussion and networking.

Survey123 User Group Meeting: Tuesday 9,  11:30 AM - 12:30 PM  SDCC - Room 8

Demo Theaters

 

You can think of Demo Theaters as mini-Tech Sessions. They are shorter, always focused on very specific workflows and run in a small setting were questions can be handled throughout the presentation.  The Demo Theaters take place in Expo area (first floor), close to the Product Islands.

 

 

User Presentations

 

User Presentations are a great way to learn the details of how Survey123 is applied to specific workflows. A good opportunity to learn directly from users like you!

 

  • Mobile data collection with Survey123: There will be two user presentations highlighting how Survey123 is being used with Spike Laser Measurement devices and with high accuracy GNSS receivers to streamline field data collection workflows.
    • Wednesday, July 10 2:30pm - 3:30pm SDCC - Room 25C
  • Going mobile for the Homeless Point in Time count: Paper presentations from multiple users sharing how they used Survey123 for ArcGIS to support the Homeless PIT count.
    • Wednesday, July 10 1:00pm - 2:00pm  SDCC - Room 29A
  • Creating impacful applications for Education: This session presents two case studies of impactful GIS applications, including employing Dashboard for ArcGIS to improve earthquake safety and preparedness in K-12 Schools; using Survey123 for real time data collection, environmental resource protection, and sanitation management. The case studies in this session exemplify the best practices for creating impactful GIS applications
    • Tuesday, July 9 1:00 pm to 2:00pm SDCC - Room 25C
  • Crowdsourcing water quality data collection with Survey123: By the Arizona Department of Environmental Quality. With more than 45,000 miles of stream and river beds in the state, this is far too much for agency to staff visit. Utilizing the tools of Survey123, three forms were created to have citizen scientists help in collecting the data needed to protect this valuable resource.
    • Wednesday, July 10 8:30 am - 9:30 am SDCC - Room 29C
  • Innovation of ArcGIS Online Apps to Enhance British Military Working Practices: The presentation will demonstrate  innovation and implementation of the ArcGIS online applications used by HQ 1 (UK) Division Geospatial Support Cell. Applications have provided the British military with a fantastic opportunity to dramatically increase their operational effectiveness.
        Wednesday, July 10 1:00 pm - 2:00pm  SDCC - Room 26A
     

 

Expo

 

The Expo area is opens Tuesday and closes on Thursday afternoon. It is located in the first floor in the San Diego Convention Center (SDCC) and features different areas including Partner Exhibits, Esri Industry Neighborhoods, GIS Product Areas and Customer Care booths.

 

  • Product Areas (Islands) at the Expo:  You will always find someone from the team at the Survey123 booth. This is the place to go when you want to have a one on one conversation to address specific ideas or issues you may have. We often refer to the Product Areas as Islands. The Survey123 team will have a station under a big sign that says 'Apps'. It will be a big sign... you will see it.
  • Partner & Sponsor Exhibits: The Survey123 software is only a third of the story. There are two good reasons to visit the Esri Partner and Sponsor Exhibits. The first one is that many of them are now experts building smart forms with Survey123 and know the tricks of the trade when implementing solutions. Swing by their booth and ask them about their own experience.  The second one is that through the Partner Exhibits you will find many hardware vendors that can teach you the benefits of using their offerings with Survey123. Think Laser Measurement Solutions, GNSS receivers, ruggerized devices and much more. Just to name a few:
  • Customer Care and Esri Tech Support: Our Esri Customer Care and Technical Support teams will be represented in the Expo. You can also swing by their stands with your Survey123 technical questions.
  • Esri Industry Neighborhoods: Everything from Public Safety to Defense, Local Government, Utilities and more.  The Industry Neighborhoods are a good place when you want to connect with people that understands how to put Survey123 in the context of your own workflows.  Ask for a demonstration of Survey123 focused on your problem or for examples of organizations using Survey123 in your industry.

 

Esri Education Summit

 

The Esri Education Summit takes place between July 7-10 at the Marriott Marquis & Marina Hotel. You will need to register separately for this one!

 


 

Pre-Conference Seminars

 

 

By now, many Pre-Conference Seminars are sold-out, but I thought I would include info about this for those of you who are planning to come next year. You need to hurry before all seats are  taken.  Survey123 will be highlighted in the Field Operations and ArcGIS PreConference seminar:

 

  • Mastering Smart Form Design with Survey123 and XLSForm
    • In this one-day hands-on seminar you will learn how to use the XLSForm specification to build smart forms with Survey123 for ArcGIS. We will introduce the XLSForm specification and will dive into advanced techniques to implement custom data business validation rules, conditional statements to control the visibility of portions of your form, auto-calculation rules for attributes in your features and much more.
      • Sunday, July 07 8:30 am - 5:00 pm
  • Field Operations and ArcGIS
    • Learn how ArcGIS helps your organization optimize field operations by: taking ArcGIS maps offline into smart devices using ArcGIS Publisher Extension and Explorer for ArcGIS; coordinating your field workers with Workforce for ArcGIS and Navigator for ArcGIS; improving locational awareness by knowing the location of key staff and equipment with Tracker for ArcGIS; and optimizing data capture with Collector and Survey123 for ArcGIS; building effective operation dashboards with Operations Dashboard for ArcGIS.
      • Sunday, July 07 8:30 am - 5:00 pm

 

The Esri Events App:

 

If you are coming to San Diego, I recommend that you download the Esri Events app. It  will help you follow your own schedule, keep track of last minute changes to sessions and vote sessions too!

 

 

If you are coming this year, welcome!  If not, be ready for next. Have a look at the many reasons why you should not miss this event!

In this blog post I will explore a few techniques to take your use of Survey123 within Integromat to the next level.  I am going to assume that you already know the basics of working with Survey123 and Integromat. If not, you may want to start with Getting Started with Survey123 and Integromat

 

Many more posts like this could be written, because what you can do with Survey123 and Integromat deserves an entire book, but we need to start somewhere. So here are a handful of ideas. I hope you discover some new things:

 

Working with Survey123 attachments

 

I will start with this one because the topic of working with photo attachments from Survey123 in Integromat has come up many times before, and it is not entirely obvious how to deal with them.  Say your Survey123 smart form includes a photo and a signature question. Can you work with them in Integromat? Can you include the photo and signature images as attachments in an e-mail? Can you back-up the images in OneDrive or Box? Yes to all of that and here is how.

 

The trick is using the Integromat HTTP module, which includes an action called 'Get a file'.  This action takes the URL of an image as an input, which you can get from the payload of a Survey123 record.

 

  1. Feed the Survey123 'watch survey' module with a smart form that includes one or more photo/signature questions.
  2. Use the Get a file action in the HTTP module to download your attachment.

 

 

The payload of 'Watch a survey' includes a collection of attachmentInfo objects. They are uniquely identified with the name of the question generating the attachment and include a url property that can you use to invoke 'Get a file'.

 

Once you have the file, you can do a handful of things with it. Of course, you can now include it as an attachment to an email.... That is a classic one!

 

 

But you can do more interesting things too. For example:

 

  • Back-up your photos and attachments in Google Drive, Microsoft OneDrive, DropBox.
  • Upload images to Flickr, Instagram, Twitter or other social media.
  • Reduce the size of an image before storing it.
  • Extract metadata from the photo, including its name, size or format.
  • Process the photo with Google Cloud Vision to identify elements within the photo or simply extract text from it.

 

 

 

Automating Survey123 custom reports

 

The Survey123 module includes an action to help you automate Survey123 feature reports. Not familiar with reports in Survey123? We have a video for that here and an introductory blog post too. The Create Feature Report action is pretty much self-explanatory: it takes an input survey, a report template and the objectId of the survey record you want to include in the report.  If the trigger of your scenario is a 'Watch a survey' trigger, then you can use again the payload of your Survey123 web hook to extract the objectId. In this case, it is within the Feature-Result object as shown in the next animation.

 

 

The use of the HTTP module to download files as described above also applies to the outputs of the Survey123 report action. That is, after you create a custom report file, you can download it and include it as an attachment in an email, or upload it to cloud storage or FTP site.   Since many cloud storage solutions include options to automatically synchronize the contents of local and cloud folders, I like to upload the Survey123 report files I create to the cloud so they show up in my computer right away. I know, I am old fashioned, but that feels like magic.

 

When configuring the Create Feature Report action you will notice that at the bottom of the dialog you can expand a window with advanced settings. In there, you will be able to configure the UTC-Offset. This setting will be your friend if you plan to include dates and times in your reports.  Keep in mind that all data within Survey123 -and ArcGIS- is stored in UTC time. Since Integromat has no real context as from where in the world the report is being invoked, you will need to adjust the UTC offset yourself.

 

The Survey123 Custom Report service is currently in Beta and scheduled to be generally available in version 3.5

 

Back to the HTTP module

 

Earlier, we looked at the 'Get File' action in the HTTP module. This time we will revisit the HTTP module but this time to describe its 'Make a request' action. I personally find this action extremely useful, because it allows your scenarios to interact with pretty much any external web service you want. Using the 'Make a request' action you can for example invoke the ArcGIS REST API to update an attribute in an existing feature, or to simply query a feature service. Lets get down to very specific scenarios:

 

  • Make a point- in-polygon query to an ArcGIS feature service to get the postal code of the submitted Survey123 record location. Then make another request to add the postal code value as an attribute of the submitted feature.
  • Use information from a submitted Survey123 record to automatically create a new assignment in a Workforce for ArcGIS project (make a request to add a new feature into the Workforce assignments feature layer).
  • After having invoked Google Cloud Vision to categorize the contents of a photo submitted via Survey123, update the attributes of the Survey123 feature accordingly.
  • Update an image_url attribute in the submitted survey so you can later create a popup in ArcGIS that shows your image directly when the popup is open.

 

 

The mechanics for using the 'Make a request' action imply good knowledge of the target web service your are invoking. I will use next a simple scenario to illustrate the essence of this technique. Say for example we want first to get the location of a submitted survey record, get the current temperature at that location using a weather service and store that temperature value as a GIS attribute of the feature that was originally submitted.

 

Here is how you get the current temperature. OK: so far we are not using 'Make a request' anywhere. We are just setting this up.

 

 

Next, we are going to 'Make a request' to the survey feature layer using the ArcGIS REST API to add the temperature to the temperature field of the submitted record. Specifically, we are going to make a request to the Update Features REST API operation. This operation, as described in the Updates Features ArcGIS REST API Developer documentation, is located at:

https://<featurelayer-url>/updateFeatures

All information needed to construct the URL above is included in the survey web hook payload. It is a matter of combining the surveyInfo.serviceUrl and the feature.layerInfo.id properties, followed by updateFeatures.

 

 

Again according to the doc, this operation only supports POST requests and the encoded parameters include:

 

  • f to define the output format as json or html. This is optional but handy so we can debug, if needed, the output within Integromat.
  • features to define the actual updates we want to make in the layer. The content of this parameter is a json object with information about the records and fields we want to change.
  • token to specify a valid token we can use to securely access the feature layer. This parameter is not needed if the feature layer is shared publicly.

 

All of that, translates into Integromat as follows:

 

 

The most error prone step is adding the exact JSON value for the features parameter. I like to copy the sample JSON that comes in the developer documentation, and then modify it for my own purposes.  It is very important to note that the sample JSON in the documentation must be enclosed with extra curly brackets. In my case, this is the JSON string that will go in the features parameter. Note that the objectId in green, and the temperature in brown are dynamically populated using outputs from other modules in the scenario.

 

{

    "attributes" : {
      "OBJECTID" : objectId,
      "Temperature" : "temperature"
    }

}

 

You will see yourself going through some trial and error until you get it right, but since you can easily look at the output of the 'Make a request' operation, it typically does not take very long before you figure things out.

 

Using the 'Make a request' action you can do a lot of things. Updating attributes is one of them, but you can really do many others.

 

A nice complement to the 'Make a request' action in the HTTP app are the many out of the box tools included in Integromat to parse XML, JSON, HTML and the many functions to handle text, dates, arrays and much more. They will help tremendously when you need parse responses from the web services you are invoking.

 

Handling Errors

 

This is all getting very interesting, but as you create more sophisticated scenarios you will face situations where you need to handle run-time errors.  If you are dealing with attachments, what happens if the end-user does not submit one? Or what happens if you hit an external web service and you get a bad output, or a time-out?

 

By default, when an unexpected error occurs, the execution of the Integromat scenario is stopped at that point but you can handle errors in other ways. You can for example ignore the error and continue executing, or you can choose to branch off the execution to do different things. This is all described in the Introduction to error handling tutorial in the Integromat Help Center.

 

The following animation shows how you can add an error handler in case the 'Get file' action fails downloading a non-existing ArcGIS attachment. In this case I chose to simply invoke the Gmail connector to send an e-mail without attachments. If an ArcGIS attachment is found, it gets added to an e-mail, otherwise the email is sent without attachment.

 

 

To learn more about error handling in Integromat, I recommend reading the Advanced error handling help topic.

 

Connecting Integromat to an ArcGIS Enterprise instance

 

By default, the Survey123 modules in Integromat will connect to ArcGIS Online, but you can actually make Integromat work against your own instance of ArcGIS Enterprise.

 

Before we start, it is important to highlight that Integromat needs to communicate with your own ArcGIS Enterprise instance, and this will not be possible if ArcGIS Enterprise is running completely behind your firewall.  You need to make your ArcGIS Enterprise instance is available to Integromat over the internet. ArcGIS Enterprise administrators are familiar with this concept, which is often resolved by exposing ArcGIS Enterprise through a reverse proxy in the DMZ, although there are other options.

 

A second step, involves explicitly registering Integromat as an app in your ArcGIS Enterprise instance. This step is necessary for Integromat to be able to securely authenticate against your ArcGIS Enterprise. Follow these steps:

 

  • Log into your portal and go to Content.
  • Add a new Item of type Application.
    • Type: Application
    • Title: Integromat (Really, you can call this anything you want)
    • Tags: Survey123 (Same here, any tag you want)

 

 

 

 

All the above is necessary for Integromat to be trusted before you can initiate a secure connection to your Enterprise instance.

 

Now, it is time to create a new Connection within Integromat to your ArcGIS Enterprise instance.  While adding a new web hook in the Watch a survey or Create a report modules, choose to create a New Connection:

 

  1. In the Create a New Connection dialog, expand the advanced settings.
  2. Add the https URL of your ArcGIS Enterprise instance, including the web adaptor name.
  3. Populate the AppID and AppSecret properties using the App Registration information from the Application item you created before in your portal.
  4. Click on continue to authenticate against your ArcGIS Enterprise instance.

 

 

You will be able to re-use this connection again and again when creating new web hooks.

 

Be aware that when working with feature services hosted in ArcGIS Enterprise you will not be able to use the technique described in this blog to download attachments into Integromat. This is due to a limitation in ArcGIS Enterprise.

 

Add and Edit Trigger Events

 

Initially, we added support for Add trigger events, meaning that the Survey123 web and field apps would be able to trigger a web hook when a new record was submitted from a smart form.  We are progressively adding support for edit trigger events as well, so you can launch your Integromat scenarios when a record is updated too.

 

Support for edit trigger events was added in the Survey123 web app in our Comet Halley Update (May 23, 2019). The Survey123 field app will catch up starting with version 3.5 (scheduled  to be released on June 27, 2019).

 

If you do not now how to update records using the Survey123 web app, check the Survey123 Tricks of the Trade: Editing records in a web form blog post.

 

While you can configure your Integromat scenarios to be triggered by Add and Edit events, please be aware that the Survey123 field app will not honor this setting until version 3.5. As of 3.4, only the Survey123 web app supports web hook edit triggers.

 

Below, you can see how you can configure Integromat to define which triggers you want to activate in your web hook.

 

 

If you happen to activate both of them, you can use the output of the Watch survey module output to tell which event triggered your scenario. This is handy in case you want to build a single scenario and add branching logic to handle adds and edits differently.

 

The trick here is that the eventType property is only included in the payload once there is an actual submission of data. You will actually not see the eventType property pre-populated in the output, until you send data.  When working with eventType, I typically put the scenario in listening mode (Run once) and submit data to populate the property. This helps me build the rest of the scenario more easily. Once the scenario is running for good, this property will change accordingly.

 

 

 

Blueprints

 

Did you know you can export your scenarios as a JSON file and import them later?

 

 

I included as a simple blueprint of the weather sample as an attachment to this blog so you can play with it. All you need to make it work is a survey with a geopoint question and an additional field where you can store the temperature. Make sure you adjust the name of your temperature field in the features parameter of the 'Make a request' module. By now I am guessing you do not need that I create another gif. I am completely "giffed-out" for today.

 

As I said at the very beginning, there is so much to write about Survey123 and Integromat, so this is just a starting point. If you feel there is a particular topic of your interest that should be covered in more detail, do not hesitate asking. Now,  let me anticipate that we still have not figured out just yet how to work comfortably with repeat data within Integromat.

 

Despite the repeat debacle, I hope this blog post will give you some ideas to have more fun with Survey123 and Integromat. I also wish some of you will share some of your creations. Share a screenshot of your scenario and a brief description of what you accomplished with it, or even better the blueprint.  Add them as comments below to inspire others!

Integromat is a powerful tool for automating manual processes. With Integromat, you can easily connect Survey123 with many other apps and services. For example, you can configure a scenario in Integromat to perform an action immediately after data is submitted or edited via a Survey123 smart form: send an e-mail or an sms, copy the data into a spreadsheet or database, create an assignment in Workforce for ArcGIS or simply perform a reverse geocode to associate that address with the record that was just submitted.

 

Integromat includes well over 300 different out of the box 'integrations' (and growing), which you can combine visually in scenarios to automate specific tasks. You will find integrations for everything ranging from Office365 to Box, PayPal, SQLServer, Google Calendar, Apple Push Notifications, InvoiceNinja, Instagram, Google Cloud Print, and of course, Survey123 for ArcGIS!

 

 

The basics, in four minutes

 

To get started with Survey123 and Integromat I strongly suggest you watch this four minute video-tutorial. It will give you a good overview of the whole thing, plus enough detail to follow along and build your first Integromat scenario.

 

 

You can create a free Integromat account to test the waters. Note that for production use you may want to upgrade to a paid subscription plan. There are multiple options, with increasing levels of support, operations, data transfer, etc.

 

Get a kick-start with a template

 

Also for starters, you can go with the different templates we have created. With them, you can automate common tasks with minimum effort. If you find a template that fits what  you want to do, you can create a scenario out of it with a simple click. Integromat will guide you, step by step, so you can configure the template with your own Survey123 smart forms and other apps as appropriate.

 

 

Integromat Survey123 Templates

 

Once you bring a template into your own scenario, you can also extend it to do even more...

 

One hour for more details

 

Derek Law and Brandon Armstrong recently delivered a free Live Training Seminar focused on automating e-mail notifications.  This seminar was recorded and features Integromat. This is good material as well if you are getting started.  You will need to sign in with your Esri credentials to access the recording.

 

Automating e-mail notifications with Survey123 Live Training Seminar

 

Integromat Help Center

 

To get the best of Survey123 and Integromat you need to know them both. As you work with Integromat and Survey123 you will realize soon that this is more than just about sending an e-mail notification when survey data is submitted.   Integromat is actually a very powerful visual environment where you can model common business workflows. Integromat is also more than 300 out of the box integrations. It includes quite flexible options to control the flow of data and decisions within a scenario, with iterators, routers, filters and error handlers.  You can also configure your scenarios to execute at regular intervals and perform operations in bulk. 

 

What I like the most about Integromat is that it makes it possible to do lots of things for which in the past you would need programming skills. This makes more people, more productive. Having said this, if you have developer skills you will also be impressed with all the things you can do.  The more  you know, the better.

 

Spend some time roaming around the Integromat Help Center. Check out their video-tutorials and technical articles to learn new techniques. It will pay off!

 

Once you get on the basics, check out theSurvey123 Tricks of the Trade: Integromat blog post for additional info!

We do not seem to observe the stars as much as we used to, but it is fascinating to look up and think how much our life as a species has depended on a good understanding of the sky. We have dedicated this update to the comet Halley. Tomorrow, over 2,250 years ago, the first documented sighting of comet Halley was made by Chinese astronomers. It was sighted May 24, 240 B.C.  There are potentially earlier accounts from 467 B.C when Chinese and Greek astronomers logged the sighting of a comet, which could also have been comet Halley. We will go with the more certain date of May 24, 240 B.C and celebrate it along with this update of Survey123.

 

The passing of comet Halley has been very well documented since these initial accounts. Initially there was no evidence that all these sightings were of the same comet. The comet's periodicity was first determined in 1705 by English astronomer Edmond Halley, after whom it is now named. It visits us regularly every 75 years. It came last in 1983 and will return in July 2061.

 

This update is focused exclusively on the Survey123 website. Clear your browser cache!

 

Here is what is new:

 

Enhanced Integromat support

 

Integromat is an amazing online automation platform. With it you can easily automate tasks, visually connecting all sorts of apps together. We strongly believe that facilitating your work with platforms like Integromat can tremendously help you embed Survey123 within larger enterprise workflows, making sure your Survey123 data flows nicely into the way people work within your organization.

 

Many of you have been experimenting with Integromat and Survey123 since we first added support for webhooks in 2018. The uptake of webhooks with Survey123 has been pretty spectacular!  Some of the most common task automations we have seen include:

 

  • Automatically send an email or SMS when an inspection or incident report is submitted from Survey123.
  • Instantly add a new row into an Office 365 Excel spreadhseet when data is submitted from Survey123
  • Create a Workforce for ArcGIS assignment and a Google Calendar event when data is submitted from Survey123
  • Automatically reverse geocode the location a submitted survey and store the address in a GIS attribute of the submitted survey

 

We have worked closely with our friends at Integromat to add a new Survey123 Integromat module. This means that you can now bring your Survey123 data into Integromat much more easily and do much more with it than before. Jim Moore put together this brief but informative video to give you a quick overview of this new Survey123 Integromat module. Here it is:

 

 

In our Getting Started with Survey123 and Integromat blog post you will find a brief guide with resources to learn  the basics of working with Survey123 and Integromat.

 

For those of you already familiar with Survey123 and Integromat, you will find that this new module, lets you do some more advanced things you have been asking for.  For example:

 

  • Easily work with photo, signature and markup attachments. Include them as attachments in emails, upload them to Box, Microsoft OneDrive or Google Drive.
  • Automate report generation through the Survey123 Feature Report service. For example, for every survey submitted, automatically trigger a new  report using a custom template and then email or update that final report.
  • Trigger Integromat scenarios when Survey123 is used to update existing records*. Add a router in your Integromat scenario to do certain things if a record is submitted, and other things if the record has been updated.
  • Setup an Integromat scenario on top of ArcGIS Enterprise.

 

All of the above are covered in the Survey123 Tricks of the Trade: Integromat blog post.

 

* Please note that the field app will not trigger web hooks on edit events until our upcoming 3.5 update.

 

Delete and update submitted records (Survey123 website)

 

You can now use the Data tab in the Survey123 website to delete and update submitted records. This comes very handy if you need to do some quick cleanup of test data during your survey design process, or to make adjustments to already submitted data to your survey.

 

This is how it works: First login to the Survey123 web site, select a record in the table shown in the Data tab and then use the Edit button in the panel on the right to enable editing on the individual response form. Make your changes and submit.  The following animation shows the key steps:

 

 

You can use the Edit and Delete options for both surveys created in Survey123 Designer as well as Survey123 Connect. However, if your surveys were created in Survey123 Connect, be aware that the Edit button will be disabled when working against a survey with a repeat.

 

Update submitted records from a Survey123 web form

 

You may be already aware that you can update existing records in your feature layers using the Survey123 field app. Starting with this update you can update existing records with Survey123 from your web browser too! This is possible by opening your web form with URL parameters.  TheSurvey123 Tricks of the Trade: Web form URL parameters blog post introduces the concept of web form URL parameters. Now you can use URL parameters to edit existing records.

 

The syntax is fairly straight-forward:

 

  • Set the mode to edit: mode=edit
  • Pass the objectId of the record you want to update: objectId=13 for example.

 

For example:

https://survey123.arcgis.com/share/815b69a16e2f474e93da7e2eb0255e5d?mode=edit&objectId=13

 

Editing is not supported in web forms published prior to this update. If you want to edit records with surveys you published in the past you have two options:

 

    1) Dynamically upgrade your survey at run-time by adding the version=latest URL parameter

    2) Republish your survey to force a version upgrade at publishing time.

 

The key to all of this is to somehow dynamically construct the URL with the objectId of the record you want to update. In the Survey123 Tricks of the Trade: Editing records in a web form blog post this is all discussed in more detail.

 

Other enhancements and fixes

in the Survey123 website and Survey123 web app

 

 

  • Save Survey As... now works also against surveys that you do not own. 

 

This is very handy if you want to clone survey designs within your ArcGIS organization. Create a survey first using Survey123 Designer, then share the survey with someone else by granting access through the Viewer group sharing settings in the Collaborate page. Next, have that second person login into the Survey123 website and show 'Surveys I can view results for' in the survey gallery. The Save As option will be available on the survey tile menu to clone the survey. 

 

 

The Save As... option in the Survey123 website is only available for surveys authored in Survey123 Designer. If you want to clone a survey authored in Survey123 Connect, use the Save As... option in Survey123 Connect

 

  • You can now quickly switch between ArcGIS accounts in the Survey123 website without having to logout.

 

  • An option to optionally propagate selection filters between the Data and Analyze pages has been added.

 

Filter Options in Data Page

 

  • BUG-000113962 Unable to type certain Japanese character in the hints and explanations of headers in the Survey123 for ArcGIS Web Designer.
  • BUG-000120077 Unable to export data from Survey123 website when logged in as a member having 'User' Role and ‘Creator’ User Type.
  • BUG-000120887 When opening an existing Survey123 for ArcGIS form from ArcGIS Online using the 'Manage in Survey123 Website' option, the error message, "Your account () is not licensed for Survey123. Please ask your organization administrator to assign you a user type that includes Field/Essential Apps..." is returned.
  • BUG-000120055 Reports may fail to generate from a survey response that contains a repeat with many high resolution images.
  • BUG-000120222 Unable to generate a Survey123 for ArcGIS Report if there is an IFstatement containing a choice with an apostrophe.
  • BUG-000120851 Survey123 for ArcGIS web app displays the World Geocoding Service (WGS) on geopoint questions, despite removing WGS from the organization's Utility services.
  • ENH-000115658 Allow for the functionality to configure right to left labels and choices in Survey123 forms.
  • BUG-000113592 Questions label disappear from the survey when default language is changed from English to French or Spanish in a browser.
  • BUG-000112106 Survey123 for ArcGIS web app does not respect the language settings of date type questions in the browser.
  • BUG-000114761 Field Matching and string-length(.) constraints cause a green "It's an invalid answer." subtext to persist in Survey123 for ArcGIS web site under the questions that are constrained that does not disappear after fulfilling the constraints or leaving the fields blank.
  • BUG-000115247 Survey123 for ArcGIS web application does not respect the default language settings in the browser.
  • BUG-000117184 Submitting an Image through the Image Upload Question in Survey123 through the Web Designer in a Desktop Browser fails if the Survey was created with the Web Designer
  • BUG-000116320 The Survey123 for ArcGIS web app displays Image and Geopoint questions in English even if the web browser is set to another language in Internet Explorer.
  • BUG-000120426 Conditional calculations do not populate as expected for select_one questions in the Survey123 for ArcGIS web app.
  • BUG-000118413 Reverse geocoding results are different between Survey123 Connect and Survey123 web application.
  • BUG-000120387 Survey123 web app fails to record time when a time between 12PM and 1PM is selected
  • BUG-000121290 Loading a survey in a web browser with a Toolbar Background Color assigned causes the default color scheme to show momentarily before switching to the chosen color.
  • BUG-000120589 When editing surveys published in Connect the prompt to save changes on exit is not necessary if changes were published, does not work if changes were not published.

 

 

About the next update

 

 

As mentioned above, this update is exclusive to the Survey123 website. As usual we wanted to release both updates for the website and Survey123 field app together, but we decided to pull back on the Survey123 field app at this time. We are doing our best to bring some important fixes you have reported to the Survey123 field app soon. The current schedule is set for an update in 5 weeks (late June 2019). Our Survey123 for ArcGIS Early Adopter Program is making early beta builds of our next update available for testing.  The Survey123 website will also be updated in late June.