Skip navigation
All Places > Survey123 for ArcGIS > Blog
1 2 3 Previous Next

Survey123 for ArcGIS

154 posts

I discovered a handy workflow for calling and passing arguments to Python scripts using Survey123’s webhook capability, Integromat, and Google Cloud Functions. The possibilities are endless, but some use cases include:

 

  • Making scripts available and convenient to users without a Python environment on their machine
  • Calling a script each time a survey is submitted or updated (e.g., perform a field calculation)
  • Update records in a separate feature layer based on survey responses

 

In this example, I’ve made a simple web form and Python script that checks an ArcGIS Online org for new items and sends a raw list object of these items via email.

 

This post assumes basic familiarity with Survey123 Webhooks and Integromat. There are great Integromat intro posts from Ismael Chivite here and here.

 

Step 1: Create a Google Cloud Function to run your script

  1. Go to https://console.cloud.google.com/, log in with your Google account, and follow prompts to begin a Google Cloud Platform trial.
  2. You’ll be provided with a default project titled “My First Project.” From the dashboard of this project, in the Resources pane, click Cloud Functions. You can also go directly to functions at https://console.cloud.google.com/functions
  3. At the top of the page, click Create Function. Name your function and accept defaults except Runtime, which you’ll change to Python 3.7.
  4. Click the requirements.txt tab, above the code editor pane. This is where you’ll enter any Python modules required by your script that aren’t included in the standard Python distribution. For this example, we will need to add only the arcgis module.Add modules to requirements.txt
  5. Now we add our Python code! Click the main.py tab and paste your code. Be sure to wrap everything in a function, pass ‘request’ as an argument to your function, and assign variables used by the function with request.args.get(‘example’), matching the field names from the Survey123 form you’ll create in the next section. See my example below.
  6. Enter your function name under ‘Function to Execute’ and click Create. It will take a few minutes for your function to deploy.

 Python code for main.py

 

Step 2: Create and publish Survey123 

Now, let’s create a form to call the Python script.

  1. Open Survey123 Connect and create a new survey with fields for each argument you’ll be passing to the script. These field names must match what you provided to the request.args.get() method in the script – in our case, “startDate” and “toEmail”.Survey123 Connect fields
  2. Publish your survey.

 

Step 3: Set up a scenario in Integromat and connect a webhook to your Survey123 form

You’ll need to set up two modules in Integromat: The Survey123 module to watch for a new survey submission, and the HTTP module to make a request that calls the script on Google Cloud Functions and passes arguments to it.

  1. Add the Survey123 module to your scenario and connect it to your new survey as described here.
  2. Next, add the HTTP module but leave it alone for now. Connect the two modules:Integromat scenario
  3. Return to your Google Cloud Function, click the Trigger tab, and copy the URL for your function:Google Cloud Function URL
  4. Paste your URL into the HTTP module in Integromat and add a question mark at the end, followed by your first field/argument name and an equals sign, as shown below.http module integromat
  5.  Now grab your first argument from the dynamic fields window by expanding Attributes and clicking the field name:get dynamic field
  6.   Repeat for additional arguments, separating with an ampersand (&). The final URL in our example looks like this:integromat http url
     

Step 4: Call your script with the survey

  1. Turn on your Integromat scenario (or click Run Once)
  2. Open your survey in a web browser, fill in your arguments, and submit. You just called your script!

The Survey123 Early Adopter Community (EAC) helps you access the latest Beta builds of the software. You can sign in with your existing Esri account and gain access to software downloads, documentation on upcoming features and discussion forums.

 

The Early Adopter Community is critical to Survey123. It allows the development team to share with you early access to the software for user testing and feedback. This feedback is used to refine new features and fixes before they are made available under general release. EAC offers you a great opportunity to try out your own surveys and workflows with the Beta software and anticipate any issues that may arise in upcoming releases.

 

We have just updated EAC with new Beta versions of Survey123. Below is a list of some of the Beta features and when we estimate they will be available.

As a general rule, we prefer feedback regarding our Beta releases through the Early Adopter Community Forums. We like to keep discussions in GeoNet for the released version of the software. Keeping things separate avoids potential confusion.

 

Survey123 Feature Reports (PDF, Previews and Merged Reports)

 

We released Survey123 Feature Report capabilities back in July 2019. EAC will give you access to the Beta builds of the Survey123 website, which include new options to:

 

  • Generate outputs in PDF format. We want to make sure this works well with your existing report templates and that you do not have any issues with fonts once you open the PDF on your computer.
  • Preview Report designs, so you can put together your report templates without incurring ArcGIS Online credit costs. The output preview documents will be always generated in PDF format and include a watermark.
  • Merge multiple reports in a single file. This new option is available when you want to generate report files in bulk. When chosen, all your feature reports will be combined into a single PDF or docx output file.

Please note that when generating reports (other than previews) through the Early Adopter Community your account will be charged with credits. Use the preview option if you do not want to be charged.

You can try out all of the above through survey123beta.arcgis.com. When testing the new PDF and report preview capabilities, please report your findings (positive and negative) in the new Survey123 website forum in the EAC website. We are targeting late January 2020 for the release of both of these features.

 

Survey123 Field App (Survey update notifications and more)

 

Through the Early Adopter Community website you can access Beta builds of the Survey123 field app for iOS, Android, Windows, MacOS and Linux. The Beta build of the Survey123 field app cannot run on the same device as the released version, so you will need extra hardware to test it or remove the released version from the device first.

 

The latest Beta builds available include support for:

 

  • Survey update notifications. This work is inspired by our most popular Survey123 Geonet Idea: Notify users a Survey123 form has been updated. This will facilitate having field users more easily refresh their surveys if new updates are available in ArcGIS Online or Enterprise. This feature is planned to be made available in late February or early March 2020. 

 

  • No more random log outs. This is a fix to an issue some of you have reported where field users would get randomly log out, forcing users to have to type their user credentials again. The fix to this problem is now included with the latest Beta builds.
  • There are also a number of additional features available for testing. Information about the specific bug fixes addressed in each Beta update and new features are detailed in the Announcements section of the EAC website. 

 

The Beta builds of the Survey123 field app are also useful for you to test your existing surveys. As much as we do our best to ensure backwards compatibility with older versions, your own testing can help highlight issues we may never find on our own.

I work for a local government with an Office 365 subscription that includes Power Automate (aka MS Flow) and I wanted to see what I could do with it. After Zoltan Kovacs posted his 3rd blog on using Power Automate to send an email using HTML from a survey I was talking with my colleague (a developer) who told me that it was possible to generate a PDF from HTML using Power Automate (PA).

 

Wow! Really? 

 

It's true. It's possible. There is a convert file action that will convert HTML to PDF. 

 

convert file

 

The Survey123 Feature Report generates a nicer output and it includes attachments and its much easier to create a Word doc template than it is to create HTML. My method does not include attachments from Survey123 because it hasn't been exposed to Power Automate by Survey123 yet. My method also requires you to know HTML and CSS because you will need to fiddle with it. You can save a Word document as HTML (two options, more on that later) to get started but I just went old school and typed it out.

 

Here is the flow. You will notice the second step is cut off. It is a parallel branch that would have been too wide to capture and see it. 

 

entire flow with renamed steps

 

I'll explain my steps here and include tips along the way.

 

Step 1 is where you trigger the flow with your survey. 

 

Tip: Any step or action you add to your flow can be renamed. Clicking on the 3 dots on the side will show a drop down menu. Rename is one of them. Adding comments is another.

 

include comments

 

Step 2: parallel branch where I initialize and set variables inside a switch case operation

 

This survey records water meter replacements. The choice value is stored with a code. I wanted the report to show the label and not the value. I know that Survey123 has jr:choice-name but I built this survey 3 years ago and I didn't want to change it. I looked at using a SharePoint list to lookup the value but I couldn't figure it out (I felt like I was this close to a solution! but I had to walk away and move on). 

 

I tried using the If expression as it seemed similar to the If expression in Survey123. However, unlike Survey123 it cannot contain nested expressions.

 

There is a switch case operation. My solution was to initialize a variable and then use the switch case operation to set the variable to a nice looking label for every value. 

 

 

set variable

 

The reason why I cannot show the entire parallel branch in the screen shot is because I have 4 branches and each branch has 4 switch cases. Two of the switches have multiple cases (17 in one of them!). It is wider than my screen can show when I click into them. 

 

Tip: Have all of your parallel branches figured out in the beginning because it is currently not possible to add one in later without needing to redo your entire flow. 

 

Step 3 and 4: Read part 2 from Zoltan Kovacs for how to set this up.

 

Step 5: Create the HTML file (add step: create file action in OneDrive for Business)

 

A side note about HTML: It's the trickiest part because … well … it is HTML. While testing I typed the HTML myself because I wanted a simple form.

 

I did have a Word doc of this form. In Word, if I saved it as web page, opened it in Notepad++ it was over 1300 lines long! If I saved it as a web page filtered it was about 300 lines long. Filtered means it strips out all of the Word tags making it impossible to turn it back into the same Word document again. 

 

A side note about challenges using the PA user interface: This report I created is not very long. About a half of a letter sized page. However, I was unable to see all of it in the PA interface. One of the challenges with the interface is that the dynamic content is in a floating window all of the way down at the bottom right.

 

dynamic content

 

It doesn't follow you as you scroll up. If you want to insert the content you have to mouse click exactly where you want it, scroll down, click on the content and then scroll back up to see if it put it where you want it. If you didn't mouse click exactly (even if you think you did) it will appear at the top of the file content. While it is possible to click on the dynamic content and drag it up through the file it is slow and it may need to be done multiple times to get it where you want it.

 

The next screenshot is the beginning of the HTML. The folder path is where the HTML file is saved in OneDrive for Business. This is one of the connections you need to set up in advance. I believe you cannot go deeper than one folder off of the root folder.

 

Tip: I found that it was possible to insert code in the HTML while using Notepad++. For example, the dynamic content for anything from Survey123 is triggerBody()?['feature']?['attributes']?['field name']. In Notepad++ I used

@{triggerBody()?['feature']?['attributes']?['field name']}.  Once it was copied into the PA window it changed into the normal looking green dynamic content.

 

Tip: I'm using inline CSS but it is possible to reference an external stylesheet. I have not tried this yet.

 

The file name can be a dynamic value. I was never able to figure out how to do this in Integromat. The green icon with the checkmark is coming from the survey. In this particular case it is the service address.  I added .htm

 

create html 1

 

PA uses different colors for dynamic content. So far I've seen green for Survey123, blue for the built-in functions used to generate the time stamp and purple for the variables I set.

 

dynamic content colors 1

dynamic content colors 2

 

Step 6: Create the PDF file

 

When you create the file in Step 5 from above the dynamic content will now include ID. It is all that is needed for this step.

 

create pdf file

 

Step 7: Save the file to OneDrive

 

When the HTML file is created in Step 5 it actually saves the file to OneDrive. That doesn't happen in Step 6. You have to explicitly save it using the Create File operation (I've already renamed it in this screenshot). Choose the folder and add the file name and file content from the dynamic content window. 

 

save pdf file to onedrive

save pdf to onedrive using dynamic content

 

Step 8: Send email (V2) preview

 

I didn't really have to send an email to myself with the attachment as it was saving it to OneDrive. It was part of my testing.

 

send email

 

I went through a lot of trial and error and Google searches a couple of weeks ago figuring this out even though I know (through the early adopter program) that a PDF option is coming to Survey123.

 

I'm a GIS specialist who dabbles in the development world. I'm no Power Automate or webhook expert. There is probably a different way to do this but I wanted to share this with the community. It might inspire others to find a different solution or be inspired to try something new.

 

p.s. I finally finished writing up this post and I've already found a possible alternative to using HTML!

[Updated Dec 18]

 

One more update to Survey123 to close out the year! A day like today in 1901 Walt Disney was born. Good opportunity to remember him!

 

Snow White and the Seven Dwarfs, Pinocchio, Dumbo, Bambi, Cinderella, Peter Pan, Sleeping Beauty, One Hundred and One Dalmatians, Mary Poppins, The Jungle Book... Walter Elias Disney was a visionary. He brought animated cartoons to the next level, introducing new techniques, modernizing classic fairy tales and synchronizing music with his creations. He truly transformed a marginal form of communication into an entire new industry. Nobody has won more Oscars (22) or being nominated so many times (59). A pioneer of the American animation industry.

 

The 3.7 release is now available across all supported platforms. You can download this update from the iOS and Google Play stores or from our Survey123 Resources page. The build numbers are 3.7.55 for the field app and 3.7.59 for Connect. The Survey123 web app has also been updated.

 

This is a minor, yet important update, as it brings a handful of critical enhancements and fixes.

 

Inbox Enhancements

 

A lot of work in this update went into making sure you can work with the Inbox more reliably, faster and with many more records. The Survey123 field app Inbox allows you to update existing GIS records. If you are not familiar with the Inbox, I suggest you take a quick look at this video:

 

 

For more details, see the Prepare for editing existing survey data—Survey123 for ArcGIS | Documentation help topic.

 

With this release, the performance of the Inbox has been greatly improved, allowing you to more quickly download records onto your device; twice as fast as before! The Inbox can also now handle many more features. With this update you will be able to comfortably work with a few thousand records in your Inbox, even if you are downloading complex lines, polygons or have many related records. Overall, the Inbox is now much faster and less memory intensive. You will note a great improvement making this feature more reliable, particularly on low-end devices.

 

Based on our own tests, we can define an upper limit for the Inbox around ~40k (for points) and ~8k (for lines or polygons). Having said this, the intended and practical limit of the Inbox is lower; around a few thousand records. The philosophy of the Inbox is that of a 'to do' list where you can download surveys for field-users to process. For example, in an asset inspection workflow, each of the records in the Inbox, would represent an inspection for the end user to complete. You will not want to have the Inbox populated with every asset you have. Instead, you will want to use a filter to have the Inbox show only assets where inspections are due. You can apply Inbox queries using the username of the logged in user to tailor the Inbox contents for every person in your field crew. For the benefit of field users, try to keep your Inbox light, only showing records to be processed. Stay away from using the Inbox to download every record in your dataset, particularly if you have many tens of thousands of records.

 

Enhanced Numbers and Calculator appearances

 

Work with the numbers and calculator appearances was initially triggered by data-loss issues reported through GeoNet (Using Numbers Appearance Type Sometimes Will Not Store Value in Field). We addressed this issue and also took our time to revisit the usability of these controls.

 

The numbers and calculator XLSForm appearances are designed to accelerate data capture, providing specialized keypads that replace the generic screen keyboard provided by the operating system in your device.

 

As their names suggest, the numbers appearance will show a numeric keypad, and the calculator appearance will show... you guessed it: a calculator. These specialized keypads are also shown close to the question on which the data will be entered, as opposed to the bottom of the screen. This allows for faster data entry.

 

Throughout this release we performed many usability tests in various representative forms and against multiple form factors and devices. We refined the look, feel and behavior of these keypads with a focus on making data capture more efficient and intuitive.  If you have used these keypads in the past, you will notice a noticeable improvement. If you are new to them, you may want to give them a go!

 

You can use the numbers and calculator appearances with integer, decimal and text questions. To learn more, see the Appearance—Survey123 for ArcGIS | Documentation help topic.

 

 

 Geode sub-meter GNSS receivers are now supported

 

With this update we are adding support for Geode sub-meter GNSS receivers from Esri Partner Juniper Systems. The Geode receiver is compatible with Windows, iOS and Android devices. You can learn more about how to connect the Survey123 field app to an external GNSS receiver through Marika's Direct GNSS external receiver support in Survey123 3.3 blog post. On top of support for Geode, Survey123 also works nicely with the Juniper Systems Mesa 2 rugged tablet (available with either Android or Windows operating systems).

 

External GNSS receivers, such Geode, allow you capture sub-meter accuracy location data. Additionally, using XLSForm expressions you can easily use location metadata from the device to power data validation rules as well as to simply store the metadata for further analysis in the back-office.

 

 

Check out the Survey123 Tricks of the Trade: XLSForms and location quality blog post or Pulling data from geopoint questions to get the best out of your Geode with the Survey123 field app.

 

Other fixes and minor enhancements

 

This update includes a lot more other good fixes and enhancements. I wanted to briefly highlight a few:


Survey123 web app (aka web form)

 

Fixes to Survey123 web forms apply to newly published surveys. If you have an existing web form already published and want to benefit from these fixes, you can re-publish your form.

 

  • BUG-000126543 - Encountering error "Failed to submit. - Cannot set property 'z' of null' when attempting to submit survey in edit mode (error occurs when submitting a record to a survey which created on top of a standalone table).
  • BUG-000111892 Location for a Geopoint question can not be selected in a Survey opened in the browser of an iOS 10.3.3/10.3.2 device.


Survey123 Connect and Field App

 

  • BUG-000127248 - Location Expressions in watermarks display full JSON object [Updated Dec 18]
  • BUG-000115279 Hosted feature layers created from Survey123 Connect, do not produce shapefiles once exported with valid/visible geometry.
  • BUG-000102165 The default values of non-relevant questions are not preserved when the values are configured with a 'relevant' expression in Survey123 Connect for ArcGIS.
  • BUG-000125142 Survey123 for ArcGIS gets stuck at "Getting Service Information" when trying to submit a survey over VPN after the survey had been open for over 10 minutes and the connection to the internet or VPN have been disconnected during this time.
  • ENH-000115846 Review how Survey123 makes query requests from the Inbox to increase performance.
  • BUG-000116741 Survey123 Field App looks like it's frozen when refreshing Inbox with many records (more than 1000).

  • BUG-000123911 After increasing the maximum feature count for a feature service, Survey123 3.5.164 for iOS crashes and can't sync more than 1000 records to Inbox.
  • ENH-000119205 Provide the ability for Survey123 to load more than 1000 records in the Inbox.
  • BUG-000125947 Microsoft Surface Pen is incompatible with the Survey123 field app.
  • ENH-000119711 Add survey version to Survey Summary page.
    • The version number of your survey, as described in the settings worksheet of your XLSForm, will be displayed in the survey details page in the Survey123 field app.
  • ENH-000120747 Use appearance to enable/disable regional formatting (thousand separator) for a question.
    • As of this release, no thousand separators will be automatically added to your integer and decimal questions. The thousands-sep appearance can be used to add thousand separators to decimal questions.
  • Various UX enhancements to the map control when editing the geometry of geotrace and geoshape questions.
  • Survey123 Community pane is now hidden in Connect. A Community tab has been added.


Internationalization

 

  • BUG-000115321 Survey123 Connect. The Cancel button is not translated in the 'Select your active ArcGIS Portal' window in Survey123 Connect for ArcGIS 3.0 which is installed on German operating system (OS).
  • BUG-000115322 Survey123 Connect. The "Close" button is not translated in the "About Survey123 Connect for ArcGIS" window of the Survey123 Connect 3.0 which is installed on German OS.
  • BUG-000098942 Survey123 Connect: Untranslated "Show Details...", "Hide Details" and "OK" buttons when validating input.
  • ENH-000121448 Survey123 Field ap. For time and dateTime questions, the placeholder words for 'Date' and 'Time' should be displayed in the chosen language.
  • BUG-000126692 Calculation field returns the error message, "Invalid Number" on validation in Survey123 Connect for ArcGIS on French operating system.

 

Documentation

 

  • ENH-000118417 When using the Report (Beta) function on the Survey123 website to display a geopoint, there is no documented method of setting the map extent for the record.
  • ENH-000126633 Addition to the Survey123 documentation: While Survey123 can perform calculations on text fields, they will not be honored in the Survey123 webform; Use concat() or join() instead.

 

Next Steps

 

We have a number of new features and enhancements lined up for upcoming releases. If you want to know more about them, join our Survey123 Early Adopter Community (EAC). You will find documentation about on-going work, forums for your feedback and of course, the Beta software.

 

Available for testing now through EAC:

 

  • Survey123 Feature Report enhancements: PDF output and a new preview mode to generate free PDF watermarked outputs. 
  • Survey123 Web Form JavaScript API: A straight-forward JS API for web developers to embed, style and interact with Survey123 web forms within their web applications.
  • New map capabilities: Support for mobile map packages, web maps and vector tile packages in the Survey123 field app.
  • Windows setup for the Survey123 website and REST API (for Enterprise users).
  • Support for custom JS functions in the XLSForm pulldata function (Survey123 Connect and Field App): Embed custom JS code in your XLSForm expressions.
Check out Part 1 and Part 2 of the series.

In the final part of this series we'll see how to set up an email template for our webhook containing dynamic elements from the survey data and where to access additional webhook settings.

 

The steps below build on the webhook started in Part 1 & 2 but can be applied to any other webhook.

1.  Click on Add an action under 'Convert time zone' of the If yes branch of your condition.No alt text provided for this image
2.   Search for ‘send an email’ and select the action from the result list. No alt text provided for this image
3.   Complete the To and Subject fields (you should be able to search for your O365 contacts). No alt text provided for this image

4.   Compose the message

 

What formats and options do we have to include different content? By default, your message is sent as plain text - no hyperlinks or bold/coloured text, but you can include dynamic content like your converted time and values from your submitted survey data. What if you want more?

 

At the bottom left of the Send an email step, there is an option called Show advanced options. Hidden here you have the availability to switch your email format to HTML (and include HTML formatting tags to change the look of your email), send the email on behalf of someone else, add attachments, set CC and BCC recipients, set the importance of your message and so on.
No alt text provided for this image
That's general formatting out of the way, so we can now focus on including dynamic content in our message template. The following example will add a hyperlink to a web map that will show the location of the reported close call when it's clicked on. We will use a web map application that is shared with the relevant AGOL group(s), thus notified people can see the reported close call in a map.

 

The web map application to use may contain many other layers and we only want to show the one that contains the survey data and want to zoom in onto the close call location. How do we do this? Documentation to the rescue! The online help for Web AppBuilder for ArcGIS has a section about using URL parameters - we just need to have a look at the following sections:
  • Open a saved app
  • Set layer's visibility
  • Query a feature

 

Using the information from the help we can construct the url:

 

https://<your portal url>/apps/webappviewer/index.html?id=<itemId>&showLayers=<layerID>&query=<layerID>,<queryField>,<queryValue>


Where to find the values required?
  • itemId: Open the item's page in AGOL and look at the end of the URL in your web browser or scroll down to URL on the item's page and copy the hash value.

 

No alt text provided for this image

 

  • queryField: this is the field from your layer to use for the query. (We use the OBJECTID in our example.)

 

  • queryValue: value coming from your survey data, so your link is dynamic and will work with any close calls. (For testing purposes you can use a static value.)

 

It is highly recommended to test your URL in a web browser to make sure your web map application loads with the layers you need and shows the queried feature. It would look something like this:

 

https://XXXXXXXX.maps.arcgis.com/apps/webappviewer/index.html?id=a0b000c00d0e0ff0gh000iiij0kkl00f&showLayers=service_00a0b00000c00000d000e0f0g0000h0i_0123&query=service_00a0b00000c00000d000e0f0g0000h0i_0123,OBJECTID,95 

 

Once the URL is tested and you are happy with the results, it can go in to the email message with the rest of the dynamic content (change to 'Yes' under Is HTML first in advanced options). An example can be seen below.

 

No alt text provided for this image

 

The sent email would look like this: 

 

No alt text provided for this image

 

Clicking on the hyperlink opens the web map application, pans to the feature and displays a pop-up:

No alt text provided for this image

 

With using dynamic content in custom functions in the body of your message you can create a template that could be dynamically updated with the correct close call reason for example: No alt text provided for this image

 

Additional webhook settings on the Survey123 website

Most of your webhook settings can be controlled in your Power Automate (aka MS Flow) account, but there are a few settings that can only be accessed on the Survey123 website.

 

1.  Open your web browser and go to https://survey123.arcgis.com

 

2.  Log in and find the survey with the webhook in question

 

3.  Click on the ellipsis (...) button and select Settings

No alt text provided for this image

4.  Click on Webhooks on the left-hand side to see all webhooks for your survey

 

5.  Click on Edit (the pencil icon) under Operations to see more details No alt text provided for this image

6.  On the new page you can make changes to your webhook:

  • Edit its name
  • Change when it should be triggered: new records submitted, existing records edited
  • Event data: information included in the payload. Hover over the grey '?' icon after each item to see the information they include
  • Turn the webhook off or back on

 

No alt text provided for this image

NB: Changing the payload URL can break your webhook, only do it if you are sure you know what you're doing.

 

Summary

In this series of articles you learnt how to

  • Create a new webhook in Power Automate (MS Flow) for a Survey123 survey
  • Add conditions, convert and format the survey time
  • Send a notification email with dynamic content as part of the webhook
  • Find additional settings for your webhook

 

I hope you enjoyed the series! Feel free to share the articles with your network, leave feedback or message. 

You can download the article series in PDF format from here.

In this blog post I will explore a handful of XLSForm techniques to help you better control the organization and layout of questions within a form. You will learn about groups, grids and pages, but also other tricks to refine the size, placement and look and feel of questions within your form.

 

--------------Groups--------------


Groups: XLSForm Basics


Groups organize questions into sections, helping users more easily navigate complex forms.Our Survey123 for ArcGIS: Using Groups to split your survey into sections - YouTube  video is about the quickest way there is to learn the basics of XLSForm groups. I strongly suggest you watch it, it's only 2 minutes long.

 

 

As described in the video, you can create sections in your form by enclosing a set of questions within a begin group/end group block. You need to specify a value in the name column of the begin group row, and this name must be unique. If it helps you navigate the spreadsheet, you can also add a name to the end group row, but it is not compulsory.

 

nametypelabel
texthh_idHousehold ID
begin grouphh_groupBasic Household Information
geopointhh_locationGPS Location
hiddenhh_location_acHorizontal Accuracy
texthh_addressAddress
end group

 

You can add as many groups as you like to a form, and you can nest them too: you can add groups within groups!

                

On occasion, you may want to display groups collapsed and let the end-user expand them if needed. This can be achieved by setting the appearance column to compact.

 

It is not possible to dynamically expand or collapse a group with an XLSForm expression.

Use the compact appearance to make your group appear collapsed by default.

              

 

Formatting Group Headers

 

Since groups are fundamentally used to help users navigate long surveys, it is important to take pride in the headers. The group headers, if formatted appropriately, can really make a big difference for people.


The font of group labels by default is a bit bigger and bolder than the labels of questions. This helps the group header stand out. Using HMTL formatting within the label column of the begin group row, you can bring your own style to the group header. For example, you can center the label in the screen, change its color, and size.

 

In the example to the right, the group headers have been given different background colors to help users navigate more quickly between them. Note that the first two groups, in red and blue, are collapsed but users can choose to expand them.

 

The Style your survey—Survey123 for ArcGIS | Documentation help topic shows the most common HTML tags that you can use to format your labels. When setting colors, you can use either their HTML code name or their hexadecimal representation. At W3Schools you can use the handy color picker for HTML colors. All the properties described in the documentation are supported in the Survey123 mobile and web apps except for the background color, which is only supported in the field app. 

 

The next XLSForm sample shows the syntax for defining the background color of the group header.

 

Body bgcolor is only supported in the field app. Webforms will ignore this property.

              

 

nametypelabel
texthh_idHousehold ID
begin grouphh_group<body bgcolor="#C00000"><font color="white">Household Information</font></body>
geopointhh_locGPS Location
hiddenhh_loc_acHorizontal Accuracy
texthh_addrAddress
end group

 

Dynamic Labels for your Groups

 

When working with labels for your groups (and questions, actually), remember that on top of the formatting syntax described above, you can also dynamically change the label content. For example, say you have a health household interview survey where we you need to capture all symptoms detected in each of the household members. You can present the symptoms group with a label that matches the name of the household member for which you are capturing the data. This could help eliminate confusion. Below is the actual example. Look how the group header in black changes depending on the household member name entered.

 

 

Check out Understanding Dynamic Labels in Survey123 for ArcGIS to learn more about this technique.

 

Visibility of Groups


One final note on groups relates to their visibility within the survey, which is controlled through the relevant XLSForm column. If an expression written in the relevant columns evaluates to true, the group is shown, otherwise it is hidden to the user. 

 

A common bad practice with form design consists on repeating the same relevant statement to a large collection of consecutive questions. This slows down your survey, because the same expression needs to be evaluated again and again for each of the questions. If a sequential set of questions shares the same relevant statement, it is good practice to group them and apply the relevant expression to the begin group row only. If the group is relevant, all questions within the group will show. If the group is not relevant, all questions inside the group will be hidden. There is no need to duplicate the relevant expression again and again.

 

--------------Pages--------------

 

Introduction to Pages

 

Pages are particularly useful when working with large forms, helping you logically organize your questions and giving end users a sense of progress across the entire survey.


Even for smaller surveys, pages can be useful. In the example below we have a Special-Use application permit form.  I added a welcome image and text on the first page to create a cover for the form. Note that the first page does not expect any input from the user. It is just a welcome page, or cover page to the survey. Next, I organized the actual questions within my form into pages, giving each page a descriptive title at the top. As users move from page to page, a progress bar at the bottom shows how close they are to the end of the survey.

 

 

In the Survey123 field app, users can easily move through pages in the form one by one. Alternatively, a page menu lets users jump to any of the pages in the form with a simple selection. In the example shown below, the page menu helps the user jump from page 3 to 32 with just a couple of taps.

 

 

Pages: XLSForm basics

 

If you want to add pages into your form, firstly you need to set the style of your XLSForm to pages. This is done through the settings XLSForm worksheet as shown below.

 

form_titleform_idinstance_namesubmission_urlstyle
Zika Household Surveypages

 

Once the style is set, add one group for every page you want and set the appearance of the group to table-list. For example

 

nametypelabelappearance
texthh_idHousehold ID
begin grouphh_groupHousehold Informationtable-list
geopointhh_locGPS Location
hiddenhh_loc_acHorizontal Accuracy
texthh_addrAddress
end group

 

When working with the pages style, it is good practice to have all questions within a page. That is: make sure all your questions in the form are within an existing group with an appearance set to table-list. If you leave questions outside of a page, you will get erratic behavior across the web and field apps where a single question may appear on every page.

 

You can combine groups and pages. That is, you can include groups within a page. Groups do not become pages unless you set their appearance to table-list.

 

All the header HTML formatting tricks described for groups above, also apply to pages. The same goes for the behavior with the relevant expressions. You can hide pages by using relevant statements, however keep in mind that the behavior across the web and field app is different. In the web app, non-relevant pages will be truly hidden, you will not even see them in the page navigator controls. This will cause the total count of pages in the page navigation control to vary as pages become visible or hidden. In the case of the field app, as of version 3.7, the pages will still show in the page navigation control, but will be shown blank. This is a known issue (BUG) in the field app.

Hidden (non-relevant) pages in the field app will be shown blank. This is a known issue as of version 3.7.

                

 

--------------Grids--------------

 

Grids: Introduction


Typically, questions within a form are arranged to take the full width of the screen. It is possible however to arrange multiple questions side by side using custom grids (theme-grid in XLSForm jargon). This is particularly useful when you are designing a form for use on a tablet.

 

With this type of grid layout, you can maximize the number of questions shown within the width of screen, reducing the need for users to scroll up and down. Grids can also help you mimic more closely the look and feel of legacy paper forms. Copying the layout of paper forms is not necessarily a good idea, but in some cases achieving some level of resemblance can make nostalgic users happier.

 

In the example shown to the right, you can see how questions can be arranged side by side. Crafting such a survey, defining exactly the placement of questions takes some thought and trial and error, but the experience it provides to the end user is significantly better than with a regular layout. This type of design is generally put together with a specific form factor in mind. That is, you design your survey for a particular minimum screen size. If a grid design like the one shown is loaded in a smart phone, the form is not going to look right. I have seen some people creating two flavors of the same survey: one for tablets and one for smart phones.  It clearly adds an overhead during design, but if your survey questions are not going to change much it is a good thing to consider.

 

The theme-grid style is not supported in the web form. It is only supported in Survey123 Connect and field app.

          

 

Grids: XLSForm Basics

 

To arrange questions in a grid layout, you first need to set the style of your survey to theme-grid. This is done in the settings sheet of your XLSForm.

 

form_titleform_idinstance_namesubmission_urlstyle
My Surveytheme-grid

 

Once the theme-grid is set, all groups within your survey will contain questions using a four-column grid layout. You can change the number of columns in a grid by simply adding the w (as in width) property to the appearance column of your group. For example, w2 means two columns, w8 means eight columns.

 

nametypelabelappearance
texthh_idHousehold ID
begin grouphh_groupHousehold Informationw2
geopointhh_locGPS Location
hiddenhh_addrAddress
texthh_cityCity
texthh_zipZIP Code
end group

 

Unless specified otherwise, questions within a group will always take one column of the grid. In the example above, the GPS Location and Address will be placed side by side in one row, and the City and ZIP Code will complete a second row.

 

Using the w property for the individual questions within the group, you can choose how many columns you want each question to take. In the example below, I gave two columns (one full row, since the grid has two columns) to the GPS Location question, and another full row (two columns) to the address. I then arranged the City and ZIP Code questions side by side giving them a w1 appearance.

 

nametypelabelappearance
texthh_idHousehold ID
begin grouphh_groupHousehold Informationfield-list w2
geopointhh_locGPS Locationw2
hiddenhh_addrAddressw2
texthh_cityCityw1
texthh_zipZIP Codew1
end group

 

 

You will also note in the screenshot above that the Household ID question is taking the full width of the screen. That is expected. When using the theme-grid, questions outside of a group always get rendered as usual, taking the full width of the screen.

 

Grids: A few extra tips and things you should know

 

  • Survey123 Connect includes a handy XLSForm sample called Grid Style Groups. It shows some advanced techniques, including arranging repeats and geopoint questions side by side, combining pages with grids, etc.

 

 

  • Working with grid layouts gives you a lot of flexibility, but mastering it takes time and trial and error. Always test your designs on a mobile device that you are targeting to make sure things look as you expect.
  • Do not expect the grid layout, or questions within a grid, to automatically adjust to the size of the screen. If you do not give enough space to your questions, they will look ugly. For example, in practice, you can't work with more than 2 columns on a smartphone. Even when working with 2 columns you need to be careful when working on small screens: you need to keep the labels of questions and choices short and avoid question types like geopoint or dateTime which tend to take more space. Do not take this the wrong way, you can use 2 columns in a smart phone, but with caution.
  • I know I said this before, but I will say it again: the theme-grid is not supported in the webform. It is only supported in the Survey123 field app.
  • You can combine pages and grids. Simply set the style of your form as follows:

 

form_titleform_idinstance_namesubmission_urlstyle
My Surveytheme-grid pages

 

  • When using theme-grid, questions outside of a group (grid) will always take the full width of the screen.
  • If you do not define a width (w parameter) for a group, the number of columns in the grid will be four (as in w4)
  • If you do not define a width (w parameter) for a question within a group, it will default to w1.
  • Non-relevant questions will still take space in your layout. If a question is non-relevant, you will see an empty space in your grid.
  • Be careful with or_other. If you use a select question with the or_other option, your question will always take one extra cell (column). This is in anticipation that the end-user selects the 'other' choice. In that case, a new free text question appears, taking the extra space.
  • You cannot apply a width parameter to a repeat, but you can apply it to a group within a repeat.  If you want to learn how to arrange repeats side by side, have a look at the Grid Style Groups XLSForm sample in Survey123 Connect.
  • The table-list appearance does not work in combination with theme-grid or pages.
  • The body::esri:style XLSForm column can help you refine the height of questions. This can be useful, particularly when working with the theme-grid. For example, if you put a text and a geopoint question side by side, the layout will show unbalanced, because the text question by default is not as tall as a geopoint question. You can change the height of geopoint, geotrace, geoshape signature and multiline text questions. Learn more at Esri custom columns—Survey123 for ArcGIS | Documentation 

 

 

When using the body:esri:style column, keep in mind that some properties are not compatible with theme-grid. For example, borderColor is not supported (do you need border colors? If so Vote for this GeoNet idea), and repeats inside of grid style groups do not support backgroundColor either.

Part 1 of this article can be found here.

In Part 2 we'll have a look at how we can add the survey's time to the webhook and how to convert time zones and format the date/time string.

Add survey time to your notification email

Survey123 submits date and time from your survey form in UNIX format. This means that the date & time value will be in milliseconds. By default, you cannot really display this in any usable format unless you convert it first. There are many time converters available on the internet, like this one to do that. However, we don’t need to use these in Flow as we can set up a function for this.The steps below are based on the webhook we created in Part 1 - but can be applied to any existing or new webhook (assuming that your survey has a date/time field in it).

1.   Click on Add an action under the If yes branch of your condition No alt text provided for this image
2.   Search for ‘add to time’ and select the action from the result list.
No alt text provided for this image

3.   To edit the action, click into Base time and switch to the Expression tab in the context dialog:
No alt text provided for this image

4.   Enter the expression* into the function box (fx) and save your webhook:
addseconds('1970-1-1',div(triggerBody()?['feature']?['attributes']?['survey_datetime'],1000))

NB:
This example assumes that the date/time field in your survey is called 'survey_datetime'. If it's different in your survey, you need to modify the expression accordingly.

* I am aware that a similar expression exists in this blog article, but all the date formatting strings I've tried gave me an error in Flow. I believe Flow was updated since that article and no longer accepts the formatting string as part of the expression. When leaving the formatting string out, the expression was accepted and works just fine.

So, what does the above snippet do?

The addseconds() function adds the seconds specified to a base time. The base time for Unix is 1st January 1970, hence the '1970-1-1' part as base. Then in the second parameter we call the dynamic value from the ‘survey_datetime’ field – but as this is in milliseconds, we need to divide this by 1000 as the addseconds() function needs a value in seconds. That’s why we added the div() function, which provides this division for us.

 

You can add a static hour/day/month/etc. value to this calculation specifying the Interval and Time unit in the step. (For example if your device's clock is misconfigured.) We don't need to add anything in our case, so we just set it to '0 Hour'. 
No alt text provided for this image

Convert time zone and format string

As discussed above, your survey's date & time is submitted in UNIX format (also known as UNIX Epoch, UNIX timestamp or POSIX time) which starts at midnight (UTC) on 1st January 1970. If your local time zone is different from UTC, you may want to convert the calculated time into your correct time zone.

The below steps assume you have completed the time calculation step in the previous section.

1.   Click on Add an action under the If yes branch of your condition.
No alt text provided for this image
2.   Search for ‘convert time zone’ and select the action from the result list to add it to your flow. No alt text provided for this image
3.   Click into Base time, then in the Dynamic content tab of the context dialog select Calculated time – this is the function we just created in the top half of this article. No alt text provided for this image
4.   Set up the Source time zone (it must be UTC) and your Destination time zone, then choose a Format string you want to use. 
No alt text provided for this image

 

 

In Part 3 we will discuss how to include the calculated/converted time and other dynamic content in the notification email and where to find additional settings for your webhook.

Update: Microsoft recently renamed Flow to Power Automate - link.

 

Note: Creating webhooks in ArcGIS Enterprise was introduced in v10.7 and the authoring process in that environment is different and will be discussed in a future post.

 

Starting with Survey123 for ArcGIS 3.0, webhooks can be set up and activated when a survey response is submitted. What does this mean? Basically, when a survey is successfully submitted, different workflows can be triggered based on the content of the submission: sending notification emails, adding results to a spreadsheet and so on.

 

There are a wide variety of workflow services available, such as Power Automate (aka MS Flow), Integromat, tray.io, Zapier, IFTTT, n8n.io, CloudHQ and the list goes on. Most (if not all) of them can be used to incorporate Survey123 as a trigger to build your automated process. Out of these, Integromat and Power Automate have a Survey123 connector making it easier to implement your processes. This article guides through an example using Power Automate.

 

What will you need to create Survey123 webhooks?

  • ArcGIS Online (AGOL) Organisational account
  • Your Survey123 form published into AGOL
  • Office 365 account to access Power Automate

 

Below is a workflow for creating a webhook that triggers an alert email if the submitted data meets a condition (for example the weather is Adverse).

 

1.  Open Power Automate (go to your O365 account and search for the app), go to My flows and click on + New > Automated - from blank :

 

2.  Give your webhook a meaningful name, so you know which one to edit/disable later if needed when you have more of them.

 

3.  Search for ‘Survey123’ under Choose your flow’s trigger, select When survey response is submitted and log in to your AGOL account (you only need to log in once, the connection is stored in your Flow profile). No alt text provided for this image

 

4.  Select your survey from the drop-down and click on New StepNo alt text provided for this image

 

5.  Choose an action: for example ConditionNo alt text provided for this image

 

and add your survey field(s) and the value(s) you want the webhook to be triggered for. You can select your survey fields from the Dynamic content part for the condition: No alt text provided for this image

 

Under the If yes branch, add a new step (click on the ‘+’ sign) and search for Send an email (Outlook). Set up the required details and save your flow – use both Save buttons: one at the bottom and one in the top right corner! No alt text provided for this image

 

You should be able to see the webhook triggered on the webhook’s page under RunsNo alt text provided for this image

 

If you click on an item on the list, you can see the details of that run. You can use this to see if a run failed (and why). No alt text provided for this image

 

6.  Download your survey in the Survey123 app and test your trigger. If you downloaded the survey before the trigger was added, delete the survey and re-download it. Otherwise your webhooks won’t work.

 

In Part 2 we will discuss how to add the survey time to your webhook/flow and convert time zones.

 

For the 2019 Esri Southern African User Conference I challenged myself and looked for a solution to a requirement that one of our clients have.

In a nutshell the client wants a survey that follows a preset workflow, for this example we will create a solution for a workflow that involves inspections done at regular intervals when a house is being built.

 

Survey Worflow

Let's consider the following status values for a building inspection survey:

 

Here are some user requirements for this survey:

  • The initial status is always Excavation
  • The inspector should not be able to skip a status
  • The inspector can only move on to the next status in the chain, not backwards
  • Inspection questions for a specific status can only be edited when that status is the active status

 

How will we go about forcing this workflow in Survey123?

There are some hidden challenges in getting this to work. The biggest hurdle is that in Survey123 you cannot compare a field to it's own or previous value, so you cannot tell Survey123 the next status can only be Foundation when the current status is Excavation. This has lots of design implications and one would be tempted to stop looking for a solution right here. But this is where you need to think a little bit out of the box...

 

So how does the end result work? Lets take a look:

 

 

 

In the above screenshot the following can be seen:

  • The survey initialises with only the Excavation status being visible in the select_one question. The inspector is not able to select any other status.
  • The survey only has the Excavation Checklist enabled
  • The inspector cannot skip a status or go back to any other status
  • The inspector cannot complete any other checklist of the survey - only the active status' checklist

 

The Solution

 

Lets look at the bits and pieces to make this work:

 

We need a way to only allow the inspector to choose one "next" status and this needs to be based on the "current" status. So we need a select_one that is filtered according to the "current" status:


 

Notice that we added a filter column called "prevval" to the choices for the "level" list. In this example "level" refers to the status of the inspection. Also you will notice that the levels run from 1 to 6 and the corresponding "prevval" filter is one level behind. So if the "prevval" or current status is zero (0) then the list will filter and only show Excavation as an option.

 

Ok, time for the interesting part of this solution. Since we cannot filter the list called "level" with itself we need to find a way of retrieving the inspection status into a new variable. This is done by quering the current status value from field "status" from the feature layer with a JavaScript pulldata function:

We do the max function so that for newly created records we start with a level of zero(0) - the pulldata returns nothing since there isnt a record created yet to query so we convert it to a 0 with the max function.

 

Basically this is it! There are some other work to be done, such as writing the JavaScript file to query the data from your hosted Feature Layer. If there's enough interest I will post the source files after cleaning it up a bit.

The 2020 Esri International User Conference is still eight months away, but the deadline for submitting presentation abstracts is approaching fast in November 8!   You do not need to have a fully developed story to submit your paper. A brief abstract describing what you intend to present is plenty. We are looking for inspiring stories showing how you are using #Survey123 to make a difference in your organization.

 

User Presentations

 

Every year many dozens of users present at the Esri UC. Definitively a great opportunity to share work with your peers. We all learn when information is shared! You can check the Recent Proceedings from Esri events to see what people have shared in the past.

 

Gidelines to submit your abstract for the 2020 UC can be found at Call for Presentations | Esri User Conference 

 

In short:

 

  • Author/Presenter name(s)
  • Author/Presenter name(s)
  • Presentation title (10 words or fewer)
  • Company name
  • Complete address, phone, and email information
  • Brief presenter biography (25 words or fewer)
  • Abstract (500 characters or fewer, including spaces): Do not include information in your abstract that will not be in your presentation.
  • Keyword set (list only 3–5 keywords): Keywords are used to organize presentations into appropriate sessions, so choose words that clearly describe the main ideas of your work.

 

 

Become a Sponsor

 

Be aware that user papers and presentations must be noncommercial. If you plan on advertising a product, service or company, there other venues for you. You can Become a Sponsor or Exhibitor | 2020 Esri Partner Conference As a sponsor, you can have your own booth in the Showcase area, collaborate to put together Special Interest Groups, etc.

 


 

 

 

This is my contribution to notify to an user (if not using survey123 in disconnected mode) than the form needs to be reload.

 

 

 

How it‘s working:

 

 You publish a table in AGOL with a field contained the version info (VAL_VERSION), each time you open the form in the field survey123 a function *js send and ID, retrieve the version and the calculation perform the comparison

 

 

 

This example uses a javasript function to send and ID and retrieve the version

 

  

 

What you need:

 

 

 

      
  1.  The javascript code in the zip (contain, the *.xls and the *.js function)
  2.   
  3. Create a reference table or a feature service hosted  in AGOL( don't forget to share the table to the same group using yours Survey123)
  4.   
  5. The url of the reference table to retrieve the official value. The table is publish in AGOL. I provide a example of the table, see the cvs to create the table on AGOL.
  6.   
  7. Add an  folder named "extensions" in the survey123 project  containing the *.js.

 

 

 

Note :To make the JS functioning, each time you modify this the *.js file you need to close and reopen the survey123 connect app

 

 

 

Detail:

 

 

 

All the configuration are done from the xls

 

 This is the detail of the values in the XLS ant in the JS function :

 

 Javascript  function:

 

 function QueryFeatureLayerByField(featureLayer,Queryfield,Value,outFields,token,debugmode)

 

 Pulldata *.xls:

 

 pulldata("@javascript","myFunctionsFR_Demo.js","QueryFeatureLayerByField",${myURLversion},"ID",string(${ID_version}),"*",pulldata("@property","token"),true)

 

  

 

                                                                                                                                                       
variable in the function *.jsvaluespulldata correspondancedetails
featureLayer"https..../0"${myURLversion}table's URL
QueryfieldID"ID"the ID field for the query
Value1string(${ID_version})the value to retrieve in the query , ID = 1
outFields*"*"fied retrieve from the  query. ou can choose all the field * or a specific field “VAL_VERSION”

 

 

 

Survey123 connect:

 

 Result:

 

 

 

 

Exemple of the table:

 

 

 

 

 

This method could be also use to retrieve other info (from the survey himself, past summit ) to populate default value info.

 

 

 

thanks for the *.js code and the idea  to @Chelsea Rozek  for the post  Force update to new survey

 

and the post https://community.esri.com/message/823053-pulldata-json-to-query-feature-service-with-token

The Survey123 Early Adopter Program (EAP) helps you access the latest Beta builds of the software. You can sign in with your existing Esri account and gain access to software downloads, documentation on upcoming features and discussion forums.

 

The Early Adopter Program is critical to Survey123. It allows the development team to share with you early cuts of the software feedback can be gathered. This early feedback is used to refine new features and fixes before they are made available under General Release. The EAP offers a great opportunity for you to try out your own surveys and workflows with the Beta software to anticipate any issues that may arise in the upcoming releases.

 

We just updated the EAP with a Beta version of Survey123 3.7, which is planned to be made available in December this year. Not all the new features and work you see in the EAP will make it to 3.7.Only features that have received positive feedback make the cut for general release. Below is a list of some of Beta features for which we want to actively gather your feedback, and some thoughts on when we estimate the features will join the production release of Survey123.

 

As a general rule, we prefer feedback regarding our Beta releases through the Early Adopter Program Forums. We like to keep discussions in GeoNet for the released version of the software. Keeping things separate avoids potential confusion.

 

Survey123 Feature Reports

 

We released Survey123 Feature Report capabilities back in July 2019. The Early Adopter Program will give you access to the Beta builds of the Survey123 website, which include:

 

  • A new option to generate outputs in PDF format. We want to make sure this works well with your existing report templates and that you do not have any issues with fonts once you open the PDF on your computer.
  • A new option to generate Preview Reports, so you can design your report templates without incurring ArcGIS Online credit costs.

Please note that when generating reports (other than previews) through the Early Adopter Program your account will be charged with credits. Use the preview option if you do not want to be charged.

When testing the new PDF and report preview capabilities, please report your findings (positive and negative) in the new Survey123 website forum in the EAP website. We are targeting 3.7 for the release of both of these functions.

 

Survey123 Field App

 

Through the Early Adopter Program you can access Beta builds of the Survey123 field app for iOS, Android, Windows, MacOS and Linux. The Beta build of the Survey123 field app cannot run on the same device as the released version, so you will need extra hardware to test it or remove the released version off the device first.

 

Our 3.7 release is mostly focused on bug fixes and performance improvements. Information about the specific bug fixes addressed in each Beta update are detailed in the Announcements section of the EAP website. Some of the main areas of work include:

 

  • The numbers and calculator appearances went through an entire overhaul. The main goal of this work is to address data loss issues reported. We also revisited the overall user experience and came up with a new design to accelerate data capture. We believe we got the data loss issue completely resolved. Can you prove us wrong? Your feedback regarding the new look and feel and usability will also be welcomed.
  • Inbox performance improvements. Our own tests show pretty substantial improvements in terms of memory consumption and speed. This means a better user experience and reduced chances of a crash, particularly on low-end devices. We want you to put points, lines and polys to the test in the Inbox, particularly when working with related records. We are not looking at the Inbox as the means to load 10K records ion your device, but working with a few hundred and even 1000 or 2000 should be now much smoother.
  • Added support for Geode external GNSS receivers.

 

The features above are planned for our 3.7 release. We are actively working to address random logout issues in the field app. It is our goal to upload new Beta builds so this networking issues can also be validated through the EAP.

 

The Beta builds of the Survey123 field app are also useful for you to test your existing surveys. As much as we do our best not to break backwards compatibility with older versions, your own tests can highlight issues we would never find on our own.

 

Survey123 Web Form JavaScript API (New)

 

  • This is for any JavaScript developer looking to embed Survey123 forms within a web application. Using this API you can easily have the form interact with your own web app. You can apply your own CSS styling, calculate questions and hook to web form events to make your web app and survey work like one.

 

This is the very first time we expose this feature through the Early Adopter Program. We are initially aiming to release in early 2020, but as usual, the ultimate release date will be dictated by your own feedback through the Early Adopter Program.

 

On-Premises Survey123 setup (Updated)

 

  • A Windows-only setup for those of you who want to deploy the Survey123 website and Survey123 REST API locally in your own on-premises environment.

 

We have had our On-Premises Survey123 setup available in the Early Adopter Program for some time. We are progressively making adjustments and improvements driven by your feedback. Currently, the beta version is ready for anyone who wants to test the website and API locally while keeping a connection to the internet available. If you plan on deploying in a completely disconnected environment (with no access to the internet), keep in mind that Survey123 Connect will not be able to work in that environment (the website and API will).

 

We are aiming towards a release of this feature (subject to positive feedback) within the first half of 2020.

 

Standard Map Types

 

  • Standard map type functionality was introduced in 3.6 as a Beta feature. This is the only Beta feature included in the released builds of the software.  Standard map type functionality needs to be explicitly activated in Connect and the field app, allowing you to work with web maps, mobile map packages and vector tile packages. 
  • Please note that our Beta builds in the Early Adopter Program do not include any new work in this area when compared with the 3.6 release. For this reason, if you are testing this feature you may want to wait for upcoming updates in the Early Adopter Program.

 

 

Other topics

 

You will find other Beta features such as how to use custom JavaScript functions in the field app or how to work with the Survey123 Feature Report service REST API.

Camera systems can help make homes and businesses safer. This technology has become cheaper and more accessible to consumers than ever before. As these systems multiply, they have become an important tool for capturing evidence of criminal activity in our communities. But with so many cameras it can be hard for police to know where these cameras are located or who to contact for access.  

 

Camera registration programs are a new initiative many police departments are introducing to enlist the communities help in crime fighting activities. These programs allow citizens and businesses to register their surveillance cameras with the local police department. Registration in these programs gives police permission to access footage from these cameras if a crime takes place nearby, which can provide much needed evidence in support of investigations.

 

Registering a camera does not provide police remote accessed to the camera or a live video feed, but it does give them a good database of security cameras in the area without having to go door to door to solicit this information.

 

By using the Survey123 for ArcGIS, police departments are easily able to setup a voluntary camera registration form that citizens can use to share the location, characteristics and contact information of private surveillance systems.

 

There are a number of Voluntary Camera Registration forms built with Survey123 out there. For example:

 

 

This article describes how to create your own Voluntary Camera Registration form using Survey123 for ArcGIS. If first explains how to design and publish the form, then provides tips to keep gathered information safe, to prevent unauthorized access. The last sections briefly discusshow you can include a link, or embed the form within a website, social media, etc.

 

You will need an ArcGIS Online account with publishing permissions (Creator User Type), and about 45 minutes of your time to follow instructions in this article.

 

Designing your form

 

To create a new form using Survey123 for ArcGIS, you will need an ArcGIS named user account with permissions to create content and publishing feature services in ArcGIS. If your organization does not have ArcGIS Online yet, you can create an ArcGIS Trial | Free 21-Day Trial subscription for free.

 

Before you start, it is best to spend some time figuring out what questions you want in your form.  Some of the examples above (St Charles, Redlands or Marietta) can serve you as inspiration.

 

Try to keep the form simple.  Write down your questions on paper and think carefully about the exact wording you want to use. Once you have your draft form design on paper, run it through a small group of colleagues and incorporate feedback. Once you have some consensus, it is time to build a first iteration with Survey123.

 

Next, purely as an example, I will provide step by step instructions that you can follow to learn basic techniques to add questions into your form.

 

  • In a web browser, navigate to survey123.arcgis.com and sign in with your ArcGIS Online credentials.  This will open the survey gallery.
  • Click on Create New Survey and select Survey123 Designer. Designer lets you visually create surveys right from within your web browser. It is very easy to use and plenty for what we need.  Survey123 Connect is a more advanced authoring tool, but we will not be using it here.
  • Choose a Name and tag for your project and optionally set a thumbnail. Then click Create.

 

Tip: If the animation below is too small, click on it to enlarge it.

 

 

After your initial survey project setup finishes, the Survey123 designer will open. You can now start adding questions to your form. We are going to organize our form with a header, two question groups, and a closing section for a disclaimer and terms of use acceptance.

 

  • Lets start by adding and labeling the groups as shown in the animation.

 

 

Groups are a great way to break down your form into sections. They can help you make your form more readable. You can add as many groups to your form as you like and even add groups within groups.  For a simple form like this we will keep all questions and groups within a single page.  For larger forms, you may want to break your survey into pages.

 

  • Add the Name, Email and Phone questions into the Contact Information group. Note that you can use hints to add subtitles into your questions and also mark questions as required. For the Name and Phone questions, use a Singleline Text type of question. For the Email, use the Email type of question, which has built-in email validation.

 

 

Tip: You can use the Preview option in the lower-right corner of Survey123 designer to look at your form as you build it. Note that you can preview your form in desktop, tablet and mobile form factors.

 

In the Security Device Information group we are going to add questions to capture the location of the camera and a few key characteristics.

 

  • For the location of the camera, we will add a map question (Geopoint question type). Make sure you mark this question as required. Optionally you can use the left panel to select the default basemap and zoom level for your map.

 

 

  • In addition to the map, it is a good idea to add a question for users to enter the address where the camera is located (Single Line Text), as extra validation.
  • Add a question of type Single Choice to capture the number of cameras at that location. We could add also a Number type of question, but to keep things simple we will just provide three choices: One, Two and Three or more. You may also want to arrange the choices horizontally to reduce the need for users to scroll up and down.

 

 

Next we are going to add a question to find out if it would be possible to have access to the footage and if so, what is the retention policy. We will use a rule to control the visibility of the footage retention question.

 

  • Add a question of type Single Choice. Change its value to Can the footage of this camera/s be accessed and shared? Then add Yes, No  to the choice list. You may want to also add I do not know.
  • Add a second Single Choice question type. For the label, use Footage retention policy. For the choices set them to: A few hours, One day, One week and More than one week.
  • Select the Yes/No question above in the preview and click on the Set Rule button so you can control when the Footage retention policy is shown.

 

 

You can apply rules to choice questions such as Single Choice, Dropdown and Likert.  When defining a rule, you can control visibility on a single question and entire groups.

 

Tip: Save your work from time to time. It does not hurt! The Save button is on the lower-right corner of Survey123 designer.

 

Now we are going to add questions for people to optionally provide additional information about the camera. system. On top of the basic types (text, choices, numbers), Survey123 forms also support photos and even documents.  People for example may want to upload a zip file with blueprints of the building highlighting the exact location and coverage of the camera, etc.

 

  • In the question panel on the right, look for the Image question type and add it to your form. Set the label to Attach a photo of what your camera can see.  You can add multiple Image questions if you want to let users add more than one photo.
  • Add a File Upload question type. Note that you can define what file extensions will be accepted. This question will let users upload documents such as zip files, pdf documents, etc.

 

 

The basics of your form are set. We will now work on the disclaimer section and the header.  The disclaimer section is important according to your own lawyer. Make sure you check with her first!  I will use in this tutorial some boilerplate text as an example but do not quote me on this one! The exact Terms of Use and Disclaimer text you will want to use are something between you and your lawyer.

 

  • Add a Single Choice question to the bottom of your form, outside the group. Use the Title and Hint to paste your disclaimer text.  Reduce all choices to just one: I agree and make sure you mark the question as required. In this way, users will not be able to submit their data unless they agree to the terms.

 

 

The header is also important because that is a good place to insert some context about your Voluntary Camera Registration Program. This is where you can tell users why the Police Department is looking to collect camera information, etc. You can also use the header to bring the logo of your organization.

 

  • Scroll to the very top of your form and click on the Description text in the preview. Then replace the description text on the left panel. Note that you can insert links to other websites, images, change the size and alignment of your text, etc.  Make this look good because it is the first thing people will see.

 

Your form is complete, but since manners matter, we are going to refine the look & feel and also configure a Thank You screen that will appear after a form is submitted.

 

 

 

  • To the right of the Appearance tab, you will find your survey Settings dialog from which you can customize your Thank You screen. Note that you can include links to other websites and also embed your organization's logo.

 

Testing your form

 

The Publish button, in the lower-right corner of the screen lets you publish your form so you can start capturing data with it.  By default, published forms are only available to the author of the survey. That is, you need to login with the credentials of the person that created the survey in order to submit data.  This is by design, so you can test your own form before you share it with other people.

 

  • Click on the Publish button to publish your form.
  • Open the survey link and submit some test data.

 

 

I strongly suggest you give your form a quick test, submitting data and making sure all looks good, before you share it with other people. Once your data is submitted, you can check it through the Data tab in the Survey123 website. You will be presented with both table and map views of your data. Note that you can further edit information submitted. For example, you can update and delete your test records, right from within the Survey123 website.

 

  • Once data has been submitted, open the Data tab of your survey project and explore the submissions.
  • Learn how to update and delete submitted data.

 

Do not be shy, run your form by people in your office and get feedback on it. You can make further adjustments to your survey at this point, and then publish the changes again. Once you feel confident with your form design, you are ready to share it.

 

Sharing your form (to submit data)

 

Using the Collaborate tab, you can share your survey design for others to submit data. You want to be extremely careful following these steps or otherwise you could be compromising the data that users will submit.

 

  • In the Survey123 website, click on the Collaborate tab.
  • On the left side of the screen, make sure that the Submitter tab is selected.
  • In the Who can submit to this survey section, select Everyone.
  • Scroll down a bit to the What can submitters do? section and switch the default option to Only add new records.
  • Click on Save to publicly share your form.

 

 

The What can submitters do? section is critical. You want to make sure you enable only access to adding features. If you also enable the option to update features, then a malicious person could technically query, download and even modify data submitted through your survey. 

 

In short, what we have done so far is to make our survey publicly available, so anyone, even without an ArcGIS account can submit data, but we have disable all options for non-authenticated users to query and download the data. That is, we have shared a survey that can only be used to submit data: your data will not be compromised.

 

Next, you can get the link to your survey and use it within social media, your own website, etc.

 

  • Scroll up to the top of the Collaborate dialog and copy the link to your survey.

 

 

You may notice that you can also get a QR code for your survey and even easily copy and paste HTML code to embed the survey within a website.

 

Sharing your form (for people to look at survey results)

 

Data submitted to your survey will only be available to you, the author of the survey, unless you explicitly share your survey results with others. This can also be done through the Collaborate tab.

 

  • While in the Collaborate tab, open the Viewer section in the left panel.
  • Select the groups within your organization that will have access to the survey data
  • Click Save.

 

 

Any person with an ArcGIS account belonging to the groups you select in this dialog will be able to log into the Survey123 website to look at the results of the survey. Viewers will be able to download, filter and map all submitted data.   Think carefully who you want to grant access to this data. You certainly may not want to allow Everyone to access this data. Keep it safe.

 

Socializing your form

 

A key aspect to make your Voluntary Camera Registration campaign successful is to promote it within your community. Connect with people in your organization to find the most appropriate way to present your  registration form. Using links provided in the Collaborate tab, you can:

 

  • Include links to your form in social media: Facebook and Twitter for example.
  • Include a link or the survey itself within an existing website (use the embed option).
  • Include a link to your survey in an e-mail campaign to residents and business owners in your community.
  • Add a QR code to easily open your form from printed marketing materials.

 

Maintaining your data

 

All data submitted through your form will be kept secure in ArcGIS Online (or your own ArcGIS Enterprise organization if you happen to run on ArcGIS Enterprise). I am not a big fan of exporting data out into Microsoft Excel or other formats because the export output becomes obsolete as soon as someone adds a new record. Having copies of the data proliferate can also become an issue. Whenever possible, it is best to have everyone work against a single source.

 

You can use the Survey123 website to visualize, analyze and curate all data submitted. The Data and Analyze tabs within the Survey123 website are pretty easy to use, yet powerful.

 

I will emphasize here that you can not only view, but also curate all submitted data from the Data tab of your survey in the Survey123 website. All records submitted can be updated and deleted right from the table and individual response dialogs within the Data tab.

 

We did not talk much about the Analyze tab in this article, but you will find more info in the Understanding Your Results with the Analyze Tab post.

 

You can also use other tools within ArcGIS to look, analyze and curate your data. ArcGIS Pro, Web AppBuilder, etc are great examples.  All data in Survey123 resides in a standard ArcGIS Feature Layer, so anything you can do with a Feature Layer, you can also do to Survey123 data.

 

I hope the above was easy to follow. If you have additional questions or feedback, do not hesitate adding your comments below. Thanks!

A minor update to the Survey123 website and field app has just been made available. This new release includes a handful of critical issues:

 

  • BUG-000125792: Japanese and Chinese keyboards not working in the field app on iOS 13.
  • BUG-000125769: The Survey123 field app image gallery is unsorted on Android devices.
  • BUG-000125687: Attempting to publish or save a survey in the Survey123 website containing a multiple choice question with the choice name containing more than 32 characters fails with the error, "Errors when parse form."
  • Null geometry is submitted by the Survey123 web app if there’s no Geopoint question in the survey or the Geopoint question is left blank.
  • Corrupted characters being displayed in some languages in the field app on iOS 13.

 

The latest build number for the Survey123 field app on Android and iOS is now 3.6.157. The Windows version of Connect and then field app has not been updated. 

 

The source code of the Survey123 Enterprise Template in AppStudio for ArcGIS has also been updated to version 3.6.

 

In preparation for the 3.7 update, which is planned to be released before the end the year, we will shortly make the Beta builds of the Survey123 website and the field app available in the Survey123 Early Adopter Program.

Josh_Shelton

UAS Flight Log

Posted by Josh_Shelton Oct 8, 2019

UAS Flight Log 

Using Survey123 to perform UAS preflight checklists and track UAS flight projects

By: Josh Shelton, GIS Manager, Pend Oreille County, Washington

      Eric Roth, County Surveyor, Pend Oreille County, Washington

 

UAS Launch Logo

 

Sometimes the most elegant solutions are the simplest. This was the thinking behind using Survey123 to transform what was a pen and paper menial task into a more efficient, portable and user-friendly digital workflow.

In this article, we will discuss our motivations to use Survey123 at Pend Oreille County to support UAS (Unmanned Aerial Aircraft) operations. We will also share details on our implementation, approach to engage with stakeholders and lessons learned.

 

Understanding the need and benefits of using a UAS in Pend Oreille County

 

Pend Oreille County, Washington is a scenic rural county in the northeastern corner of Washington state nestled against the state of Idaho and Canada. Pend Oreille County encompasses approximately 1500 square miles of lush coniferous forest and is divided by the Pend Oreille River, a major tributary of the Columbia River.

 

Pend Oreille County utilizes UAS primarily for gravel pit & stockpile quantity calculations. The benefit here is that our crews spend a third less time collecting data than it would take utilizing conventional survey methods. We also use UAS for roadway pre-design topography maps, bridge inspections, flood tracking, and land slide tracking. By using UAS for these situations, our personnel can operate the equipment from a remote location keeping them safe and out of harm’s way.

UAS Launch Location

UAS launch site prior to flight

Eric Roth, Pend Oreille County

 

Unlocking the potential of Survey123 – Empowering our Users

 

Being a rural county and an organization with less than 200 employees, at the Pend Oreille County GIS group we are constantly trying to think outside the box when it comes to improving workflows while maintaining our need to be fiscally responsible for our constituents.

 

In an effort to better facilitate collaboration and information sharing, the Pend Oreille County GIS staff formed an internal GIS user group in 2009. This group discusses new technologies, showcases new projects and shares ideas empowering our users to improve the way they work with GIS.

 

This is exactly what happened when the GIS Department showcased Survey123 to the group for the first time. We discussed the power and simplicity of Survey123 and showcased how easily smart forms can be created without the need for advanced GIS or developer skills. When our county surveyor saw the technology he immediately began thinking of ways he could leverage Survey123 to improve his own workflows. This ultimately led to the creation of the UAS Flight Log.

 

FAA regulations mandate that all UAS (Unmanned Aerial System) pilots perform and document a thorough preflight check on their equipment and surroundings. UAS flight logs must also be documented.

Commonly, preflight checks and flight logs are manually recorded on paper. Completing these paper forms is cumbersome and can lead to errors and even data loss. There are several reasons why Survey123 smart forms are a natural replacement for obsolete paper forms: 

  •  Survey123 runs on the same mobile devices that everyone already has. There is no need for an additional piece of paper when the form can be completed on your own mobile phone or tablet.
  • Survey123 smart forms streamline data capture, allowing UAS pilots to focus in areas of the preflight check and flight log that require more attention.
  • Certain critical information such as location, times and photos are particularly easy to capture with Survey123 in a systematic and accurate way. That is the opposite of what happens with traditional paper forms.
  • Survey123 forms can be easily synced with our ArcGIS organization. This reduces chances for data loss, eliminates transcription errors, and facilitates immediate access to this information for visualization, reporting and sharing.

 

The nuts and bolts of our survey design

 

The survey was developed entirely using the Survey123 web designer at survey123.arcgis.com. The basic premise was to take the preflight checklists already being performed and recreate these in the survey. The added benefit here is that one could integrate post-flight observations as well as log our flight location and record the event with photos. The survey design is split into 4 logical sections:

UAS Flight Log
Form Summary

 

  • Flight Details – Here we record the following:
  1. Date of capture – This auto populates further simplifying the process
  2. Location – Uses device GPS to provide location or the user can select from the map.
  3. Purpose of flight – Comment field allowing for a customized description
  4. Photo of the UAS at the launch location – This helps to prove project location and UAS used, as well as the condition of the launch location
  5. Weather conditions – Comment field for customized descriptions
  6. Name of RPIC (Remote Pilot in Command) – Comment field for customized entry
  7. UAS pilot name, if different than the RPIC – Comment field for customized entry
  8. VO (Visual Observer(s)) names, if applicable – Comment field for customized entry
  • UAS Details – Here we record the Make and Model of the UAS.
  1. UAS Manufacturer – prepopulated dropdown containing the manufacturer of the UAS being used
  2. UAS Model – prepopulated dropdown containing the model of UAS being used
  • Pre-Flight Check List – This is a simple check list where the following items are “checked off” as they are inspected:
  1. Batteries charged and secure?
  2. Aircraft Hardware OK?
  3. Equipment & Gear OK?
  4. Remote Control OK?
  5. Propellers OK & Secured to UAS?
  6. Software & Firmware Versions Up to Date?
  7. Remote Control is Powered ON?
  8. UAS is Powered ON?
  9. UAS Compass is Calibrated?
  10. Camera is ON & Calibrated?
  11. UAS GPS is Functioning?
  12. Flight Control App is Running & Connected to UAS?
  13. Pre-Flight Notes – Here any ancillary information can be noted
  • Post-flight details – here we record flight start/stop times and note any post-flight observations
  1. Begin Flight – record the time the flight began
  2. End Flight – record the time the flight ended
  3. Post-Flight Notes – Records any additional observations

 

For those of you interested in seeing what we've done, here is the XLSForm

 

Using UAS for road and bridge inspection

Eric Roth, Pend Oreille County

 

Lessons Learned and next steps

 

From a GIS management perspective, the ability for our non-GIS personnel to develop and put into production their own data collection tools has been essential to success. Inspiring and empowering county personnel to implement GIS technology for their own needs has allowed GIS personnel to focus efforts on other tasks which help our organization improve the information, tools, and services we provide and utilize. It’s been said that necessity drives innovation and in this case the need to simplify a simple redundant workflow is proof that there is room for improvement in the seemingly most simplistic of tasks.

 

The use of Survey123 for UAS operations validated major benefits.

 

  • Ease of use – The intuitive design of the Survey123 mobile app facilitates rapid, efficient data capture with minimum training.
  • Flexibility – Using Survey123 web designer, smart forms can easily and timely be modified to match new requirements.
  • Reliability – Paper forms can easily be damaged or misplaced and handwritten notes are often illegible or difficult to decipher. Survey123 form data is directly stored in ArcGIS as soon as the mobile device as network connectivity. Using ArcGIS Online provides us with security and peace of mind that the data will be there should the need arise where we need to go back and review prior projects.
  • Portability – The Survey123 app is supported on a wide range of devices and platforms allowing us to easily make use of existing devices such as smartphones, tablets, etc.
  • Offline capability – Our operating areas are rugged and remote. Reliable network access is not always available making offline collection a necessity.
  • Longevity – Since the records are stored digitally within our ArcGIS Online organization and can be manually archived, we can guarantee access to the data into the future increasing the value and return on our investment.

 

Not only were we able to achieve our goals but we’ve helped reduce the time it takes to prepare and perform a flight. We can now get up to date imagery of our public works projects, use the imagery in conjunction with Drone2Map to calculate volumes of sand and gravel piles before road projects and in preparation of sanding roads in the winter. We’ve also helped to improve the safety of our employees by using the UAS to perform inspections on bridges that are often difficult and unsafe to survey manually. This critical return on our investment has helped to solidify the sustainable use of UAS technologies in our everyday workflows. Talks are currently underway to expand the use of UAS technologies to other departments in the hopes that it will add further improvements to our workflow processes.

 

Josh Shelton is the GIS Manager for Pend Oreille County in Washington state. Josh has been doing GIS and supporting the county for over 11 years. Pend Oreille County GIS supports all county departments and strives to help empower people to use GIS technologies to better themselves, the organization and the communities in which we live. Josh loves the outdoors and has an enthusiasm for technology, it’s this passion and drive which enables him to continually push the boundaries and think outside the box providing technologies and solutions to county personnel and their constituents.

 

Eric Roth is the County Surveyor for Pend Oreille County. Eric has been a licensed surveyor in Washington state for over 15 years and has been working for Pend Oreille County for the last 3 years. His love of surveying and technology led him to create the first UAS program in the county. Since the program’s inception, Eric has been working on ways to make the program more efficient, safe and cost effective.