Skip navigation
All Places > Survey123 for ArcGIS > Blog > 2019 > June
2019

Yeap! World UFO Day is approaching so we want to use this out-of-this-world release to remind you. This is a really substantial update to Survey123, including many, many fixes, new features and enhancements. We are really excited about 3.5 and we hope you will too! 

 

Survey123 Feature Reports

 

Many of you are already familiar with the Survey123 Feature Report capability, which we have had in Beta for quite some time. With this update, the Survey123 Feature Report service is no longer Beta. It is fully supported by Esri Technical Support and it is here to stay.

 

The Survey123 Feature Report service is designed to help you generate beautiful reports of your Survey123 records. You can define the contents and look and feel of your reports using a report template in Microsoft Word format. Once your template is uploaded to ArcGIS Online, the Survey123 Feature Report service can generate individual reports for records submitted from Survey123. 

 

The Feature Report functionality can be triggered from the Data tab of the Survey123 website as shown below. You can create reports for a single selected record, or in bulk, for a larger selection set.

 

 

You can control the layout, contents and look and feel of your output reports through report templates. One or more templates can be associated with a survey, so users in your organization with view access to the survey data can generate reports on their own.  Report templates are authored in Microsoft Word. You can add headers, footers, tables and format the text  in your template as you like.   Feature Report Templates include placeholders to bring responses from your survey record into the report. This includes text, dates, numbers, choices and checklists, photos, annotated images, signatures and of course, maps!  The syntax to bring data from your survey into a template is described in our Feature report templates help topic.  A full blog post dedicated to the feature report syntax is on its way.

 

Feature Report generation is available as a premium ArcGIS Online service. Users who want to use the service will need to be granted Feature report privileges. By default, all roles and role templates included in ArcGIS Online include this capability, but if you use custom roles you will need to turn it  on.  This can only be done by an administrator of an ArcGIS Online organization by including Feature report access to the custom member role. The next animation shows how:

 

 

As of today, and until the next release of ArcGIS Online (planned for October 2019), you can generate as many reports as you want for free. Starting October 2019, feature report generation will consume 2.5 credits per report file generated.

 

One final note for ArcGIS Enterprise users: The Survey123 Feature report service is designed to work and is fully supported with surveys and data hosted in ArcGIS Online. However you can also take advantage of this service with surveys hosted in ArcGIS Enterprise 10.5 or newer, with some limitations:

 

  • Your ArcGIS Enterprise instance must be accessible to the Survey123 ArcGIS Online Feature report service. That is, you must configure your  ArcGIS Enterprise web adaptor to expose access to your Enterprise instance from the internet. Please note that portals with Integrated Windows Authentication will not work with the report service.
  • The inclusion of any type of feature service attachments (photo, signature, annotated images...) is not supported when your data is hosted in ArcGIS Enterprise. This is a limitation listed in the backlog of ArcGIS Enterprise.

 

Bearing the limitations above, the use of the ArcGIS Online Feature Report service from your ArcGIS Enterprise instance will not incur into any ArcGIS Online credit costs.

 

 

Web surveys: Survey open, survey closed! 

 

Up until this update, if you wanted to disable access to a public survey,  you would simply unshare it. In this way, people could not longer submit data to your survey, but you could still view and analyze its results. The challenge with unsharing a survey is that when attempting to open your survey users would get a generic dialog asking for a user and password, and that is the last thing someone would expect.

 

You can now configure a "Survey closed" dialog and define when to show it. In the 'Survey closed" dialog you can add your own text, images and links describing to survey takers why survey responses are not being accepted. Through the Collaborate tab, you can also define the exact time when the survey will be closed or open.  For example, you can design and publish a survey today, schedule it to start accepting responses in November 1, 2019 and stop accepting responses in December 13, 2019.

 

 

This feature is currently only supported for web surveys.  The Survey123 mobile app will not honor the "Survey closed" custom dialog until a later release.

 

Web surveys: Avoid multiple submissions

 

This new feature is also built with web surveys in mind. The idea here is to prevent users from submitting a survey multiple times.  By default, users can submit data to your web survey as many times as they like, but you can now configure your survey to limit submissions to one. This is done through the Collaborate tab as well.

 

For clarity, the way this works is not bullet proof for public surveys.  If working with a public web survey, this setting will persist a cookie in the user's web browser. If the cookie is present, the user will not be able to submit again. A smart user can delete the cookie by clearing the browser's cache or running an incognito session in the browser.  Also, for public surveys, we can't really do anything if one person submits from different devices (a phone, a desktop computer in the office, a laptop at home).

 

 

If you really want to confidently block multiple submissions from the same user, the best option is to secure your survey to force users to identify themselves.  At that point, we can more consistently block multiple submissions.

 

Working with related records (repeats) in the Survey123 website Data view

 

Through the Data view in the Survey123 website you can interactively inspect all submitted records as a map, table and a form.  In this release, the table view and map views have been enhanced so you can navigate to related records in the table view and display or hide related records in the map. You can also apply filters to the table view.

 

 

In the example above, we have a Zika Household survey example. At each household, we have one or more household members and for each of them we document all trips they did in the past 4 weeks.  The member households and travel locations are modeled in the smart form as a nested repeat, and in the feature service relational model as two nested related tables. Look carefully at the table view and you will see that each of these layers/tables is shown in an independent tab. You can switch between the tabs, look at the data, apply filters and select records.  This gives you much more flexibility to explore your survey data!

 

Survey owner? Double-click to edit (Survey123 website)

 

One more enhancement on the Survey123 website Data tab. Double click on a cell in the table view... and voila, now you can edit the value.You can even edit related records. This function is only available to survey owners.

 

 

Photo Watermarking (Connect & Field App)

 

Over time we have received a number of requests to add watermarking capabilities into Survey123. This is important in some field data collection workflows; for example when performing certain damage ***assessments, documenting work performed or inspecting assets. 

 

The basic idea is to burn some critical information into the photo while it is taken from the device. Information such as the current date and time, the compass direction or the latitude and longitude. We have extended the XLSForm syntax so you can do all of this and much more.

 

In the example on the right we have a street service report survey. This is used to document work performed.  Note how the photo is watermarked automatically to include in the bottom-left corner of the image the unique identifier of the job. This job id comes from one of the questions in the form.

 

In the bottom-right corner you will see the date and time when the photo was taken as well.

 

You can control almost anything you want in these watermarks: the position, color, font, halo effects and of course the content, which can be dynamically calculated or retrieved from the form itself.  You can even use a logo in your watermark expressions!

 

To learn more about watermarks, check out our Survey123 Tricks of the Trade: Photo Watermarks blog post and our Watermarks help topic

 

Grid Theme (Connect & Field App)

 

Grid theme has been a long awaited feature, so we are really proud of announcing it today. Using grid theme you can control the layout of your survey to arrange multiple questions in a single row. This is important when you want to make your surveys resemble more closely paper forms, and when you want to take advantage of the wider screen of a tablet or desktop computer.  Using grid themes wisely can help you create much more usable forms to accelerate data capture because you can reduce the need to scroll up and down through your form.

 

Below is an example of a Certificate of Roadworthiness. No kidding, that word actually exists, you can google it .  Note how the form includes multiple sections: A first section at the top for the Tester's License Number, where we arranged a single question all the way to the right. Then a second section for the main title, with a single note centered in the screen. In section A (vehicle description) where we place questions, notes and even a signature pad side by side.

 

 

When taking advantage of grid themes, you will want to test thoroughly in your own field devices. A form with the layout shown above, for example, will work well on a tablet, but not on a smartphone.  Combining grid themes and multiple pages in your form is a really powerful combination too.  Grid themes and pages are worth their own blog post, so wait for one coming up soon. If you cannot wait, check our Appearance help topic for details on the XLSForm syntax, or the newly added sample in Survey123 Connect.

 

As of this release, grid themes are only supported in the field app, but we are working towards bringing them into the web as well.

 

Other miscellaneous topics (Connect & Field App)

 

You are going to find many other enhancements throughout Survey123 Connect and the field app in this release.  I simply want to highlight a handful, although you can find the complete list in our What's New help topic.

 

  • In Survey123 Connect, you can now preview your surveys in different form factors.

 

 

  • Constraint and required messages now appear inline, helping you identify the exact questions in your form that need attention. This is particularly useful when you have long forms, or forms with multiple  pages.

 

  • Many visual controls in the field app and Connect have been refreshed bringing a cleaner look.
  • The Survey123 field app now sends information about the type of submission (add or edit) to webhooks. This was introduced in our previous release for the Survey123 web app, and now we close the circle with this update for the field app.

 


Critical Fixes and Enhancements

 

There are a lot of new new features in this release, but in truth our biggest commitment for this update has been quality.  If you have been testing Survey123 through our Early Adopter Program you will recognize that many of the features we make official today had been on the works for quite some time. Our 3.5 release has been for the most part, devoted to fixing issues and applying some popular enhancements.

 

We want to thank all of you who have worked with us to validate many of these fixes over the past 10 weeks.  Here is a list of the most important bugs addressed.

 

Survey123 website and web app

 

  • BUG-000118801 An existing secured ArcGIS Server feature service added to ArcGIS Online with credentials saved is not be able to attach images when using the web browser to submit the survey (the data is submitted, but not the attachments).
  • BUG-000119623 Survey123, non GlobalID/Guid primary key field gets automatically populated with a GlobalID when submitted from the web form.
  • BUG-000120589 When a survey is published from Survey123 Connect, then modified in the Web Designer to check on "Allow the respondent to add another record" the option does not appear in the Web App.
  • BUG-000121051 Pulldata calculation on select_one question does not work when accessing the survey in a browser.
  • BUG-000122048 Survey123 doesn't submit survey through mobile browsers if the survey has pages and signature fields.
  • BUG-000121719 In the Survey123 for ArcGIS website, the total number of features displayed changes when logged in using the credentials of other people in the organization apart from the owner.
  • BUG-000122052 When running a report on a Survey123 for ArcGIS form that contains the Other option as a response from a Multiple Choice question, the Other option is repeated after each of the Other selected responses.
  • BUG-000122750 The sum() aggregate function does not concatenate string inputs and records a "NaN" value when submitting responses from a web browser.
  • BUG-000123131 Exporting results to a file geodatabase fails if a survey contains more than one nested repeat from the Survey123 for ArcGIS website.
  • BUG-000114635 Photos do not appear in the Analyze tab of the survey in survey123.arcgis.com when logged in as a custom role account.
  • BUG-000121812 The custom basemap is not visible in the Survey123 for ArcGIS web form when the basemap is not shared publicly.

 

Survey123 Connect and field app

 

  • BUG-000121623: Survey123 Connect for ArcGIS 3.3.51 and Survey123 Connect Classic 2.9.3 crash when using a pulldata() function that references a CSV with 1048,576 records.
  • BUG-000100441: Survey123 Connect: Truncated "Search" string and wrongly placed search icon in search box.
  • BUG-000103155 Surveys created from the Survey123 for ArcGIS Web interface that contain geopoint questions allow field users to zoom in further on the map than is allowable in the Survey123 for ArcGIS mobile app.
  • BUG-000103942 Large surveys crash the Survey123 for ArcGIS iOS field app when specific answers are provided on an iPhone 6 without saving data.
  • BUG-000113684 Surveys with multiple repeats with the repeat counts added crash upon opening in the Survey123 for ArcGIS (iOS) field app.
  • BUG-000113734 The survey creation performance degrades when greater than 5-6 repeats are added to a single survey form.
  • BUG-000114396 The Survey123 field app successfully submits the survey even when all the required questions in the Repeat field are not answered.
  • BUG-000097367: Truncation in Date field in Survey123 and Survey123 Connect.
  • BUG-000094992  Truncation in Publish survey in Survey123 Connect.
  • BUG-000094993: Truncated string in Settings-Style in Survey123 Connect.
  • BUG-000115227 When the language of an Android device is set to the Korean language, switching the device keyboard to Korean to type Korean characters into an autocomplete field in a Survey123 for ArcGIS survey is not possible.
  • BUG-000118014 Unable to use a Japanese keyboard in Survey 123 for ArcGIS (3.1.158) on Android devices. This behavior appears to be restricted to the Survey 123 for ArcGIS (3.1.158) release. This issue is not reproducible on iOS.
  • BUG-000118208 In the Survey123 field app, when a constraint is applied to a select_one question, it is not validated until submitting the survey.
  • BUG-000118366 Unable to send surveys in the Survey123 for ArcGIS field app when Layer ID has values other than the default value.
  • BUG-000119632 Once relevancy exposes a repeat in Survey123, the information in the repeat will get submitted regardless of whether the repeat becomes non-relevant again.
  • BUG-000120023 Select_one questions with an expression in Required or Relevant fields do not validate in Survey123.
  • BUG-000121057 In the Survey123 3.3 mobile app, a select_one question in a nested repeat conditional based on a select_one question within a single repeat will not display after changing the answer to the single repeat select_one.
  • BUG-000119414 In Survey123 Connect for ArcGIS, nested repeats are not preserved when scrolling through the Form Preview.
  • BUG-000121921 Data not displaying in first record of nested repeat with relevant expression referencing a question outside of the nested repeat when open from Drafts.
  • BUG-000122112 When required fields are not answered, the survey does not honor question ordering when alerting the user of required, unanswered questions.
  • BUG-000122534 Survey123 for ArcGIS hangs indefinitely when Form Item is moved or deleted from ArcGIS Online. An error message must be thrown instead.
  • BUG-000122255 The Survey123 for ArcGIS field app does not retain the font size setting on iOS and Android devices.
  • BUG-000121156 Submitting surveys from the Survey123 for ArcGIS field app retains related records from a repeat question, even after removing the attribute values for the same related records before submitting the survey.
  • ENH-000106588 Provide the ability to expand a Group in Survey123 by clicking the Group header text, not just the arrow.
  • ENH-000115797 Survey123 should allow users to have control over color schemes for the group box on a survey form.
  • ENH-000118020 Lessen the touch precision needed for the Group expand/collapse button in the Survey123 field app.
  • ENH-000120369 Clarify the messaging when deleting a survey from Survey123 Connect to be more explicit that AGOL/Portal items will be deleted.
  • ENH-000121453 Provide additional functionality for there to be two "Text" question types be adjacent on one line in the "Form Preview" page of the Survey123 Connect for ArcGIS, instead of only one "Text" question type on top of another "Text" question type.


Next steps

 

We are now furiously working on 3.6. Join our Survey123 Early Adopter Program to test some of the new features we are working on, including:

 

  • Support for custom webmaps and mobile map packages.
  • On-premises setup for the Survey123 website and Survey123 REST API.
  • Support for polygons and polylines.
  • And more...

 

For developers customizing or branding the Survey123 field app, we will be making version 3.5 available with AppStudio for ArcGIS 4.0 Beta in the next weeks.

 

Do you believe?

Here we are once again, so close to the Esri User Conference!  Here is a short guide to help you find Survey123 related events at the conference.

 

Technical Sessions:

 

Technical Sessions are one UC's favorites. Always well attended, Tech Sessions run for about 60 minutes and cover technical aspects of Esri products.  We always try to keep a good chunk of time at the end for a live question & answer section. Tech Sessions start on Tuesday morning and run until Friday.  They all take place in the second floor of the San Diego Convention Center (also known as SDCC).

 

If you want to start planning your sessions, go to our UC 2019 Online Detailed Agenda. You can search sessions by topic and presenter and if you are logged-in you can add sessions to your agenda.

 

Here is a list of some of the Survey123-related Technical Sessions we are offering:

 

  • Survey123 for ArcGIS, an introduction: This session covers the basics of the Survey123 website, including designing smart forms with the web designer, publishing and sharing surveys, generating high quality reports for submitted data and other capabilities available through the Survey123 website. The session will end with a sneak peak of smart form capabilities going beyond what is possible through web designer.  I recommend attending this session even if you are already familiar with Survey123, because there are many new things that have been added to the Survey123 website in the past few months.  Philip Wilson, James Tedrick and Zheng Liu and will be leading this session; fun and good content guaranteed!
    • Tuesday, July 9 10:00 am - 11:00 am SDCC - Ballroom 08
    • Wednesday, July 10 2:30 pm - 3:30 pm SDCC - Ballroom 06 E
  • Survey123 for ArcGIS: Working with webhooks: Learn how to use web hooks with Survey123 to automate all sort of tasks aftere a survey is submitted. Automate e-mail notifications, instantly move Survey123 data into databases and spreadsheets, integrate Survey123 with image recognition software and much more.  Ismael Chivite, Brandon Armstrong and Calvin Jung will show you the way!
    • Wednesday, July 10 10:00 am - 11:00 am SDCC - Ballroom 09
  • Survey123 for ArcGIS: Advanced Topics: This session will introduce advanced capabilities of Survey123 including the new theme-grid layouts, editing existing features and designing surveys on top of existing feature layers, workflow automation with Survey123 and web hooks and other exciting features. There will be lots of live demos and time for live questions.  Ismael Chivite, Elvin Slavik and James Tedrick will present this session.
    • Thursday, July 11 1:00 pm - 2:00 pm SDCC - Room 10
  • Survey123 for ArcGIS: Extending the mobile app: Awaken the developer in you. Join this session to learn how you can extend the Survey123 mobile app with your own Javascript functions, the new extensibility framework or even brand and create your entirely new Survey123 mobile app. Presenters: Ismael Chivite, Elvin Slavik and Erwin Soekianto.
    • Thursday, July 11 2:30 pm - 3:30 pm SDCC - Room 31 B/C
  • ArcGIS Apps for the Field: An Introduction: In this session we will describe how all ArcGIS mobile apps work together. A great place to learn how Survey123 works along with Workforce for ArcGIS, Navigator for ArcGIS, Collector and even Operations Dashboard for ArcGIS.  Scheduled presenters include Scott Ball and James Tedrick. 
    • Tuesday, July 9 4:00pm to 5:00pm SDCC - Room 2
    • Wednesday, July 10 8:30am to 9:30am SDCC - Room 2

 

Survey123 User Group Meeting

 

We are happy to announce that this year we will have our own Survey123 User Group Meeting. It runs on Tuesday at lunch time from 11:30 AM to 12:30 PM, but no lunch is provided. This is for the ones with a strong Survey123 spirit!  Our User Group will be a great opportunity to meet other Survey123 Ninjas. This is fundamentally about connecting with other people and the team.  We will have a couple of 7-minute lighting talks to warm-up and then the rest of the time is for group discussion and networking.

Survey123 User Group Meeting: Tuesday 9,  11:30 AM - 12:30 PM  SDCC - Room 8

Demo Theaters

 

You can think of Demo Theaters as mini-Tech Sessions. They are shorter, always focused on very specific workflows and run in a small setting were questions can be handled throughout the presentation.  The Demo Theaters take place in Expo area (first floor), close to the Product Islands.

 

 

User Presentations

 

User Presentations are a great way to learn the details of how Survey123 is applied to specific workflows. A good opportunity to learn directly from users like you!

 

  • Mobile data collection with Survey123: There will be two user presentations highlighting how Survey123 is being used with Spike Laser Measurement devices and with high accuracy GNSS receivers to streamline field data collection workflows.
    • Wednesday, July 10 2:30pm - 3:30pm SDCC - Room 25C
  • Going mobile for the Homeless Point in Time count: Paper presentations from multiple users sharing how they used Survey123 for ArcGIS to support the Homeless PIT count.
    • Wednesday, July 10 1:00pm - 2:00pm  SDCC - Room 29A
  • Creating impacful applications for Education: This session presents two case studies of impactful GIS applications, including employing Dashboard for ArcGIS to improve earthquake safety and preparedness in K-12 Schools; using Survey123 for real time data collection, environmental resource protection, and sanitation management. The case studies in this session exemplify the best practices for creating impactful GIS applications
    • Tuesday, July 9 1:00 pm to 2:00pm SDCC - Room 25C
  • Crowdsourcing water quality data collection with Survey123: By the Arizona Department of Environmental Quality. With more than 45,000 miles of stream and river beds in the state, this is far too much for agency to staff visit. Utilizing the tools of Survey123, three forms were created to have citizen scientists help in collecting the data needed to protect this valuable resource.
    • Wednesday, July 10 8:30 am - 9:30 am SDCC - Room 29C
  • Innovation of ArcGIS Online Apps to Enhance British Military Working Practices: The presentation will demonstrate  innovation and implementation of the ArcGIS online applications used by HQ 1 (UK) Division Geospatial Support Cell. Applications have provided the British military with a fantastic opportunity to dramatically increase their operational effectiveness.
        Wednesday, July 10 1:00 pm - 2:00pm  SDCC - Room 26A
     

 

Expo

 

The Expo area is opens Tuesday and closes on Thursday afternoon. It is located in the first floor in the San Diego Convention Center (SDCC) and features different areas including Partner Exhibits, Esri Industry Neighborhoods, GIS Product Areas and Customer Care booths.

 

  • Product Areas (Islands) at the Expo:  You will always find someone from the team at the Survey123 booth. This is the place to go when you want to have a one on one conversation to address specific ideas or issues you may have. We often refer to the Product Areas as Islands. The Survey123 team will have a station under a big sign that says 'Apps'. It will be a big sign... you will see it.
  • Partner & Sponsor Exhibits: The Survey123 software is only a third of the story. There are two good reasons to visit the Esri Partner and Sponsor Exhibits. The first one is that many of them are now experts building smart forms with Survey123 and know the tricks of the trade when implementing solutions. Swing by their booth and ask them about their own experience.  The second one is that through the Partner Exhibits you will find many hardware vendors that can teach you the benefits of using their offerings with Survey123. Think Laser Measurement Solutions, GNSS receivers, ruggerized devices and much more. Just to name a few:
  • Customer Care and Esri Tech Support: Our Esri Customer Care and Technical Support teams will be represented in the Expo. You can also swing by their stands with your Survey123 technical questions.
  • Esri Industry Neighborhoods: Everything from Public Safety to Defense, Local Government, Utilities and more.  The Industry Neighborhoods are a good place when you want to connect with people that understands how to put Survey123 in the context of your own workflows.  Ask for a demonstration of Survey123 focused on your problem or for examples of organizations using Survey123 in your industry.

 

Esri Education Summit

 

The Esri Education Summit takes place between July 7-10 at the Marriott Marquis & Marina Hotel. You will need to register separately for this one!

 


 

Pre-Conference Seminars

 

 

By now, many Pre-Conference Seminars are sold-out, but I thought I would include info about this for those of you who are planning to come next year. You need to hurry before all seats are  taken.  Survey123 will be highlighted in the Field Operations and ArcGIS PreConference seminar:

 

  • Mastering Smart Form Design with Survey123 and XLSForm
    • In this one-day hands-on seminar you will learn how to use the XLSForm specification to build smart forms with Survey123 for ArcGIS. We will introduce the XLSForm specification and will dive into advanced techniques to implement custom data business validation rules, conditional statements to control the visibility of portions of your form, auto-calculation rules for attributes in your features and much more.
      • Sunday, July 07 8:30 am - 5:00 pm
  • Field Operations and ArcGIS
    • Learn how ArcGIS helps your organization optimize field operations by: taking ArcGIS maps offline into smart devices using ArcGIS Publisher Extension and Explorer for ArcGIS; coordinating your field workers with Workforce for ArcGIS and Navigator for ArcGIS; improving locational awareness by knowing the location of key staff and equipment with Tracker for ArcGIS; and optimizing data capture with Collector and Survey123 for ArcGIS; building effective operation dashboards with Operations Dashboard for ArcGIS.
      • Sunday, July 07 8:30 am - 5:00 pm

 

The Esri Events App:

 

If you are coming to San Diego, I recommend that you download the Esri Events app. It  will help you follow your own schedule, keep track of last minute changes to sessions and vote sessions too!

 

 

If you are coming this year, welcome!  If not, be ready for next. Have a look at the many reasons why you should not miss this event!

In this blog post I will explore a few techniques to take your use of Survey123 within Integromat to the next level.  I am going to assume that you already know the basics of working with Survey123 and Integromat. If not, you may want to start with Getting Started with Survey123 and Integromat

 

Many more posts like this could be written, because what you can do with Survey123 and Integromat deserves an entire book, but we need to start somewhere. So here are a handful of ideas. I hope you discover some new things:

 

Working with Survey123 attachments

 

I will start with this one because the topic of working with photo attachments from Survey123 in Integromat has come up many times before, and it is not entirely obvious how to deal with them.  Say your Survey123 smart form includes a photo and a signature question. Can you work with them in Integromat? Can you include the photo and signature images as attachments in an e-mail? Can you back-up the images in OneDrive or Box? Yes to all of that and here is how.

 

The trick is using the Integromat HTTP module, which includes an action called 'Get a file'.  This action takes the URL of an image as an input, which you can get from the payload of a Survey123 record.

 

  1. Feed the Survey123 'watch survey' module with a smart form that includes one or more photo/signature questions.
  2. Use the Get a file action in the HTTP module to download your attachment.

 

 

The payload of 'Watch a survey' includes a collection of attachmentInfo objects. They are uniquely identified with the name of the question generating the attachment and include a url property that can you use to invoke 'Get a file'.

 

Once you have the file, you can do a handful of things with it. Of course, you can now include it as an attachment to an email.... That is a classic one!

 

 

But you can do more interesting things too. For example:

 

  • Back-up your photos and attachments in Google Drive, Microsoft OneDrive, DropBox.
  • Upload images to Flickr, Instagram, Twitter or other social media.
  • Reduce the size of an image before storing it.
  • Extract metadata from the photo, including its name, size or format.
  • Process the photo with Google Cloud Vision to identify elements within the photo or simply extract text from it.

 

 

 

Automating Survey123 custom reports

 

The Survey123 module includes an action to help you automate Survey123 feature reports. Not familiar with reports in Survey123? We have a video for that here and an introductory blog post too. The Create Feature Report action is pretty much self-explanatory: it takes an input survey, a report template and the objectId of the survey record you want to include in the report.  If the trigger of your scenario is a 'Watch a survey' trigger, then you can use again the payload of your Survey123 web hook to extract the objectId. In this case, it is within the Feature-Result object as shown in the next animation.

 

 

The use of the HTTP module to download files as described above also applies to the outputs of the Survey123 report action. That is, after you create a custom report file, you can download it and include it as an attachment in an email, or upload it to cloud storage or FTP site.   Since many cloud storage solutions include options to automatically synchronize the contents of local and cloud folders, I like to upload the Survey123 report files I create to the cloud so they show up in my computer right away. I know, I am old fashioned, but that feels like magic.

 

When configuring the Create Feature Report action you will notice that at the bottom of the dialog you can expand a window with advanced settings. In there, you will be able to configure the UTC-Offset. This setting will be your friend if you plan to include dates and times in your reports.  Keep in mind that all data within Survey123 -and ArcGIS- is stored in UTC time. Since Integromat has no real context as from where in the world the report is being invoked, you will need to adjust the UTC offset yourself.

 

The Survey123 Custom Report service is currently in Beta and scheduled to be generally available in version 3.5

 

Back to the HTTP module

 

Earlier, we looked at the 'Get File' action in the HTTP module. This time we will revisit the HTTP module but this time to describe its 'Make a request' action. I personally find this action extremely useful, because it allows your scenarios to interact with pretty much any external web service you want. Using the 'Make a request' action you can for example invoke the ArcGIS REST API to update an attribute in an existing feature, or to simply query a feature service. Lets get down to very specific scenarios:

 

  • Make a point- in-polygon query to an ArcGIS feature service to get the postal code of the submitted Survey123 record location. Then make another request to add the postal code value as an attribute of the submitted feature.
  • Use information from a submitted Survey123 record to automatically create a new assignment in a Workforce for ArcGIS project (make a request to add a new feature into the Workforce assignments feature layer).
  • After having invoked Google Cloud Vision to categorize the contents of a photo submitted via Survey123, update the attributes of the Survey123 feature accordingly.
  • Update an image_url attribute in the submitted survey so you can later create a popup in ArcGIS that shows your image directly when the popup is open.

 

 

The mechanics for using the 'Make a request' action imply good knowledge of the target web service your are invoking. I will use next a simple scenario to illustrate the essence of this technique. Say for example we want first to get the location of a submitted survey record, get the current temperature at that location using a weather service and store that temperature value as a GIS attribute of the feature that was originally submitted.

 

Here is how you get the current temperature. OK: so far we are not using 'Make a request' anywhere. We are just setting this up.

 

 

Next, we are going to 'Make a request' to the survey feature layer using the ArcGIS REST API to add the temperature to the temperature field of the submitted record. Specifically, we are going to make a request to the Update Features REST API operation. This operation, as described in the Updates Features ArcGIS REST API Developer documentation, is located at:

https://<featurelayer-url>/updateFeatures

All information needed to construct the URL above is included in the survey web hook payload. It is a matter of combining the surveyInfo.serviceUrl and the feature.layerInfo.id properties, followed by updateFeatures.

 

 

Again according to the doc, this operation only supports POST requests and the encoded parameters include:

 

  • f to define the output format as json or html. This is optional but handy so we can debug, if needed, the output within Integromat.
  • features to define the actual updates we want to make in the layer. The content of this parameter is a json object with information about the records and fields we want to change.
  • token to specify a valid token we can use to securely access the feature layer. This parameter is not needed if the feature layer is shared publicly.

 

All of that, translates into Integromat as follows:

 

 

The most error prone step is adding the exact JSON value for the features parameter. I like to copy the sample JSON that comes in the developer documentation, and then modify it for my own purposes.  It is very important to note that the sample JSON in the documentation must be enclosed with extra curly brackets. In my case, this is the JSON string that will go in the features parameter. Note that the objectId in green, and the temperature in brown are dynamically populated using outputs from other modules in the scenario.

 

{

    "attributes" : {
      "OBJECTID" : objectId,
      "Temperature" : "temperature"
    }

}

 

You will see yourself going through some trial and error until you get it right, but since you can easily look at the output of the 'Make a request' operation, it typically does not take very long before you figure things out.

 

Using the 'Make a request' action you can do a lot of things. Updating attributes is one of them, but you can really do many others.

 

A nice complement to the 'Make a request' action in the HTTP app are the many out of the box tools included in Integromat to parse XML, JSON, HTML and the many functions to handle text, dates, arrays and much more. They will help tremendously when you need parse responses from the web services you are invoking.

 

Handling Errors

 

This is all getting very interesting, but as you create more sophisticated scenarios you will face situations where you need to handle run-time errors.  If you are dealing with attachments, what happens if the end-user does not submit one? Or what happens if you hit an external web service and you get a bad output, or a time-out?

 

By default, when an unexpected error occurs, the execution of the Integromat scenario is stopped at that point but you can handle errors in other ways. You can for example ignore the error and continue executing, or you can choose to branch off the execution to do different things. This is all described in the Introduction to error handling tutorial in the Integromat Help Center.

 

The following animation shows how you can add an error handler in case the 'Get file' action fails downloading a non-existing ArcGIS attachment. In this case I chose to simply invoke the Gmail connector to send an e-mail without attachments. If an ArcGIS attachment is found, it gets added to an e-mail, otherwise the email is sent without attachment.

 

 

To learn more about error handling in Integromat, I recommend reading the Advanced error handling help topic.

 

Connecting Integromat to an ArcGIS Enterprise instance

 

By default, the Survey123 modules in Integromat will connect to ArcGIS Online, but you can actually make Integromat work against your own instance of ArcGIS Enterprise.

 

Before we start, it is important to highlight that Integromat needs to communicate with your own ArcGIS Enterprise instance, and this will not be possible if ArcGIS Enterprise is running completely behind your firewall.  You need to make your ArcGIS Enterprise instance is available to Integromat over the internet. ArcGIS Enterprise administrators are familiar with this concept, which is often resolved by exposing ArcGIS Enterprise through a reverse proxy in the DMZ, although there are other options.

 

A second step, involves explicitly registering Integromat as an app in your ArcGIS Enterprise instance. This step is necessary for Integromat to be able to securely access information in your ArcGIS Enterprise. Follow these steps:

 

  • Log into your portal and go to Content.
  • Add a new Item of type Application.
    • Type: Application
    • Title: Integromat (Really, you can call this anything you want)
    • Tags: Survey123 (Same here, any tag you want)

 

 

 

 

All the above is necessary for Integromat to be trusted before you can initiate a secure connection to your Enterprise instance.

 

Now, it is time to create a new Connection within Integromat to your ArcGIS Enterprise instance.  While adding a new web hook in the Watch a survey or Create a report modules, choose to create a New Connection:

 

  1. In the Create a New Connection dialog, expand the advanced settings.
  2. Add the https URL of your ArcGIS Enterprise instance, including the web adaptor name.
  3. Populate the AppID and AppSecret properties using the App Registration information from the Application item you created before in your portal.
  4. Click on continue to authenticate against your ArcGIS Enterprise instance.

 

 

You will be able to re-use this connection again and again when creating new web hooks.

 

Add and Edit Trigger Events

 

Initially, we added support for Add trigger events, meaning that the Survey123 web and field apps would be able to trigger a web hook when a new record was submitted from a smart form.  We are progressively adding support for edit trigger events as well, so you can launch your Integromat scenarios when a record is updated too.

 

Support for edit trigger events was added in the Survey123 web app in our Comet Halley Update (May 23, 2019). The Survey123 field app will catch up starting with version 3.5 (scheduled  to be released on June 27, 2019).

 

If you do not now how to update records using the Survey123 web app, check the Survey123 Tricks of the Trade: Editing records in a web form blog post.

 

While you can configure your Integromat scenarios to be triggered by Add and Edit events, please be aware that the Survey123 field app will not honor this setting until version 3.5. As of 3.4, only the Survey123 web app supports web hook edit triggers.

 

Below, you can see how you can configure Integromat to define which triggers you want to activate in your web hook.

 

 

If you happen to activate both of them, you can use the output of the Watch survey module output to tell which event triggered your scenario. This is handy in case you want to build a single scenario and add branching logic to handle adds and edits differently.

 

The trick here is that the eventType property is only included in the payload once there is an actual submission of data. You will actually not see the eventType property pre-populated in the output, until you send data.  When working with eventType, I typically put the scenario in listening mode (Run once) and submit data to populate the property. This helps me build the rest of the scenario more easily. Once the scenario is running for good, this property will change accordingly.

 

 

 

Blueprints

 

Did you know you can export your scenarios as a JSON file and import them later?

 

 

I included as a simple blueprint of the weather sample as an attachment to this blog so you can play with it. All you need to make it work is a survey with a geopoint question and an additional field where you can store the temperature. Make sure you adjust the name of your temperature field in the features parameter of the 'Make a request' module. By now I am guessing you do not need that I create another gif. I am completely "giffed-out" for today.

 

As I said at the very beginning, there is so much to write about Survey123 and Integromat, so this is just a starting point. If you feel there is a particular topic of your interest that should be covered in more detail, do not hesitate asking. Now,  let me anticipate that we still have not figured out just yet how to work comfortably with repeat data within Integromat.

 

Despite the repeat debacle, I hope this blog post will give you some ideas to have more fun with Survey123 and Integromat. I also wish some of you will share some of your creations. Share a screenshot of your scenario and a brief description of what you accomplished with it, or even better the blueprint.  Add them as comments below to inspire others!

Integromat is a powerful tool for automating manual processes. With Integromat, you can easily connect Survey123 with many other apps and services. For example, you can configure a scenario in Integromat to perform an action immediately after data is submitted or edited via a Survey123 smart form: send an e-mail or an sms, copy the data into a spreadsheet or database, create an assignment in Workforce for ArcGIS or simply perform a reverse geocode to associate that address with the record that was just submitted.

 

Integromat includes well over 300 different out of the box 'integrations' (and growing), which you can combine visually in scenarios to automate specific tasks. You will find integrations for everything ranging from Office365 to Box, PayPal, SQLServer, Google Calendar, Apple Push Notifications, InvoiceNinja, Instagram, Google Cloud Print, and of course, Survey123 for ArcGIS!

 

 

The basics, in four minutes

 

To get started with Survey123 and Integromat I strongly suggest you watch this four minute video-tutorial. It will give you a good overview of the whole thing, plus enough detail to follow along and build your first Integromat scenario.

 

 

You can create a free Integromat account to test the waters. Note that for production use you may want to upgrade to a paid subscription plan. There are multiple options, with increasing levels of support, operations, data transfer, etc.

 

Get a kick-start with a template

 

Also for starters, you can go with the different templates we have created. With them, you can automate common tasks with minimum effort. If you find a template that fits what  you want to do, you can create a scenario out of it with a simple click. Integromat will guide you, step by step, so you can configure the template with your own Survey123 smart forms and other apps as appropriate.

 

 

Integromat Survey123 Templates

 

Once you bring a template into your own scenario, you can also extend it to do even more...

 

One hour for more details

 

Derek Law and Brandon Armstrong recently delivered a free Live Training Seminar focused on automating e-mail notifications.  This seminar was recorded and features Integromat. This is good material as well if you are getting started.  You will need to sign in with your Esri credentials to access the recording.

 

Automating e-mail notifications with Survey123 Live Training Seminar

 

Integromat Help Center

 

To get the best of Survey123 and Integromat you need to know them both. As you work with Integromat and Survey123 you will realize soon that this is more than just about sending an e-mail notification when survey data is submitted.   Integromat is actually a very powerful visual environment where you can model common business workflows. Integromat is also more than 300 out of the box integrations. It includes quite flexible options to control the flow of data and decisions within a scenario, with iterators, routers, filters and error handlers.  You can also configure your scenarios to execute at regular intervals and perform operations in bulk. 

 

What I like the most about Integromat is that it makes it possible to do lots of things for which in the past you would need programming skills. This makes more people, more productive. Having said this, if you have developer skills you will also be impressed with all the things you can do.  The more  you know, the better.

 

Spend some time roaming around the Integromat Help Center. Check out their video-tutorials and technical articles to learn new techniques. It will pay off!

 

Once you get on the basics, check out theSurvey123 Tricks of the Trade: Integromat blog post for additional info!