Sneak Peek: Survey123 and Webhooks (Integromat)

IsmaelChivite · ‎03-05-2018

----------------------------------------------------Note start------------------------------------------------------------------------------------------

The original content of this post is obsolete. All functionality described is now generally available as described in this video:

----------------------------------------------------Note end------------------------------------------------------------------------------------------

We are happy to announce that Survey123 will introduce support for Webhooks shortly. If you are not familiar with webhooks, think that you will be able to integrate Survey123 workflows with Micosoft 365, Google Sheets, Box, Evernote, GMail, SMS (Simple Message System), Apple Push Notifications, Salesforce and hundreds of cloud services more!

What is a Webhook? The concept of a Webhook is simple. A WebHook is an HTTP/s callback: an HTTP/s POST that occurs when something happens; a simple event-notification via HTTP/s POST.

The beauty of webhooks is that they are a commonly accepted pattern by many applications out there, so a whole ecosystem of different providers allows you to work with webhooks and automate processes involving things such as Microsoft 365, Evernote, Google Sheets, SalesForce...and now Survey123.

What can I do with Webhooks and Survey123? As you will find soon, many different things, but I will give you a simple scenario: Imagine you want to send an email to someone when a high priority incident is reported through a particular Survey123 form. Imagine too, that if the incident is low priority then you want the incident to be logged in a Google Sheets spreadsheet. By configuring a webhook with Survey123, you can trigger a flow that orchestrates Gmail and Google Sheets to do what I just described. The flow could look something like this:

The flow above was built using Integromat. Integromat is a cloud application that helps you visually orchestrate different cloud services. Integromat supports initiating these flows, or scenarios as they call them, with a webhook triggered by Survey123.

How can I do this myself? I am slow typing, so I recorded a step by step video so you can get a hang of it. You will be able to do this by using the Early Adopter version of Survey123.

Do I need to use Integromat? Not necessarily. Integromat is one of many possible solutions for using webhooks and Survey123. Other solutions include Zapier, Microsoft Flow, Tray.io, IFTT etc. You can in fact trigger a webhook that calls your own custom developed web service, an ArcGIS geoprocessing service, an FME service etc...

Does Esri plan to have connectors for Zapier, Microsoft Flow etc? Very likely, but we are still working out what is the specific set of connectors we will create. We want to use this Early Adopter Program to figure out which connectors you would like. For now, you can use the generic webhook connector!

To what Survey123 event can I associate a webhook? For now, only to the submit operation in a form (Web Form or Survey123 field app). If you have use cases for other triggers, let us know.

Updated June 17, 2018: Are there any restrictions to the use of webhooks using Survey123? Yes: Webhooks will only be triggered if the end-user is logged in into the Survey123 field app or Survey123 Web Form. Survey123 webhooks will not be triggered if the form is shared publicly and the user submitting the form is not logged-in.

When is this functionality planned to be made available? We are targeting version 3.0, which will release around July 2018. Our Survey123 Early Adopter Program now lets you experiment with this feature. We are looking for your feedback through the Early Adopter Program!

IsmaelChivite · ‎08-27-2019

Hi

webhooks are not implemented in Collector for ArcGIS. At the moment, webhook support is only implemented in Survey123. You may want to vote on this Geonet Idea: https://community.esri.com/ideas/13257

Anonymous User · ‎08-28-2019

Thanks a lot for the reply, Ismael.

I am trying a create a simple e-mail notification system for Portal/Enterprise Database using your e-mail-notification-system, so that when a field staff collects a specific survey point on Collector, the concerned department would get notified. So, far example if they observe a threat - Department A will get notification

and if they observe some species - Only Department B will get a notification and so on.

Do you think if its possible to set up the notification as explain in the scenario above using your script?

Thanks in advance.

ShuilingYu · ‎09-09-2019

Hi James,

Thanks for the convenient functionality of webhook.

Right now I am using Integromat to generate multiple daily field reports. But I found two potential issues when using Integromat:

If the user didn't fresh and submit the up-to-date survey, Integromat won't be triggered, then I have to do the same procedure in Integromat manually.
After a few weeks of using Integromat, I found the ArcGIS Online connection in Integromat expired every 2 weeks (error message: refresh_token required), which means I need to reconfigure the some modules (e.g. re-add ArcGIS Online connection, re-select Survey123 form) in Integromat and get up-to-date Survey123 form without being reminded.

If any of these 2 cases happened, the automation scenario ends up with manual data gathering, so I am wondering is any Python resources I can use to do the same thing (watch a survey, create feature report based on Survey123 templates)? In this case, I have more control of the data in my Survey123 form.

Thank you in advance and I look forward to your response.

JamesTedrick · ‎09-10-2019

Hi Shuiling,

Thanks for the feedback. We are researching the issue with tokens expiring.

On your first point, I'm not sure what you mean by "user didn't fresh and submit the up-to-date survey"?

You can generate a report using the ArcGIS API for Python and schedule a python script to generate reports, say, every hour.

ShuilingYu · ‎09-10-2019

Hi James,

Thanks for the ArcGIS API for Survey123 information, I will do more research about this.

For the first point, if I published a new version of Survey123, while the field-user didn't download the new version but still submit results based on an earlier version, they can still submit the results. However, in this case, Integromat (watch a survey module) won't be triggered if new results are submitted.

Thanks.

JamesTedrick · ‎09-10-2019

Thanks for the clarification,

Yes, the user needs to have downloaded the form after the webhooks settings have been made.

AdminValorInmueble · ‎10-16-2019

Hello, I appreciate all the help you can give me for the following:

I have two surveys, related to the same feature service. I have done two workflows in integromat, the first one is activated when a record is created and the second when it is edited. I have no problems with the first scenario, but in the second, the flow only captures the data that is edited, for example, I have values such as tracking numbers, assignment names, etc. these values are filled in the first survey, and I retrieve them in hidden questions in the second survey, but I cannot extract them in integromat because they appear empty.

I have researched but this has become a headache

I appreciate any help, thanks!

JamesTedrick · ‎10-21-2019

Hi Jeffry,

Survey123 will only submit the values that have changed in an editing workflow- because it is in an editing workflow, Survey123 does not know if it has received the full feature (as fields may not have questions). You would need to query the feature service based on objectId/GlobalID to retrieve the full feature as part of the webhook flow.

AdminValorInmueble · ‎10-24-2019

Hi James, thanks for your prompt reply.

I think I understand you, but I'm not sure how to do it. This is my scenario:

Is it necessary to add some other tool?

Regards,

JamesTedrick · ‎10-25-2019

Hi Jeffry,

The workflow you present should be fine- the Feature Report action receives sufficient information form the Webhooks to run. If the e-mail is going to a person listed in the survey, you would want an additional HTTP request to retrieve the feature from the feature layer to ensure you have the e-mail address.