Survey123 Webhook Sometimes Doesn't Fire

[CalvinHarmin]

I have a handful of Survey123 forms (AGOL hosted) that have webhooks registered to deliver a payload for a Power Automate cloud flow trigger. 

95% or more of the time everything works fine. But I'm noticing this week that there are many that simply don't fire the webhook payload to Power Automate. For example,

• a form submitted at 1:32PM triggered correctly
• a form submitted at 1:36PM - no record of any trigger
• a form submitted at 1:37PM triggered correctly

There have been 6-7 of these instances in the past couple days. I hate that I have to baby this thing and make sure stuff is coming through. The Survey123 forms are for our Town residents to request services; I process them in Power Automate to ingest them into our Town's work order management system, and the customer gets a confirmation email, etc.

Is anyone else experiencing something similar now or in the past? I started a ticket with ESRI just now but wanted to reach out to the community as well. Thanks in advance!

P.S. I actually have two identical webhook payloads for each form, one goes to our Testing environment flow (which doesn't follow through on processing requests), one to our Production environment flow. Neither one triggers in these instances.

[ChristopherCounsell]

If they're using the field app and have previously downloaded a survey, adding a webhook is an update to the form and it needs to get downloaded.

If you're adding multiple webhooks to a survey form, maybe check the survey webhook Id and the power automate connection? People have indicated they can get overwritten if you don't rename them:

https://community.esri.com/t5/arcgis-survey123-questions/power-automate-flow-not-triggered-when-surv...

If your workflows involve editing data, including adding new repeat entries, you also need to configure the webhook to trigger on updates/resubmissions.

[CalvinHarmin]

I don’t think any of that is relevant to me unfortunately — the webhooks are firing successfully about 50 times each day, when a new form is submitted. It seems that one or two simply don’t fire off the webook for some reason.

To follow up in you’re first sentence actually: if a user uses Survey123 app, the client sends the webhook, rather than AGOL? If using a browser, is the browser client also sending the webhook payload?

I assumed this would be coming from AGOL and not the clients. Actually it must— the json includes the objectid which isn’t created until the submission hits the AGOL server and registers a new record in the hosted feature layer?

[ChristopherCounsell]

Payload comes from agol

The request to deliver a payload comes from survey form. You 100% need to
re-download the survey form or update it after adding a webhook.

I don't believe it matters for browser forms.

[CalvinHarmin]

In our case the residents of the Town are only 'provided' access to the survey form through an embedded iframe on our Town website, so I'm not sure if there is really any visible mechanism for them to open the survey in the Survey123 app unless they somehow yoinked the url for the survey from the HTML source or browser devtools. We don't provide any link "to open this in a separate page" or anything like that. So I feel like this is not likely our issue, but I can't guarantee it.

[JakeMatthys]

Yes, we have experienced this issue (community post ). 

We are now at Enterprise 11.3 and this issue does still occur, but very infrequently.  Probably occurring on the order of once a month or quarter and we are receiving 3000+ surveys per month.  We have band-aided the issue by running a job hourly for those webhooks that are mission critical and time sensitive to process any records that may have been missed.  For my organization, we felt that the effort to research and resolve this issue outweighed the cost, so we are living with the minor inconvenience that is causes us. 

 - Jake

[CalvinHarmin]

Thanks for the secondary datapoint, I am planning on doing the same strategy as you. Our survey has a hidden field 'Status' that is populated with 'Submitted' by default; this is changed to 'Received' if my flow runs successfully, and 'Error' if the flow fails for some reason.

So I think I can just compare if any record is 'Submitted' and the 'survey_completed_datetime' field (after converting epoch millisecond time to UTC) is more than X minutes from utcNow() then I can consider it an orphan 😞 and needs to be processed.

I sure do wish AGOL had some kind of logging. I wonder if they do on the backend that we don't have access to.