Survey123 web form inconsistently triggering Integromat Webhook

[PhilipMcCarthy]

Hi All

I have a number of Survey123 forms that I've built and have configured webhooks in Integromat to output submitted data for each into a Google Sheet, and in some cases save submitted photos to Google Drive.

During testing using the Web App, everything worked just fine - data went through the google sheet and photos saved to google drive. I am not using the Field App

However after deploying to users I'm discovering that some of their submitted records don't appear to be triggering the webhooks (happens across multiple forms), though all records are appearing in the 'Data' tab for the form in Survey123.arcgis.com. I can't reproduce the problem, I've arranged for these users to submit test records whilst watching the webhook and it gets triggered ok each time. Everytime I submit a test record it goes through ok.

I've examined the logs for the Integromat webhook, and the missing records do not appear in the history and there are no incomplete executions. Its like the webhook is randomly not getting triggered.

To say this is alarming is an understatement, I have no explanation and the random nature of the issue is making it extremely difficult to resolve. Attached is a screenshot of the 'webhook' section in a Survey123 form that has this issue. I just put in a Microsoft Flow webhook to be notified about any new records so I can see how that service performs.

Thanks, Phil

[JessieShapiro]

After further investigation, using an Android versus iPhone does not seem to solve to problem. The webhook is still inconsistently not being triggered even when a survey is submitted through an iPhone. 

[AndreaRyffel]

Thank you for letting me know, Jessie. It looks like these are two different problems, then. I'll set up a new question with my Android vs. Iphone issue.

[BrackenDavis]

I have had similar problems when sending the webhooks to FME Server.  What I've noticed is that if I trigger the webhook more than twice in a few minutes, the 3rd + edits/new records (triggers) don't get sent to FME Server.  

 @Anonymous User, have you heard anything on solving these issues with the webhooks?

[PaulSweeney3]

We use survey123 and integromat extensively and it seems generally very reliable however recently I have users mention they have a small percentage of forms submitted, with a webhook attached , but no record of it on integromat. The record is successfully submitted to Survey123 but it did not trigger in integromat. I am unsure as to who I should raise a support ticket with.  I note the comment above about multiple submissions in a short period of time. I think this could be a similar pattern but not able to reproduce it consistently.  @Anonymous User 

[m3]

I too am having this problem, using Power Automate, Integromat, Android, iPhone, web form.  Not seemingly related to submission rate here

[JamesSerendip1]

I am really disappointed to see so little response from ESRI about this.  I had put many hours into developing data back-up systems for my Survey123 data collection apps on MS Power Automate, and then abandonded it for unsolvable inconsistencies -- submitted ESRI trouble ticket, and after several days of the support folks getting nowhere, I decided to switch to Integromat ("Make") which has been great, but lately I am experiencing the exact same issue presented here.  All devices using current updated versions.  In fact, there is only 1 device being used in the field currently, and on a given day I may get 4 submissions from that device out of which 1 or 2 will trigger the web hook, and the others will not.

This is not a minor issue!  To be able to easily access a back-up of the data is integral to our use of Survey123 as a data gathering platform, and so for it to have these inexplicable and unsolvable inconsistencies is not acceptable.  And the fact that a given user can successfully trigger thousands of webhooks a day is completely immaterial to it being a critical error for other users!  As a response, that to me says "user error" which mayvery well in fact be the case!  But here are multiple folks, obviously with some experience in using these systems, still finding errors that essentially invalidate the integrity of our data, making the entire Survey123 platform somewhat useless.  Alas, I also tried to pursue the issue with Integromat and got absolutely nowhere.  After 3 weeks of no response and multiple requests I eventually received a video from a tech going over basic concepts that was completely unhelpful, and recorded speaking so fast I couldn't even make out what he was saying.  I reply to say as much, and that this did not resolve my issue and I still needed support.  That was 3 months ago and I NEVER got another response, so that isn't an option.  Plus, from this thread it seems obvious that the issue is with Survey123.

I am submitting another ticket about this issue with ESRI, and if I find any resolution I will pot it here, and if I do NOT receive any resolution I will post that here as well as a caveat emptor!

[AmyNewsam]

Hi All, I am also experiencing this problem with Power Automate and Survey 123.  The data so far has successfully shown up in the table, but the web hooks are not consistently processing.  Please do post a solution if one is found.  I will also go through Esri support and post any solutions that may arise.

[DuncanC]

Did anybody ever find a resolution to this?  I'm running into the same problem.  Webhook fires inconsistantly.  Seems to be more likely to not fire if it's a second submission within the hour.  Nothing shows up in my make logs as incomplete, that record just shows up in the Survey123 data view without my make automation being triggered.  Worked fine for a month.  Now I've had 2 issues in 2 days.

[AmyNewsam]

It seems to be working for us now... looks like it was just failing when I tried to test it (perhaps because I'm the owner?) and the solution is using an incognito window.  But it sounds like that's not the same issue that others are experiencing.