Webhooks not triggering randomly

1689
22
03-16-2021 01:49 PM
PaulSweeney3
Occasional Contributor II


We use survey123 and integromat extensively and it seems generally very reliable however recently I have users mention they have a small percentage of forms submitted, with a webhook attached , but no record of it on integromat. The record is successfully submitted to Survey123 but it did not trigger in integromat. I am unsure as to who I should raise a support ticket with. Is it more likely to be an integromat issue or survey123?

Tags (1)
0 Kudos
22 Replies
TylerBragg2
Occasional Contributor

I've been having this happen to me as well, and it seems to be more often in the last few months.  I have a paid subscription to Integromat and have 28 active scenarios running at all times, some of them being somewhat complex, some being fairly simple.  I actually had this happen last week and again today on a scenario that has been running for about a year now with very few problems.  The user reported the problem to me but I can't seem to nail down the source, except that it must be in the webhook.  I checked the history of the scenario in Integromat and there are no errors, and more importantly, no record of the trigger at all.  I've tried to look on the Survey123 side and I can't seem to find a way to validate that the webhook trigger was sent to Integromat.  Then the next submission of the survey will work perfectly.  

PaulSweeney3
Occasional Contributor II

I think it's fair to say there is an issue so we re not all imagining it. Id imagine given the scale and number of requests its likely from time to time a webhook would fail . It would be great if someone could just confirm that , at least then we could work with it. As a precaution on some of my more important scenarios I have started creating round up scenarios that basically search the layer and process everything in the day again.  It's extra executions but it gives piece of mind . I appreciate its not ideal for all scenarios but it does work well polling layers is defo more reliable if it suits the work flow.

0 Kudos
PaulSweeney3
Occasional Contributor II

Also what I intend to do is set up a Scenario in power automate and add the webhook to one of my existing forms in an integromat scenario. That way I can test if both Webhooks trigger or fail.  My gut feeling it's the s123 app or an operating system issue due to connectivity I.e poor mobile data service 

0 Kudos
TylerBragg2
Occasional Contributor

That's not a bad idea at all.  But, I'm thinking we could rule out connectivity issues as the cause because the webhook is not initiated at the mobile level; rather, it should be issued by ArcGIS Online at the feature service level(at least in my case, unless you are using Server, then from there).  That's where a log file from AGOL would be nice so you can compare the submit time/date of the feature and the log file of webhooks to see if it skipped one.  I'm thinking that's what's happening.  But what you mention would be a good way to validate that.  Or, I suppose there could be a blip with Integromat where it was NOT listening for a certain time period.  Then AGOL could have sent the webhook but Integromat never saw it because it wasn't listening.  In that scenario, Integromat wouldn't show it in the scenario history because it truly didn't receive it, even if it was because it "wasn't listening." 

PaulSweeney3
Occasional Contributor II

 

Hi @TylerBragg2 

Really is that something you  know for definite.  I was always under the impression the app submits the webhook once it recieved a response from the server. That's why you had all the additional info on the user, survey information etc.  I know you can generate webhooks from the feature service layer but I thought this was a separate process. I will admit I am no expert on this stuff so happy to be wrong on it. 

0 Kudos
AndrewBowne
Occasional Contributor III

@TylerBragg2 wrote:

That's not a bad idea at all.  But, I'm thinking we could rule out connectivity issues as the cause because the webhook is not initiated at the mobile level; rather, it should be issued by ArcGIS Online at the feature service level(at least in my case, unless you are using Server, then from there).  That's where a log file from AGOL would be nice so you can compare the submit time/date of the feature and the log file of webhooks to see if it skipped one.  I'm thinking that's what's happening.  But what you mention would be a good way to validate that.  Or, I suppose there could be a blip with Integromat where it was NOT listening for a certain time period.  Then AGOL could have sent the webhook but Integromat never saw it because it wasn't listening.  In that scenario, Integromat wouldn't show it in the scenario history because it truly didn't receive it, even if it was because it "wasn't listening." 


I'm not necessarily sure that is true.  I am using Portal ver. 10.6.1 with federated Enterprise 10.6.1.  webhooks weren't supported with Enterprise until 10.7.  

I believe that Survey123 is where the webhooks are administered and brokered to Enterprise (or AGOL).  I am in agreement with @PaulSweeney3 .  S123 is the initiator of the webhook.  I think the big question here is - where would logging exist?  If Integromat doesn't get the payload, it'll never know that it's being initiated.  Since the webhook is presumably not initiated by Enterprise/AGOL then it really has no knowledge of the execution either.  In my opinion the logging would have to exist on the S123 Admin level(https://survey123.arcgis.com/) where the initiation would have to be logged and a callback result would have to be logged as well.  What I am not seeing is any sort of logging that currently exists for anything.  Perhaps we could get some further insight from the Esri folks on this... @JamesTedrick @IsmaelChivite @Philip-Wilson 

0 Kudos
KeithOlsen
New Contributor III

What Integromat function searches existing records on the featureclass? I've only used Integromat to respond to webhook triggers from Survey123. Would you be using something else to generate a webhook on a set schedule?

0 Kudos
PaulSweeney3
Occasional Contributor II

The Http module along with the REST API Can be used to query a feature service with a typical query eg where objectid > 100 and return the json which you can parse and use in more modules in your scenario.  You can schedule it to run at certain intervals.  I think the intervals depend on your subscription.  It's pretty useful once you pick it up. 

0 Kudos
KeithOlsen
New Contributor III

Thanks! I'll check it out.  --Keith

0 Kudos
JamesTedrick
Esri Esteemed Contributor

Hi @PaulSweeney3 @TylerBragg2 @KeithOlsen ,

We have observed scenarios where the webhook for a submission did not trigger in cases where the form included a number of attachments (images) and either a callback was specified to open another application or the user put Survey123 in background manually.  We believe that this has been addressed for the next release of Survey123 - I would encourage you to get the current beta at the Early Adopter Community and test to see if the issue is resolved.