Portal Webhooks - Peer not authenticated

[JackGeo]

Hello, I have created a web-hook in Portal that subscribes to all groups events. When a user updates a group name in portal this web-hook is triggered and sends the payload to my API. My API validates the name and then updates the group name via the Portal REST API's if it is not in the correct format. This will then trigger the web-hook again as expected but won't update the group name now that it is valid. However a 3rd web-hook event is triggered and fails with the error message 'peer not authenticated'. This third event is only triggered after the update of the group name via the Portal REST API where 2 events are triggered rather than the expected 1. 

This does not occur in our non-SSO environment and the 3rd webhook trigger never appears in the 'notificationStatus' history. I find the error confusing as the web-hook is able to communicate with my custom API to send the payload and my custom API is able to send successful requests via the Portal REST API's back.

Has anyone else experienced this and/or has a proposed solution?  

EDIT: This issue is not related to SSO, we have narrowed down the problem. If a web hook is triggered and another trigger occurs before a response has been received for the 1st trigger then the 1st notification will time out regardless if a response has been received in time. The response from the 1st trigger is displayed on the notification status page proving that it did receive it. 

This looks like a bug to me, would be grateful if someone could confirm. It seems like a common use case where a web-hook is triggered whilst another one is still awaiting a response. 

[JackGeo]

Just provided an edit with a basic scenario that causes the issue we found previously. 

[KevinHibma]

Hi - I'm trying to understand the workflow. Do you have more than 1 webhook created that would be listening to groups? Or is there just a single webhook, but your receiver is responding to the webhook and performing actions on the group which cause that same webhook to fire again. 

If you have a 2nd webhook configured, what events is it listening for? Just group, or "*", or?

I haven't seen the error you've mentioned, nor did I have any luck trying to reproduce it. However, I'm not sure I've set my test up in the same way you have yours.

[JackGeo]

Hi Kevin, thank you for the response. This is just 1 web-hook listening to all actions e.g. /groups 

To reproduce:

1. Setup basic API with an endpoint that receives a payload, 'sleeps' for 10 seconds then returns a 200 ok response. 

2. Create web-hook to listen for all group actions e.g. '/groups'. For payload URL, use the API endpoint created in step 1.

3. In portal, open a group

4. Edit the group name multiple times within a second apart

5. In the portal web hooks notification status list you should see an error appear 

The aim of this scenario is that the 1 webhook is triggered whilst it is still responding to a previous trigger.

Hope that makes sense. 

[KevinHibma]

Hi Jack,

I was working on this today and cannot reproduce the error you're seeing. I have my receiver setup to return "true" - you can note my copy/paste results below where the time stamps are all in the same second.

I tried testing in a couple of ways - 

1) Multiple browser tabs and setup to click save real quickly.

2) I put a breakpoint on my receiver, essentially it's up and running, but forced it to stop short of responding. I sent off a few triggers before cleaning the break point.

I looked through our error messages, and I don't see something where we throw peer not authenticated. (I'm not saying it's impossible that's from Portal, I however couldn't find an instance of when/why we throw it). Is there any chance this is a message coming from your receiver? Do you get this with other receivers? Any chance you could setup a different one, or even a 3rd party service and perform the test. Reference: https://github.com/Esri/webhooks-samples 

2022-06-10 09:04:03		true		{"success":"true"}
2022-06-10 09:04:03		true		{"success":"true"}
2022-06-10 09:04:03		true		{"success":"true"}
2022-06-10 09:03:42		true		{"success":"true"}
2022-06-10 09:02:51		true		{"success":"true"}
2022-06-10 09:02:51		true		{"success":"true"}
2022-06-10 09:02:33		true		{"success":"true"}
2022-06-10 08:57:20		true		{"success":"true"}
2022-06-10 08:57:20		true		{"success":"true"}

[JackGeo]

Hi Kevin, thanks for that detailed response, 

Looks like this could be related to the older version we're using. Are you able to try it with Enterprise 10.7.1? 

Good idea about trying a different receiver though. I will experiment the next opportunity I can.