topic Re: AGOL Feature Service Webhooks in ArcGIS Online Questions

AGOL Feature Service Webhooks

RonParis — Tue, 07 Jul 2020 09:35:39 GMT

Referencing the recent blog post which has introduced webhook to hosted feature services in AGOL. I was wondering if it was possible to use this to return the updates that have been made to the feature class.

At the moment, it will tell me when a feature has been created, but I want to be able to pull the attributes of the feature that was created.

TIA

Re: AGOL Feature Service Webhooks

JimmyKnowles — Tue, 07 Jul 2020 22:04:25 GMT

I can't help with your question, but anyone know why that blog post disappeared and can only be accessed with a password? Wanted to dig into feature service webhooks after reading the blog post, but now can't view it.

Re: AGOL Feature Service Webhooks

TylerBragg2 — Mon, 20 Jul 2020 12:00:09 GMT

I'm seeing the same password screen. I've been told by my ESRI rep that they just released new functionality for webhooks with feature classes in AGOL but they have no documentation yet. I'm trying to find some information to get me going with this, but keep coming up blank.

Re: AGOL Feature Service Webhooks

Anonymous User — Sat, 11 Dec 2021 09:53:01 GMT

Hi all, here is a link to a documentation page on webhooks: Create (Feature Service)—ArcGIS REST API | ArcGIS for Developers. Apologies on the broken blog link, I shared that to a couple of places myself but I believe it is undergoing some revisions. The doc link has essentially the same information though written more in the API referenece style.

Here is what the payload looks like:

 "payload": "[{\"name\":\"7_30_webhook\",\"layerId\":0,\"orgId\":\"<OrgID>\",
\"serviceName\":\"test\",\"lastUpdatedTime\":1596148958015,
\"changesUrl\":\"https://services.arcgis.com/OrgID/arcgis/rest/services
/test/FeatureServer/extractChanges?
serverGens=[48170963,48170964]&async=true&returnUpdates=false&
returnDeletes=false&returnAttachments=false\",\"events\":[\"FeaturesCreated\"]}]"

With this in mind, it is not possible to return specific features in the payload at this time, however this is really great feedback.

Update:Josh Harris‌ & Tyler Bragg‌ - after some further research and a pointer from Mikie Kelly‌, though you still cannot access individual features in the webhook payload, you can query them through the extract changes URL that is sent in the payload. This is alluded to in the documentation here: Webhooks - Payloads (Feature Service)—ArcGIS REST API | ArcGIS for Developers. Here is a quick example below:

Making a new edit in ArcGIS Online:

Then use this endpoint provided in the payload and append a token:

"https://services.arcgis.com/<OrgID>/arcgis/rest/services/test_offline_layer_replicas/FeatureServer/extractChanges?serverGens=[48420093,48420265]&token=<token>"

This returns a status URL: {"statusUrl": "https://services.arcgis.com/<OrgID>/ArcGIS/rest/services/test_offline_layer_replicas/FeatureServer/jobs/2847cc8c-9595-4415-84b1-07a963e82df8"}

The status URL returns a Result URL that is a JSON file with the attribute/feature information about the edits made:

Here is the contents of the JSON file for the edit made in the web map above:

{"layerServerGens":[{"id":0,"serverGen":48420265}],"transportType":"esriTransportTypeUrl","responseType":"esriDataChangesResponseTypeEdits","edits":[{"id":0,"features":{"adds":[{"geometry":{"x":-13086458.650021132,"y":4013282.62561917},"attributes":{"OBJECTID":20,"GlobalID":"446B3304-9AE2-4D71-9E6E-E61486B0D95E","CreationDate":1596645398869,"Creator":"<myUsername>","EditDate":1596645398869,"Editor":"<myUsername>","TextFldEdit":"Hello","Float_Field":20.2}}],"updates":[],"deleteIds":[]},"attachments":{"adds":[],"updates":[],"deleteIds":[]}}]}

So to summarize, it will be more configuration at this time but you are able to access individual attributes related to each edit/change that triggered the webhook.

--------------------

What would your specific use case be once you obtained the attributes through the web hook?

Hope this helps,

-Peter

Re: AGOL Feature Service Webhooks

RonParis — Fri, 31 Jul 2020 08:03:30 GMT

Hi Peter,

Thanks for the information, that's very useful.

For us, being able to return specific features using webhooks would allow us to write straight to a hosted database. i.e. webhook>base URL>database.

We use webhooks for Survey123 and the solution works very well. Just hoping to replicate using hosted feature services AGOL!

Re: AGOL Feature Service Webhooks

TylerBragg2 — Fri, 31 Jul 2020 10:57:26 GMT

Hi Peter Klingman‌,

Regarding attributes in the payload of the webhook - we use those attributes, in several different cases, to attribute automatic notifications. For example, I recently set up feature level webhooks for one of our ArcGIS Workforce projects layers. When a new assignment is created in the Workforce Project, it triggers a scenario in Integromat that sends a text to the Field Crew Supervisor with the address of the assignment and the priority level. In order to get the address and priority level, I have to conduct an HTTP POST request to the feature service and look for the feature with the newest creation date, then extract the data. It would be nice if the feature level webhook would at least pass along the GlobalId of the feature (based on the "changeTypes" parameter, which is, in my case, FeaturesCreated). I think ideally though, it would be a great addition to the feature level webhooks functionality if there could be a code block added that customized the webhook to include attributes of the user's choice in the payload. I can't complain though - I'm excited just to have the feature level webhook functionality at this point!

Re: AGOL Feature Service Webhooks

Anonymous User — Fri, 31 Jul 2020 18:20:27 GMT

Josh Harris‌ & Tyler Bragg‌ thank you for the detailed descriptions above. Very helpful!

Re: AGOL Feature Service Webhooks

TylerBragg2 — Wed, 05 Aug 2020 18:01:35 GMT

Peter Klingman‌, Thanks for the update about the "extractChanges" functionality. I can say that I have already tried to use this option, and I think there may be some issues with it (or the issue is with me). I couldn't get it to work consistently.

Specifically, I am using the feature level webhook to trigger a scenario in Integromat when a few feature is added to the feature layer (This particular feature layer is the "assignments" layer in one of our Workforce projects). The Integromat scenario then sends a text message (email to text) to the field supervisor to make him aware of the new assignment in Workforce. I want to include the address and priority level of the assignment in the message so he has an initial idea of the location and severity of the problem, which is why I need attribute information. As I was creating this workflow, I initially set up the scenario to use the extract Changes functionality that you mention above, which returns the Status URL, which contains the Result URL json file. I tested all my HTTP requests in Postman, and everything worked perfectly. However, when running the scenario in Integromat, I kept receiving an error when it tried to open the Result URL json file after the webhook trigger. I can't remember the exact wording of the error, but it basically said that the json file (Result URL) did not exist. When I went back to review the error (just moments later), it worked just fine and opened it right up. So I was thinking that there may be a time delay in the creation of the Result URL json file after the extractChanges is triggered, and before that file is actually generated my Integromat scenario has already tried to open it, causing the error. I don't know if this is actually what was happening, but it was the only explanation I could come up with. Is there a way to determine the amount of time it takes for AGOL to generate this Result URL json? Maybe it was an issue with the way I had things set up though.

So as a workaround, instead of using the extractChanges function, once the new feature is created and the webhook is triggered, my scenario makes an HTTP POST request to the feature server, queries where CreationDate was in the last 2 minutes, then grabs the attributes from the feature returned in that query. It's working, but not ideal, especially in an event where two assignments may be submitted back to back within 2 minutes of each other (not highly likely, as it takes a minute or two to enter all the appropriate information for the assignment).

I would have included screenshots of this issue, but I abandoned the idea when I discovered the workaround, and now I'd have to go back and set the whole thing up for using extractChanges to see what was happening again. It would be nice to get this working, as I could then parse the Result URL json, grab the "globalID," then make an HTTP POST request back to the feature service and grab the attributes of the feature with that globalID (instead of looking for features created within the last 2 minutes).

Also, the link that you provided above regarding the 'Create Webhooks' API Documentation, as you stated, is more of the API reference information and doesn't actually direct you as to how to go about getting the feature level webhook setup for the feature layer.

Thanks for following up on this!

Re: AGOL Feature Service Webhooks

Anonymous User — Mon, 10 Aug 2020 19:55:34 GMT

Thanks for the follow-up response on this Tyler, and glad you found a workable solution for the time being. I haven't proven this out myself yet, but I imagine the best way to address it would be to check the status of the extract changes job and once it is "Completed", then open the Result URL file. I hope this helps, feel free to post here or reach out to me at pklingman@esri.com with any additional questions or feedback.

-Peter

Re: AGOL Feature Service Webhooks

Anonymous User — Mon, 31 Aug 2020 21:49:29 GMT

Tyler Bragg‌

The webhooks blog is live now, and shows how to use the sleep function in Integromat to handle the delay in generating the json file found at the result URL.

Create a hosted feature service webhook

Cheers,

-Peter

Re: AGOL Feature Service Webhooks

TylerBragg2 — Tue, 01 Sep 2020 11:41:14 GMT

Peter Klingman‌,

Thanks for the update! I'll check this out and I'm sure I will learn a few things. I wasn't aware of the "sleep" function in Integromat, so I'll have to look into that as well. Awesome!

Tyler

Re: AGOL Feature Service Webhooks

Anonymous User — Thu, 03 Sep 2020 22:11:29 GMT

Sounds good! I didn't know about the Sleep function either, was pretty excited to learn about that.

Re: AGOL Feature Service Webhooks

Derek_Bedarf — Fri, 08 Jan 2021 22:43:44 GMT

Hi Peter,

Is there a way to use the extract changes example you have above to take the coordinates of a newly created feature and reverse geocode to a field? I'm trying to think of a way to auto-populate a field with an address derived from only a point placed on the map.

Thanks. -Derek

Re: AGOL Feature Service Webhooks

RonParis — Wed, 20 Jan 2021 08:54:23 GMT

Hoping you can advise on this one @Anonymous User...

I've started using Integromat to automate and grab the resulting json file and give me the specific attributes I am after (referenced at the bottom of the blog post). The only issue I'm encountering is that my feature service needs to be publicly shared for this to work. Is it possible to build into the Integromat scenario an authentication so that this can be achieved without needing to publicly share an editable feature service?

Re: AGOL Feature Service Webhooks

Anonymous User — Thu, 21 Jan 2021 00:41:22 GMT

Hi @RonParis - I had to check the blog to be sure, but it is surprising that it was written for a publicly editable hosted layer. Since we are now more explicitly discouraging publicly editable layers, this would be important to amend.

You will need to generate a token that will be appended to the URLs. You can do this in Integromat with the HTTP module. I did test this quickly and it returns a token that can be used to access a secure service. Then you would append the token to each of the service URLs to access the endpoints. Below are the parameters I used in the request body, also documented in the generateToken link.

I hope this helps ~ it definitely needs some fine-tuning but I hope this gets you started!

-Peter

Re: AGOL Feature Service Webhooks

Anonymous User — Thu, 21 Jan 2021 00:48:15 GMT

Hi @Derek_Bedarf - this sounds possible to me. You have access to the attributes of the feature (here is the blog on how to accomplish that component), so you would need to obtain the X/Y from the edit, then chain together some HTTP modules to reverse geocode the coordinates and then update the feature or apply the edit, back into the desired field. Hope this helps!

Re: AGOL Feature Service Webhooks

RonParis — Thu, 21 Jan 2021 12:25:39 GMT

That's great, thanks @Anonymous User. I've managed to get the token to generate and then append to the URLs along the scenario, utilising the substring function to grab the token from HTTP response. It's not the prettiest but it works!

Re: AGOL Feature Service Webhooks

Anonymous User — Thu, 21 Jan 2021 17:46:02 GMT

Glad it worked!

Re: AGOL Feature Service Webhooks

julian_svcs — Fri, 22 Jan 2021 02:37:14 GMT

Hi Peter.

I am trying to run the extract changes for an ArcGIS Online feature layer. When I try and access the job's resulting URL I get an error stating "Invalid Sync model type. syncLayers json object is required.".

Do you know what is causing this?

Thanks, Julian.

Re: AGOL Feature Service Webhooks

Anonymous User — Sat, 23 Jan 2021 00:13:55 GMT

Hi Julian,

The one record of this error I am seeing internally indicated it was caused by a mismatch between the layerServerGens and Layers parameters in the extractChanges call.

I would make sure the same layers are referenced in each parameter (e.g. 0 in both parameters vs. 0 in one parameter and 0 and 1 in the other). If they are and you are still running into the issue, let me know and I can help get a support case going for an analyst to take a closer look at the issue.

I hope this helps,

-Peter