ArcGIS Connector for Power Automate - Getting Started with Webhooks

[SeanKMcGinnis]

If you have heard people reference webhooks, but weren’t sure of what they are - webhooks allow web services to provide other services with near real-time information using HTTP POST requests.

Feature layer web hooks are triggered by an event, such as creating, updating, or deleting records in a hosted feature layer. The ArcGIS Connector for Power Automate has triggers that can create and configure webhooks to allow Power Automate flows to use the response payload and data to be used by the other actions.

Webhook Triggers

ArcGIS for Power Automate Connector - Triggers

The ArcGIS Connector for Power Automate has three triggers that listen for different types of changes within ArcGIS Online feature layers (links will take you to the trigger documentation)

• Updated record in a feature layer
• Deleted record in a feature layer
• New record in a feature layer

Before creating the trigger, review the feature layer's Editing settings an confirm the following options are checked:

• Enable editing
• Keep track of changes to the data (add, update, delete features)
• Keep track of who edited the data (editor name, date and time)

Back in Power Automate, select the appropriate trigger and choose the feature layer from the dropdown.

Select the desired feature layer from the dropdown

The next step is to select the 'Fetch updates, changes, or deletions from feature layer' action to handle the payload from the webhook.

Choose the Fetch Updates action

When the action is added to the flow, it will appear as a single action in the flow.

Original state of the action when adding it to the flow

When populating the 'Get changes from' with the 'Changes URL' Dynamic content, you will notice the interface changes and gets wrapped inside an 'Apply to each' because the webhook payload may have multiple records included depending on how actively edited the feature layer is and what the webhook's interval is set to.

After completing this step, the flow can now work with the data returned as the feature layer changes.

I hope this helps you get started creating triggers on your feature layers. If you have any questions, please do not hesitate to reach out.

[PhilLarkin1]

Thanks for the blog post. This should simplify flows. 

Any word on if future iterations of ESRI's feature service web hooks will include attribute info for deletes? See Community thread. https://community.esri.com/t5/arcgis-online-ideas/feature-service-webhook-should-include-all-fields/...

[SeanKMcGinnis]

Hi @PhilLarkin1 - This was deployed as part of the March 2022 release. The team worked with the ArcGIS Online team to extend the webhooks for deleted records and now the payload provides all of the attribute information for the deleted record.

[PhilLarkin1]

@SeanKMcGinnis Awesome! Thank you 

[SeanKMcGinnis]

@PhilLarkin1 - please let me know how it goes

[SeanKMcGinnis]

@PhilLarkin1 -  I ran a test to confirm, below are the screenshots:

After successfully running the flow

Showing the raw output from the webhook's payload - it includes all the attributes

[PhilLarkin1]

This will cut a few branches from the flow tree. Excited simplify my flows. 

Looks like, based on the unix timestamp in your screenshot, EditDate and Editor are not showing the user/time the feature was deleted. Is this accurate?

[SimonNigh]

When will this be available for Enterprise please?

[SeanKMcGinnis]

@PhilLarkin1 - there is not a deleted time returned, it is passing the attributes from the record. The closest thing I could find when looking through the flow was when the webhook was triggered. I know it is not the exact time that the item was removed, but depending on the frequency the webhook checks for edits it could be "close".

[SeanKMcGinnis]

@SimonNigh - the team is woking with Microsoft to identify Power Automate security best practices to allow for the connector to authenticate against Enterprise and are planning to have the capability before the end of this year.

[erica_poisson]

@SeanKMcGinnis  - do these new webhooks help to simplify the workflow detailed in this blog from Esri Canada? 

https://resources.esri.ca/getting-technical/webhooks-microsoft-flow-and-the-arcgis-rest-api-automati... 

Thank you,

[SeanKMcGinnis]

Hi @erica_poisson - yes, these triggers will help simplify the process. You will no longer have to request and manage the token, construct the query request and HTTP handling to get the data.

When the trigger is fired, it will have the edits, updates, or deletes as part of the payload that can be used in the downstream actions of your flow.

[Anonymous User]

Hi  @SeanKMcGinnis

This is great, do they work in reverse by creating, updating and deleting records in a feature layer when a row is added, modified or deleted in a table in PowerApps?

I understand Create a record in a feature layer,  Delete a record from feature layer &  Update a record in a feature layer actions in PowerApps may fulfil this functionality. In which case is there an expression that can convert Latitude, Longitude or GeoJSON strings into a Geometry Object for this entry in PowerAutomate?  

Thanks

[SeanKMcGinnis]

Hi @Anonymous User,

I am not sure I fully understand your question. If you are asking if there is a way to update a feature layer when there are records created, modified, or deleted from a table that is managed by a Power Apps-based application? If that is what you are looking for, there are many triggers that can listen for edits in DataVerse or other data storage accessible in the application. From there, you would be able to use the 'Create a record in a feature layer', 'Update a record in a feature layer', and 'Delete a record from feature layer' actions to manage the data in a hosted feature layer.

There are no helper functions at this point to convert values to a geometry, but you can create them as variables from values stored in other tables or extracted from a geocoded record, but issues might arise depending on the complexity of the geometry and the length of the string.

[JiaJia]

Hello @SeanKMcGinnis ,

I tried the workflow but found it unstable.  I create an app through the AGOL app-builder including an editable layer. My goal is to trigger an email with the the attributes info of the newly created feature when it was created. It could include several scenarios: for example, sometimes the user may edit the other info in the app before they create a new feature, sometimes they would delete the feature created by mistake and make another right one.  These cause problems: the Fetch Updates, Changes, or Deletions from Feature Layer method can't always retrieve all the attribute info of the changed rows. Even when I only create a single feature, sometimes it can't retrieve all the attributes (please see the attached screen shot). Is there way that I can improve this workflow and get the attributes that I wanted? thanks so much!

[SeanKMcGinnis]

Hi @JiaJia 

I am afraid I need a little more detail to better understand the use case you are sharing and the problems you are facing. If I understand you correctly, you are:

• Using an editable feature layer in an ArcGIS Online app-builder based application that allows users to create, update, and delete records
• The desired outcome is an email is sent when a new record is created using the ArcGIS Connector for Power Automate and a trigger listening for a feature layer webhook

Can you please expand on some of the scenarios and problems you laid out?

• "Sometimes the user may edit other info in the app before they create a new feature" - What is it that they are editing and where is it being stored if they have not created a feature? 
• "sometimes they would delete the feature created by mistake and make another right one" - If the record was created, deleted, and then a new one created, how would you expect the Connector to be aware which is the correct record to include in the email?
• When you say you "can't retrieve all the attributes' why is that issue being presented? Is it because upon record creation, the creator did not add values to all of the fields and the payload is including the level of attribution that was populated when initially saved?

[JiaJia]

Hello @SeanKMcGinnis ,

Thanks for your quick reply! Your understanding is correct.  We only need to trigger the email when a new feature was created in the editable feature layer in AGOL. And the email needs to include the attribute info of the new feature. Meanwhile, we want  to grant the user maximum flexibility to edit/delete the records in this feature layer without triggering any emails. These scenarios cause the problems:

#1 Assuming we already created a number of features and they were displayed in the web app. Then the user found some mistakes in the previous records, he revised those records in the app. In the next, he will create a new record and trigger emails to all the stakeholders. I expected the workflow by Power Automate could fetch the data of several rows after his activities. For example, the revised rows and newly created ones? I tried the work flow, but the newly created feature was not always fetched by the fetch method (Fetch Updates, Changes, or Deletions from Feature Layer).

#2  "If the record was created, deleted, and then a new one created, how would you expect the Connector to be aware which is the correct record to include in the email?", My thought is to sort all the fetched rows by the order of edit time (or OBJECTID number), the most recent time (or the highest number of OBJECTID) would be the newly created feature, as we only trigger email after creating a new feature. 

#3 I applied the Editor widget from APP Builder to our web app, so the user can edit, update and delete the feature layer and its attributes at any time. It could be the reason the attributes of the new feature can't be fetched immediately. The Editor widget needs two steps to create a feture: 1) draw the geometry 2) fill in the attributes and click the save button to update the records. It seems like after the 1st step, the Fetch method was triggered. So no attributes have been fetched at all? Also I wonder if the webhook payload have limit on text length? I found sometimes the fields with many words can't be fetched (e.g. the dscpt field usually include 100+ words)

All the above process are performed through ArcGIS Online. The editable layer is from AGOL hosted feature service. So all the data are stored in AGOL and has AGOL default webhook configurations. I hope to test the workflow first in AGOL, then move to  our Enterprise Portal. There may be more issues regarding configuring the Enterprise webhook. 

Anyway, thanks so much for discussing those with us!

[Anonymous User]

Thanks for the tip @SeanKMcGinnis. This seems to have done the job to create the variable to pass into the 'Create a record in a feature layer' action, for point features at least.

[SeanKMcGinnis]

Hi @JiaJia,
Ok, thanks for confirming my understanding of the workflow you are looking to support. I will answer below based upon the numbering of the points you raised.

#1 - If the user is updating a record and then creating a new record, the payload from the webhook will be classified as 'FeaturesCreated' and 'FeaturesCreated' with the respective geometry and attribution. By creating the trigger using the 'When a record is created in a feature layer' trigger, it is only going to pass back the 'FeaturesCreated' data to the 'Fetch updates, changes, or deletions from feature layer'. If the workflow you are making is looking to get information from multiple records, the Flow will have to query for them using a known identifier (i.e. UniqueID) using the 'Get data from feature layer' action.

#2 - If your expectation is to work with multiple records instead of the newest one created, I would again use the 'Get data from feature layer' to return all of the records and do the appropriate data engineering. Given some of the data issues you are experiencing, I would put a conditional statement in the Flow to check for a minimum set of attribution before sending the email.

#3 - The way the Editor widget is building the record could be introducing the problem. But, I am not sure of the way that it handles the creation of records. To my knowledge there are no limits imposed by the webhook or Power Automate, but will raise it to the Engineering team to confirm.

For now, the workflow is going to have to live in ArcGIS Online if you are looking to use the ArcGIS Connectors for Power Automate. We do not have Enterprise capabilities yet, but they are currently in development.

[JiaJia]

Hi @SeanKMcGinnis ,

Thanks for your reply! I triggered the above workflow by Editor Widget both in Web App Builder and Experience Builder. It indeed introduced some issues. Sometimes the Fetch method retrieved nothing from the updated feature. I guess there must be a time lag between the updating the geometry and attributes through Editor widget. Do you have any idea to fix the issue caused by Editor Widget?

I also have questions regarding feature attachment: 

Is there any way that the Fetch method (Fetch Updates, Changes, or Deletions from Feature Layer) can retrieve the attachment in the updated feature? Will 'Updated record in a feature layer' trigger the workflow when the attachment of the feature was changed? I tested it through Editor widget when updating the attachment, it caused problems.

Thanks so much for your answers!

[SeanKMcGinnis]

I am glad they help @JiaJia.

Regarding your questions on feature attachments, the team is in the process of releasing an update to the ArcGIS Connector for Power Automate that will have attachment capabilities. Keep an eye on the community and we will post when they are in Preview.

[BrianLeroux]

@SeanKMcGinnis  Will this ever be extended to work with feature services that we do not own? I would love to have this notify me when there are new storm reports added to the USA Storm Reports  https://services9.arcgis.com/RHVPKKiFTONKtxq3/ArcGIS/rest/services/NOAA_storm_reports_v1/FeatureServ...

[SeanKMcGinnis]

@BrianLeroux - that would be a question better directed towards the Feature Layer or ArcGIS Online teams. The ArcGIS Connector for Power Automate is leveraging the webhooks that they are delivering via the service. I see the utility in something like that, but I can also see some of the security or load concerns that might introduce.

[Jay_Paleja]

Hi @SeanKMcGinnis 

Great to see functionality developing in the ArcGIS Connectors for Power Automate. Is there a timeline on when the fetch updates, changes or deletions connector will have attachment capabilities?

[SeanKMcGinnis]

Good day @Jay_Paleja,

Attachment capabilities are currently in the Microsoft Preview region. Capabilities include triggers based on attachment webhooks and actions that will allow flows to get attachments.

[EricPescatore]

This is very interesting stuff but I'm having difficulty finding any documentation, videos and what appears to be very little activity on the ESRI communities forums.

Again the documentation does describe but really doesn't provide full details on engaging the triggers and actions.

The workflow I'm doing:

When a Survey123 form is submitted i want to update a feature class with a value that was generated from the form.  In my case I am looking to update a field with the Object ID (Since the object ID is generated server side when the form is submitted, Survey123 cannot invoke the ObjectID into another field in the database. So the workaround seems pretty simple. Trigger "When a Survey Response is Submitted" > Action "Update a record in a feature layer".   In this case, we would pull from the payload the objectID and populate another field in the database with that value. I'm not working with anything complicated, we just trying to use ObjectID as the unique id/related key. 

But in all my test thinking I could just quickly jump in an use the ObjectID, I am finding myself hitting an error within Power Automate yet the flows are working with a failed status. 

The error Received: "The API 'arcgis' returned an invalid response for workflow operation 'Update_a_record_in_a_feature_layer' of type 'OpenApiConnection'. Error details: 'The API operation 'UpdateItemInFeatureLayer' requires the property 'body/objectId' to be of type 'Number' but is of type 'String'.'"

As much as I like that flow is working, it still shows failed so I have concerns about the validity and long-term use. Since we are dealing with two different SaaS/PaaS the Power Platform and ArcGIS Online. Is ESRI Support the place to go with questions or does this fall under something not supported given its O365 Product using ESRI connections? 

[SeanKMcGinnis]

Good Day @EricPescatore 

I am glad you are finding interest in the capabilities and please know we are working on building more enablement videos and documentation.

The issue you are encountering is the data you are passing in to the field is not what it is expecting to see. The ObjectID is a number, but the field you are populating is expecting a string. The conversion you would use Microsoft Power Automate's 'string'  expression to do the data type conversion. The expression would look like:

string('ObjectID Value from Webhook Payload')

 NOTE: The 'ObjectID Value from Webhook Payload' text would be replaced by the dynamic content coming from the survey's webhook payload.

In regards to your concern around long-term viability and support - the ArcGIS Connectors for Power Automate are fully supported by Esri support and you can contact them if you encounter issues. I would also recommend looking into the resources Microsoft has freely available to help build your skills in using Power Automate. One that might be of particular interest given the issue encountered is the Introduction to Expressions in Power Automate.

[JiaJia]

Hi @SeanKMcGinnis ,

Our organization finally implemented ArcGIS Connector in a workflow, it worked finely. But it seems to lose the ArcGIS account connection every several days. I found the limitation in Power Automate documentation.  It says "Connections will time out after 14 days and must be refreshed manually by initiating a sign-in to ArcGIS" . Is there any way we don't need to manually refresh a sign-in to ArcGIS account. Survey 123 Connector seems not to have this issue. Thanks!

[SeanKMcGinnis]

Hi @JiaJia 

We have a lot of people asking that question - why is the connector expiring in 14 days, why don't I encounter that when using the Survey123 connector, and is anything being done to make this a better experience.

The reason users are encountering this issue is due to technology disparities between different platforms. There is a difference in the way Microsoft and ArcGIS Online grants the refresh tokens and that is what is causing the tokens to expire after 14 days. We are actively working with the ArcGIS and Microsoft teams to come up with a resolution - it is one of our highest priorities.

The near term fix, which is planned to be included in the next release (which is currently in Microsoft's Preview Environment), is using a different OAuth method which will extend the refresh token to 90 days to give us additional time to work through a secure and sustainable solution between the different parties.

The reason you are not encountering this with Survey123 - the submitter's token is being included in the webhook payload. Since the ArcGIS Connector for Power Automate is not authenticating outside of the connection, there is no other way to get a token that can be passed securely given the permissions needed to edit a feature layer.

 I hope this helps explain why you are bumping into these issues and where we are in addressing the issue.

[ChristopherNourry1]

I have a question - regarding a hosted layer and view layers created with filtering by certain attributes for each. Is there a way to have one trigger watch the base hosted feature where we can fire off an action instead of having to make 15 individual power automate flows? We tried both and we could only get the triggers to work on each view layer. It makes things a bit more cumbersome. 

[SeanKMcGinnis]

@ChristopherNourry1 - there is not a way to create a single trigger to listen to multiple feature layers/views. My suggestion would be to add that as an idea and we will review it as a possible solution when we come into our next semester planning.

[Kuyler_JonesCivil]

I'm running in to trouble with Table layer types. When using the Fetch updates, changes, or deletions from feature layer action, whenever I select a feature layer that is a table, I get the following error when trying to save.

Flow save failed with code 'DynamicOperationRequestClientFailure' and message 'The dynamic operation request to API 'arcgis' operation 'GetFeatureLayerResponseSchemaForWebhook' failed with status code 'BadRequest'. This may indicate invalid input parameters. Error response: { "message": "Cannot read property 'toLowerCase' of undefined", "error": {} }'.

Is there a special process for table layers? Or are they simply not supported?

[MeganHicks]

@SeanKMcGinnis 

Thanks for all the information on this post, it's been very helpful to get set up with this new connector!

I am running into an issue with the Fetch Updates action. The flow is able to see that a change occurred in the hosted feature service, and if I have it set to send an email directly from that trigger, it works just fine.  But when I add the step to Fetch Updates from the updated record, the flow fails. The output error is "Unable to synchronize replica", code 400. I checked in my hosted feature service that all editor tracking is enabled. 

Any thoughts on what to change?

Thank you!

[SeanKMcGinnis]

That is an interesting issue and since it is a 400 error coming back from the service, it leads me to believe it is something going on at the service level and not in the connector. I did some research and there are a few community posts (Post 1, Post 2) where others have encountered this and it is the result of the service’s source data.

My suggestion is to check the two community posts and if it continues to open a support ticket. If you do open a support ticket, could you please send me an email at smcginnis@esri.com so I can follow along in case others encounter the same issue?

[JoeyCoscia]

Hi @SeanKMcGinnis ,

Thank you for being a resource in this esri community thread.

I am following your Power Automate workflow in attempt to post using the trigger: New record in a feature layer.

I have confirmed that following options are in the Editing settings are checked:

• Enable editing
• Keep track of changes to the data (add, update, delete features)
• Keep track of who edited the data (editor name, date and time)

The flow is simply not triggered when adding a new record into the feature layer.

Is there any advice or work-arounds you can propose to troubleshoot in order to get this trigger to fire?

Thank you.

[SeanKMcGinnis]

@JoeyCoscia - my first question in troubleshooting is how is the feature layer being edited (desktop client, web app, Field Maps, etc)? My next question is the application performing the edits going against the feature service itself or was an editable view created for the application to work with?

For a technical way to test, I use Webhook.site and perform an edit, update, or delete to see if the webhook is firing to another client.

[MeganHicks]

@SeanKMcGinnis 

It does seem like the issue I raised above is due to a problem with the HFS. Creating a new version of the data/HFS has fixed the issue but ideally we can fix the issue with the old HFS. Working with ESRI on that.

In the meantime, I am working on creating my email flow with the ArcGIS Connector for Power Automate. When I've used the Survey123 connector, I've used the method from Ismael here to convert times in the emails. It does not seem to work quite the same with the ArcGIS connector. Any idea on how to convert ESRI time (in seconds) to a normal time and date? 

Thank you!

[SeanKMcGinnis]

@MeganHicks  Can you expand on what is different or what you are seeing in the conversion? How is the date/time coming back to you?

I wrote up a post on how I Convert Epoch Time to a Human Readable Date Format, maybe give that a shot?

[JoeyCoscia]

@SeanKMcGinnis 

Thank you for your prompt response.

The feature layer being edited was through an AGOL web app.

The issue was with the 5 webhook limit that is imposed on a feature service. I had to go into the webhook manager and delete some other connections!

[SeanKMcGinnis]

@JoeyCoscia - I am glad you found a solution to the problem. I am wondering, was it returning an error message? If it wasn't, it makes me think we need to make the creation process a little more robust and provide a more meaningful message.

[BrunoGomesdeSouza]

Hi @SeanKMcGinnis 

On August 2022, you talked about the possibility of at the end of 2022 Power automate could allow these arcgis Power automate new functions in ArcGIS Enterprise. Do you have some news about it?

Thanks  a lot

[SeanKMcGinnis]

Hi @BrunoGomesdeSouza  - there are so many threads where people are asking for info about this, it looks like I missed one. The ArcGIS Enterprise Connector for Power Automate is currently in Microsoft's Preview environment. We are finalizing our testing on the connector and will be promoting it to General Availability in the next week or two.

[BrunoGomesdeSouza]

Hi @SeanKMcGinnis 

It's amazing!

You gave me a good one news.

Thank you so much!

[JamieHewitt]

Finding this information has been a welcomed surprise! In playing with the flows, I'm able to add the record to my feature layer, but it's not updating on my map. I'm hoping to be able to have a SharePoint list trigger a Power Automate flow that will pull the an address from a SharePoint list and add it to my feature layer and display the new point on my map. In an ideal world, that new point would then be included in my buffer layer and would have my 1000' buffer around it. So far, I'm only able to get the new address added to the layer, but nothing ever shows up on my map. I do have the refresh set for both my feature layer and my analysis layer. What am I doing wrong?

[BrunoGomesdeSouza]

Hi @SeanKMcGinnis , how are you?

Any news about Power automate new functions in ArcGIS Enterprise?

Did you have released?

Thank's

[JonDolphin]

Hi Bruno, that's great. We work with ArcGIS Enterprise 10.9.1, does your procedure supports this version or is just for 11.0 and above? When I came to step 7, for testing the connection, after Test new connection  the Test Operation button is disabled, is not clickable and I cannot figure out why. Have you got some idea? Thank you!! 

[AnthonyLatini]

@JonDolphin @BrunoGomesdeSouza - Hello! We are currently testing the ArcGIS Enterprise connector in our Preview region. Once we have finished testing, it will be available for general availability. This should be within the next 4-6 weeks as we need to work through Microsoft's submission process. 

Some notes about the ArcGIS Enterprise connector in Power Automate. We are working on this connector in two different phases. In the first phase, we are supporting CRUD-based operations and a preview action of fetching updates and changes on feature layers. Later on this year, we will be adding support for geocoding, geoenrichment, routing, networking, and webhooks.

We will also be working with the ArcGIS Enterprise team to fully incorporate webhooks. 

Once we have deployed the connector to general availability, we will provide instructions on how to use the preview version of webhooks in Enterprise 11.1 and above. 

Environments supported: 11.0 and above / 11.1 is required for testing webhooks

Actions supported: 

Note: This screenshot does not include the Fetch updates, changes or deletions from feature layer which will be a preview action.

Let me know if you have any questions! 

[SarahRijneke]

Are there any ArcGIS specific Power Automate tutorials available for further information? 

I've had no issues setting up a flow using the Survey123 When a survey response is submitted trigger. I was hoping to use the When a record is updated in a feature layer trigger to automatically send an email when a feature is updated and a specific condition is met but I don't see any options to select and filter specific attributes from my feature layer. For example, one of my conditions will include a status of "completed" i.e. I only want an email to be sent when a feature is updated and the status field is set to "completed". I've had no problem implementing this with a Survey123 trigger but in this case I'm looking for an update to a feature layer that is edited in the Field Maps App.

[mlinn]

@SeanKMcGinnis  Thanks for all the great info!

I'm trying to use the "When a record is updated in a feature layer" trigger to send an email when a feature is updated and a certain condition is met. 

It worked initially, then, without changing anything in the flow or feature layer, it stopped working. The flow appears to run fine, however sometimes it registers the condition as false in cases where it is true, so an email is never sent.

Any tips? Thanks!