ArcGIS for Power Automate - v2023.3 - Change log and updates (September 18, 2023)

1506
17
09-18-2023 07:04 PM
AnthonyLatini
Esri Contributor
1 17 1,506

ArcGIS Connectors for Power Automate is updated regularly to provide new features, improved speed and usability, and bug fixes. The 2023.3 update is a quality release. Performance improvements, new actions, and bug fixes have been made for the following features. 

This release will start deploying on September 18th, 2023. It may take up to 48 hours to push out to all regions (up to two weeks for government-level sites). 

If you do not see the changes, you may need to clear your cache and restart your web browser to see the changes in Power Automate.

As always, please let us know of any improvements or issues you want to chat about.

 

---

ArcGIS Enterprise

If you had gone to this year's User Conference in San Diego, you might have heard the team mention support for ArcGIS Enterprise inside of Power Automate. Well, it's finally here! We are rolling out the ArcGIS Enterprise connector in phases.

The first phase, which is now available in Power Automate, contains all of the same CRUD-based operations that are found in the ArcGIS connector. For example, you can start adding, deleting, or updating records and attachments in your feature layers just as you would using the ArcGIS connector. 

AnthonyLatini_0-1694816387447.png

The second phase of the ArcGIS Enterprise connector will contain the remaining actions that pertain to geocoding, geoenrichment, and networking. We are also working on adding webhook support for ArcGIS Enterprise 11.2 and above. In the current phase of the connector, you must manually set up the webhook on your feature layer.

Details on how that is done can be found in here.

Note: This is a preview action as support for webhooks will not be fully integrated until the release of ArcGIS Enterprise 11.2. 

Some notes about the ArcGIS Enterprise connector before I move onto the next topic. 

Requirements

- The ArcGIS Enterprise connector supports ArcGIS Enterprise 11.0 and above

- To use the webhook preview, you must have ArcGIS Enterprise 11.1 or above

- Your Enterprise instance must have access to the internet (or at least access to Power Automate itself). The connector will not work behind a firewall or in a disconnected environment. 

All of the set-up information can be found here in our documentation. 

DocumentationSet up the ArcGIS Enterprise connector—ArcGIS for Microsoft 365 | Documentation

 

Find Address Candidates

As we have been working through various flows and feedback from you, a common ask was having the ability to use custom geocoders in the geocoding actions, Find Address Candidate and Geocode Addresses. By default, those two actions return the top match from the ArcGIS World Geocoding Service, and you can pass those values into another action or application.

Dynamically, it will only return the top result. For example, when I geocode 15 Maple Street, I receive the best match after the flow has finished. I deliberately did not pass any other information other than the street name. As a fun note, this is one of the most common street addresses in the United States. 

In the results, it appears that the World Geocoder gave me a location in England. Even the score that was returned isn't 100 (meaning it wasn't a perfect match). 

AnthonyLatini_1-1694818177489.png

 

Now let's pass that same variable into the new action Find Address Candidates. The first difference in this action is the ability to change the geocoder. This list is populated dynamically from your ArcGIS Online (and eventually Enterprise) organization. For this example, I will still use the ArcGIS World Geocoding Service.

AnthonyLatini_3-1694822103650.png

When I use the same address, the results window looks a bit different. In this result, it is now sorted by the Top candidate

AnthonyLatini_4-1694822297453.png

 

If I peek at the raw outputs, I can see that there are multiple results for this address. 

AnthonyLatini_5-1694822354104.png

 

From this action, I can now pass either the top candidate or create a flow that filters out the results based on a score of 80 or above. I encourage you to play around with this action and use it in your flows as it will open up some new possibilities with custom geocoders in your organization. 

AnthonyLatini_0-1694822711523.png

 

AnthonyLatini_1-1694822717567.png

 

Token Expiration / Manual Sign In

A workflow that was a huge hurdle to overcome in ArcGIS Connectors for Power Automate was manually refreshing your connection whenever the token expired. For example, if your organization had a 14-day timeout on your login, you would need to manually refresh the credentials to keep your flows active. That is no longer the case and has been implemented since early May 2023. 

We have been running internal tests and have confirmed that even with a token timeout of 90 days, as long as you run a flow with the same credentials, the token will refresh automatically, and you won't need to sign in manually. 

 

Attachment Support

While this has been in the connector for a bit of time, I wanted to highlight it here in this post. 

  • Added support for the use of Attachments in ArcGIS Connectors for Power Automate

Attachment Triggers

  • When an attachment is created
  • When an attachment is deleted
  • When an attachment is updated

AnthonyLatini_0-1695087519105.png

Attachment Actions

  • Add attachment
  • Query attachment
  • Get attachment
  • Delete attachment
  • Get attachment from URL

AnthonyLatini_1-1695088258630.png

 

----

 

Bug fixes and polishes

 

Bug Fixes

  • Fixed a bug related to the Get geometry from feature layer action - Exporting to EsriJSON does not return the correct values
  • Fixed a bug related to the Fetch updates and changes actions - Fetch updates do not filter results to the selected Feature Layer
  • Fixed a bug related to CRUD operations for feature layers - Create a record fails when only "Add" is enabled in settings
  • Fixed an issue where Fetch updates, changes, and deletions was returning unexpected changes in the results body
  • Fixed an issue related to triggers failing for certain layers
  • Fixed an issue related to inconsistency in runtimes in longer flows
  • Fixed an issue when a flow contains empty geometry and would not complete
  • Fixed an issue when updating a record requires a “number” but is a “string”
  • Fixed an issue when a query returns no records even with the appropriate query parameters
  • Fixed a rare issue with Create a feature layer when a layer added to ArcGIS Online would merge data fields together

UI/UX polishes

  • Fixed an issue where the watermarked text was not appearing for Ring Buffers under the Geoenrichment action

 

Quality of Life updates

  • Users can now choose what editing options they want available for their feature layer. Previously, it was required to have add, delete, and update checked for editing.
  • Updated multiple error messages into a human readable format
  • Token expirations are handled by the user's organization and refresh tokens do not need to be manually updated

---

The ArcGIS Connectors for Power Automate connector continues to grow and expand. We are working on the next phase of ArcGIS Enterprise support and will be rolling that out in the next few months. We also have been compiling your feedback and are working on some new helper actions and finding ways to make automation fun and exciting.

As always, we love to hear your thoughts and feedback on the connector (whether it is positive or negative). We thrive on understanding how you use Power Automate and the ArcGIS Connector inside your organization. Let's make automation part of your group's daily routine! 

 

17 Comments
TL2
by
Occasional Contributor III

@AnthonyLatini 

 

Can you confirm "The ArcGIS Enterprise connector supports ArcGIS Enterprise 11.0 and above"

I am currently using Enterprise 10.9.1 and am able to run get data from feature layer successfully.

 

TL2_0-1695215260959.png

 

AnthonyLatini
Esri Contributor

@TL2 - While different instances of ArcGIS Enterprise may work (meaning anything below ArcGIS Enterprise 11.0), we only officially support ArcGIS Enterprise 11.0 and above. 

acarmody
New Contributor III

Can this ArcGIS Enterprise workflow be made available for the ArcGIS Online connector in the next update?

It is limiting that the connector controls the webhook - there doesn't seem to be a way to edit it in power automate? and it can't be modfied on the feature layer (except renaming). I've tested changing the 'change type' to wildcard (*) - the power automate flow ceases to hear any feature changes. 

If I'm completely wrong about this please disregard and please could you share a how-to?

Many thanks.

SeanKMcGinnis
Esri Contributor

Hi @acarmody - the ability to create a flow by manually creating webhook following a similar pattern with ArcGIS Online is possible, but is not a direct product capability. The flow creation is less of a configurable component and more of a technical experience which goes against the intent for a no code/low code process builder. 

The reason we are handling webhooks in the post you reference is due to differences in the way ArcGIS Online & ArcGIS Enterprise have implemented webhooks. The intent is to create a more consistent experience, modeled after the ArcGIS Connector, with the 11.2 release of ArcGIS Enterprise.

It makes sense that the flow would become unresponsive if you changed the 'change type' to a wildcard would fail. The action has no concept of how to handle an asterisk. Could you please provide a little detail on what you are trying to accomplish by listening for all webhook types?

Maybe submit it as an idea for consideration in future releases.

acarmody
New Contributor III


Hi @SeanKMcGinnis , thank you!

Our organisation has  AGOL layers that need to be watched for any/all changes. I'm rebuilding our Integromat webhooks in Power Automate. Hoping to keep using 'wildcard' so only one flow is needed for each layer. 

erica_poisson
Occasional Contributor III

Hi @AnthonyLatini  @SeanKMcGinnis -

Can I get some confirmation here on token life - above it seems to be saying 90 days, which I thought was the case for GCC. I'm having some issues with my flows using the ArcGIS connector failing due to authentication issues well before their 90 days is up. 

Can you please share any insights here?

Thank you,

Erica 

TL2
by
Occasional Contributor III

@erica_poisson 

I have a simple flow for both ArcGIS and Survey123 connectors, it runs every hour and alerts me if the connection failed.  I don't trust the token life either.

erica_poisson
Occasional Contributor III

@TL2  - could you share a screen capture of this flow? I'd be very interested in setting something similar up. This continues to be a massive pain point of using this technology. I have reminders all over the place to check tokens, but it seems like the length keeps changing! 

TL2
by
Occasional Contributor III

TL2_0-1701185015250.png

Note, it "fails" every time but I make the Terminate to run if it fails and log Successful.  If it's actually successful, it sends the email.

SeanKMcGinnis
Esri Contributor

@erica_poisson & @TL2 - thank you for the comments and sharing the way you are working with the token situation.

There are two tokens that we are managing within the ArcGIS Connectors for Power Automate. The first is the session token which is what authenticates the user in the flow and is updated each time the flow runs. The second token is the refresh token, which manages the connection settings validating whether the user can make a request back to Online or Enterprise to get a session token. The expiration of this token is managed buy your Organization and by default is set to 14 days, but is configurable.

Each time a flow runs we update refresh token, which in turn starts the clock to timing out against your Organization or Enterprise. If the flow has not run within the duration of your refresh token timeout, it will fail. The way I have been getting around this is I have a scheduled flow for each of my connections that runs on a daily basis to query against a specific feature layer. This forces the refresh token to be refreshed before it times out, preventing me from getting unexpected connection issues.

I hope this helps and am open to any feedback or questions.

SeanKMcGinnis
Esri Contributor

@TL2 - that is the Survey123 connector which uses a different authentication pattern than the ArcGIS Connectors for Power Automate and timeout every two weeks. The ArcGIS for Microsoft 365 team is working with the Survey123 team to update their authentication pattern to more closely follow our pattern.

erica_poisson
Occasional Contributor III

@SeanKMcGinnis I think the community would greatly benefit from some sort of blog to walk them thru how to set up what you've described above. I certainly know I would need some step-by-step guidance to configure such a flow. 

erica_poisson
Occasional Contributor III

@SeanKMcGinnis -

I recently went thru the work of clearing out all of my duplicate connections for ArcGIS and Survey123 in Power Automate. I created two new connections and rebuilt all of my flows to ensure that my Connections list was clean and I stopped having so many issues with flows failing and needing to be fixed. This has worked great until today. 

I got an error from one of my flows that uses the Survey123 Create report action. The failure was for a missing token. When I go into Power Automate > Connections, I see that my Survey123 Connection shows a Status = Connected. 

I know that if I go into the broken flow and click the ... on the top corner of the Create report action, that I can use +Add new connection to fix the flow, however I also know this adds a duplicative Survey123 connection to my list of available Connections which is annoying/not ideal/difficult to manage. 

Do you have any suggested strategies for dealing with this situation? My Organization has ArcGIS Online and I am not able to find a way to increase token length thru Organization settings. 

I've posted about this as a question in both the Power Automate and Survey123 communities in the hopes of a quick response. Any help or advice on dealing with this would be amazing. The Create report action could be great, but this is incredibly frustrating and makes it unreliable and not really a tool worth using because of it. 

Question posted in Power Automate community here

Thank you,

SeanKMcGinnis
Esri Contributor

Hi @erica_poisson,

Unfortunately, the Survey123 connector is still using the older pattern of authentication. Our team is working with the Survey123 team to update the authentication pattern to the one currently in use by the ArcGIS Connectors for Power Automate. I do not have an estimate time when this will be completed, but when deployed (and promoted to GCC), it will address the timeout issue.

As updating the connection so you no longer have duplicates, I go to the Survey123 Connection and click the 'Switch Account' button and re-enter my credentials. This renews the authentication tokens and prevents creating another connection.

Switch Account process will allow you to reauthenticate the connection and keep all Flows running.Switch Account process will allow you to reauthenticate the connection and keep all Flows running.

 

daveostrander
Occasional Contributor II

Hello @SeanKMcGinnis  

I am trying to understand the rules regarding tokens for our current connection. Unfortunately, we are still at 10.9.1 for our ArcGIS Enterprise and will likely need to be at that version for some time. However, I did create a custom connecter using the Esri technical article that works well with Survey123 and ArcGIS Enterprise Portal (I know it's not supported exactly). I am just now getting back to testing/working with this connection, and if memory serves correctly, I have had flows where the token expired on my trigger (no longer showing dynamic content and listing the item ID instead of the survey name), but it still functioned when surveys were submitted. So my question is a multi-fold one: will a custom ArcGIS Enterprise connector connection expire (if so, how long before that happens), and within my flow, can the trigger expire and continue to function? Also, you mentioned a workaround to keep connections alive.

You stated:

"If the flow has not run within the duration of your refresh token timeout, it will fail. The way I have been getting around this is I have a scheduled flow for each of my connections that runs daily to query against a specific feature layer. This forces the refresh token to be refreshed before it times out, preventing me from getting unexpected connection issues."

I built two scheduled flows in an attempt to follow what you were laying out above. I have a scheduled flow with my custom connector connection, which queries all my published surveys. The other scheduled flow makes an HTTP POST request to generate a token, parses the JSON to retrieve the token as dynamic content, and makes an HTTP GET request to query all of my published surveys. That one doesn't work because the token I get doesn't seem to work with the header's Authorization: Bearer <access token> scheme. Assuming both are working, would either of these scheduled flows have the effect of keeping the custom connector alive? And where does one get a Bearer access token if it differs from a portal-generated token? I am just seeking as much clarification as possible since there seem to be different types of tokens and connectors/connections at play. Thank you for any guidance you can provide with this.

daveostrander
Occasional Contributor II

Hello @SeanKMcGinnis  

Since my last post, I have done some further research into OAuth2. From that, I have come to the understanding that performing the authorization and token requests in a flow will not work to keep my ArcGIS Enterprise custom connector alive because a flow produced in this way would require credentials to be entered manually at the redirect page. Instead, this workflow is really what the custom connector does for me on its own. So, my question now is, will the ArcGIS Enterprise custom connector refresh itself indefinitely (whether or not any flows run), or do I still need to have a recurrence flow that runs daily, which queries the connection using an action type flow? In my case, the flow I set up queries all of my published Survey123 surveys. So far, in my week or two of testing, I haven't seen the custom connector lose connection, requiring a manual sign-in to reauthorize it. However, even losing connection at 90 days would be bad for a production workflow where the public submits surveys at will. Again, thank you for any guidance you can provide on this.

daveostrander
Occasional Contributor II

Hello @SeanKMcGinnis 

At this point in my testing, the token on my get surveys action query connection has finally expired (elapsing the 20,260-minute mark), and as a result, my recurrence flow fails with a "missing token" error. So, it would seem that setting up a recurrence flow to query the connection to keep it alive doesn't seem to have panned out for me here.

However, my trigger-based flows that run when surveys are submitted using webhooks from the survey123 website still function as expected even though they have also expired (showing the survey item ID instead of the survey name and not displaying dynamic content). This development is promising! So, I think I am only down to one question: Will my custom ArcGIS Enterprise Connector work indefinitely as long as my flows are only trigger-type flows using webhooks? I guess the next test for that won't be for 90 days when the refresh token will max out (if I understand that correctly). I'd prefer not to wait that long to find out, so any guidance you can provide on the expiration of the ArcGIS Enterprise custom connector would be greatly appreciated. Thanks in advance!