Working with Survey123 attachments in Integromat

5034
18
01-26-2021 04:32 PM
IsmaelChivite
Esri Notable Contributor
4 18 5,034

Survey123 smart forms let you upload signatures, photos and files of different types into ArcGIS. This content is stored in ArcGIS as ‘attachments’. Using the out of the box Survey123 module in Integromat, you can build automated workflows that use these Survey123 attachments.

Common automation workflows using attachments include:

  • Automatically send an email when a survey is completed and include photos in the survey as email attachments.
  • Instantly backup photos and files submitted via Survey123 into Microsoft OneDrive or other cloud storage.

This blog introduces Integromat techniques to help you work with Survey123 attachments. It is assumed you are already familiar with task automation using Survey123 and Integromat. If not, check the following resources:

Downloading files with the Integromat HTTP module

Before you can work with Survey123 attachments in Integromat, you must download them first. The animation below shows how to use the HTTP module and its Get a file operation to download a Survey123 attachment.

002-Integromat HTTP Module Token.gif

 As you can tell from the animation, the Survey123 attachment URL is provided by the Survey123 module. If your survey feature layer is secured, you also need to provide a valid ArcGIS token, otherwise Integromat will not be able download the file. The token can also be fetched from the Survey123 module.

Once the file is downloaded, you can work with it. In the animation below the file is added into a new email message as an attachment.

003-Integromat Email Single Attachment.gif

 

Similarly, you could use other Integromat modules to upload the file into cloud storage, convert the file into another format or extract some file metadata.

The scenario described so far is the simplest possible: A survey configured with a question where the user can only upload one photo or file.

As the author of a Survey123 smart form, you can decide how many photos or files should be uploaded with questions in your survey. The screenshot below shows an image question in the Survey123 designer configured for users to upload a minimum of 1 photo and a maximum of 4.

001-Multiple Attachments Designer.png

The blog post Survey123 tricks of the trade: Photos describes more advanced techniques to capture multiple files or photos with a single question using Survey123 Connect.

If the survey question is configured to accept multiple photos or files, things get a bit more complicated. Let’s have a look at how to handle that.

Attaching multiple files to an email

If a survey question is configured to accept more than one file or photo, you will want to use the Integromat Iterator and Array aggregator modules to handle the collection of files.

Attachment collections are modeled in the Survey123 module as arrays. Let’s have a look at how you can use the Iterator module to download multiple attachments.

004-Integromat Iterator.gif

Note that the Iterator module must be configured to target the array of attachments you want to process in the Survey123 module. Also, the Get a file operation will no longer target the attachment URL in the Survey123 module. Instead, it will target the attachment URL in the Iterator module.

If we were to run the Integromat scenario at this point, one email would be sent for every attachment found in the survey record. This is not what we want. We want to send a single email with multiple attachments in it. This is where the Array aggregator module comes into play.

005-Integromat Array Aggregator.gif

 The key considerations to configure the Array aggregator are:

  • Set the Iterator as the source module.
  • Set the Target structure type to the module property that will receive the output of the Array aggregator. In our case, we want to use the aggregated array to populate the attachments property of the Microsoft 365 Email module.
  • When configuring how the target structure will be populated, make sure you use the Map option, so you can get the Data property from the HTTP Get a file operation and the File Name from the name property of the Iterator.

Note that as you finish configuring the Array aggregator, a gray indicator will wrap all the modules involved by the iterator. The Microsoft 365 Email module will be outside of that group, indicating that it will run only when the output of the Array aggregator is ready.

The next step is to have the Microsoft 365 Email module use the output structure of the Array aggregator to populate the email attachments.

006-Integromat Email Attachments.gif

 As shown in the animation, you need to enable the map option of the attachments property before you can target the output of the Array aggregator.

I hope this guide was useful to you!

 

 

 

 

 

 

 

 

 

18 Comments
AprilChipman
Occasional Contributor III

Can this be used in conjunction with attaching and emailing a Survey123 report? I have that setup in several Integromat scenarios, and it works wonderfully! But there are a few that it would be nice to attach the image or file *and* the report.

AndrewBowne
Occasional Contributor III

Absolutely.  I'm doing exactly this in several of my scenarios.  Simply add the Create Report module to your scenario, then add an HTTP module to get the report file, finally attach the result to your email..

AndrewBowne_0-1613587757621.png

AndrewBowne_1-1613587855722.png

 

 

DanteLee
Occasional Contributor II

@AndrewBownehow is it configured in the email module?

I have 2 fields that collect attachments. I'd like to attach them all to a single email notification

  1. Gets a single file
  2. Gets multiple files

I can set up each one separately, but struggling to figure out how to do it in one scenario!

DanteLee_0-1619564620576.png

 

AndrewBowne
Occasional Contributor III

You should be able to add a new attachment to your email module.  The first one can get that single file, the second attachment can reference the array aggregator of multiple files.  

Here is an example I use.  I am attaching two files: the first is a barcode image, the second is a PDF attachment.

AndrewBowne_0-1619611913985.png

While it's not exactly what you are doing, the concept should be the same.

DanteLee
Occasional Contributor II

I can do it that way if I have multiple single files, but no dice working with the array of multiple files and other single attachment like that. @IsmaelChivite?

AndrewBowne
Occasional Contributor III

I've got an idea.  I don't know if it'll work but could be worth a try....

Wrap each of your HTTP Get Modules in an iterator and aggregator.  You already have that for the second one so all you need to do is the first.  Yes, I know there is only one file, but it should create an array aggregator with just that file.

Then in the email merge both the arrays together in the attachments section.

merge(ArrayAggregator1;ArrayAggregator2)

See this link on Array Functions (merging).

https://support.integromat.com/hc/en-us/articles/360005771833-Array-Functions

 

DanteLee
Occasional Contributor II

Oh lordy - the merge function did it. 😵 I had some idea that's what I needed, but I was missing that piece! Thanks @AndrewBowne 

MichaelBruening
Occasional Contributor

It's been a minute! I am trying to pull attachments from repeats and it is not as straight forward as I thought it was going to be. I have a report form that has a related point that allows users to drop locations of photos being taken while completing a site visit. This will later be used within a web builder application to show where photos have been taken and the direction they were taken. I had thought about using QuickCapture and the Oriented Imagery to do this, but I am trying to also use the Photographs as part of a custom report.

The data schema is a point for the initial report and a repeat with a geopoint with attachments applied (repeat relationship #9). There are several other repeats within the form, but none of them have attachments. In the past I have been able to complete

  1. HTTP Get request for the related records
  2. Parse the JSON
  3. Iterate through each related record
  4. HTTP Get request for the attachments of the related records
  5. Parse JSON
  6. Iterate through each attachment
  7. Save into OneDrive

At least that is how I remembered doing it a little over 2 years ago (I switched companies and couldn't bring existing materials with me, so I am relearning the processes) within Integromat. However, I am receiving an error once the scenario reaches the first Parse JSON and states "Not valid JSON".

To ensure I had the correct JSON, I went into the REST services page for the hosted feature service and generated the JSON using the (GET) method. I used this JSON to create the data structure for use within the Parse JSON module; just like I have in the past. I completed this same exercise for the querying of the attachments.

MichaelBruening_0-1640190285590.png

The JSON string being returned does not appear in the format that I would expect from the data structure.

MichaelBruening_1-1640190734949.png

Is there a new way of extracting the attachments from within a repeat? The only other thing that is bugging is that I thought at one point we had to declare parameters within the "Query String" of the HTTP request? Which would be everything after the "queryRelatedRecords?" portion of the URL?

MichaelBruening_2-1640190849290.png

Thanks in advance for any insight.

Mike

MichaelBruening
Occasional Contributor

So, I had the "f" variable set to "html" instead of "json" within the query URL. I'm okay with it just being something that simple.

rachelm
New Contributor III

Has anyone tried attaching the Survey123 report to a feature in an online feature service, after using HTTP Get File request? This is something that we currently execute in a gp tool and we would like to transfer this to Integromat. 

My Get File command works to attach the report in an email. But when I try doing an addAttachment request using the Rest API, it fails. 

 

add Attachment Integromat.PNG

error.PNG

  

AndrewBowne
Occasional Contributor III

Yes, I am successfully doing this.  Your HTTP Make Request parameters are a bit off.  They should look something like this:

AndrewBowne_0-1647948899904.png

 

Make sure you specify the "Field type" Parameters.  

LiamHarrington-Missin
Occasional Contributor II

Hi @AndrewBowne  and @rachelm,

Excuse my noob status but what you are attempting to do is something I'd be keen to recreate.  However, without a full blown @IsmaelChivite type walkthrough I'm a bit lost.  

Here is my attempt at doing what I think you are talking about using your screen grabs to guide me. The goal is to create a pdf report and then attach it to the source feature.  Very cool stuff.  I thought I was being clever by adding objectID to the URL to attach it to the correct "resource" but I think what I have done is broken it.  

If one of you has a moment can you cast an expert eye over my attempt and tell me what I am doing wrong.  Thank you.

Capture.PNG

 

AndrewBowne
Occasional Contributor III

You're close...  really close.... in your Make Request, add: "/addAttachment" at the end of your URL.  

If you wanted to kick it up a notch, you could use elements from the Watch Survey module's response to build your HTTP request.  Something like this:

AndrewBowne_0-1647969171175.png

 

rachelm
New Contributor III

@LiamHarrington-Missin  also I removed the properties under the Query String section and only worked with the Fields section below Body Type.

Thanks @AndrewBowne for solution! 

LiamHarrington-Missin
Occasional Contributor II

That did the Trick! 

Thank you @rachelm  and @AndrewBowne.  For reference this is the working version.

LiamHarringtonMissin_0-1648034155875.png

 

MikeOnzay
Occasional Contributor III

I'm sharing my workflow as this had me stuck for weeks. This blog and the subsequent comments weren't quite clear enough for me to understand what I needed to do. I wanted to attach two photos and send them via email. Each photo is in a separate image question in Survey123. While the beginning of the blog describes the workflow for one image, the next part about attaching multiple images, talks about using iterators and aggregators, which will NOT work with separate image questions. 

My current solution ***

email flow.JPG

http file parameters.JPG

Use Map for each attachment and add .jpg to the end of each name or you will get a "file.html" file.

add jpg extension.JPG 

***

This scenario only works if both questions are answered. What do I do if only one or the other question is answered? I've tried using the MAKE operations "resume" and "ignore" when only one of the questions are answered. I've thought about using a router with different paths when only the door hanger question is answered or only the violation question is answered, or both are answered. That doesn't scale well if I have multiple separate questions.

Changing the survey to only one image question with a multi-line appearance is not an option. I need to be able to do different things with each image. 

I could ask the user to keep a "no photo" image on the device and load it when there isn't an actual photo to ensure the scenario runs. 

How can I set up my scenario to work with separate image questions in MAKE?

Willie_C
New Contributor

Is it possible to attach both feature report and attachment(s) to the email?

WilliamRockwell1
New Contributor III

@Willie_C yes. You have to use array aggregators and iterators. In my example below, this attaches a Survey123 feature report and any attachments uploaded to the Survey. 

Special Events Scenario.PNG