Include additional data in Integromat Webhook Payload Schema

492
5
Jump to solution
03-29-2021 11:27 AM
AndrewBowne
Occasional Contributor III

@IsmaelChivite  @JamesTedrick @Philip-Wilson 

Is it possible to send additional payload information from Survey123 to Integromat that is NOT in your feature service schema?  For instance, I want to include some additional information in my survey to be used as filter criteria for a module however I do not want to store this information as part of my feature service.  

  1. I have added a couple text questions in my survey to collect some data. They are NOT part of my feature service schema.  I have set their bind::esri:fieldType to "null". 
  2. The data appears to be populated and sent as part of the JSON.  I verified it with using Survey123Connect and viewing the result output.
  3. On the Integromat side, I do not see these additional text elements being included.  The only attributes I see in the payload are the ones that match the feature service.

Thanks for any insight! 

0 Kudos
1 Solution

Accepted Solutions
IsmaelChivite
Esri Notable Contributor

Hi. This is an interesting question. The software right now drops your null questions from the payload as you have observed. This is expected behavior and you can't do much about it.  Now: should these values be dropped? Why? I wonder what other people's thoughts are on this.

At first sight, it looks like a good idea to include the null values but I wonder if there are any drawbacks people can think of.

In the meantime, to get you going, I suggest you try the following:

  • Remove the null field types
  • Publish and have Connect create these new fields in your layer
  • Manually remove the fields created in the previous step
  • Download survey and submit

Now that your fields are not null, you will see them in the payload. Data from these fields will be sent by Survey123 to your layer but since there are no fields to stored that information, the feature layer will ignore that data.  A bit of  a hack, but I think it will work.

View solution in original post

0 Kudos
5 Replies
IsmaelChivite
Esri Notable Contributor

Hi. This is an interesting question. The software right now drops your null questions from the payload as you have observed. This is expected behavior and you can't do much about it.  Now: should these values be dropped? Why? I wonder what other people's thoughts are on this.

At first sight, it looks like a good idea to include the null values but I wonder if there are any drawbacks people can think of.

In the meantime, to get you going, I suggest you try the following:

  • Remove the null field types
  • Publish and have Connect create these new fields in your layer
  • Manually remove the fields created in the previous step
  • Download survey and submit

Now that your fields are not null, you will see them in the payload. Data from these fields will be sent by Survey123 to your layer but since there are no fields to stored that information, the feature layer will ignore that data.  A bit of  a hack, but I think it will work.

0 Kudos
AndrewBowne
Occasional Contributor III

@IsmaelChivite - Thanks for the reply.   I submitted an Idea for this here: https://community.esri.com/t5/arcgis-survey123-ideas/include-null-field-types-in-webhook-payload/idi...

My suggestion would be to put all null datatypes into their own section of Event Data.  That way users can find the specific information being passed as null datatype and also have the option to toggle it off if they don't want it as part of the payload.

The only drawback I can foresee is increasing the payload file size however it would be negligible in my opinion.  

As for the workaround - I'm considering trying this.  I am wondering if there will be any issues when republishing the service after the fields have been removed.  Or, republishing the survey once the fields have been removed from the feature service.  I'm guessing that if the fields are not part of the service and I try to publish the survey again, it will error out.

IsmaelChivite
Esri Notable Contributor

If your feature service does not have the fields, Connect will ask for your permission to add them. Let Connect create them and after that, you remove them.  Adding fields into the layer will not cause data loss. When you publish, make sure you read all the warnings.

Also, as a reminder, Survey123 Connect as a BUG that causes it to hang if you add a field into a repeat and republish. This should be corrected in a future release but in the meantime, if you run into this, make sure you add a fake field outside the repeat to workaround the problem.   Al these fields Connect creates you can delete later.  Thanks for posting the idea. Lets see if it gets traction! :)

GFernando
New Contributor III

Thanks @IsmaelChivite, for this suggestion. This may help our team with some data sovereignty issues. As we'll be able to email any personal info that we'd like to collect (rather than storing it in a feature layer)

0 Kudos
GFernando
New Contributor III

Also, I can't think of any drawbacks of including null types in the payload.

0 Kudos