Select to view content in your preferred language

Integrating Field Maps with Survey123 using globalid and joining in AGOL or ArcPro

1385
8
Jump to solution
10-03-2023 09:53 PM
BryceHancock
Frequent Contributor

Hello,

 

I have configured a Field Maps popup that opens a Survey123 form and populates the globalid from Field Maps to a field in the Survey123 form. Popup URL:

arcgis-survey123://?itemID=e4012bc59ca1407a8712a65862aa1454&field:incoming={globalid}

I found a post in the community that described the need to remove the curly brackets when passing the globalid to the Survey123 form in order to join the Field Maps feature data to the Survey123 form data on the backend (cannot find the post now).  I use the following calculation in the Survey123 form to remove the curly brackets and populate the field FEATURE_ID with the globalid, sans curly brackets. 

BryceHancock_0-1696394821838.png

if(regex(${incoming}, '^\{[\w\-]*\}$'), substr(${incoming},1, string-length(${incoming}) - 1), ${incoming})

On the backend I wish to join the Field Maps feature data using globalid with the Survey123 form data FEATURE_ID, but when I go to join the two fields, the Field Maps feature data globalid has curly brackets and FEATURE_ID does not, so the two fields will not join. The workaround I am using now is, I created a new field in the Field Maps feature data, named feature_id and run the field calculator replace function to remove the curly brackets from the globalid field and populate the feature_id field. I can then join the Survey123 FEATURE_ID to the Field Maps feature_Id. 

Obviously this is not ideal from an efficiency perspective as I have to remove the join and recalculate whenever there is new data collected. It would ideal to create a static join, so in the AGOL Webmap and/or ArcPro so when the map is opened the updated data is displayed. How would I accomplish this? Do you have to remove the curly brackets from both datasets in order to join the two? Is there another way to join the two datasets rather than using globalid?

Any help greatly appreciated!

0 Kudos
1 Solution

Accepted Solutions
DougBrowning
MVP Esteemed Contributor

My last post there https://community.esri.com/t5/arcgis-survey123-questions/mapping-with-survey123-within-a-polygon-or-... lays it all out.

Depending on the project I try to make some meaningful id.  We prepopulate our points and have pregenerated ids that are usually State_ProjectName_PlotName type format.

Others we may visit multiple times I year so there I use Arcade to add a date stamp to the PlotID in Field Maps and then send that ID over.  That one is tricker though since in the past since Arcade is not a real field.  But now with the FieldMaps form you can Calculate an ID in FM then send that over to 123 via a URL.

My formula is Concatenate($feature.PointID, "_", text(ToLocal($feature.FieldEvalDate), "Y-MM-DD"))

For past data you can add the new ID field and the calculate the key based on the join using globalid.  You would need to get the relationship class in there though and I am not sure how to do that without republishing.  So if you are live time this is all of course much harder and it gets complicated from there.

Hope that makes sense its a advanced workflow for sure.

 

View solution in original post

8 Replies
DougBrowning
MVP Esteemed Contributor

I would suggest not using globalid at but instead your own keys that are meaningful.  I list several reasons why in this post.  https://community.esri.com/t5/arcgis-collector-questions/related-tables-for-offline-data-collection/...

Biggest one is global ids change when moving data to SDE later, copying to a diff data source, having to fix issues that came up etc.  It is way easier to fix keys if you have some type of formula.

Another big one is the bug (that is still there going back to 2019!) where the 123 and Field Maps apps use lowercase globalid but Pro does upper case.  Then Pro will go and change them on you.  Total nightmare.

see https://community.esri.com/t5/arcgis-survey123-questions/globalid-in-lowercase-from-collector-to-sur... and https://community.esri.com/t5/arcgis-pro-questions/globalid-inconsistent-display/td-p/291405 

What I do is leave any built in repeats that use the globalid but have a 2nd relate that I manage.  Works well.  See here for more  https://community.esri.com/t5/arcgis-survey123-questions/mapping-with-survey123-within-a-polygon-or-...

Hope that is useful.

BryceHancock
Frequent Contributor

Hi Doug, 

Thank you for responding. What did you use for your new key to establish the relationship between Fieldmaps and Survey123? How do you deal with existing records where there is a join between fieldmaps and Survey123 on GlobalID? My fieldmaps feature datasets point, line, and poly, only have the GlobalID field in the attribute table. Would I add a datetime field and another to fieldmaps, concatenate them in the url to pass to the Survey123 form, or use some other field type in Fieldmaps to pass to Survey123 in the url? I am confused on the exact process and I cannot find any resources with specific examples. Thanks for any help you can provide.

0 Kudos
DougBrowning
MVP Esteemed Contributor

My last post there https://community.esri.com/t5/arcgis-survey123-questions/mapping-with-survey123-within-a-polygon-or-... lays it all out.

Depending on the project I try to make some meaningful id.  We prepopulate our points and have pregenerated ids that are usually State_ProjectName_PlotName type format.

Others we may visit multiple times I year so there I use Arcade to add a date stamp to the PlotID in Field Maps and then send that ID over.  That one is tricker though since in the past since Arcade is not a real field.  But now with the FieldMaps form you can Calculate an ID in FM then send that over to 123 via a URL.

My formula is Concatenate($feature.PointID, "_", text(ToLocal($feature.FieldEvalDate), "Y-MM-DD"))

For past data you can add the new ID field and the calculate the key based on the join using globalid.  You would need to get the relationship class in there though and I am not sure how to do that without republishing.  So if you are live time this is all of course much harder and it gets complicated from there.

Hope that makes sense its a advanced workflow for sure.

 

BryceHancock
Frequent Contributor

Hey Doug,

Thanks for your prompt response. This is very helpful. I like the idea of using Arcade to create the key and then passing it through the url. I currently have the FieldMaps spatial and Survey123 tabular joined on a FeatureID field that is in both FC's. The FeatureID field is the GlobalID field calculated to remove the curly brackets. I had to get creative because of the curly bracket formatting of the GlobalID from Fieldmaps. I could not join the two datasets with the brackets, so I ended up using a calculation in the Survey123 form to remove the brackets when they are received by the url, and then I have to run a field calculation in the Fieldmaps attribute table in Arcpro to populate the FeatureID with the GlobalID without brackets, and rejoin the data. It's ugly, the equivalent of duct tape and bubble gum to fix a workflow, and requires constant manual maintenance. So, excited to try this out and it should not cause any issues with the old data, not live now, so should be an easy fix! Thanks again for your help!

BryceHancock
Frequent Contributor

Hi Doug,

Where do I implement the arcade expression in Fieldmaps designer? I have a field named FEATURE_ID in my FC attribute table it contains the unique identifier that links the fieldmaps and survey123 data. In fieldmaps designer, I added the FEATURE_ID field to the form and created a new Arcade expression for the field:

return guid('digits')

I run the Arcade code in the code block and it works, but when I open my map in fieldmaps on my tablet and add a new feature, the form pops up but the FEATURE_ID does not populate. I thought maybe I needed click the check in the upper right to get it to populate and show the integration url, but the field did not populate. 

BryceHancock_0-1713816807150.png

I see that I can add an arcade expression under the 'Logic' tab for the editable, required, and visible options, but I just added a new expression outside those options. Do I need to add the expression to the required option in the logic tab, shown above? Just not certain where to implement it.

Also, wondering if there are any issues using the "guid" arcade expression in the FEATURE_ID field. Can there be a calculated guid field and globalid field in the same feature class, or will that cause problems with the data? I received no error when I added the guid to the existing feature class with the globalid, but maybe there is currently limited error handling for arcade expression in a feature class, so no warning.

0 Kudos
DougBrowning
MVP Esteemed Contributor

I would not think you would set read only or visible since calculated can not be edited.  As for being blank sorry I have no idea as that is a pretty simple looking expression.  It may not calculate until you save.

Are you back to precalcing the ID then sending to 123?  If so you can just send the GlobalID in the URL to 123 since it is a text field on that side.  At least it works as text not sure on a GUID type.  That is how 123 does it repeats parent globalid field copies to parentGlobaID field in the child.

Sorry you are losing me.  You just need to two sides to match in the end.

0 Kudos
BryceHancock
Frequent Contributor

Hi Doug,


I wrote the following Arcade expression to calculate a new unique FeatureID in the Fieldmaps designer. The FeatureID is the calculated date/time to the millisecond, using the following expression:

var currentDateTime = Now();
var day = Text(Day(currentDateTime));
var month = Text(Month(currentDateTime));
var year = Text(Year(currentDateTime));
var hour = Text(Hour(currentDateTime));
var minute = Text(Minute(currentDateTime));
var second = Text(Second(currentDateTime));
var millisecond = Text(Millisecond(currentDateTime));
var concatenatedDateTime = month + day + year + "_"+ hour + minute + second + millisecond;
IIf(IsEmpty($Feature.FEATURE_ID),concatenatedDateTime, $feature.FEATURE_ID)
 
The expression calculates the FEATURE_ID as the date/time to the millisecond, if the field in the feature service is empty.
 
I then use the url to pass the FEATURE_ID from fieldmaps to a FEATURE_ID field in Survey123, so I can join the two datsets on FEATURE_ID on the backend. Do you see any potential issues with this workflow?
 
The last piece I need is to create a callback on the Survey123 form to the origin Fieldmaps map, triggered once the Survey123 form is submitted.
 
Thanks
0 Kudos
DougBrowning
MVP Esteemed Contributor

One thing to watch out for is if they use Copy in Field Maps.  That can get ya with IsEmpty so test that.

0 Kudos