Survey123 assistants (Beta)

[MarikaVertzonis]

The following information is out of date.
Please refer to the latest information on assistants at Survey123 assistant (preview) and Survey123 analysis tools (beta)

AI assistants can be used in Survey123 to do the following:

• use a conversation to help you design a survey in the web designer
• extract information from images, audio, and text, in the web app
• translate survey designs in the web designer or in Connect

Survey123 assistants leverage large language model (LLM) and machine learning technologies. Specifically, Microsoft Azure OpenAI service is used for the Survey123 assistant and Microsoft Azure AI Translator Service for the translation assistant. Survey123 assistants follow the data security and privacy guidelines of ArcGIS Online.

These capabilities are disabled by default and only enabled when an organization administrator has opted in. To enable these features, organization administrators need to:

1.  Navigate to the AI assistants section on the ArcGIS Online organization settings page and turn on the Allow use of AI assistants by members of your organization option.
2. Go to the Survey123 website, open the Organization -> Settings tab, and under the Extensions section turn on the Survey123 assistant and Translation assistant.

Whilst these assistants are in beta, please join our early adopter community to learn more and provide us with feedback. Before clicking the following links, be sure to register and sign into the Early Adopter Community.

• Survey123 assistant (web designer) to design a survey, to process images captured and extract recognizable information, or for audio transcription and text analysis
• Translation assistant (web designer)
• Generate translations (Survey123 Connect)

Similar but different, are smart assistants in the field app. Configured for image questions, smart assistants can perform image classification, object detection, and object redaction. These capabilities rely on machine learning models that are located on the device and do not use online generative AI services. For more information see Smart assistants.

[LouisaM]

I have enabled both the Survey 123 assistant and Translation assistant, but I still cannot access the Survey Assistant feature in the Survey 123 beta, 

[LouisaM]

@MarikaVertzonis I have enabled both the Survey 123 assistant and Translation assistant, but I still cannot access the Survey Assistant feature in the Survey 123 beta, can you kindly assist?

[PierreloupDucroix]

Hello, currently the photo extractor AI assistant is returning HTTP 500: UNKNOWN_ERROR when trying to process an image, any idea ?

[ZhifangWang]

@LouisaM ,

Can you please confirm that your Survey123 organization settings are as below? If yes, can you share a screenshot of the Design tab on the website?

[ZhifangWang]

@PierreloupDucroix ,

Does the same error occur consistently? If you see it again, could you submit feedback using the thumbs up/down icon so we can investigate?

[PierreloupDucroix]

@ZhifangWangit occurs every time and I tested multiple photos.

And yes I enabled AI settings

[PierreloupDucroix]

I just sent a thumb feedback

[PierreloupDucroix]

Hello @ZhifangWang , FYI it is now working, I needed to enable AI assistant also in my AGOL organization parameters

Is this functionality expected to work in the Survey123 mobile app ?

[LouisaM]

@ZhifangWang it worked thank you

[Tiff]

Hello, is the survey assistant that helps  process images captured and extract recognizable information, available also in Survey123 Connect? I added a deep learning model to my survey built in Survey123 Connect and I can only extract EXIF metadata and attributes. I would like it to populate an intuitive image description for me like web designer can.

[ZhifangWang]

@Tiff ,

The "extract information from image" capability mentioned in your referenced EAC post currently works only in the Survey123 web app. However, it can be authored in both Survey123 Designer and Survey123 Connect.

To generate an intuitive description of an image, you could use an expression such as: pulldata("@ai","image2text",${<imageQuestionName>},"Describe this image in 20 words.") in the calculation column of a text question. Since this relies on a large language model, no deep learning model is required in this case.

Is this what you were looking for?

[Tiff]

@ZhifangWang Hello! Thank you for your response. Your explanation and expression look to be exactly what we are looking for. However, my calculation doesn't seem to work. Is there a setting or parameter I need to specify? 

I used the exact string you had and replaced the image question name:
pulldata("@ai","image2text",${photo},"Describe this image in 20 words.")

It does not seem to be populating in the Connect app nor in the web app (survey in browser).

[ZhifangWang]

@Tiff ,

As mentioned previously, this capability currently only works in the Survey123 web app in browser and does not work in the Survey123 field app (mobile app).

To use this image analysis capability in the Survey123 web app, you need to ensure the following are satisfied:

• As an organization administrator: enable the "AI assistants" setting in Organization->Settings tab in ArcGIS Online.
• As an organization administrator: enable the "Analysis tools" setting in Organization->Extensions tab on the Survey123 website.
• Use the 3.23 version web app. This can be done by:
  
  • Re-publish your survey in 3.23 version Survery123 Connect.
  • append `?version=latest` parameter to the Survey URL in the browser.

Please let us know if this could work for you.

[Tiff]

Hi @ZhifangWang, I understand now! I was looking at the two image capabilities in Survey123 Connect, and although pulling from metadata is different from AI Assistant, I get that doing image analysis/description capabilities in Connect still require the AI assistants toggled on. I do not have it toggled on yet, but am looking more into this and will let you know how it goes!

I am looking more into the AI Assistant feature for all products. Right now, the documentation states that none of the assistants use credits (but documentation assistant may refer to a workflow that does). Do you know if this will stay the same? For example, if out of beta, they will also not use credits? I am concerned if we turn on all of these tools for the organization and eventually it uses credits, it will come at a surprise to admin and users. Thank you!