Survey123 and Speech Recognition integration

[Yuhash]

The idea to integrate speech recognition technology in field data collection has been proposed (see below), but is there anyone who can share or speak to their current workflow integrating Survey123 and Dragon NaturallySpeaking (or similar speech recognition) software? Has anyone customized their application to enable the transcription directly within Survey123 or is the voice recording transcription performed post-data collection?

Add Speech Capabilities such as Dragon Naturally Speaking to Control and Interact with Map Applicati... 

https://community.esri.com/ideas/13487 

[JamesTedrick]

Hi Yuriko,

On both iOS and Android, speech recognition is built-in to the keyboard and works to fill in values in a form.

[Yuhash]

Hi James,

This discussion branches off from this thread (https://community.esri.com/ideas/13487-speech-recognition-software-for-collecting-data-in-the-field) regarding the integration of voice commands/controls. Siri isn't yet integrated with Workflow but could Siri Shortcuts voice commands be employed to navigate the form controls? Dictation to text is nice but what is desired is the voice command controls functionality that NaturallySpeaking provides. I will mark this thread as answered and hope this comment directs folks to the ESRI Idea discussion.

[BillSimper]

I really like this idea, admittedly a big ask for S123 developers. Bird surveys are notorious for massive amounts of data that needs inputting in a ridiculously short span of time, ideally without taking your eyes off the action.

In an ideal world the app would work something like:

surveyor says "Collect" <form opens>

"species, red-tailed hawk" <species select_one question populates with 'Red-tailed hawk'>

"quantity, two" <quantity integer field populates with '2'>

"nest present, no" <nest present select_one toggles to 'no'>

"save to Outbox" <form saves to Outbox, chime sounds to notify user that task accomplished>

Of course, to be useful in most field conditions the voice recognition needs to work without having an active internet connection (possibly via user-programmable vocabulary, e.g. ""black-capped vireo" != "dance the rio").

[BryceHancock]

I have a Juniper CT7G running Android version 6 and the Gboard voice recognition feature does not work in Survey123. Also, Gboard is not capitalizing the first letter in a sentence in any of text fields. Any ideas on how to fix this? The voice recognition and capitalization work fine in all other apps.

[Anonymous User]

Hi Bryce,

The Gboard keyboard capitalize issue with the first letter of sentence is a known limitation of Gboard that we have documented for our 3.4 release as a known limitation. Unfortunately the issue is with the Gboard keyboard and not Survey123 app as the keyboard is not recognising the type of input field being used and applying the correct auto-capitalization. The issue has been logged with Google. For now, the only workaround would be to use another keyboard.

As for the Gboard voice recognition feature does not work, can you give more information, do you see the icon to enable voice recognition, what happens when selected? Also, have you tried other keyboards and do they work? It could be related to the same problem above that the Gboard keyboard does not recognise the type of input field being used and therefore not enabling the microphone.

Phil.

[BryceHancock]

Thanks for your prompt response Philip and sorry for my delayed one. The icon to enable voice recognition is not visible on the keyboard in Survey123 but it is visible in all other apps. Thank you.

[Anonymous User]

Hi Bryce,

Thanks for the extra info, can you confirm what version of Survey123 are you using, and have you tested with 3.4 beta available on EAC: https://earlyadopter.esri.com/project/version/default.html?cap=e69ef91f45744b98882c651f7b518eb7&artt...

Also, could you send some screenshots of the difference between Survey123 keyboard that pops up, and other apps?

Thanks,

Phil.

[BryceHancock]

[BryceHancock]

I am using Survey123 version 3.3.64. I will give version 3.4 a shot. The first photo shows MS Excel app with the keyboard microphone icon.The second photo shows Survey123 without the keyboard microphone icon, instead numbers show at the top of the keyboard.