Object detection deep learning model in Survey123

[AnushaGTI]

Hi,

We developed a Survery123 smart form for field data collection and planning to train a deep learning model to identify features in an image using the Survey123 smart camera functionality. 

The article referenced here (https://developers.arcgis.com/python/samples/train-a-tensorflow-lite-model-for-identifying-plant-spe...) explains training a single-label classification model.  Our use case requires to build an object detection/multi-label classifier to be integrated with Survey123 application.

Does Survey123 support object detection and multi-classification models as well?

Thanks in advance,

Anusha

[IsmaelChivite]

@AnushaGTI    Please check our Survey123 Early Adopter Documentation.  You can use an object detection TensorFlow Lite models with the Beta build of Survey123.  This is scheduled for General Release in June 2023. Below is a screenshot from our Early Adopter doc:

Note: At the moment, we do not provide ArcGIS tools to train object detection models for Survey123, but you can do it outside of ArcGIS.   There is on-going work directed to help you train your own object detection model using ArcGIS tools (tentatively June 2023). If you want to start your work right away, please post questions in the Early Adopter Forum.

[AnushaGTI]

@IsmaelChivite Thanks  for your reply. 

Based on your comment on training the models outside of ArcGIS, what kind of libraries/packages are supported for integrating back into Survey123? (Tensorflow Lite, PyTorch, etc). Also, do the model files need to be in a specific format to be integrated back with Survey123 for inferencing (.dlpk vs .tflite/.pt)?

Thank you,

Anusha

[BoruiZhu]

Hi @AnushaGTI 

Survey123 Smart assistants is now available with Survey123 June 2023 release (v3.18), and you can find related documentation here: ArcGIS Survey123 Smart assistants reference.

TensorFlow Lite model (.tflite) is the only model format that can be consumed by Survey123, you can find more information in the Models section.

Thank you,

Borui