Survey123 Tricks of the Trade: Face redaction with smart assistants
This post explores the use of a Smart Assistant to help you redact human faces from a photo taken with the Survey123 mobile app. Familiarity with XLSForm and Survey123 Connect is assumed.
Some context before we start
You likely take photos when using Survey123: to report incidents, inspect assets or document your work. Now, are these photos allowed to contain personally identifiable information? Some regulations such as CPRA, CCPA, VCDPA and GDPR qualify human faces as personally identifiable information (PII). The presence of PII in your photos limits their use and how you secure and manage them.
You can configure Survey123 image questions to help you redact faces from your photos. Face redaction will happen right on your device, even if offline and before any data is sent to your ArcGIS organization.
Before I go into the details, let’s be clear about of couple of important things:
- PII encompasses more than just faces. A license plate in a car, for example, can be considered PII. What I will describe next in this post will assist in the process of removing faces from a photo, but that is just one step in completely removing PII. Full PII redaction is the responsibility of the end user.
- As much as a smart assistant can do a good job, it will not be perfect. Nobody is perfect and less perfect are machines. Smart Assistants help, but do not replace your review and professional judgment.
If removing PII from photos is important to you, Survey123 Smart Assistants may be of help. Read on, learn, and experiment. Then discuss with your colleagues how well, or not, this technology is useful to your workflows.
Technically speaking, what I describe in this article uses face-detection routines included with iOS and Android devices. These routines leverage, depending on your operating system, pre-trained machine learning models from Apple and Google.
It is very likely that you are using these face-detection routines already: Did you ever wonder how the camera in your mobile phone is so good at auto-focusing on the faces of people? Turns out, your camera is using pre-trained face detection machine learning models to put everyone in focus.
According to Apple and Google, these face detection routines do not send your photos to their respective clouds. That is, face-detection happens on your device. In case you want to learn more, here is the relevant privacy statement from Google. For Apple, check their Device Analytics & Privacy and Data & Privacy statements.
Survey123 smart camera features
As described above, the face detection features used by Survey123 are powered by APIs from Apple and Google. By default, access to these API's is disabled, but you can enable it as follows:
- In the Survey123 mobile app, tap on your profile picture in the top-right corner
- Open the Settings page
- Tap on Privacy and Security
- Enable enhanced camera features
If you do not enable Smart camera features in your mobile app, face redaction as described in this article will not happen.
Configuring your Survey123 form to trigger face redaction
To trigger face redaction on an image question, add redaction=@faces to the bind::esri:parameters column of your image question.
Even better, try adding this instead: redaction=@faces&cameraPreview=true This will blur faces once the photo is taken and also while in the live camera preview.
The Add Smart Redaction to your Survey help topic describes a few more parameters you can use: You can control the redaction effect (pixelate, blur, blockout or a custom symbol), hide and show labels in the live camera preview, and the scale and color of the redaction boxes.
Understanding face redaction in the mobile app
Depending on how you configured your image question, automatic face redaction will occur after you take the photo, or also before you take it, while in the live camera preview. Either way, after you press the camera shutter button, your photo will be presented so you can review it. The review process is key.
While reviewing your photo, you can adjust the redacted areas: Add new, remove, or edit existing ones. This is your chance to completely eliminate all PII from your photo:
- Tap on the image to add a new redaction box
- Tap on a redaction box to move it or resize it
- Drag a redaction box into the trash can symbol at the top to remove it
As you can see, the smart assistant helps you get started with the PII redaction process. Smart Assistant will blur all the faces it can find in your photo. You can then manually redact other information as appropriate to your workflow.
Note: Once you exit the redaction dialog, your photo will not be editable from the Survey123 mobile app anymore. Redacted areas are not recoverable. That is, you cannot restore an area that has been redacted.
Troubleshooting
- The steps outlined in this article assume that you author your survey using Survey123 Connect version 3.18 or newer
- Your mobile app also needs version 3.18 or newer
- In addition to configuring your XLSForm as described above, you must enable Smart camera features in the Survey123 mobile app for face redaction to occur