Smart Sketching in Survey123: Stroke by stroke.

9423
57
10-09-2017 02:43 PM
Esri Frequent Contributor
8 57 9,423

In this blog post I will describe how you can configure smart sketching in your surveys, step by step.  Smart Sketching was introduced in version 2.4 of Survey123 and allows you to embed questions in your form where users can draw or sketch. Depending on how you configure your survey, you can let users sketch on top of an empty canvas, on top of predefined diagrams, and even on top of photos.

There are multiple scenarios where sketching can be handy:

  • When performing inspections, a user can take a photo of an asset and sketch on it to highlight issues. Alternatively, the inspector could annotate over a diagram representing the asset. 
  • When submitting a traffic accident report, a police officer can use the sketch capabilities in Survey123 to illustrate the positions of the vehicles involved and skid marks.
  • Free-form sketches in Survey123 allow conservationists to highlight certain features/marks of a whale which can be used later to track the individual specimen across different observation reports.

Sketches and annotated images and photos are automatically persisted by Survey123 as a geodatabase attachment, ensuring you can access them from other ArcGIS tools.

For the instructions below, some familiarity with XLSForms and Survey123 Connect for ArcGIS is assumed. A series of video-tutorials on XLSForms are available in our YouTube Survey123 playlist.

              

The Basics: Draw and Annotate appearances

In order to add sketching capabilities into your own survey, you need to use the draw or annotate appearances on survey questions of type image.  It is easier than it sounds: In your XLSFile, using Survey123 Connect, simply add a question of type image, and then set the value of its appearance column to either draw or annotate.

In Survey123 Connect, create a New Survey using the Advanced template and add a question as follows:

typenamelabelappearance
imageaccident_diagramVehicle Collision Diagramdraw

Now use the survey preview in Survey123 Connect to exercise the sketching widget.

The draw appearance enables a full screen drawing pad where you can use pencils of multiple colors and thickness for your sketches. The draw appearance is ideal when you want end users to draw an idea on top of a white canvas. Note that you can:

  • Add an arrow by drawing a straight line.
  • Add text by tapping on the screen.
  • Add shapes of any form through free-drawing. Circles and rectangles will be automatically smoothed for you.

The annotate appearance is similar to draw, except that it allows the user to set the background of the canvas with a photo.   Use the annotate appearance when you expect users to first take a photo of something, and then sketch on top of it.  The same sketching tools available in the draw appearance are also available for annotate.

Advanced Techniques: Using Defaults and Calculations

Rather than starting with an empty canvas, you can configure your survey to present users with a custom background image in the sketching pad.  Before you can reference your image, you must copy it into the media folder of your survey directory.   In Survey123 Connect for ArcGIS, while looking at the preview of your survey, click on the 'Open survey content folder' button.

Once the survey folder is open, look for the media directory and copy all your images in there.  To play safe, copy images in either .png or .jpg format only.  Be considerate with the size of the images: You want images big enough to look good in phones and tablets, but try to reduce their size as much as possible or otherwise the initial download of your survey into devices will take longer than usual.

Most tablets and phones will do pretty well with images smaller than 1920x1080 pixels.

Now that your images are in the media folder. You can reference them in the survey:

  • Default: If you always want to display the same background image in the sketching pad, reference your image using the default column. You do not need to enclose the name of your file in quotes, but you will want to make sure you include both the name and file extension. In the next example, the image coyote.png will always be used as background for the draw widget.

typenamelabeldefaultappearance
imageanimal_observationsHighlight  where animal was hurtcoyote.pngdraw

  • Calculation: If you want to dynamically switch the background image based on previous responses in your survey, use the calculation column.  A typical scenario for this is that where you present users with a choice question and then you switch the background according to user selection.  Say for example, you first let user select the type of tower inspection being inspected. The drawing canvas background will change depending on the type.

Lets do this step by step:

  1. Ensure you have all your images in the media folder.  For simplicity, make sure the extension for all the images is the same. That is, make sure they are either all .png or .jpg.
  2. Create a list in the choices worksheet where the value of the name column for choices in your list matches the name of filenames in the media folder.
  3. Set the expression in the calculation column of your draw or annotate question as:

concat(${ChoiceQuestion},'.png')

In the example above, I assume that your choice question is a select_one type of question named ChoiceQuestion. I also assume that your images are in .png format. Remember that all the concat() function does is to concatenate the value of the selected choice in your select_one question, with the .png extension.

You could certainly get more fancy with the calculation, but the heart of how this is done is described above.

It is always easier to learn with an actual example. The Transmission Tower example is available under the Community category in Survey123 Connect:

  • Log into Survey123 Connect
  • Click on New Survey
  • Select the Community category and then choose the Transmission Tower sample

A few extra tips and things you should know

  • All sketches will be saved as geodatabase attachments once submitted to ArcGIS. This means that you can always easily access sketches associated with a submitted feature.
  • If you use the default or calculation columns to predefine the background of the drawing canvas, keep in mind that iOS (iPhones and iPads). You must reference images making sure that the capitalization in your XLSForm is exactly as in the filenames.
  • Once the user creates the sketch, a preview of the sketch will be made available in the survey. Note that at that point the user can change the name of the image.  Calculating the name of the image filename is not possible at this time.
  • Support for defaults and calculations on draw and annotate images is missing in the WebForm. Only the Survey123 field app will honor your defaults and calculations.
  • [Fixed as of 2.6] As of version 2.4, it  is not possible to let users choose a background for the drawing canvas using an existing image from the photo gallery. This is something that we want to add in the future. As of 2.4, you can only use images from the media folder or let users take a photo on the spot and use it as background.
  • If you are using the Inbox to edit existing features, be aware that as of 2.4 the Inbox does not support images of any type (signatures, photos, drawings or annotated images).
  • If you bring a photo from your gallery or camera into the Draw and Annotate widgets, the EXIF data in your image will be stripped out and lost once submitted  to ArcGIS.

As usual, we are eager to hear your feedback. This is a new feature and would love to hear your feedback so we can take next steps with it.

Enjoy!

57 Comments
Occasional Contributor II

The new draw and annotate options are great. Really useful for inspection workflows.

I'm wondering if there is (or will be) functionality that allows you modify a part of the sketch? Say you sketch three shapes and then want to change or delete the first of these shapes, can you select it and then modify/delete?

Cheers,

Finnian

Esri Frequent Contributor

Hi Finnian,

Currently we do not support modifying the sketch once the image has been saved as jpeg, and we do not support being able to edit or remove shapes/text not in order, ie currently you can only remove each shape/text in order of last created.

I have added your request as a new feature enhancement.

Regards,

Phil.

Occasional Contributor II

Thanks for the information Philip. Hopefully the enhancement will make it's way in at some point.

Cheers,

Finnian

New Contributor II

The new draw and annotate features are very helpful for us, as well as being able to zoom when using the camera to capture an image in a survey on an iPad!

I am currently using the annotate appearance on an image question.  The user will always take a picture with the iPad camera and then have the ability to draw and comment on that captured image.  We do not want a default background image (which is currently all white).  Is there a way to default the image question to activate the camera automatically, rather than having to tap on the camera icon to access the camera? 

Esri Esteemed Contributor

Hi Stacey,

Thanks for the suggestion!  I've logged it and consider it as a possible enhancement for Survey123.

Occasional Contributor

I am trying to duplicate this procedure for one of my forms but I do not have the draw or annotate option in my appearance drop down.  Is there something else I need to do?  Thanks

Occasional Contributor

I actually figured it out, the survey I was using did not have the advanced capabilities! That said I created a grid image that I wanted to use to annotate. It works fine in 123 on my Windows Surface Book, but when I download it to an iPad Mini the graphic does not show up at all.  The size was less than 1920 x 1080 so I am not sure as to why it is not showing up.  Thanks!

Occasional Contributor

I've created a survey in which the user can write/draw on a generic background image. I am running into a problem where the Web form cannot find the background image (File "sketch_background.jpg" could not be found...). I have tried using IE, Firefox, Chrome and Safari on a desktop and mobile; it works with the desktop and mobile apps but not the Web form. It is saved in the media folder in the survey's "My Survey Designs" folder. I thought I screwed up the PNG creation, so I tried it using a JPEG with the same result. 

Has any else run into this problem?

Any help you can provide is greatly appreciated,

Chris Anderson

Florida Fish and Wildlife Conservation Commission

PS - Link to the survey: https://survey123.arcgis.com/share/fa76bc2b245b483196185dd78930a3d0

New Contributor

Hi guys

i´ve problem to download the example archive, the page say that the archive dont exist

Thanks for your help

New Contributor II

Is it (or will it be) possible to annotate/draw on a preset image using the webform?  Looks like there is a statement above stating that defaults are not available in webforms for draw/annotation type images, but is there a way to edit the html to enable this functionality?   Seems like I can get the image to appear in-line above the drawing window on the form, but not as a background.  Maybe someone with more survey123 or html experience knows a way around this.  Thanks.

Occasional Contributor

Nathan,

You can set a default image to a sketch. I use one in my survey. You will need to put it in the media directory of the survey you are creating. Then, in the XLS Form, you reference the image in the default column of the sketch question. When the user clicks on the pen/pencil in the survey, they will be able to draw on you background image.

My background image is a color PNG and is 577px X 721 px.

Chris Anderson

Florida Fish and Wildlife Conservation Commission

New Contributor II

Thanks for the reply Chris, is this working for you when filling out the form from a webpage?  Would you mind posting a link to the form?

Occasional Contributor

Not in the Web form. The sketch question is only available using Survey123 Connect.

by
Occasional Contributor

Are there any plans to support images when using the Inbox to edit existing features?

  • If you are using the Inbox to edit existing features, be aware that as of 2.4 the Inbox does not support images of any type (signatures, photos, drawings or annotated images).
Esri Frequent Contributor

Hi Jim,

We do not currently support editing of images via the Inbox (or any attachments for that matter). This is in our backlog and I have added your request to the issue, hopefully it is something we will aim to support in the future.

Phil.

New Contributor III

I tried implementing this function with a default image outside of a repeat. If the appearance is set to 'annotate' or 'draw' I am getting the name of the image with extension in place of the draw or annotate button. It has to be deleted in order to sketch on the background image. If a default image with extension is placed inside of a repeat, it works as intended. Is this a bug or does the sketch functionality always need to be inside a repeat?

With repeat:

Without repeat:

Thanks!

Esri Esteemed Contributor

Hi Kade,

Which version of Survey123 are you using?  This appears to be an issue that was addressed in the 3.1 release.

New Contributor

This is a great function, but is it possible to disable the text box that automatically pops up with every shape or line that is drawn? My survey only requires sketched lines and circles, so the keyboard that pops up on a tablet or phone after each shape is drawn just covers up part the image (a basic floor map of a building) and disrupts the process of sketching paths and circling regions. Is this a default option that can be changed?

Esri Esteemed Contributor

Hi David,

Yes, you can disable the automatic text box by clicking on draw settings (the second from right icon that has an 'A' and head) and turn off the text.

New Contributor II

Hi Phil, any updates on a time frame for support for editing the images (either while the user is in the survey, to add to the sketch, or editing existing surveys from the inbox?) Thanks! Sue

Occasional Contributor III

Does the default only work within a repeat? When I remove the repeat from the Transmission Tower Inspection example the Smart Sketch function goes away and only a text appears.

Esri Frequent Contributor

Hi Carl,

No the default should work both outside and inside repeats? Can you provide your survey xlsx file so we can take a closer look and the default image from media folder you are using?

Phil.

Occasional Contributor III
typenamelabelhintconstraintconstraint_messagerequiredrequired_messageappearancedefaultreadonlyrelevantcalculationchoice_filterrepeat_countlabel::language1hint::language1media::audiomedia::imagebody::accuracyThresholdbind::esri:fieldTypebind::esri:fieldLengthbind::esri:fieldAliasbody::esri:inputMaskbind::esri:parametersbind::saveIncompletebind::typebind::esri:warningbind::esri:warning_message
begin grouplocation_infoLocation Information                          
dateTimeinsp_dateInspection Datenow()
textinspectorInspectorproperty('email')
textaddressAddress
geopointGPSGPS
select_one install_typeinstall_type<b>Type of Installation</b>horizontal-compact
end group                            
begin groupwater_meterWater Meter                          
begin repeatdiagram
imagewater_meter_diagramDiagramPlease fill out measurements drawwatermeterdiagram.png
end repeat
end group
Esri Frequent Contributor

Hi Carl,

What version of Survey123 Connect and App are you using? I am unable to reproduce any error with annotate appearance on an image question, either inside or outside of a repeat, and using the .png you provided.

I have tested with release 3.3 version and 3.4 beta and found no issues. I tested the Transmission Tower survey from the Gallery as well as created a simple example survey. Do you have any other calculations, defaults or other settings applied to the image question, as I can not see form the screenshot of your xlsx file, if possible share the actual xlsx file.

Regards,

Phil.

Occasional Contributor III

Great function. Just wondering if it is possible to be able to take a photo and then use that photo within the same survey to annotate?

Regards,

Craig

Esri Frequent Contributor

Hi Craig,

Yes it is possible. You will see in the above animation I shared that once the canvas is open there is a camera button and browse button along the bottom that allow either taking a new photo or selecting an existing one from gallery, to insert onto canvas and annotate on top of.

Phil.

Occasional Contributor III

After updating to latest version of Connect, annotation does work.

New Contributor

Hi Philip,

I´m trying to implement dynamically switching the background. As far as I understood the explanation above it depends on only one value. But what in case you want to have the ground plan of the 3rd floor of a specific of many buildings?

I did a cascaded select so far. First you select the building, then you select the floor, but then I´m stuck.

What I was thinking is, that I combine the two fields, like all buildings around here have numbers as well as the floors. So I get for example building 130 floor 3 -> 1303 and then concatenate this value to an image. 

But that´s the point where I´m stuck. Hope you can help.

Kind regards

Sebastian

Esri Esteemed Contributor

Hi Sebastian,

The approach you describe should work.  I've developed a sample that shows this, but can't attach it to a comment in a blog post.

New Contributor

Hi James,

it works. I set a hidden field where I concatenated building and floor and then named the picture like described above. Thanks for your reply.

New Contributor

Calculating the name of the image filename is not possible at this time

Is there any update or suggested work around for renaming an annotated image automatically (eg. annotationfile_date.jpg) rather than asking the user to do so?

Thanks,

Joy

Esri Frequent Contributor

Hi Joy,

Sorry, there is no update on this issue, the image filename can not be calculated at this point in time, it is still in our development backlog. For now the user must manually change it.

Phil.

Occasional Contributor III

Hi,

Is this functionality only supported in the app? I tried in a browser and my default image doesn't load to allow for drawing on. Wishing to use this as a citizen science data gathering tool.

Craig

New Contributor II

Hey! 

As of right now it’s only available in the app. It’s super frustrating and difficult for those not tech savvy to try and complete the surveys. I’ve been trying for a year with little luck and desperately need this function to work on the web browser.

Good Luck!

Yvette

Esri Esteemed Contributor

Hi Craig,

Thanks for noting this.  I've logged an issue to investigate default support for annotate on the web.

Occasional Contributor III

Thank you James and Yvette.

Occasional Contributor III

Hi James Tedrick‌,

Some of our clients would love to have this option too.

Thanks,

Gee

New Contributor II

Hi,

is there a way to adapt the canvas size to the picture without lose resolution? or is there a way to take annotations directly on photos taken with camera without losing size and resolution?

thanks!

New Contributor

Hi! 
Is there any way not to smooth circles or rectangles and to let the areas just as they were drawn? 

Thanks! 

Esri Esteemed Contributor

Hello,

Yes, to disable smart sketching click on the 'head' icon in the palette (magenta square highlight in the image):

New Contributor II

Hello,

I have had my survey up for almost a year and the past few months, the default image comes up extremely blurry  on smartphones in the app. Why did this change? It was completely clear from Nov. 2018-approximately Jun 2019. When accessed on a tablet or computer it isn't blurry; just the phones. I am using the newest version of the app (waiting for this annotation/default image feature to be available on the web app).

It'd be great to hear if anyone has found a solution for this! 

Thanks!

Esri Frequent Contributor

Hi Yvette,

What make and model phones are you using? And what OS version do they have installed? Does it only occur on iOS or Android or both?

Did the blurry images start occurring after a specific version update, or after a phone OS update?

Regards,

Phil.

New Contributor II

Good morning!
It is blurry on both Android and iOS. The first results with the blurry image showing up was submitted July 5, 2019.

I'm guessing it's from a specific version update.

Best,

Yvette

New Contributor

I am having the same issue as Yvette with blurry/low res images in sketch, but for me it is occurring on all devices - including windows desktop and iPad.  The issue started in July for me as well.

Esri Frequent Contributor

Hi Yvette,

Thanks for the additional information, we will look into this issue. It would appear it is related to the 3.5 release which was available from around the 27th June.

Can you please provide a copy of your xlsx file and the default image from media folder you are using for the annotate appearance? If there are any other files in media folder also share those. This will help greatly with further investigation.

Do you see the same blurry/low res images when using the capture new photo option, or select existing image from gallery, or is it only with the default in the background?

Regards,

Phil.

PS - Joy, if you could also share your xlsx file and default images (or other files in media folder) that would be great. 

New Contributor

Hi Phil,

Thank you for looking into this issue. Newly captured images or files added from the device do not appear blurry, just the pre-loaded background images.  There are multiple background images and all are affected in the same way, across multiple surveys.  

The xlsx file for one of the surveys can be accessed here.  Below is one of the pngs in the media folder, vs a screen shot of what shows up in the app and attachments. 

Joy

original image in media folderimage in survey123 app

New Contributor II

Good morning,

Thank you for looking into this. You can view the xlsx file here. The pngs are below.

(Image png from media folder)

Screenshot from Iphone.

Thank you for your help.

Best,

Yvette

Esri Frequent Contributor

Thanks Yvette and Joy for sharing your png files and xlsx file, we are looking into this issue and will get back to you soon.

Phil.

Esri Regular Contributor

Hi Joy,

We are able to repro the same here and will look into this further. In the meantime, could you contact Tech Support and log a salesforce issue for this so that we can address the fix?

Esri Regular Contributor

Hi Yvette,

 

We are able to repro the same here and will look into this further. In the meantime, could you contact Tech Support and log a salesforce issue for this so that we can address the fix?