Skip navigation
All Places > ArcGIS Survey123 > Blog > 2017 > November

IMPORTANT NOTE: Content in this blog post has been revised in the A simple e-mail notification system for Survey123 for ArcGIS v2 blog post.  The new blog as an improved version of the content and Python script.


I constantly get requests from people who want to configure an e-mail notification system on top of Survey123 for ArcGIS.  The idea is simple: notify someone via e-mail every time a new survey is submitted.


There are different approaches to do this. I have seen some organizations configure ArcGIS GeoEvent Server in ArcGIS Enterprise for this purpose. It works pretty well and provides a great deal of flexibility, but not everyone has the luxury of having GeoEvent at hand. I have seen clever database triggers been written and also FME Server workflows for automating e-mail notifications.


In this blog, I will describe a technique for doing this with a Python script and Windows Task Scheduler. Essentially, Windows Task Scheduler will run a Python script at a regular interval that you can define. The Python script will connect to your survey's feature service, look for any added records, and then connect to your own corporate e-mail server to send notifications accordingly.


Many thanks to Mark Smith from Esri's Applications Prototype Lab who patiently worked on this Python script.


While the Python script may not format the e-mail message exactly as you need, I hope that at least will give you a starting point from which you can write the exact logic that  matches your needs.  Below are step by step instructions so you can configure this Python script with your own survey.


Configuring your ArcGIS Online feature layer


For the script to work you need to enable an option in your survey's feature layer called 'Keep track of created and updated features' .  This is not to be confused  with Editor Tracking.  This option helps the Python script determine if any changes have been made in your feature service.


  1. Login into ArcGIS Online and open the Details page of your survey's feature layer.
  2. Switch to the Settings tab and make sure the 'Keep track of created and updated features' option is checked as shown in the screenshot below.



Configuring the Python script


Next we need to configure the Python script. Since you will want this script running unattended every at a regular interval, I suggest you choose a computer which will always running and connected to the internet.  Your laptop, which you may disconnect from time to time or take home is not a good choice. Choose either a desktop computer which is always on or even better a server managed by your IT department.


The script requires Python 3.x to run and the 'requests' Python module installed.  If the computer already has ArcGIS Pro installed, you already have Python 3.x and the 'requests' module installed. However, ArcGIS Pro is not required to run this script so you can install Python 3.x and then the 'requests' module.



Now you can download and configure the script as follows:


  1. Download the Detect Changes and Notify Python script and its associated init.json file and save them together within a folder in your computer.
  2. Edit and save the init.json file. At the very least, you will need to change the properties highlighted  in blue below. 
    • recipients: This is the e-mail that will be receiving notifications. You can create an email alias if you want several people to receive the email.
    • from: This is the e-mail of the person/organization that is sending the e-mail. You can also set it to a DoNotReply email address.
    • server: This setting refers to your email server connection properties which are defined as a comma separated list of strings. 
      • The first parameter is the host of your email server which can be specified by hostname or IP address. It will look something like for example or
      • The second parameter is optional and defines the port where the SMTP server is listening. Common ports are 25 and 587 but it could really be anything depending on how your mail server is configured.
      • The third and fourth parameters are also optional and are used to set a user and password to access your email server.
To properly get the email  server connection properties, you will really want to contact your IT department and describe what you are trying to achieve as many email servers are configured with strict security policies that will prevent the script from successfully connecting and using your corporate e-mail server.  Folks in your IT department should know how to give you the right hostname and port for  your email server. Handing over the source code of the Python script may be of help too.

If you want to configure this script using your Gmail,  You will need to configure your Gmail account with 2-step verification, and then setup a Gmail App password.  Once you have done that, use your complete Gmail e-mail address as the user (third parameter) and the App password for the fourth parameter.  When using Gmail you do not need to specify a port number.
    • fsURL: This is the URL of your survey's feature service. 
    • serviceuser and servicepw: The credentials of an ArcGIS Online account with access to the feature service.



          "recipients": [""],
          "from": "",
          "subject": "Changes detected in your service",
          "text": "You are receiving this e-mail because records have been added, updated or deleted in:\n",
          "server": ["", "", "",""]

          "fsURL": "",
          "serviceuser": "username",
          "servicepw": "password",
          "fsLayerNum": 0,
          "viewerURL": "",

          "layergens": "layergens_ms.json"


After saving changes to the init.json file, it is time to give the script  a quick test:


  1. Run the script once so it can capture the current state of your feature service.
  2. Submit one record to your feature service and run the script again.


If any errors occur during execution, an error text file will be created in the same folder where you saved the Python script.


Scheduling the script


The Windows operating system includes a simple utility called Task Scheduler. It is quite easy to setup. Once you have determined  when the script will be triggered for the first  time, you can repeat its  execution at regular intervals. For example, every 5 minutes. Obviously, the computer where  you setup the task will need to be running all the time, although you can configure the task to run regardless of who is logged in.


The configuration of tasks in the scheduler is pretty much self-explanatory, but here are some specific instructions that can save you some back and forth:


  1. General: Check the option to run with highest privileges and set the task to run even if you are not logged-in.
  2. Trigger: If you want to quickly test your task, you can simply select your task in the gallery and then hit Run in the Selected Item panel on the right.  When configuring the task for real, I suggest you select the startup trigger and that you also configure the task to run indefinitely every five or ten minutes or so.
  3. Actions: You will need to be particularly careful with this one. The Program/Script setting needs to point to the Python executable (Python.exe).
    • If using your own copy of Python, refer to the installation directory of Python where you will find Python.exe
    • If using Python from ArcGIS Pro, it will be under the Pro installation directory. For example: C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe"

You also need to indicate the location of your Python script as an argument. Do not forget to include the .py extension in the path. If the path to your file includes spaces, then you need to enclose the path with quotes. Lastly, set the 'Start in' property so it points to the  directory in which you are storing the file.



Beyond the basics


The technique described in this blog post is a bit rudimentary, but it  may do its job reasonably in some simple scenarios. You can manipulate the Python script to properly format the email message to be sent, and also to apply some logic to determine if a n email needs to be sent, and to who.  Since you are in 'Python land' already, you can get creative and import the arcpy.mapping module to do all sort of sophisticated things in the script. You can for example take the incoming feature added into your feature service and do a straight (buffer) or network distance (closest facility) search to determine who will be notified.


We are working on some more flexible ways to make notifications via e-mail, SMS, voice call etc simpler with Survey123. We plan on adding some specifics about this before the end of the year in our Early Adopter Program. In the meantime, you may want to play with the approach outlined here as it may work just well for you.

Sometimes the most difficult part of writing a blog is coming up with a good title. I am not sure I quite got the right one this time, but since I have your attention I will describe now in more length a common problem that I see people having in Survey123, and some approaches to it.


The 'Ticket Number' problem:  The use of Survey123 for ArcGIS to submit incidents, requests or workorders is quite common. In these cases, it is very likely that you will want to uniquely identify every survey submitted, so it can then be tracked throughout its complete life cycle: from the moment the survey is submitted, to when it is reviewed, assigned, rejected and/or completed. I will generically refer to this as the 'Ticket Number' problem.  Say for example you create a survey to help handle the whole life cycle of a 311 call, from the moment you receive a call from a citizen, to the moment it is, validated, prioritized, assigned, acted upon and closed.  To follow along the 311 request, you need to have a unique 'Ticket Number' so all parties involved can find the right information about the event. Ideally, this Ticket Number should be somewhat user friendly.


There is not an obvious way in Survey123 to create such 'Ticket Numbers', at least not at first sight.  Lets look at some options.


Auto increment fields (and ObjectIDs): In common database jargon, some people solve the problem with what is known as Auto Increment fields. Through an Auto Increment field, a database is able to assign a new and unique identifier (typically a number) to every new record added into a table.  This works well, because it does not matter from where you add a new record: when it gets to the database it will be assigned a new identifier... a new 'Ticket Number'.


In ArcGIS, every record submitted to a feature service automatically gets a 'Ticket Number' through an attribute called ObjectID.  Every record submitted to ArcGIS, from Survey123 or any other app, will have its own unique ObjectID. Now, there are a handful of consideratons to keep in mind with ObjectIDs:  


  • ObjetIDs are managed automatically by ArcGIS and as such, you can never change them. 
  • ObjectIDs only get created when the record has been submitted to the feature service.  This is a problem when the life cycle of the event starts even before there is a chance to add a record into the feature service. Think for example that a city employee wants to use Survey123 to record an infraction of the city's code.   The person receiving the ticket must know what the Ticket Number is even if it has not yet been officially recorded (submitted) into the City's database (feature service).  If the city employee happens to be working on a remote area with no connectivity, there will be no chances of waiting for ArcGIS to give back a Ticket Number... 
  • ObjectID fields are created new every time you create a new table, so there is really no way you can reuse old ObjectIds from a back-up table, or when you append records from one table to another. 


The ObjectID or Auto Incrementing field approach give the most user friendly Ticket Number, but the necessity to be connected and the fact that it is completely managed by the database rule this option out in some scenarios.


UUIDs: A Universally Unique Identifier (UUID) is, as its name  indicates, a universally unique identifier.   They are also known as GUIDs.   Unlike auto increment fields and ArcGIS's ObjectIDs, UUIDs can be created even if disconnected. That is, you do not need to wait for a record to be added to a master table before you can get a unique identifier.


In Survey123, you can use the uuid() function within the calculation column of your XLSForm to create a UUID. A UUID looks something like this:




UUIDs are not pretty to look at, and from a practical perspective, impossible to remember. This is the reason why they do not quite fit the bill for the 'Ticket Number' problem either.  I wanted to bring this up anyways, because in some cases where you need a unique identifier, even if it is not human friendly, the use of the uuid() function could be handy.


Purely for completeness, you should also know that ArcGIS feature services support the notion of GlobalID and GUID fields where UUIDs can be stored. In the case of GlobalID fields, the UUIDs are created automatically by ArcGIS when the feature is created in the feature layer, and guaranteed to be unique across all records in the table. If working with  GUID fields, UUIDs can be assigned and edited at will and no uniqueness is required. That is, you can calculate and set a GUID value using the uuid() XLSForm function.


Hand-made Ticket Numbers: As long as you understand certain limitations, it is possible to create more human friendly 'Ticket Numbers' using functions supported in Survey123.


It would be great if we had some sort of magic auto-incrementing function that would give us unique numbers sequentially.  This would allow us to start with an easy number like 0000 and then increase it to 0001, 0002.... easy to remember! Unfortunately, this is not that easy when working disconnected because there is no way for an offline device to know what is the next sequential number available.  Even if we were able to create sequential numbers within a device, we cannot guarantee that other devices will not create the same sequential numbers.


Since time is always moving forward, a first approach is to use the current time, say down to the second, to generate a number. This can be accomplished by using the format-date() function, which is described in a previous post on Dates and Time in Survey123.


textTicket Numberformat-date(now(), '%y-%m-%d %H:%M:%S')


The above will generate a number similar to this: 17-09-26 13:22:23 where the first two characters are the year, the next two the month, then the day, hour, minute and second.  If you look carefully at the expression within the format-date() function I have added some dashes and spaces a separators, but you can get rid of them if you like.


Using the current time is certainly going to give us a unique number for the device you are on, but this does not mean that it will be unique across all devices capturing data. First of all, there may be people creating new survey at exactly the same second.  If working across multiple time zones, the chances of coincident numbers increase.


Not everything is lost, because you can augment your Ticket Number with some additional data to reduce the changes of creating duplicates. You can for example use the username of the currently logged in user. This can be achieved by using the function property('username'). This would leave your Ticket Number expression as:



textTicket Numberconcat(format-date(now(), '%y-%m-%d %H:%M:%S'),property('username'))


Obviously, using the logged-in username is only possible if your survey is secured. This approach will not work with public surveys where users can submit data anonymously.


Depending on your particular scenario, you may be able to use a combination of one or more other attributes in your survey that would make your Ticket Number meaningful and unique.


Technically, a handmade Ticket Number is not guaranteed to be unique although in practice, if you build it correctly, it will do the job just fine.


When working with hand-made Ticket-Numbers or with the uuid() function, you also need to consider certain workflows that could get on your way.


  • Hand-made ticket numbers and uuid() are defined through an expression in the calculation column of a survey. Calculations will execute when your form loads for the first time and will populate the Ticket Number accordingly. Once the Ticket Number is set, it will not be calculated again unless the calculated value is altered or removed. This is all good and consistent with how calculations should behave, except if a user decides to create a copy of an existing survey from the Sent folder. In that case, the Ticket Number will not be recalculated  so you will end up with a duplicate. Using the Sent box settings in Survey123 Connect, you can prevent this from happening by disabling the Sent folder for your survey.
  • Similarly, you will want to decide if you want to make the Ticket Number be a read-only field or not. If users are given the ability to override a Ticket Number manually, you can no longer guarantee uniqueness.  You can easily flag the readonly XLSForm column of your question to 'yes' if  you want to make your Ticket Number question read-only.


I hope that the above gives you some food for thought, even if you did not find a solution as you would have imagined it first.

Jack Dangermond credits Ralph Nader with being the person who inspired the creation of GIS Day. He considered GIS Day a good initiative for people to learn about geography and the uses of GIS. He wanted GIS Day to be a grassroots effort and open to everyone to participate.


GIS Day was first held in November 1999 by the National Geographic Society, Association of American Geographers (AAG) and Esri. It is celebrated since then the third week of November, coinciding with National Geographic Society's Geography Awareness Week.


Our 2.5 release includes updates to the Survey123 website, Survey123 Connect for ArcGIS and the Survey123 field app across all supported platforms. Make sure you refresh your browser cache and get the latest Connect and field app from the Survey123 download page.


Survey123 website


Remember to clear you  web browser cache to make sure you get the greatest and latest from this update:



Survey Publishing Enhancements (Modify Schema)


With Survey123 web designer, you can visually create surveys right from your web browser.  Our vision with web designer is to provide the simplest experience possible for building powerful smart forms.  Survey123 web designer, in fact, does a lot for you under the covers including automatically building a fully functional ArcGIS feature service where data from your survey will get persisted.  Up until this release, you could not control the exact characteristics of the fields in your feature service. This is not typically an issue except when you want to download the data to Microsoft Excel, CSV or other formats like Shapefiles.


Starting with this release, we added an option in the Publish Survey dialog that lets you modify the schema of your survey's feature service.  You can control the exact name that will be given to fields, define the values to be stored when using choice lists and even the maximum length of text fields.



Please note that you will not be able to change  the schema of fields that already exist in your feature service.  For  example, when you publish your survey for the first time, you will be able to modify the schema for all questions in your survey. However, if you publish the same survey again, you will only be able to modify the schema for new questions in your survey.



Delete a survey while preserving the data


Starting with this release, the Delete Survey dialog in the Survey123 website includes a handy option that will let you optionally preserve all data you captured.   If you choose to delete all collected data, your survey will be removed along with its feature service.  If you choose this option, you may want to first make a backup copy of your data first. 



Changes to the Data tab


The Data tab in the Survey123 website lets you explore all collected data using a map, a table and an individual response format. We made a handful of changes:


  • A new refresh button lets you more easily update the map and table to get the latest data from your survey.
  • You can now more easily drag the table view over the map to split the screen in between the two views.
  • The Individual Response view includes a new settings dialog from which you can control the size of photos retrieved into your report.
  • Click on photos in the Individual Response reports to open a detailed view which will let you also see EXIF metadata, if available, including location, direction, camera and size.



Other enhancements and fixes in the Survey123 website


  • A new keyboard shortcut for saving your work in web designer has been enabled: Ctrl+S (Cmd+S in Mac).
  • Multiple fixes allow the Analyze and Individual Response reports to better handle surveys with Groups and Repeats.



Survey123 Field App and Connect Enhancements


Disable the Sent box in the Survey123 field app


Through a new survey-level setting in Survey123 Connect for ArcGIS you can now disable the Sent box in the Survey123 field app. This setting has been requested by a handful of organizations who, for security reasons, want to ensure that data already submitted to ArcGIS is completely erased from the device.  Disabling the Sent boxis also handy if you want to prevent users from copying previously sent surveys to create new ones.


If you already have published a survey and you want to apply this setting, field users will need to delete their local copy of their survey first, and then download the survey again.  Simply refreshing the survey from the Download page will not take effect, unless the Sent box in the device is already empty.

For more info, check this help topic.


Extract Exif metadata from photos using the pulldata() function


The pulldata() XLSForm function has been extended so you can extract Exif metadata from photos.  This allows you to store the Exif metadata as attributes of your features, making it possible to use this information easily once stored in ArcGIS.

Information such as the date-time and location where the photo was taken are stored as Exif metadata. You can also retrieve information about the camera used to take the photo, its dimensions etc  

The syntax for extracting Exif data is as follows:

pulldata("@exif", ${image}, "gpslatitude")


The first parameter indicates that the pulldata() function will be used to extract Exif metadata. The second parameter indicates the question in your XLSForm with the image. The third parameter defines the name of the Exif metadata to be extracted. Depending on the camera used to take the photo, Exif metadata can include more than 100 tags.


For a detailed list of common tags available, look at the Exif XLSForm included in Survey123 Connect for ArcGIS:


  1. Open Survey123 Connect for ArcGIS
  2. Click on New Survey
  3. Find the Exif survey in the Samples category
  4. Set a title to your new survey and create it
  5. The list of tags is located in the choices worksheet


Survey123 Connect publishing enhancements


Survey123 Connect will no longer overwrite your survey feature service when attempting to add new questions to an already published survey.  A new dialog will prompt you to confirm that new fields should be added to your feature service.



A side effect of this enhancement may cause you some confusion when attempting to re-publish surveys with notes in your XLSForm.   If the Publish dialog indicates that new fields will be added for your note questions, simply go back to your XLSForm and remove the value of your note question in the name field of the survey worksheet.  Notes should not include a name! You can learn more about notes in Understanding Notes in Survey123 

New XLSForm Community Surveys and Samples in Survey123 Connect


The Create New Survey dialog in Survey123 Connect for ArcGIS includes many functional XLSForms in the Community and Samples categories.  You can use these surveys as a starting point to your own. This update includes new XLSForms:


  • Exif (Sample): Included in the Samples category, illustrates how to use the pulldata() function to extract EXIF metadata from photos in your survey.
  • Smart Sketching (Sample): This sample illustrates how to use the draw and annotate appearances on image questions to enable smart sketching on an empty canvas, a photo or a predefined image.
  • Spike (Sample): This functionality is in Beta and shows how to integrate Survey123 with Spike. More information atSneak Peek: Survey123 for ArcGIS and Spike® integration 
  • Cardenal Amarillo (Community): A simple survey used to capture observations of Gubernatrix cristatain Argentina. This survey is a good example of how to include questions in your survey to record audio. It also shows how to include audio files in your survey so they can be played from your device.
  • Pesticide use Log (Community): A survey for helping private pesticide applicators record the location and weather conditions where pesticide is applied.


Other Survey123 Field App and Connect Enhancements and fixes


  • A slider control in the camera widget now lets you apply optical zoom to take detailed photos.
  • Read-only questions are now supported in the Survey123 field app Inbox.
  • BUG-000107126: Disallow the ability to bypass certificate verification warnings in Survey123 for ArcGIS field app.
  • BUG-000108290: Unable to collect geopoint locations on Android devices using a Trimble GNSS bluetooth receiver when configuring the device with a mock location and GPS-only enabled.
  • BUG-000100273:  Importing data from Survey123 for ArcGIS into a geodatabase in Oracle returns an error "The Field is a keyword in the destination DBMS (cannot rename).[rowid].
  • BUG-000108893: Using Survey123 for ArcGIS field app, if a survey is taken while Offline, then the device is switched to Online, pending surveys in the Outbox will fail to send with Code 0.
  • ENH-000100907: Provide documentation for silent and/or unattended installation for Survey123 Connect for ArcGIS.
  • ENH-000104929: Support Single Sign On (SSO) experience for Survey123 Connect when using SAML-based Identity Provider (ADFS).
  • Clearing and recalculating a dateTime question that uses the now() function now updates the value to the current date and time instead of the date and time from when the form was first loaded.
  • The source code of both Survey123 Connect and the Survey123 field app have been updated in AppStudio for ArcGIS.

Surveys are for the most part about questions, but very often you need to provide some context before users can actually give you a good answer. And that is where notes come into play.


Notes allow you to insert text and images in between questions in your Survey123 smart form. Here are the main three reasons why you may want to use notes in your form:


  1. Information: Notes are useful when you want to provide users with some information that will help them capture data more appropriately. A note for example, could provide a reminder in an invasive species  survey that the inventory of invasive species should be performed in a radius of 100 meters. In a population census enumeration survey, notes can also include certain tips for the enumerator to introduce questions to respondents. 
  2. Warnings: Notes can be used to warn users about inconsistencies in the data captured, before it is  submitted. For example, if we do not want people to capture the location of an observation while on the move, we can make a warning note visible whenever we detect that the geopoint object has a speed larger than 0 meters per second. We could also use a warning note if we detect some  logical inconsistencies in the captured data.  Of course, we could use constraints in Survey123 to prevent all this bad  data to be submitted, but note provide a nice way to bring the user's attention into these  problems.
  3. Look&Feel: Finally, notes are a great way to bring banners and or logos  into your form to make them look exactly as you want.


The Basics


The use of notes is supported in both Survey123’s Web Designer as well as Survey123 Connect. The easiest way to create a note is to use Web Designer, where you can easily format your text using custom font sizes, colors, bullets for your paragraphs and even embedded images and links. Through Web Designer you can also control the alignment of your text in the notes: centered, right aligned and left aligned.


Here is how you can add notes to your survey in Web Designer:



Notes in XLSForms


Do  not set a name for your note  questions!   Notes do not expect user  input so they do not  need it. If you set the name  of your label, then  Survey123 Connect will create a field  in your feature service  that  will always be empty.



In XLSForms, notes are modeled as a note type of question.  The most important rule when using notes is that they do not require any value in the name column of your XLSForm. That is right: unlike other types of questions in XLSForms, the name column is not mandatory. In fact, it is typically better that you leave it empty.



The reason for this is simple: When you publish a survey with Survey123 Connect, the values in the name column of the survey worksheet define the attribute columns in the feature layer where we will persist your data.  Since notes do not expect any user input, there is no need to  give them a name.


Other than the above, notes behave like most other XLSform questions:


  • HTML formatting is supported in notes. You can use  HTML formatting for way more than cosmetic reasons. You can for  example add links. In the next example, I added a link that will trigger a phone call to the Water Department.


note<center><a href="tel:9091234567">Call Water Department</a></center></p>


  • Media: You can associate media to a note question. Use the media:audio and media::image columns for this. In the next example, a note is used to allow the user to play an audio file from the survey.  Images and audio files referenced must be stored in the media folder of your survey.


notePlay this audio before counting <i>Gubernatrix cristata</i> callsgcristata.mp3


  • Relevant: Relevant expressions can be used to make notes appear and dissapear. In the next example, A warning is shown to the user if the location is fixed while on the move. When adding warnings, I like to


calculatespeedSpeedpulldata("@geopoint", ${location}, "speed")
noteDo not move while fixing location!0.2 < ${speed}


  • Calculations: You can use calculations to display text in the main body of a note. In the following example, a note is used to indicate the number of choices selected in a select multiple question.


select_multiple violationsviolationsCode Violations
noteViolations observedcount-selected(${violations})


When calculating a note, you may want to have  your text all be shown in one single line, rather  than split in two like above. If that is the case, you can do something like the following:


select_multiple violationsviolationsCode Violations
note</b>concat("Violations: ", count-selected(${violations}))


The only reason I use </b> in the label is because the label column is mandatory for a note. <b> will not render any text, so the main body of the note will read a a single  line of text, which is what  I want.

Filter Blog

By date: By tag: