Latest Contributions by IsmaelChivite

If you have used Survey123 for ArcGIS to make a difference, tell us about it. Send us photos, screenshots and a bit about your story. You can send up to three images here.  Stories will be considered as potential content for the Esri User Conference 2016 Plenary Session's Your Work section.  Time is running out: the deadline is May 31st. ... View more

  [Updated June 16, 2020] [Updated September 18, 2023] [Updated October 25, 2023]   Note: In October 2023, ArcGIS Online introduced new date and time field types. We will add support for these new fields in Survey123 in 2024.  This article will be updated accordingly once we have support for the new field types. Until then, everything below applies only to esriFieldTypeDate.   In this blog post I will describe how you can work with dates and time in Survey123 with XLSForm. I will also cover important concepts for you to understand how dates and time are modeled in ArcGIS.   Before we start   To best follow instructions and concepts in this blog, I suggest you open the Dates & Time survey sample in Survey123 Connect. If you have a first look at this sample survey, it will much easier for you to follow content in the blog.   Open Survey123 Connect Tap on New Survey Select the Samples category Scroll until you see the Dates & Time survey sample or use the search function. Create a new survey from this sample     Next, spend some time going over the survey. I suggest you explore it first in Connect and you also bring it into your phone or a web browser to see what it feels like. The structure of the blog pretty much follows the sample survey.   Basic Types and Appearances   The Basic Types and Appearances section of the survey sample explores the different ways in which a date or time question can be presented to the end user. Combinations of the question type and its corresponding appearances, will help you control the best user input style that fits the needs of your workflow.   The first question in this survey is of type date and it uses no appearance. A calendar control is shown for users to pick a date.  Note that the question shows the current date by default. This is accomplished by setting the default column in the XLSForm to today().   The following 3 questions illustrate the use of appearances: Year, Month-Year and Week-Number.   Week-Number is the most exotic. Week numbers are often used in government and business for fiscal years and timekeeping. With this appearance, the calendar widgets will let the user pick a date and display the week number in the form.  You will also see the week numbers on the side as you select the date. The week numbers are calculated following the ISO-8601 standard. The Year and Year-Month appearances are handy when you simply want to present the user with a choice to pick a year, or a year-month.   As you exercise the calendar control, it may not be obvious, but you can actually tap and hold the arrows to quickly navigate back and forth in time.   The DateTime and Time questions are shown at the end of this first section. Note that the questions automatically show the current date and time. This is accomplished by adding he now() function into the default column.   Understanding Dates and Time in ArcGIS   The Date, DateTime and Time types of questions as well as their corresponding XLSForm appearances, help you control the user experience that you want to present to the end-user capturing the data, but it is also important to understand how that date will be stored in ArcGIS.   User input to the Date and DateTime questions will be persisted in ArcGIS in a field of type Date (esriFieldTypeDate).  Despite the name of this field type, ArcGIS does not store just dates in Date fields: ArcGIS date values always include time as well. Hence, there is a perfect match between DateTime questions in Survey123 and ArcGIS date fields, but when it comes to date questions in Survey123, you need to understand that actually the time will also be stored.   But what time? Well, it depends:   The time will be noon (12:00PM) if the user has manually selected a date, or if you set the date using the today() function. The time will vary if you set the date using the now() function. Now captures the current time, even if your date question does not show the time to the user in the form.   The screenshot below shows how you can configure a date field in your layer to display time. Note that in the Map Viewer, I had to open first the Fields dialog, then click on the date field and finally turn on the "Show time" option.   Once you do this, your popups will display time information!   Time values in Date and DateTime questions  are always persisted as Coordinated Universal Time (UTC). This is important when you have people working in different time zones. ArcGIS clients will translate the stored UTC values into your local time zone. The only exception is ArcGIS Pro. ArcGIS Pro will show the time in UTC (unless configured to do otherwise).   So far, I described how date and datetime questions behave. Lets talk next about Time questions, because they are special. For Time questions, Survey123 does not use the ArcGIS date type (which we know now that actually stores date and time). Survey123 time questions store values as a string. We did it this way, because otherwise you would get dates as well, and this could be confusing: Say you want to capture the time when a store opens.... you would not want the date in there...  Now, because time is stored as a string, there is no information regarding the time zone in it. As such, you need to keep in mind that there will be no time zone conversion.  Later in the survey, we will explore how to capture times differently.   Date and DateTime questions store values as date objects including time. Time questions store values as text. Formatting Dates   All graphical controls in the Survey123 apps will honor your locale regional settings. For example, if your browser or smartphone's locale is set to Spanish, then dates will be formatted the Spanish way (which is the correct one, by the way... just kidding).  In the next screenshot I show what the calendar control will look like in a phone set with United States vs Spain regional settings. Note that when using the Spain regional settings, the day goes first, then the month and then the year, versus the US version where the month goes first then the day and then the year.     Additionally the format-date and format-date-time function allow you to convert date objects into strings.   In the following example, we  use the wsdatetime question to capture the date and time at which a water sample was taken. The wstime question calculate will be hidden from the end-user and is used to extract the time entered and persist it as a string in military format.  Note that the output of the wsdatetimequestion is stored in ArcGIS as a date object in UTC, versus the wstime which will be stored as a string using local time in your device.   type name label calculation dateTime wsdatetime Sample Date/Time   calculate wstime Time format-date(${wsdatetime},'%H:%M')   The qualifiers in the format-date function are as follows: %Y 4 digit year %y 2 digit year %m 0-padded  month %n numeric month %b month name abbreviated %d 0-padded day of month %e day of month %H 0-padded hour (24-hr time) %h hour (24-hr time) %M 0-padded minute %S 0-padded second %3 0-padded millisecond ticks (000-999) %a Three letter short text day   Date and Date-Time Defaults   Survey123 accepts three possible values in the XLSForm default column:   today() This function returns the current date in your device. Typically you apply this function to questions of type date. The time part of this date will always by noon (12:00pm). now()  This function returns the current time in your device. Typically you apply this function to questions of type datetime. Finally, you can also apply a fixed date in ISO 8601 notation. For example: 2020-06-23T00:00:00-07:00. 2020 for the year. Always 4 numbers. 06 for the month. Always 2 numbers 23 for the day. Always 2 numbers T is simply a separator, for time. Always T. 00 for the hour. Always 2 numbers 00 for the minutes. Always 2 numbers 00 for the seconds. Always 2 numbers -07:00 is the UTC offset, which in this case is minus 7 hours.   Technically, you could also apply default value using Unix epoch notation (in milliseconds), but this would only work in the Survey123 field app. For consistency across the field and web apps, it is best to use ISO 8601 notation.   Constraints and Calculations   This section of the sample survey will be useful to understand how you use formulas with dates and date-times, so you can validate user input, as well as pre-compute date values. The questions in the survey are self-explanatory, The best is that you have a look at the XLSFile to better understand how things are done.   Constraints: A common constraint you will want to enforce is one that ensures a date is in the past or the future. That is pretty straight-forward: .> now() will ensure the user input is in the future for example. If you want to reference another question in the survey, you can use something like this: ${birthdate}<today(). This will ensure that the birthday question has a value in the past.   In constraints you may be asked to ensure a value is always newer and older than a fixed date. The date function is very useful in these cases. For example, you can set a constraint such as . > date('2006-01-01') or ${manufacture_date} > date('2006-01-01').   If you need to work with time constraints, simply add the time component to the date() function using the ISO 8601 notation.     Calculations: Calculations are easy too, as long as you understand that dates in the Survey123 app are encoded as numbers. When users pick a date or a date-time in the survey, all values will be presented in a human friendly way, but internally the application handles them as numbers that represent the total time elapsed since 1970. A positive number represents a date after 1970 and a negative number a date in the past.  As you can imagine those are pretty big numbers, but the math is simple once you understand how time is expressed. There are basically two formats for the elapsed time:   Unix epoch is the elapsed time since 1970 expressed in milliseconds. Decimal date or French Revolutionary Time is the elapsed time since  1970 expressed in decimal days. That is, one day is 1.0, an hour is 1/24, a minute is 1/24/60, etc.   To learn more  about Decimal Time have a look at this 'How the French made a 10-hour day' article. You will love it!     UNIX EPOCH DECIMAL DATE 2 weeks into the future now() + 14*24*60*60*1000 date(decimal-date-time(now()) +  14) Calculate age from birthday int((today() - ${birth_date}) div (1000*24*60*60* 365.25)) int((decimal-date-time(today()) - decimal-date-time(${birth_date})) div 365.25) Minutes spent for lunch int(${LunchEnd} - ${LunchStart}) div (1000*60) int((decimal-date-time(${lunchends}) - decimal-date-time(${lunchstarts})) * 24*60) 15 minutes from now now() + 15*60*1000 date-time(decimal-date-time(now()) +  0.0104166675) 0.0104166675 is equal to  (1 (day) / 24 (hours) / 60 (minutes) ) times 15 (minutes). I recommend you work with decimal date format, because then your XLSForm expressions will work both in the field and web apps. The Survey123 web app ONLY supports decimal time. If you plan to use date expressions in the web app, use the decimal-date-time function. If you ever need to populate a Survey123 date or dateTime question using a UNIX epoch number, you can use an expression like this:   type name label calculation text epoch Epoch (milliseconds)   dateTime mydate The Date date-time(int(${epoch}) div 86400000)   Calculates and Relevants   This section in the sample survey is very similar to the previous one, but shows some other techniques that may be handy. For example, note how a calculate question is used to get the Lunch time computed, and then that value is used in other expressions throughout the survey.   A calculate is a type of question that will let you keep the result of a calculation without having to show it to the user in the survey.    You will remember that questions of type time are ultimately stored as strings in ArcGIS, however, as far as calculations go, you will treat them as numbers. That is actually pretty handy because otherwise, you would not be able to do much with them!   Other ways to capture Time   The last section in the sample survey shows other techniques to capture time. The 'Time from a list' example should not come as a surprise. A select_one type of question with minimal appearance is a nice way to present a list of hours to choose from. Then there are some other examples illustrating how to use regular expressions in constraints as well as input masks.  I will go into the details here, so nothing is missed. First, an example to capture military type with a period. For example: 11:35.  Note the use of an input mask here. The input mask automatically adds the period (:), so the user does not have to worry about that. The value of the input mask is 99:99 meaning that we expect to digits followed by a period and two more digits. You will see that the constraint column checks to make sure that the hours do not go beyond 23 and the minutes do not go beyond 59.  Note: input masks are NOT supported in the Survey123 web app. In the second example, we want to capture military time, but without the period. For example: 1135. In this case, I want to highlight a couple of things. First, note that I am not using an input mask like before. It is not needed, because we can set the type of question as integer (to ensure that only numbers can be typed) and also set the the maximum length of the field to 4. This is done through the bind::esri:fieldLength column. This will do two things: will ensure that only integers with 4 digits can be entered into the Feature Service, and most importantly, will limit the user input in the Survey, to 4 numbers.  I still added a regular expression in the constraint to ensure that proper values within the 24 hour day are entered. In the third example, I want to capture military time including the time zone. For example 1445Z. Z is the time zone (ZULU, which represents UTC). In 1445G, G is for Golf and represents UTC + 7.  That would be 1445 Golf Time. How awesome is that! More details on this crazy business with Time Zone Abbreviations here. Again in this case I added a mask and complemented it with a regular expression in the Constraint.   If you are not familiar with input masks in Survey123 I recommend you have a look at the Esri custom columns help topic. Regular expressions are tricky to put together, but you can find loads of samples in the web.   Start and End   So far, we have been looking at date and time questions that were visible in the survey. It is for this reason that we cared about the user experience (types of questions, defaults and appearances) as well as the validation of inputs (input masks, constraints).  Now, in Survey123 there are a couple of handy question types that will capture time without the end user knowing. These questions are start and end.  When added to your survey, like in the Sample you are looking at, they will not show in the survey to the end-user, but they will populate data in the Feature Service, that you can see from the survey123 web site or a web map.   Start and end will capture the exact date and time at which the survey was initially opened by the end user, and flagged as done. For you to see how this works, the best is that you load the survey in your phone, complete a couple of surveys and submit the data.   The start and end values are useful if you want to understand how much time people in the field take to complete surveys. To calculate the duration of the survey, you will need to download the data in FileGeodatabase or Shapefile format and calculate a new field with ArcGIS for Desktop. This ArcWatch article describes how you can calculate the duration between two date fields.   On top of start and end, which are optional, you will always get the Creation and Last Edit date and time for every row you add from Survey123 for ArcGIS. It is important to highlight here that the Creation and Last Edit columns are handled through ArcGIS Editor Tracking. The help has a good topic on Editor Tracking in case you are not familiar with it. Do not worry about  enabling Editor Tracking in Survey123, because it is always enabled by default. This feature looks at the time when edits are made in the Feature Service not when the data is necessarily being captured or changed in the field. For example, if someone is working in the field while disconnected from the network, the start and end field will truly reflect the time when the user is working on the survey. The Creation and Edit Times will reflect the time when the data was actually submitted: May be immediately after data capture, or may be a couple days later while back in the office.   This was a long post, but I hope you find it useful. ... View more

First started in 1967, International Children’s Book Day is celebrated right around the birthday of fairy tale author Hans Christian Andersen. Lets start with the basics: Survey123's Connect for ArcGIS version is now 1.3.18 You can download it from the survey123 web site by opening the New Survey dialog. The latest Survey123 mobile app is 1.3.38. You can download it from the App Stores (Google Play, iTunes and Windows). The Windows, Ubuntu Linux and Mac Desktop versions can also be downloaded from the Survey123 web site. Image Size Threshold (death to the 400 Error!) Including photos in your survey is an important feature to many of you. Now, photos can be quite large, particularly if using high resolution cameras.  When it comes time to submit your survey, large photos can be a problem. In fact, large photos are the reason for many of the Code 400 errors that you have been getting while submitting your surveys from the field. In this update, the Survey123 app will reduce the resolution of images before they are sent to the feature service, making the submit process much more reliable and quick. Images will be adjusted so they have their maximum dimension (width or height, whatever is larger) reduced to 1280 pixels. We will take the other dimension and adjust it accordingly to preserve the aspect ratio. The reason why we chose 1280 is completely arbitrary, but in our own experience you will get images that have a balance between detail and size and most of the Error 400 issues should be gone. As the author of the survey, you can control this default parameter. In Survey123 Connect for ArcGIS, you can now go into Settings/Images and select what will be the maximum size of images to be submitted: Small (320px), Medium (640px), Large (1280px) or Unrestricted. Controlling the size of images definitely helps, but we are still working on making the submit process bulletproof, so you can submit not just larger images, but also in the future audio and video. Date Calculations and Constraints Say you want to calculate the age of a person automatically based on a birthdate. Well, now you can do it. It will look something like this: int((today() - ${birth_date}) div (1000 * 24 * 60 * 60 * 365.25)) When calculating dates it is important you understand that Date values are computed as a very large number, rather than as a human readable date. This large number represents the number of milliseconds elapsed since January 1st 1970, which is positive for dates after 1970 and negative for dates before. The Date and Time widgets in Survey123 for ArcGIS translate this big numbers into human friendly dates.  Say you want to calculate a time exactly 14 days into the future: (1000 * 24 * 60 * 60 * 14) + today()) Or just 15 minutes into the past: ${dateTime} - (1000 * 60 * 15) You can use the above expressions in the Calculation and Default columns. With this update we also support Date expressions in the Constraint column. For example, if you want to restrict input values to a Date in the future you can go with something like this: .<= today() Because there is nothing best that examples to learn quick, we put together a sample XLS file that you can download and import into Connect. The online help has also been updated for your reference. Form Items Starting with this release, surveys will now be persisted in ArcGIS using a new type of item called Form – you can browse these in My Contents.   Old versions of the Survey123 app will not recognize these new Form items, but don’t worry if you have existing published surveys, as this new release of Survey123 will continue working with them. In other words: Versions 1.3.x or newer of the Survey123 app will work well with the good-old Surveys you may have already published, but if you publish a Survey with Survey123 Connect 1.3.16, then you need a 1.3.x version of the app to use it. From a practical perspective, Form items allow you to more easily get to the Feature Service behind your survey (you can get to it through the form item details page). If you happen to be using Survey123 for ArcGIS against Portal for ArcGIS 10.3.1 you will notice that your surveys will not use the Form item. A Code Sample (like in the old days) will be created instead to represent your survey. Other Improvements: Users of the Survey123 mobile app can now control the size of Text and Input controls. This is useful for folks with fat fingers, who may want to use larger font sizes and inputs. The camera is now much friendlier with Windows 10 and Windows Phone devices. We also have added options so you can switch the camera in your device. We added Barcode scanning in our previous update. Thanks to all who submitted feedback! With this new update you will find barcode scanning less temperamental, particularly in low-end Android devices. Some key enhancements were added specifically to Android: The Repeat navigation controls now display proper sizes in high resolution devices and the Back button on your phone will no longer systematically close your app. HTML formatting used in your question labels will be stripped out from your Feature Service field aliases, but preserved in your Survey so you can better control the look and feel. A new column in the XLSForm column will allow you to overwrite the field aliases in your Feature Service if you like. Input masks are progressively getting better. Learn more about them in this help topic. Last, but not least, the Create New Survey dialog in Connect now includes a gallery of Samples from which you can learn quite a bit. We cannot emphasize enough how much all of your feedback is helping us progress. Thanks again for your support. ... View more