Latest Contributions by IsmaelChivite

New Britain is the largest island in the Bismark archipelago of Papua New Guinea. William Dampier, takes credit for being the first British man to land on this island. This happened on February 27, 1700. Since then we call this island New Britain (Nova Britannia at the time). Both Wiliam Dampier and New Britain are fascinating. This island, of volcanic origin, was populated by two main indigenous groups: the Papuans who arrived tens of thousands of years ago, and the Austronesians, who joined the party around 2000 years ago. The island hosts remarkable traditional cultures. Still today, over 10 different Papuan languages  are spoken in the island and as for the Austronesians languages, we count more than 40! William Dampier is known for being the first man to circumnavigate the world three times. He grew as a pirate attacking Spanish ships in America and then became a successful navigator and explorer. Dampier documented winds, tides and currents that were critical for future explorations by James Cook and Horation Nelson. His natural history observations were also highly influential on works of Alexander Von Humboldt and Charles Darwin. With our 3.3 update, we celebrate the adventures of William Dampier and his landing over 300 years ago to the island today known as New Britain. Enhanced support for GNSS receivers in the Survey123 field app Starting with version 3.3, you can establish a direct connection between the Survey123 field app and your GNSS receiver. This is useful for several reasons: More accurate location data:  By leveraging dedicated high accuracy GNSS hardware, you can substantially improve the quality of the location data you capture with your smart forms.  There is a wide range of GNSS receivers, some of them able to produce, if used properly, sub-meter and sub-centimeter accuracy.  Better understanding of the quality of the data: Having more accurate location data is not enough. It is also important to understand how accurate that data is.  Most GNSS receivers are able to provide important information that describe how the locations you are getting were produced. This includes things like the type of fix (RTK Fixed, RTK Float, GPS, Differential GPS, SBAS...), the differential age, reference station ID and many others.  With version 3.3 of Survey123 and above, you can select which of these properties are important to you and store them as GIS attributes in the feature layer columns of your choice. For a complete list of all the location metadata you can extract from your location data with Survey123 refer to the extract geopoint values help topic. Custom validation rules: You can use all the GNSS metadata described above to create custom data validation rules that get triggered while the data collection is happening. This is critical, as it can save many trips back to the field! It is important to note that you can use all the power of XLSForms to build sophisticated expressions that take advantage of this metadata. In this initial release, we have thoroughly tested GNSS receivers built by a number of vendors including Leica, Eos Positioning Systems, Bad Elf, SXblue, and Trimble, although it technically can work with other receivers as long as they output NMEA sentences. Our choose a receiver help topic includes more information regarding currently supported hardware. To support direct connections to external GNSS receivers, you will notice we have introduced several changes in the Survey123 field app. The Settings dialog in the app now includes options to connect to external location sensors via Bluetooth and network ports. For each connection you can also set user preferences such as a custom name for your device, the height of the antenna and handy visual and audio alerts to warn users in case the connection between the GNSS receiver and your device is interrupted. Our XLSForm support has also been expanded so you can include in your form expressions to help work with the GNSS metadata in real time.  This really opens a wealth of possibilities for you to use this information to design soft and hard data validation constraints in your forms.  The geopoint help topic in our doc describes in more detail how to take advantage of this. Point to Point Measurements with Spike® and the Survey123 field app We introduced support for the Spike laser range-finder in March 2018.  Spike is an excellent complement to Survey123, allowing you to quickly bring laser-accurate measurements into your forms. Jim Moore and Brett Stokes, who recently joined the Survey123 team, put together this video to demonstrate the basics of bringing measurements from Spike into Survey123. In this release, we are bringing support for the Point to Point (P2P) measurement mode which was missing in our initial implementation. P2P is a very handy way to calculate distances between two objects because it does not require you to identify a 2D plane on which to measure. In P2P mode you can simply aim Spike at the first object, such as the side of a house, and take a photo. Then, aim at the second object, such as a tree or fence, and take a second photo. Spike will calculate the distance between those two objects and bring that value into your Survey123 form. You can also use Point to Point to measure the height of an object such as a tree, streetlight or building. If you are already using Spike with Survey123 and wish to try Point to Point measurements, make sure you upgrade your Spike app from the store with the latest version. For more details on how Spike and Survey123 work together, check out thehttps://community.esri.com/groups/survey123/blog/2018/03/06/using-spike-with-survey123-for-arcgis blog post. Date/Time question type in web designer A new Date/Time question type has been added in web designer.  This will help you capture both a date and a time in one single row, saving vertical space in your forms. Within web designer you can set a default value for this question, either selecting a static value, or the exact time when the form is open.  You can also set validation rules to constrain inputs to a time window. Better field and choice naming in web designer If you never liked auto-generated field names such as field_0, field_1... in your Survey123 feature layers, you will want to pay attention to this one. Let me start with a bit of context first:  When you publish a survey in web designer, we create on your behalf a feature layer field for every question in your form.  These fields are used later to store responses to your survey. Up until this release, the name of these fields followed a simple field_0, field_1 pattern. This was typically never an issue when using the Survey123 web site to inspect your data, but it was not ideal when exporting results into Excel, CSV, or when configuring web hooks. Starting with this update, a more friendly name will be given to these fields based on the label of your questions. The following animation compares what an auto-generated feature layer schema looked like before, compared to what you will get now. Field and choice names are now derived from your question and choice labels. Given that there are some constraints that apply to the names that can be given to fields in a feature layer, you will notice that your labels will not be used literally. For example, spaces will be replaced with underscores (_), very long labels will be shortened and special characters will be eliminated. Using the Modify Schema dialog you can optionally fine tune the names of fields and choices to your liking, although remember that you will not be able to make adjustments in the schema once the fields have been added to the feature layer. The option to Modify Schema in the Publish Survey Dialog has also been made more visible as a reminder of this important step before you publish your survey. Delete and edit existing records for survey owners As a survey owner, I am sure you always submit a couple of records to test out your survey designs, immediately checking values in the Data tab. You can now easily delete, and even edit values right from the Survey123 web site. You no longer need to use a separate tool to do that. All you need to do is to select the records in the table view, and then choose the Delete or Edit options in the right panel to make the changes you want. At this moment in time, the option to edit records will not be available for surveys published from Survey123 Connect, but we are looking to relaxing this limitation in the future. Other fixes and enhancements Survey123 web site: ENH-000107831 Provide the ability to edit the survey submission "Thank you" screen seen in the browser when the survey is published from Survey123 Connect Import custom themes from other surveys. For text questions, in the Analyze tab, you can now choose to display a list of word counts or complete responses. Much improved user experience for configuring custom report templates. BUG-000119133 Field aliases in a survey in the Survey123 for ArcGIS application are not respected in single choice and drop-down lists. BUG-000098360 When downloading data from a Survery123 survey that was made using Survey123 web designer, the field names are converted to {field_1}, {field_2}, ... instead of honoring the name in the data table. BUG-000103150 Surveys created in Survey123 web designer using drop-down choices display 'choice1, choice2, ...' when exported to a CSV file or a shapefile. BUG-000105778 Survey question choices from single choice questions do not correctly translate when exported to CSV or shapefile from surveys that were created via Survey123 web designer. BUG-000114116 When using a custom template that writes out results from related and non-related attributes from a survey, the non-related records do not populate the template table. BUG-000113548 When an audio attachment is included in the survey, the Survey123 for ArcGIS website fails to generate a report. Survey123 field app and Connect Allow users to zoom beyond level 18 if the tile package includes more detailed levels. BUG-000119481 When creating a Survey123 for ArcGIS survey with an existing feature service, the float or single field types do not auto populate in the XLSForm created BUG-000110786 When signing in to the Survey123 for ArcGIS (iOS) app configured with an IWA portal, the app uses the username and password entered at the beginning of the app session for all subsequent sign-in attempts. BUG-000119847 Date questions in certain languages are not displaying the calendar correctly BUG-000119899 Integer type questions with the "spinner" appearance type do not work ENH-000105170 In Survey123, allow users to change the Location Provider, from the internal receiver to an external receiver. ENH-000109937 Provide an indicator within the Survey123 interface to display which GPS unit the app is using. ... View more

In this blog post I will describe how you can integrate Survey123 with Spike. Spike is a laser-based measurement solution by Esri partner ikeGPS.   What makes Spike unique is that it allows you to easily measure the dimensions of remote objects, such as areas and lengths, all from a photo. Say, for example, while doing a property inspection you need to calculate the height of a balcony, or the size of a garage door.  Rather than using a ladder and measuring tape, you can simply take a photo of the building, and then calculate dimensions by drawing on top of the photo.  You can similarly use Spike to calculate the size of road signs, billboards, poles, etc.    You can embed Spike questions within Survey123 and automatically bring all measurements from Spike into your form: Tap on a Spike question, snap a photo, measure on it… and your measurements will get persisted in ArcGIS along with the rest of the survey data. The integration of Survey123 with Spike is exciting news, because it has the potential to boost the productivity of certain data capture workflows tremendously.  This short video tutorial describes the basics of how Survey123 and Spike work together: Lets next get into the details... About Spike® The Spike laser: The most visible aspect of Spike is the laser instrument. It is an eye safe Class-1 laser, designed to measure distances in a range between 2 and 200 meters with an accuracy of 5 centimeters. The laser is designed to be attached to your smartphone or tablet and paired using a Bluetooth connection. Spike units have a retail price of $499 and are sold online and through a network of resellers. The Spike app: Once the laser is paired, the Spike app uses the camera on your smartphone or tablet so you can take a photo of your object of interest. Using the distance to the photographed object captured by the laser, the Spike app can accurately calculate lengths and areas as you draw lines and polygons over the photo. The Spike app is available for iOS and Android operating systems and is certified for use against a wide collection of smartphones and tablets.  Getting familiar with Spike: Spike is straight-forward to use, but knowing a handful of tips and good practice in the field is key to get the best out of it.  The best reference to take your first steps is the Spike User Manual.  Spike can be operated through two main modes: Photo Measurement: When working in New Photo Measurement mode, you take a single photo, and then derive all measurements from it.  You can for example simply get the distance from your location to the object, or extract lengths and areas by drawing on your photo. Point-to-Point Measurement: In this mode you can calculate the distance (horizontally and vertically) difference between two objects by taking a separate photo of each object.         Photo Measurement mode:In its simplest form, the Photo Measurement mode will allow you to simply snap a photo of an object, and get back the exact distance to that object form your current location.  This alone can be useful in multitude of use cases.  Say for example you need to document the width of a road: stand on one side of the road and take a photo targeting at the other end across. After taking the photo, Spike will give  you the exact distance. Not only you can get an accurate measurement, but you can do it without risking your life; safely from one side of the road.  With Photo Measurement mode you can also measure distances (and areas) on objects by simply drawing on top of the photo you just took. For example, say you want to know what is the height of a building. Well, you take a photo of it, and then you extract the height by drawing on that photo.  If you want to derive measurements from a photo in this way, certain conditions need to be met.  When used correctly and in ideal conditions, you can expect a measurement with an accuracy of 1%.  For example, if you measure the height of an object to be 10 meters, you can expect the accuracy to be around 10 centimeters. Let’s describe next what used correctly and ideal conditions means: Measurements against a flat surface: You should only perform measurements on flat surfaces, such as a wall or a sign. The specific shape you want to measure should be on a rectangular plane. This is because you will need to define the plane in the photo before you can measure anything.  Face your target front-on: For the most accurate measurements, you should take your photo while facing your target perpendicularly. That is, you need to be right in front of the plane on which you will do your measurements. If you need to take your photo at an angle, the expected accuracy of the measurements will be reduced progressively as you get further from being perpendicular. Ideal target range: The ideal working range of the laser is between 6 and 100 meters, although technically you can push the limits and measure objects between 2 and 200 meters away. Center your target: You also want to make sure that the object you want to measure is right in the center of the camera bulls eye, because the laser always calculates distances through the bulls eye. Draw accurately: Once you have taken the photo, you will want to make sure you accurately draw your lines and areas for measurement. The Spike app has a handy magnifying window that will help you with that task.   If your photo is taken following all the tips above, but you do not take pride in carefully drawing the exact shapes you want to measure… the measurement will not be as accurate as it could be. So far all of this may sound a little bit intimidating, but once you have the Spike device in your hand things are much simpler. Not just simpler, but also fun. Next you can see Brandon having a good time while ‘spiking’ outside Esri. From our own experience, the Photo Measurement mode works best with man-made structures: walls, signs, billboards, doors, straight poles, streetlights etc.  Measuring the height of trees, rocks etc is much more challenging because you often do not have a good plane to reference your measurements.  If you do not have a good 2D plane to make your measurements on, then you should consider using the Point to Point measurement mode. Point to Point Measurement mode: Point to Point or P2P is very useful when all you need is to measure the distance between two objects in front of you and you either cannot fit the two objects within a single photo, or when you do not have a good 2D plane on which to base your measurements. With P2P, you simply need to center your camera on your first target, say for example a tree and take a photo. Then you turn from that same location to take a photo of your second target, say for example a wall. Spike will compute the horizontal and vertical distances between the two photographed objects. Like with any tool out there, it takes a bit of practice to fully understand what is the best measurement mode for the job.  Spike can greatly boost the productivity of field workers saving valuable time. It also can provide a much safer way to make measurements. For hard to reach objects, Spike can make it possible to perform measurements that could not be done otherwise. Now that I have described the basics of Spike, it is time to go into the details of how Survey123 and Spike work together. Integrating Survey123 with Spike (Photo Measurements) Starting with Survey123 version 2.6, you can incorporate Spike functionality into your form. This allows you to streamline the use of Spike within your data collection workflows. For example, if you have a survey to help map the presence of graffiti in the city, you will want to capture many things including the location, typology (Tag, Throw-up, Blockbuster…), surface, photo and size.  Spike can help you capture the size accurately, so you can estimate the cost of removal. Adding Spike questions into a survey: You can add Spike questions to a survey using Survey123 Connect for ArcGIS. In your XLSForm, add a question of type image, and then set the appearance to either spike-full-measure, spike or spike-point-to-point: spike: With this appearance, Survey123 will launch the Spike app so you can simply aim at your object of interest and snap a photo of it.  The spike appearance is useful when you simply want to get the distance to the photographed object and its estimated location. spike-full-measure: This appearance sets the Spike app in New Photo Measurement mode as described in the previous section. When using this appearance, you will be able to get the measured lengths and areas, into your form. spike-point-to-point: Launches Spike in Point to Point mode so you can calculate the horizontal and vertical distances between two photographed objects. When using the spike-full-measure, spike-point-to-point and spike appearances, Survey123 will let users easily launch the Spike app from within the form.  Once the photo has been taken and measurements added, Spike will return back to Survey123, so all measurements of interest can be extracted and stored in the survey.  The measure object with Spike help topic describes step by step how Survey123 and Spike work together. Spike photos will be stored as an attachment automatically.  This is a great feature because photos will become a verifiable record persisted with your data. Photos will show how accurately the measurement was made in the field, and if needed, the photo itself could be used in the back-office to make additional measurements. The use of Spike within Survey123 is only available to logged-in users. Spike questions can be used while online or offline, but you must be logged-in.                  Extracting Spike measurements with pulldata(): Spike uses EXIF photo headers to store information such as the location from which the photo was taken, the direction, distance and measurements made in the Spike app.  In a previous blog post we explored how to extract EXIF metadata from an image in Survey123 using the pulldata() function. This same technique can be used to extract Spike metadata. The Spike metadata in the EXIF of the image is split between standard EXIF tags and custom information stored in the ImageDescription tag. The following table describes standard tags populated by Spike: EXIF tag Description gpsdestlongitude The estimated X coordinate of the photographed object gpsdestlatitude The estimated Y coordinate of the photographed object gpsdestcoordinate The XY coordinate pair of the photographed object gpstimestamp The time when the photo was taken gpsdatestamp The date when the photo was taken gpsdestdistance The distance between the photographed object and your device imageDescription A JSON string including measurements captured by the Spike app This expression will extract the distance in meters between the photographed object and the device: pulldata("@exif", ${spike}, "gpsdestdistance") In order to extract photo-measurements from Spike, your pulldata() function will need to first get the ImageDescription EXIF tag, and then parse the returning JSON string to extract the specific dimensions you are looking for. For example, the following expression assumes the first measurement made in Spike was of type length, and gets its value in meters: pulldata("@json",  pulldata("@exif", ${spike }, "ImageDescription"), "Spike.objects[1].dimensions") The next expression assumes the first measurement made was of type area and extracts its value in square meters: pulldata("@json",  pulldata("@exif", ${spike_full}, "ImageDescription"), "Spike.objects[1].area") This expression gets the horizontal distance between two objects as measured through the Point  to Point measurement mode: pulldata("@json",  pulldata("@exif", ${spike }, "ImageDescription"), "P2P.horizontal_distance") Spike returns many other properties through the ImageDescription EXIF tag including: the pitch, roll and direction of your device when the photo was taken, the unique identifier of the specific Spike laser unit that captured the distance measurement etc. The Spike section of the Media help topic describes in detail the contents of the JSON object stored in the ImageDescription. Another way to explore the contents of the ImageDescription tag is to simply add a text question to your form and have it populated with the contents of the ImageDescription tag through a calculation.  When you do this, it is important that you set the esriFieldLength column of your text question to 1000 characters. The ImageDescription JSON string can be quite big. A great starting point to experiment with all the things you can do with XLSForms and Spike is our Spike sample. I strongly recommend that you have a look at it:   1.    Open Survey123 Connect for ArcGIS 2.    Click on Create New Survey 3.    Search for the Spike survey in the Samples category 4.    Publish it and test from your smartphone 5.    Look into the XLS file to learn all the techniques used in the sample. Integrating Survey123 with Spike (Estimate Locations) While the heart of what Spike excels at is helping you measuring dimensions over photos, you can also use Survey123 to roughly estimate the location of remote objects.  For example, you can map the location of a building without stepping out of your car, or the location of a streetlight across the street. Before we go into the details of estimating the location of remote objects, I want to emphasize that Spike is not a high accuracy surveying device.  It can measure accurately distances to objects, but when estimating the location of remote objects there are several sources of error that will come into play outside of what Spike is designed to do. To define the location of a remote object, Spike relies on the following: The location of your own phone or tablet: This is the first source of error. A typical smartphone is not going to give you a highly accurate location for your phone. Your location accuracy may vary depending on many factors and if your location is off, the estimated location of the remote object will be off too.  To correct this problem, you can connect an external GNSS receiver to your own phone or tablet via Bluetooth. If used correctly, external GNSS receivers will improve the source location used by Spike in its calculations. In the side photo, for example, Survey123 is combined with Spike and Juniper Systems Geode Sub-meter GNSS antenna. The distance to the remote object: This is going to be very accurate as long as you keep yourself within the working range of Spike (2 to 200 meters) and you aim at the remote object using the bull’s eye in the Spike app. Measuring distances to objects is what Spike is designed to do, and does it very well. The azimuth to the remote object as measured from your own device location: The azimuth is another important source of error. The azimuth is obtained from the compass in your own smartphone or tablet. The compass in your smartphone must be calibrated before use by Spike. It is critical that you follow steps in this compass calibration video-tutorial before you estimate remote locations. Even with your compass calibrated, it is normal to get deviations of a few degrees, which will increase the error as the distance to the remote object increases. The gpsdestlongitude and gpsdestlatitude properties will be populated with both the spike and spike-full-measure appearances.If you do not plan to capture any length or area measurements and all you want is the location of the photographed object, then the best is to use the spike appearance. To extract the estimated latitude and longitude of the photographed object, you will use the pulldata() function within a calculation. For example, the following expression will extract the latitude: pulldata("@exif", ${spike}, "gpsdestlatitude") You can store the latitude and longitude as attributes in your survey, and then create an XY Event Layer with them in ArcGIS Pro or other tools.   While it should be possible for you to calculate a geopoint question with the latitude and longitude values extracted from the Spike photo, there is currently a known limitation in Survey123 preventing the successful calculation when applied. This is in our backlog and we will work towards resolving the problem as soon as possible. As stated before, it is best that you exercise Spike in the field in all its capabilities. To fully understand its potential and limitations you must experience it in the field in real working conditions. If you need some ideas, here is an informative video by IkeGPS further describing some common use cases for Survey123 and Spike. ... View more