Latest Contributions by IsmaelChivite

[Last update: June 16, 2020] Starting with our July 2018 update, Survey123 supports working with Microsoft Power Automate (formerly known as Microsoft Flow). Microsoft Power Automate is a cloud-service that lets you easily build workflows to automate repetitive tasks across multiple applications and services.  Here are some examples of the type of things you can do with Survey123 and Power Automate: Notify a supervisor via e-mail and/or SMS, when a new high priority incident is reported through Survey123. Add a reminder to an e-calendar 6 months after a new installation is submitted via Survey123. Add a new row into a spreadsheet, sharepoint list or SQL Server database when a new survey is submitted. In this ArcNews story, you can learn about how the Massachusetts Department of Conservation and Recreation’s Office of Watershed Management is leveraging Survey123 and Power Automate to streamline boat inspections. To get you started I will describe, step by step, how you can automate e-mail notifications after one of your surveys is submitted. This is a simple scenario, but going through it, you will quickly realize how to build your own different flows to do entirely new things. The Survey123 Microsoft Flow connector is available in all Office365 regions except in US Government (GCC) As of July 10, the connector is ALSO available in US Government Cloud (GCC) Getting Started with Microsoft Power Automate Microsoft Power Automate is included with Office 365. The free version has limited capabilities, but you can upgrade to premium subscriptions. For details check the Microsoft Power Automate pricing page.  You can also create a 90-day free trial. Unless your subscription to Office 365 already shows Microsoft Power Automate in the main page, you will need to launch it from the Office 365 App Switcher as shown in this animation: Within Power Automate, go into My flows and select Create from blank. This will present a gallery of Connectors, from which you will be able to trigger your Flow. Setting up a Survey123 trigger in your Flow: The very first step in every flow you create is to define a trigger. Search for Survey123 in the gallery of connectors. You will notice that Microsoft includes many connectors out of the box. That is the beauty of it! As you can see, the Survey123 Power Automate connector has only one trigger: When a survey response is submitted.   After selecting the When a survey response is submitted trigger, you will be prompted for your ArcGIS  credentials.   This is because you will need to select among the surveys that you own, the one that will trigger your flow when a record is submitted. Once logged in with your Survey123 credentials, Power Automate will keep your ArcGIS connection for later use. In my case, I selected the survey called Incident Report.  My Incident Report survey has a few questions such as the location of the incident, a photo, comments and priority, which can be High, Medium or Low. Adding condition logic and actions Once your survey trigger is set, you can start adding conditions, actions and loop statements to your flow. In this example I will add a condition to check if the value of the Priority question in the survey response is equal to High.  The survey data is exposed within Flow, so you can dynamically extract values from the survey submitted and use them in your condition expression. Next, I will add an action in the event that the condition (Priority is equal to High) is true.  I want my action to send an e-mail to someone.  I will choose the Gmail connector, and sign in to grant Microsoft Power Automate permission to use my Gmail account to send e-mails. E-mail connectors will let you define the recipient, subject, body and attachments in the message.  Again, bringing values from the survey response is possible too when defining actions. Note how in the subject and body of the e-mail for example, you can include values from the submitted data, such as the Date, Priority, Comments etc.  If you get creative, you can do other things such as building a link to a map centered at the Incident's location. The body of the e-mail is made out of HTML tags. This helps you format your email, by highlighting certain text in bold, or adding breaklines when appropriate. To create a map link illustrating the location of the incident I looked at the ArcGIS  Use URL parameters to modify maps—ArcGIS  Help | ArcGIS help topic. Note how the X and Y coordinates of the submitted incident, can be dynamically added to the link in the e-mail. In a later section in this blog post I will describe some tips to dynamically get data from your submitted survey. While you could keep adding additional conditions and actions, I will leave it here, and save the Flow.  I encourage you to explore the many connectors included with Power Automate.  Testing your Flow Now that your Flow is saved, it will be triggered by Survey123 every time a survey response is submitted.  Do the test yourself: submit data with your survey from either the Survey123 web or field apps and your recipient will receive an email. Learning more about Microsoft Power Automate Power Automate provides many capabilities that are out of the scope of this post. If you would like to learn more about Flow I suggest the following: Power Users Community  Microsoft Power Automate - YouTube  Plans | Microsoft Power Automate Handling Dynamic Content from Survey123. A few tips. One of the great features in Microsoft Power Automate is that you can process incoming data submitted through your survey response. For example, in the scenario above I processed the values in the Priority question to build a condition. I also used the values in the Comment and Location question for the subject of the e-mail.   Data from your form is exposed through the Dynamic Content dialog in Microsoft Flow. You can directly bring values or apply additional expressions to them. When you bring data from the submitted survey into your Flow using Dynamic Content, keep the following in mind: If you have hidden questions in your survey (or metadata questions such as start, end, username etc), you will not see them exposed in the Dynamic Content dialog. However, you can always add them manually into Flow following this syntax: ?['feature']?['attributes']?['yourhiddenquestioname'] Date and DateTime questions from Survey123 are exposed in Microsoft Flow using the UNIX time format.  If you want to convert UNIX time into a readable date string, use the addseconds function. For example: addseconds('1970-1-1', Div(triggerBody()?['feature']?['attributes']?['mydate'],1000) , 'yyyy-MM-dd') The output of a select_one question in Microsoft Flow will not be the label of your choice, but its name.  If you want to include the labels of your selected choices in the payload of the webhook, so you can use them through the Dynamic Content in Flow, then use the jr:choice-name() function to extract the label in a calculate or hidden question. jr:choice-name(${activity},'${activity}') The first parameter represents the choice name. In the example above, I am passing the choice name selected in the 'activity' question.  The second parameter is the question in the form using the list where the choice has been made.  Very often, the first and second parameters will reference the same question, but note that the second parameter is enclosed with quotes, because it does not refer to the value (response) of the question, but to the actual question itself. Some additional technical details Technically, when you save your Flow, Microsoft Flow registers a webhook in your own survey. Webhooks are the foundation for having Survey123 communicate with Flow to make all the steps described above work. A webhook is simply a URL Callback. That is, a URL that is invoked by an app when a particular event occurs, to pass some data to another app.  In our case, the apps are Survey123, which invokes the URL, and Microsoft Flow, which is the receiver of the webhook. In the initial steps described above, when the trigger was setup by selecting the survey, Microsoft Flow created a unique URL representing your Flow. This url was registered as the webhook URL in your own survey when you saved the Flow. If working with the Survey123 field app, it is important to remember that users will need to download the survey into the app after the webhook URL has been registered.  If you want to look at the webhooks registered in a particular survey, simply follow these steps: Log into survey123.arcgis.com and select a survey Switch to the Settings tab of the survey Open the Webhooks section. You really do not need to setup webhooks manually when using Power Automate but the webhook dialog can be of interest in case you are working with other Workflow Automation services or with your own custom developed web service end points. Known limitations and roadmap: I believe that with this first implementation of the Survey123 Connector you will be able to do quite a bit, but there are some known limitations I want to highlight: Attachments are not going to be exposed in the Dynamic Content dialog. You cannot reference attachments manually either. We are working towards enabling access to attachments within Flow in a future release (planned for version 3.10). Implemented in 3.10. Data from repeats will also be missing in your Flow at the moment (planned for version 3.10). Implemented in 3.10. The Microsoft Automate connector is not compatible with ArcGIS Enterprise (tentatively planned for Q4 2020) ... View more

[Published July 17, 2018] [Updated July 21, 2018] [Updated July 31, 2018] [Updated October 8, 2018] This blog post is for those of you that have upgraded the Survey123 field app to version 3.0 and realized that your surveys can no longer submit data to your feature layers.  Here is a screenshot of the error that the Survey123 3.0 field app will show when working against a feature layer with no supportsApplyEditsWithGlobalIDs. If your surveys do not show evidence that supportsApplyEditsWithGlobalIDsis a problem, then you should ignore this blog post. A little bit of context As described in our https://community.esri.com/groups/survey123/blog/2018/07/06/bull-release-30 announcement, the Survey123 3.0 field app no longer can submit data to feature layers where the supportsApplyEditsWithGlobalIDs property is set to false.  This is a somewhat obscure property that cannot be simply be turned on with the push of a button. Instead, it switches on automatically when your feature service meets a number of requirements: All layers within the feature service must have an indexed GlobalID field. All relationships (layer to layer, layer to table, layer to attachments, table to attachments) must use GlobalID fields (as opposed to ObjectID fields). If using an enterprise geodatabase, the feature service cannot include versioned feature classes. If using Portal for ArcGIS, only versions 10.4 and newer support this property. The reason why Survey123 requires supportsApplyEditsWithGlobalIDs is that this setting is critical in order to guarantee clean transactions when uploading attachments (particularly when working in low bandwidth conditions) and also to support nested repeats. We are aware that many of you have feature services where supportsApplyEditsWithGlobalIDs could not be enabled immediately, so we made available a backwards-compatibility build of Survey123 2.9 (also referred to as Survey123 Classic), which has been made available on Windows, Android and Mac.  Apple has not yet approved this build in the store so far, but in exceptional cases Esri could provide access to older builds as a temporary measure (Please contact Esri Tech Support for that). To download Survey123 Classic (2.9), use the version switcher in the top-right corner of our Survey123 for ArcGIS | ArcGIS Download page. Survey123 2.9 (Classic) is a temporary solution for all of you who can't really adjust to the requirements of version 3.0.  In this blog post I will describe a couple of approaches to enable supportsApplyEditsWithGlobalIDs in your feature layers so you can work with 3.0 against your existing layers. Before embarking into these steps, consider if you really need your surveys to work against existing layers. In many cases, you can simply publish a new version of your survey with Survey123 Connect and have field users download your new survey. If you want to keep Survey123 3.0 (and other previous versions) working against your existing layers, then read on... How to check if supportsApplyEditsWithGlobalIDs is enabled in your feature layer The supportsApplyEditsWithGlobalIDs property is not advertised through the user experience in ArcGIS Enterprise or ArcGIS Online .  This property is only visible within the json definition of your feature service. In ArcGIS Online or your portal in ArcGIS Enterprise, navigate to the Page Details of your Form and click on the Form Layer to open the Feature Layer behind your Form. Next, open the Services Directory view of your Feature Service and display it as JSON. Look in the JSON for the supportsApplyEditsWithGlobalIDs You cannot simply edit the supportsApplyEditsWithGlobalIDs property. This property only turns true when certain requirements in your Feature Service are met as described in this blog post. Working with Hosted Feature Layers in ArcGIS Online and Enterprise If your feature service does not have attachments or relationship classes: This is the easiest scenario. To enable supportsApplyEditsWithGlobalIDs, simply enable the sync capability in your feature service. This can be done through the Settings of the feature service.  From ArcGIS.com go into your Content, navigate to your survey folder and look for the Feature Layer (not the views!). Then open the Item Details page and go to Settings. The Enable Sync capability is inside the Feature Layer (hosted) section. Do not forget to hit Save after you check Sync. Enabling the Sync capability will automatically create and index GlobalId fields in your feature layer and supportsApplyEditsWithGlobalIDs will switch to true. Once you have enabled sync, you can turn it off.  The Sync capability will not have a negative performance impact on your feature layer, but you will not be able to disable queries in your layer if Sync is enabled.  This is one of the main reasons why you may want to keep Sync disabled. If your feature service has attachments and/or relationship classes: In this case, we need to overwrite your feature service with a geodatabse schema that includes one uniquely indexed GlobalID field per every layer and table in your service.Additionally, all attachment relationships must be based on GlobalID fields..  The following steps illustrate how to do this, without losing any data you may already have. In ArcGIS  or your portal in ArcGIS Enterprise, navigate to the Page Details of your Form and click on the Form Layer to open the Feature Layer behind your Form. Download your Feature Layer data in File Geodatabase format and add the entire contents of the File Geodatabase into ArcGIS Pro. In ArcGIS Pro, run the Add Global IDs geoprocessing tool on all layers and tables in your File Geodatabase. In ArcGIS Pro, run the Migrate Relationship Class geopocessing tool against attachment relationship classes in your File Geodatabase. In ArcGIS Pro, run the Disable Editor Tracking geoprocessing tool against your File Geodatabase data. In ArcGIS Pro, set all the Editor Tracking fields in your File Geodatabase to "Read Only" using the Design/Fields view as shown in the next animation. Overwrite your Form Feature service using the Share/Web Layer/Overwrite Web Layer option in ArcGIS Pro. In your portal in ArcGIS Online or ArcGIS Enterprise, Enable Editor Tracking (in the Setting Dialog of your feature service as shown in the next screenshot) in the Form's Feature Layer. Make sure you enable Keep Track of who created and last updated features. Do not get confused with the Keep track of created and updated features. Working with Non-Hosted Feature Layers in ArcGIS  Enterprise For ArcGIS Enterprise, there are some situations where supportsApplyEditsWithGlobalIDs cannot be set to true in a Feature Service. This is the case, for example, when your Feature Service works on top of versioned data or tables not registered in a geodatabase. Installing Survey123 2.9 (Classic) would be the only solution possible unless you unversion your data or move your data into a non-versioned geodatabase feature layer. Other than the exceptions above, here are steps to enable supportsApplyEditsWithGlobalIDs in ArcGIS Enterprise for non-hosted feature services: Add all layers and tables included in your Feature Service into an ArcGIS Pro session. In ArcGIS Pro, run the Add Global IDs geoprocessing tool on all layers and tables included in your Feature Service.  It is very important that the new GlobalID fields are indexed using a unique constraint. If your feature service includes attachments: In ArcGIS Pro, run the Migrate Relationship Class geopocessing tool against attachment relationship classes included in your Feature Service. Overwrite your existing feature service to ensure that added GlobalID fields are now exposed through your Feature Service, by making them visible. If after doing all the above, you still cannot see that supportsApplyEditsWithGlobalIDs is enabled, then you can try the following: Overwrite your service definition ensuring this time that the sync capability is enabled. Enable archiving in your layers and overwrite your service definition. Technically, Archiving and Sync are NOT required but they will ensure all your GlobalID fields are properly indexed.  Once supportsApplyEditsWithGlobalIDs has been switched to true, you should be able to disable archiving and sync if you like. Please note that while sync will not cause any performance degradation or overhead on your feature service, archiving might.  If sync is enabled, you will not be able to disable queries in your feature layer. ... View more

Survey123 provides great support for working with USNG and MGRS coordinates. Since this is quite important to a number of you I thought I would write a specific blog post detailing the basics, but also some more advanced techniques. About USNG and MGRS USNG and MGRS are two commonly used grid-based reference systems used to unambiguously express geographic locations. They provide definitive advantages over using latitude/longitude coordinate notations, and addresses, so they have become the standard among different communities. MGRS for example is the NATO standard for referencing locations on the earth and it is often used by Emergency Response teams across the world.  USNG, which practically uses the same notation as MGRS, is widely used by Seach and Rescue and Emergency Response teams in the United States.  In fact, through  Directive 092-5, FEMA sets the United States National Grid (USNG) as its standard geographic reference system for land based operations and encourages the use of USNG among community partners Unlike the different notations used to express coordinates through a latitude and longitude pair (DMS, DDM, DD...), USNG and MGRS provide a unequivocal and more user friendly way to denote a location.  The following table illustrates differences between a few common notations: Map coordinate notation Example Degrees, minutes and seconds (DMS) 41°24'12.2"N 2°10'26.5"E Degrees and decimal minutes (DDM) 41 24.2028, 2 10.4418 Decimal degrees (DD) 40.446° N 79.982° W MGRS 11SMT818846858 USNG 11S MT 8188 6858 The fundamental issue with DMS, DDM and DD notations is that they are more difficult to read and people can transcribe the coordinates in slightly different ways which ultimate can lead to confusion. It may not be obvious to untrained field staff what the difference is between Decimal degrees and Degrees and decimal minutes. People may swap the order of the latitude and longitude values etc  In that sense, while MGRS and USNG may be just as cryptic at first to the untrained eye, they reduce the chances of getting the location wrong. It really does not matter where you put the spaces in USNG or MGRS, and there is no chance to flip latitudes and longitudes around... if you transcribe all the characters in order, you get the intended location right. For the reasons above and more, organizations are choosing MGRS and USNG as a common location language  and want applications like Survey123 to work naturally with them. Lets explore first how field users of the Survey123 field app can get the coordinates of a particular location. This is achieved through geopoint questions in the form, which support MGRS/USNG coordinate readouts in both the map preview and detailed modes. Reading and configuring coordinate formats in geopoint map previews Geopoint questions in your form will always display the coordinates of the selected location as well as a map in preview mode. In the survey shown on the side, for example, the current location of the device is shown in USNG notation, along with the estimated horizontal accuracy. The coordinate format to be shown in the map preview is configurable by the author of the survey as follows: Open your survey in Survey123 Connect for ArcGIS Once you are previewing your survey, go into the Settings tab Select the Map section Choose the coordinate format for the Preview Publish your survey and download into the Survey123 field app to test The available coordinate formats are: DMS: Degrees Minutes Seconds DDM: Degrees Decimal Minutes DD: Decimal Degrees MGRS USNG UTM/UPS Once the coordinate format has been selected in Survey123 Connect, it will apply to all geopoint questions present in that survey. End users cannot overwrite the coordinate format chosen by the author of the survey. Being able to quickly read a geopoint location in USNG or MGRS coordinates from the map preview in your survey can be very useful. For example, a Search and Rescue team that needs to communicate its own location or the location of an event over the radio can read the coordinates directly from the map preview. Reading and configuring map coordinates in the map detail mode The map preview as described above is intended to give the field user quick access to visualize a selected location and its coordinates.  By tapping on the map preview, Survey123 will bring the map into full screen, or detailed mode, and switch into interactive mode. When the Survey123 map is expanded to full screen, the map coordinate display is at the bottom. Note that the coordinates change as the map is panned around. For example, you can use the geosearch to find an address and immediately get the MGRS coordinates.  Or you can refresh your location using the device's GPS and get the coordinates again. If you would like to copy the coordinate readout into your clipboard, you can tap the bottom panel and that will switch the geosearch control at the top to map coordinate entry. Note again that in this mode, as you pan the map the coordinates will change dynamically in the geosearch control, from which you can easily copy them into your clipboard. The coordinate format for the detailed map is also controlled from Survey123 Connect as described in the previous section. Entering coordinates through geosearch using USNG and MGRS notation A second workflow when using the Survey123 app is that where the field user needs to define the location of an event, asset or observation, by entering its coordinates.This is done through the geosearch input in the detailed map view (map at full screen) of a geopoint question.  The geosearch input automatically accepts map coordinates in a number of formats: DMS, DDM, USNG, MGRS. Simply type in or paste the map coordinates and the map location will be set.  Persisting USNG and MGRS coordinates as GIS attributes Through the use of calculations, a survey author can configure a form to automatically persist geopoint questions in different map coordinate formats.  This is achieved through the XLSForm calculation column, which can take a geopoint object and extract its coordinates in USNG and MGRS for example.  This is what the syntax of the calculation will look like: type name label calculation geopoint location Location text usng USNG pulldata("@geopoint",${location},"USNG") hidden mgrs MGRS pulldata("@geopoint",${location},"MGRS") In the example above, the pulldata"@geopoint") function is used in the calculation column. Note that we first pass the question holding the geopoint object, which in our case is location,and then we indicate the property from the geopoint that we want to extract: USNG or MGRS. The expression in the calculation column is evaluated every time the location changes. So for example, if the user is in map preview mode and hits the GPS button, then the location is recalculated and with it its coordinates in the usng and mgrs questions.  The location could also change when the user manually sets a new location through the map detailed view, either by passing an address, a coordinate pair or simply panning the map. Note that in the example below I used a text type of question for the USNG coordinates.This means that the USNG coordinates will be displayed in the form to the end user. If you use a text type of question, you may want to flag the question as read-only, so users do not overwrite the coordinates.  In the case of MGRS, I choose hidden as the type, because I wanted to keep the calculation hidden from the end user of the survey, but still have this information be submitted as a GIS attribute to the ArcGIS feature layer. When using the pulldata("@geopoint") function you can control the precision of the coordinates you will get back. In the example I provide above, no precision parameter is passed. When no precision is set, the default precision for MGRS is 1 meter, and for USNG is 10 meters. For clarity, this is what USNG coordinates at different precisions will look like: 11S GZD and 100 km Grid Square ID, precision level 100 km 11S MS 8 1 precision level 10 km 11S MS 85 18 precision level 1 km 11S MS 851 184 precision level 100 m 11S MS 8513 1846 precision level 10 m 11S MS 85138 18466 precision level 1m When working on WGS84 and NAD83, MGRS and USNG are the same, although the notation for USNG typically  includes spaces in between the Grid designator, 100K meter grid and Northing and Easting. In the following example, I am passing a precision parameter with a value of 100, to get a 100 meter MGRS grid. I could have passed a value of 1, to get a 1 meter MGRS grid, or 0.1 to get a 10 cm grid. pulldata("@geopoint",${location},"MGRS",100) The pulldata("@geopoint") function supports many other properties as described in our Geopoints—Survey123 for ArcGIS | ArcGIS help topic. More on MGRS and USNG in ArcGIS There are a number of additional utilities within ArcGIS that will help you work with MGRS and USNG. Grid Overlay widget—Web AppBuilder for ArcGIS | ArcGIS  Add MGRS grid into your Web AppBuider web app. Add and modify grids—Layouts | ArcGIS Desktop  Add MGRS grid to your ArcGIS Pro map. Military Tools for ArcGIS | ArcGIS Solutions for Defense ... View more