Skip navigation
All Places > Survey123 for ArcGIS > Blog > 2016 > October
2016

With the 1.7 release of Survey123 for ArcGIS, we introduced a function that makes it easy to access data that is external to the survey- pulldata(). Using this function, can have a set of reference information distributed with the survey, making it easy for your respondents to enter information that doesn't change very often or is used as a baseline data point in your survey.

 

Preparing your data

The pulldata() function relies on your external data being stored in a CSV file and uploaded as part of your survey. This means that you can easily generate the data file from a wide variety of tools, including Microsoft Excel and ArcGIS Desktop. Place the CSV file in the media folder, the same place where all external files (video, audio) go.

 

Using pulldata() in your form

The full function and its parameters are pulldata(<csvfile>, <returnColumn>, <lookupColumn>, <lookupValue>).

  • csvfile is the name of the external data file without the .csv ending.  As mentioned above, it is stored in the media folder of the survey
  • returnColumn is the column in the CSV file with the value you want to be entered into the form.
  • lookupColumn is the column in the CSV file with the value you have from the form to find the correct row
  • lookupValue is the value in form that exists in the CSV file to find the correct row.

 

Example: Fish Size calculation

A common task in wildlife management is to conduct a survey of plant and animal specimens. Management of fish, in particular, factors in the size and weight of the fish to determine maturity and capture potential. The 'standard weight' of a fish can be roughly modeled against its length using a density function. The function is given as: 

Weight = a*Length^b.

We can compare this weight against the actual weight to provide a quick evaluation of fish health; under 80% of standard weight may indicate the fish is under stress.

 

The values of a and b vary between fish species. Rather than embed a set value for all fish in the survey itself, we can use an external data to store the values of a and b for each fish species, and then read the values using pulldata.

To implement this, we first have a table that stores the a and b values, along with an identifier for the fish.

codelabelab
BCFBlue catfish0.0000015533.278
BCRBlack crappie0.0000159592.968
BLGBluegill0.0000035923.304
BRBBrown bullhead0.0000633872.711

FishCalc.csv - table of standard weight parameters 


In our form, we then use the pulldata function to retrieve the values. We could use the pulldata function directly in the calculation, but for clarity, I'll use separate note questions to retrieve the values and then use them in a third question that calculates the weight:

 

typenamelabelappearancereadonlydefaultcalculation
textstationStation Location
datedateDate Sampledtoday()
begin repeatfish_repeatFish Data
select_one speciesspeciesChoose Speciesminimal
decimallengthIndividual length
decimalweightMeasured weight
noteaValue for a:pulldata('FishCalc', 'a', 'code', ${species})
notebValue for b:pulldata('FishCalc', 'b', 'code', ${species})
notestdWeightStandard Weight:round(${a} * pow(${length}, ${b}),3)
decimalshowknWeight / Standard Weightyes${weight} div ${stdWeight}
end repeat

FishEntry.xls - sample XLS form that uses pulldata to calculate standard weight for fish

 

The full form template can be downloaded to try. Thank you to the Georgia Department of Natural Resources, who provided the original values used in this form.


What type of data is a good fit for pulldata()?

The data used by pulldata() is published at the same time as the form. Because of this, the data used in pulldata functions shouldn't change too frequently. In the above example of estimating fish weight, this information would rarely change, if ever. The data can be updated by republishing the form with new data; but the field workers then need to update the forms on their devices. This suggests that the data that is used should be updated less than the typical cycle that the field workers would update the form. If you have your field workers check in every morning, for example, data that is updated once a day or less would be appropriate.

 

More uses for pulldata()

The example in this blog used the CSV file to store variables in a function; other similar uses include being able to estimate crop yield based on crop or seed type and acreage or total allowable development based on land use zoning and approximate lot dimensions. Another type of survey that benefits greatly from pulldata is a periodic inspection, such as bridge, hydrant or household address verification.  The CSV file can contain the data from the last time the inspection and then filled into the format a baseline for the updated inspection.  Some operations, such as post-disaster recovery, may include multiple rounds of surveys- a preliminary damage assessment, a more thorough building assessment after the immediate disaster, and post-disaster followups.  The data from previous surveys can accompany subsequent rounds of surveys as reference and a consistency check for the surveyors.  In addition to calculations, the pulldata function can also be used in a constraint to validate the value matches (or is within a range of) the existing value.

 

Learning more

The Survey123 for ArcGIS help has a section describing the use of pulldata.

We've also posted a tutorial video on CSV file and pulldata use in the Survey123 for ArcGIS playlist on YouTube.  

If you have any additional questions, please ask in the Survey123 for ArcGIS community in GeoNet!

This blog introduces Trek2There, a new mobile app from Esri Labs just published to the Google Play and iTunes app stores. You can think of this app as a smart compass that will always point to your destination.  Unlike an ordinary compass, always pointing to magnetic North, this app will use your direction of travel to tell you in what direction you should go to reach your destination. Best of all: you can integrate this app with Survey123!

 

Esri Labs projects are developed by Esri employees and are inspired by our interactions with ArcGIS users like you. Esri Labs projects are free to use but are not official Esri products. These projects do not go through the rigorous software development cycle so they are not holistically tested, documented or supported by Esri technical support.

 

Trek2There is particularly helpful when you are trying to find a location in an area where you cannot get driving directions. If you are working for example in a forest, or in a large agricultural property with not even dirt roads, using straight-line-navigation is often the most practical approach to arrive to a known location. 

 

Trek2There only requires two things to work:

 

  • First, the destination where you want to go provided in geographic coordinates (Latitude/Longitude).
  • Second, it requires you to be outside and moving so your device can fix your location and determine your direction of travel.

 

Let’s first try the app and then I will explain details of how to make it work with Survey123.

 

Trek2There, a quick first test

 

Start by downloading Trek2There onto your Android or iOS device.  First, you need to feed the app with the coordinates of your destination, which can you can enter manually or alternatively pass from a separate application. For now, we will use a simple web-based app to pass the coordinates. On your device, launch this web app and tap on the map to select your destination. Next you will see a new dialog open from which you can launch Trek2There.

 

Now that the app is running, start walking in any direction and shortly the arrow will rotate to show you the direction and distance to your destination.  Walking with caution and avoiding physical obstacles is all on you! 

 

As Trek2There is launched in your mobile device, please be aware that the arrow and distance will only calibrate correctly when you start moving (walk, drive etc). You need to be outside and moving before Trek2There will give you any meaningful information.  Trek2There uses your direction of travel to determine the direction in which you need to go. If you stand still, the arrow may temporarily point away from your destination, but things will go back to normal once you start moving again.

 

Trek2There custom URL scheme

 

Using the app is straight-forward as you can see. Launching it and passing the coordinates of your destination is what takes some thought but do not worry, it is not complicated at all. Trek2There can be launched remotely by invoking its custom URL scheme:

 

arcgis-trek2there://?stop=38.133453,-117.223455


If you copy the above string and paste it into a browser on your device, you will be able to launch Trek2There and set its destination at the provided latitude and longitude. You could send this URL as a link in an email, or programmatically invoke this URL from a custom developed application.

 

Launching native apps like Trek2There through a custom URL scheme -also often referred to as a deep-link- works fairly well across all platforms when you invoke the URL from a custom native application.  If you invoke the app from a web browser it will work as expected on Windows, iOS and Mac. On Android you may find issues launching the URL scheme from some mail clients and in the Chrome browser.

 

Working with Trek2There and Survey123 for ArcGIS

 

You can integrate Survey123 and Trek2There via the custom URL scheme described above. As you may already know, Survey123 forms can include a small subset of HTML. HTML formatting is used to make your questions look better, to organize notes etc  HTML can also be used to include links in your surveys and as you can now imagine... HTML links can launch Trek2There. The trick is to dynamically create these links, so you can control the coordinates passed to Trek2There.

To illustrate this concept Survey123 Connect includes the Trek2There Test sample survey.  This survey includes a choice list with all Esri offices worldwide.Using the pulldata function and a calculation, an HTML link is created dynamically so the coordinates of the Esri office in the selected country are passed into Trek2There. The actual list of offices may be out of date, but you get the idea... You may want to use a similar technique to add your own assets for example.

To open the Trek2There survey sample, install Survey123 Connect for ArcGIS in your desktop. Then click on New Survey and look for the Trek2There survey under the Samples category.  In the XLSForm you will see that a calculation is used to get the coordinates of the selected asset and dynamically generate a HTML link rendered in a note.  

First, the pulldata function is used to lookup from a CSV file the latitude and longitude of the selected element in the list. The CSV file is called EsriDistributors.csv and can be found in the media folder of the survey. To learn about pulling data from CSV files  in Survey123,  I recommend you look at this help topic.
pulldata('EsriDistributors','Lat','Country',${Country})
Both values are persisted in calculate questions so we can reuse them later on. Then these coordinates are used to generate an HTML link using a calculation in a note question.
 concat('<a href="arcgis-trek2there://?stop=',${Lat},',',${Lon},'">Launch Smart Compass</a>')
Following the pattern illustrated above, you can now use your own lists of assets and integrate Trek2There  into your own surveys. Since Trek2There is only available for Android and iOS, keep in mind that you will not be able to test the integration unless you download the survey into your phone or tablet.
.

Invoking Trek2There programmatically

In a web app or website, simply create an HTML link:

<a href='arcgis-trek2there://?stop=38.133453,-117.223455'>Launch Trek2There</a>

In QML AppStudio for ArcGIS:

Qt.openUrlExternally('arcgis-trek2there://?stop=38.133453,-117.223455')

In Java, using Android's Intent:

PackageManager manager = mContext.getPackageManager();

Intent i = manager.getLaunchIntentForPackage(
"com.esri.survey123");
i.setData(Uri.parse("arcgis-trek2there://?stop=38.133453,-117.223455"));
startActivity(i);

.

 

Customizing and Contributing to Trek2There

 

The source code of Trek2There is shared under the Apache 2.0 License, so if you like you can take this app and make it your own. Perhaps you may want to embed this functionality within your own app, or re-brand Trek2There… your choice! You can find the source code at https://github.com/Esri/Trek2There.  Esri welcomes contributions from anyone and everyone. If something is not working for you, create an issue... if you actually know how to fix existing problems or want to contribute, please follow these contributor guidelines.

 

You may wonder, how this app was built. Just like Survey123, Trek2There was developed using AppStudio for ArcGIS.  AppStudio for ArcGIS is quite a powerful product for building your own cross-platform native GIS apps. If you are a Javascript developer, transitioning into AppStudio for ArcGIS is relatively easy. If you like tinkering with Python code in ArcGIS, becoming an AppStudio for ArcGIS efficient developer is perfectly within reach.  With AppStudio you write your code once, and then you can easily compile your work so it runs on desktops (Windows, Mac, Ubuntu Linux) as well as tablets and phones (iOS, Android, Windows).

 

The easiest way to download the source code is by using AppStudio for ArcGIS Desktop Edition.  Once installed, click on New App and search for Trek2There under the Enterprise category.  You can look at the code, modify it and run it in your Mac or Windows development machine. If you have an AppStudio for ArcGIS Standard license, then you will be able to compile your code for Windows, Mac, iOS, Android and Ubuntu Linux. The AppStudio for ArcGIS GeoNet Group is quite active. A great place to address technical questions.

 

You see, even if Trek2There is not an app you ever use, if may be worth looking into its source code to get familiar  with AppStudio. After all, just like Trek2There, the source code of the Survey123 field app is also available with AppStudio for ArcGIS. Can you imagine adding your own features into Survey123? Hmmm, that may be a good idea for a separate blog post.