Select to view content in your preferred language

Out of Bounds?

4139
13
11-22-2020 07:48 PM
GlenShepherd
Esri Contributor
12 13 4,139

Setting boundaries for survey submission using JavaScript in Survey123

 

The Premise:

Let's consider workflows when survey submission should be dependent on the user's location inside/outside of a boundary and where we might want to control these submissions using our Survey123 form...

  • Working on private property
  • Field teams assigned to different land parcels
  • Collecting data in proximity to hazards
  • Conducting surveys of assets for a particular city council

One method of quality control might be to restrict submissions based on where field workers are located in regards to specific geometry (i.e. within council boundary, 10m from a major freeway, on private property, etc...)

This can be implemented by taking advantage of writing and referencing custom JavaScript functions to automate spatial queries. @Ismael Chivite's blog post, "Extending Survey123 smart forms with custom JS functions" introduces us to using the pulldata() function to implement JavaScript functions in our XLSForm by storing our JavaScript files in a dedicated 'scripts' folder. Furthermore, the JavaScript Sample survey in Survey123 Connect provides us with examples of scripts that can query the geometry of our feature services:

sampleSurvey.png

This workflow will utilise a modified version of the "url_requests.js" script from the above sample survey. To follow along with this example, you'll need the following:

  • Survey123 Connect Desktop Application (download here)
  • "OutOfBounds.xlsx" and "OutOfBounds.js" files from the .zip attached to this blog
  • A polygon feature layer available through your web portal (ArcGIS Online or ArcGIS Portal)

In this example, we're simply going to indicate whether a Survey123 mobile app user is inside or outside of a set boundary (polygon feature layer) using an intersect query, and limit their submission capability accordingly...

 

The Workflow:

Create a new survey from file, using the "OutOfBounds.xlsx" form mentioned above.
To store our script(s), we'll want to create a folder within our Survey project location titled "scripts", as follows:

Please Note: Later, when we add a pulldata("@javascript") calculation to the survey, Survey123 will look to the "scripts" folder for the name of the JavaScript file we've saved. The "scripts" folder must be manually created by you, the survey author.Please Note: Later, when we add a pulldata("@javascript") calculation to the survey, Survey123 will look to the "scripts" folder for the name of the JavaScript file we've saved. The "scripts" folder must be manually created by you, the survey author.

We'll start by adding a "geopoint" question to our survey.

Then, add a "hidden" question and place the service URL of your polygon feature layer in the Default column.

Furthermore, if our feature service is secured (i.e. not public), we will want to include another "hidden" question to incorporate the pulldata("@property","token") function and provide a token which our JavaScript function will require to access the service:

Table1.png

We're going to save the following script (attached; "OutOfBounds.js") into our "scripts folder and reference it from within our survey form to carry out the intersect query and return a string relevant to whether the geopoint lies within the feature service polygon boundary or not.

Please Note: a double forward-slash (//) denotes a non-functional comment in JavaScript. Near the bottom of this script, you will see comments where we can edit the text strings that warn the user whether they are "Within Boundary" or "Out of Bounds"

 

// "echo" checks that JavaScript is working correctly
function echo(input) {
	return input.toString();
}
// Query feature service and determine if the geopoint intersects

function featureByLocation(layerURL, location, token) {
	let outValue = "";

	if (layerURL == null || layerURL === "" || location == null || location === "") {
		return location;
	}

	let coordsArray = location.split(" ");
	let coords = `${coordsArray[1]},${coordsArray[0]}`;

	let f = "f=json";
	let geometry = `geometry=${coords}`; 
	let geometryType = "geometryType=esriGeometryPoint";
	let inSR = "inSR=4326";
	let spatialRel = "spatialRel=esriSpatialRelIntersects";
	let outFields = "outFields=*";
	let returnGeometry = "returnGeometry=false";
	let returnCount = "returnCount=1";
	let parameters = [f,geometry,geometryType,inSR,spatialRel,outFields,returnGeometry,returnCount].join("&");
	if (token) {
		parameters = parameters + `&token=${token}`;
	}
	let url = `${layerURL}/query?${parameters}`;

	let xhr = new XMLHttpRequest();
	xhr.open("GET", url, false);
	xhr.send();
	
	if (xhr.readyState === xhr.DONE) {
		if (xhr.status !== 200) {
			return "bad request: " + url
		} else {
			let response = JSON.parse(xhr.responseText);
			if (response.error) {
				return "There was a problem with the query"
			} else {
				if (response.features[0]) {
					outValue = "Within Boundary";			// Positive intersect "string"
				} else {
					return "Out of Bounds"					// Negative intersect "string"
				}
			}
		}
	}
	return outValue;
}

 

In a 'note' question, we add the following function to the calculation field:

pulldata("@javascript","OutOfBounds.js","featureByLocation",${boundaryurl},string(${droppoint}),string(${tokenpull}))

  • "@javascript" - looks for .js files in the scripts folder
  • "OutOfBounds.js" - specifies the name of which .js file to pull
  • "featureByLocation" - specifies which function to use from the script (there's only one in this example)
  • We then populate the parameters of the function (specified in Line 7 of the attached script):
    • ${boundaryurl} - our hidden question containing the layer URL
    • string(${droppoint}) - stringified geopoint question providing user location
    • string(${tokenpull}) - stringified hidden question providing the feature service token

Table2.png

Now, once the mobile device has captured a location for the geopoint question, the JavaScript function will return either "Within Boundary" or "Out of Bounds" depending whether the user's location is intersecting a feature in the feature service.

Furthermore, we might like to add extra questions for quality control of our field team:

  1. Prevent submissions if the user is outside of the boundary
  2. Add a dateTime question for users within the boundary

Please Note: These have already been added to the attached example survey form "OutOfBounds.xlsx"Please Note: These have already been added to the attached example survey form "OutOfBounds.xlsx"

 

The result for the end-user in the Survey123 mobile application may look similar to the following:

OutOfBounds.gif

 

Other quality control considerations and notes:

  • Make the geopoint question read-only.
  • Add a web map with the polygon ‘boundary’ feature layer as Linked Content in Survey123 Connect and use it as the Basemap for the GeoPoint question.
  • The "OutOfBounds.js" file in the attached .zip contains further comments regarding the specific JavaScript functions in the script example above and what they do.

 

13 Comments
MichaelBruening
Frequent Contributor

This looks very useful, but will this work within an offline environment when the user has little or no access to a data connection via cellular or Wi-Fi?

GlenShepherd
Esri Contributor

Hi @MichaelBruening

Thanks for taking the time to have a read.
No, in a disconnected environment, we wouldn't be able to successfully query the web service...
Great question though, and a use-case that would be quite sought after as well. Certainly food for thought on a "Part 2" to this blog post...

An alternative here, might be to create offline areas for a web map that includes our boundary polygon feature layer and then add this web map as linked content in the survey. Then the Survey123 mobile worker would at least be able to visualise the boundary when placing a geopoint.

This workflow is briefly illustrated here by another Survey123 user: An Offline Map in Survey123
And expanded upon in Survey123 documentation: Prepare basemaps for offline use

 

Thanks,

Glen.

MichaelBruening
Frequent Contributor

Hello Glen, this is still very helpful in areas where we have a connected environment for some of our projects. Thank you for posting and I have created some offline areas for the crews to use within Survey123; which they appreciate. This application has came a long way and continues to get better and better with each update. 

WilliamLawrenceCoG
Occasional Contributor

@GlenShepherd,  This is great stuff. The only problem I am having is when previewing the app in Connect, the boundary logic works fine, but when publish it,  and I open the shared link on my phone in Chrome, it doesn't. What am I missing? (sorry this is my first post here and I'm learning the lingo as I go). I learned that this is a known issue when the survey is made public, it disregards the Javascript and ignores it. ESRI stated the fix is scheduled to be implemented in the next release.

LeonardBarnhill
Frequent Contributor

Would it be better if the user was notified at the start of the form to save time and prevent the user from completing the form only to find out at submission that he/she was in the wrong place? Visualizing the boundary as defined above is an option but a warning message would get the users attention.

Could this be applied differently by installing the boundary map on the device as if offline then have the Survey form check the boundary before completing the form - maybe triggered when the user checks the date or time at the start of the form?

WilliamLawrenceCoG
Occasional Contributor

I think that is what this process does. When the user it out of the polygon, it displays a message that they are not in the area and prevents them from submitting the survey. You could also hide any fields based on the result of the "Are we within the boundary?" question.

GlenShepherd
Esri Contributor

@WilliamLawrenceCoG, Thanks for giving the workflow a crack!

So your survey is shared publicly and configured to open via a browser rather than the Survey123 mobile app for public data consumption, is that correct?
You're right, this isn't an isolated issue, I've encountered the same when making the survey public and attempting to submit via a browser on a mobile device. This is an enhancement currently under consideration for development:

ENH-000135167 - "Enable javascript function on public surveys"

The limitation is currently described here: https://community.esri.com/t5/arcgis-survey123-blog/extending-survey123-smart-forms-with-custom-js-f... 

There is also an idea submitted for this which you may wish to Kudo and follow in order to generate further attention to this issue.
Sorry I haven't got a better workaround for this one at this stage!

GlenShepherd
Esri Contributor

@LeonardBarnhill ,

You could certainly set the appearance of the geopoint question to be minimal/un-editable by the end user and rearrange the "Are we within the boundary" Note question to be the first row in the survey. That way users would be notified from the get-go.

The example above is a simple example but I think it displays what you've asked, the warning message is displayed below the geopoint question. In practice, I'd add to this structure with any follow-up data collection questions after the user has been notified that they're "Within Boundary".

This workflow will not work in an offline setting at this stage, as the javascript function requires an internet connection to query the service.

WilliamLawrenceCoG
Occasional Contributor

@GlenShepherd , Thank you for explaining how to support the IDEA. I didn't know you guys called them Kudos. Do you have any other suggestions for doing something like this, as in letting the user know if they are within the boundary or not? Even something other than Survey123. Sorry, I am not in our GIS department, I am just brainstorming here.

GlenShepherd
Esri Contributor

Hi @WilliamLawrenceCoG ,

In a shared publicly scenario, keeping the current limitation in mind, it looks like your best option might simply be having the geopoint question established with a map that displays the polygon boundary and perhaps add a note for the end-users such as "if your location is not within the boundary indicated within the map, it will not be retained".

Then you may wish to use an intersect tool in either ArcGIS Desktop or ArcGIS Online to filter the collected data according to whether it is within the boundary polygon(s).

 

WilliamLawrenceCoG
Occasional Contributor

@GlenShepherd , Thanks for the idea. I will run it by the GIS department to see what they can do. I am really hoping this functionality gets implemented this year as it would be very beneficial for multiple users.

ThorstenHohenstrater
Regular Contributor

Nice Work Glen!

bsklaohfl
Regular Contributor

@GlenShepherd thank you for this documentation. Can I use this workflow with a geoshape vs geopoint? Will I need to modify the geometry in the Javascript?

Thanks in advance,

Brooke

About the Author
Esri Australia Client Success team member