Citizen science street level imagery and mapping with Mapillary
The idea behind Mapillary is a simple but powerful one: Take photos of a place of interest as you walk, bike, drive, or however else you move across the landscape using the Mapillary mobile app. The app takes photographs automatically, which you then upload to the Mapillary database. Once there, they are wonderfully combined into a ground photo view that is a bit like Google’s StreetView, showing you a digital virtual path of how you traversed the landscape. Mapillary is part of the rapidly growing crowdsourcing citizen science movement, which seeks to generate “volunteered geographic information” content from ordinary citizens. In fact, Mapillary is helping to generate critical infrastructure and natural resource inventory in places around the world that have no national mapping agency or local GIS data. Mapillary is therefore helping to create a data-informed citizenry that can more effectively plan resilient, safe, and thriving communities. As part of the growing set of artificial intelligence tools, Mapillary can also automatically extract map features from images—light poles, trees, benches, curbs, and so on. While I have written about Mapillary in the past, and regularly include Mapillary in my field workshops, in this essay I wish to provide an update for people already familiar with this tool, and introduce new people to these exciting capabilities.
Mapillary is much more than a set of tools–it is a community, with its own MeetUps and ambassadors, and it is an Esri partner. At the time of this writing, over 710 million images have been contributed, covering over 8.6 million kilometers. Currently, the site’s leaderboard shows that the top 50 users have submitted over 1 million images each, with the leader at 18 million images and nearly 300,000 km. I currently have submitted 2,400 images covering 24 km. I have a long way to go: Ah! More fieldwork!
There are many uses for Mapillary in education, and I have explored all of the following with students at the secondary, university, and informal education (libraries, museums, after-school clubs) level over the past few years in a wide variety of settings and institutions. First, I use Mapillary to help students explore places of interest from thousands of users around the world. The Mapillary map page linked to images allows instructors and students to play sequences of images in a flowing video style that provides a powerful immersive experience of thousands of landscapes and places around the world. What clues do the vegetation, land use, building type, weather, and place names give about the climate, ecoregions, biomes, history, and culture of the area? These images and maps can be powerful sources of inquiry, prompting investigations using other sources and drawing on content knowledge in history, environmental studies, geography, earth science, and even language arts, as I explain in this video.
To examine the map and images, from the main page, under the Imagery tile, select “Explore coverage.” A global map will open with the Mapillary data collected shown in green. For example, if you zoom to Melbourne Australia, you will see a large circular feature that I collected in Royal Park, shown below. You can see the photographs I took on a fine late winter day as I was walking to the University of Melbourne to teach a GIS workshop. Look at those fantastic Australian trees! You can also tick the “Play” button in the image to “travel” around the circle as I did, in your case, virtually, using the images. Note how each image indicates where on the map it was captured and what direction from straight ahead I took it. You can also play the sequence in full screen mode with the map in the corner, turn on object detection,....
Try walking along my route through Royal Park in Melbourne by clicking on the forward and backward arrows in the above immersive view collected with Mapillary!
Example set of Mapillary points and images I collected in Royal Park in Melbourne Australia.
Second, I use Mapillary with students in the field to create data, and encourage faculty reading this to do the same with your own students. You will need to register for a free Mapillary account to do this. After obtaining your free account, download the Mapillary app for your phone. You can use this Mapillary app to create photos and maps to document a field trip to your local wetland, rainforest, prairie, or urban neighborhood, and if you cannot get off campus, use the tools to walk every pathway on campus. The Mapillary app is free and fun to use, and can spark discussions such as “how does the app determine my location?” and “how does the app know what direction I am pointing my phone?” As you collect tracks, they will be visible on the web map along with the global community’s tracks, and also, your own tracks will be visible as “uploads” on your phone app (shown below).
My set of Mapillary images currently online.
For students who become familiar with creating Esri ArcGIS Online maps and Esri Storymaps, Mapillary images can be embedded in these types of multimedia maps. Start simply by downloading one image from Mapillary, such as mine, here, in Melbourne, using the Download Image while logged into Mapillary: https://images.mapillary.com/VdjXFuVzWX9Y1se9qN89rA/thumb-2048.jpg. Add this into a Map in ArcGIS Online. Make a story map and experiment with this image. Or, link to the image online. See below for example.
Example Mapillary image embedded in a map note in ArcGIS Online. Try it yourself with my image: https://images.mapillary.com/VdjXFuVzWX9Y1se9qN89rA/thumb-2048.jpg.
Mapillary provides map data as a subscription and downloads are requested through Mapillary for Organizations. Mapillary for Organizations is a workspace that anyone with a Mapillary account can create. Within an organization, there can be multiple individual accounts. Thus, it provides a way to organize capture projects and request data for your area of interest, rather like putting together a team for mapping purposes. To use data for educational purposes, you should focus on data that you or your students have collected.
You can download your Mapillary map data and bring it into ArcGIS Online. The data gets extracted as a GeoJSON file, which you can add to your ArcGIS Online map as I will explain below. Go to your sequence in the web mapping app > click the three dots in the bottom right corner, as shown below:
The 3 dots on your collected track that allow you to download your data for use in ArcGIS.
After clicking on the 3 dots, select: > Advanced options" > "Download lines" to get the trace of your track:
Downloading your Mapillary track.
After selecting Download Lines, a GeoJSON file will open up in a web browser tab. Right click somewhere on in the white space where there is no text, and > "save as." Change the filetype to "all files" and then add the ".geojson" extension to the file name. Alternatively, in a web browser where you are displaying the GeoJSON file, copy all of its contents, paste it into Notepad, and name the file appropriately, such as Melbourne_track.geojson. Your system will likely add “.txt” to the end of the file name. If so, rename the file and take off the “.txt” extension.
Once you have your GeoJSON file, go to ArcGIS Online > Add Data > Add from file, and point to your geojson file. Symbolize the tracks in the area on user name so you can determine which track is your own, as I did, below:
Alternatively, you could add your GeoJSON file as a file to your ArcGIS Online content, thereby creating a feature service from it. Then, you can add it to your map, and filter on the user name to only see your own track, as shown in this web map of mine, here. Now that it is a feature service, it is an even more powerful layer than simply a map element, that you can now use as input to your spatial analysis tools, such as buffer, overlay, and more.
Next, for enhanced geo-visualization, try bringing your Mapillary track into the 3D scene viewer in ArcGIS Online, as I did here; see below.
Mapillary track in a Esri ArcGIS 3D scene.
Close up of Mapillary track in 3D scene viewer using a hiker as the symbol.
Want to dig deeper? You can even extract features from the images; see more information here on map features, and the help page about how to bring data to ArcGIS Online. When you do so, you are using Artificial Intelligence in action! See an example of features in the benches example below, along with my track. The benches have been extracted with the Mapillary algorithms from the images on my track.
Example set of Mapillary features in ArcGIS Online. Mapillary uses computer vision, a form of artificial intelligence, to automatically identify and extract map objects like these benches.
Next, use these guidelines to start building story maps with Mapillary sequences. Essentially you will get the embed code for the Mapillary image, its thumbnail, and the geographic coordinates. As they are working through the procedures, show them this example set of stories and this story map of a refugee camp for inspiration.
For students who become familiar with using Esri’s Web App Builder, you might also encourage them to try the Mapillary Widget, which allows for the viewing of Mapillary street-level images.
I encourage you to use these Mapillary tools to enhance your fieldwork, teach about apps, Web GIS, and crowdsourcing, and to improve the spatial thinking of your students.
For more information, see my related essay on the Mapillary blog.