To begin with
When conducting a field survey, shall we use location tracking?Location Tracking is a feature available across the entire ArcGIS Online (or ArcGIS Enterprise) organization, providing users the ability to record current and past locations.
For those who have never used it before or wonder what location tracking is and how to use it in ArcGIS Field Maps, recently,ArcGIS resource collection has been added to the ArcGIS Field Maps start app guide, so please refer to it.
On the other hand, management methods for location tracking features and access methods to data using ArcGIS API for Python are explained in Help as Managing ArcGIS applications - Configuring Location Tracking. Also, a notebook describing the same content is published as Configuring Location Tracking and Track Views For Your Organization, and that translated notebook is also published, so if you want to know how to access via API, please refer to them.In this blog, I will explain the points of implementing a sample notebook created on ArcGIS Notebooks on ArcGIS Online. The sample notebook is
published on GitHub in a refined form, so if you want to check the details more closely, please refer to it.Accessing track data acquired with Field Maps
- Querying data for a specific date and time and trying Spatially Enabled DataFrame
- Saving queried data as a feature layer Note: Although described in the above translated notebook, ArcGIS Online tracks are only stored for 30 days. This time, in order to use the data even after the storage period has passed, it is saved on ArcGIS Online as a feature layer.
- Note: Location tracking requires a paid
ArcGIS Developer Subscription
, which allows use of "ArcGIS Online Organization Plan (Development and Test Use Limited)". I conducted tracking with ArcGIS Field Maps using this plan.Figure: Sample Notebook
Point explanation
From here, explanations of the code written in the notebook will be central. If you are uneasy about how to use ArcGIS Notebooks on ArcGIS Online, I recommend reading the following article first.
Introduction to ArcGIS API for Python: Notebook for Beginners
The following is automatically generated when creating a new ArcGIS Notebook on ArcGIS Online and accesses the GIS module as is.
from arcgis.gis import GIS
gis = GIS("home")
1) Accessing track data acquired with Field Maps
When location tracking is enabled in an organization, a new item called Location Tracking Service is created. The track data acquired with Field Maps is updated to this service.The Location Tracking Service consists of two layers; the first layer Tracks stores each user's past location history, and the second layer Last Known Locations stores each user's latest location information using Tracker mobile apps such as Field Maps.
These layers can be easily accessed through LocationTrackingManager; however, this time we will use only the Tracks layer (
tracks_layer
).from arcgis.apps.tracker import TrackView
# Access LocationTrackingManager
location_tracking = gis.admin.location_tracking
# Tracks layer can be accessed from property
tracks = location_tracking.tracks_layer
2) Query by specifying date and time with location_timestamp field from Tracks layer
Since the time recorded in Tracks is UTC, caution is needed when creating query conditions.
(To convert from JST time to UTC time, set UTC time by subtracting 9 hours from JST time.)
This time, tracking was conducted with Field Maps between 15:30 and 19:00 JST on January 10, 2022.
Therefore, setting query conditions converted from JST time by subtracting 9 hours in MapViewer's where clause becomes as follows.
(location_timestamp BETWEEN timestamp '2022-01-10 06:30:00' AND timestamp '2022-01-10 10:00:00') AND (1=1)
Using the above condition, perform query with FeatureLayer.query(), then return value's property FeatureSet's property sdf as Spatially Enabled DataFrame (SEDF).
# Set query condition in UTC time
wh = "(location_timestamp BETWEEN timestamp '2022-01-10 06:30:00' AND timestamp '2022-01-10 10:00:00') AND (1=1)"
# Return value of FeatureLayer.query() is FeatureSet,
# FeatureSet property sdf can get SEDF
query_tracks = tracks.query(where = wh, out_fields='*').sdf
3) Save queried data as feature layer (Feature Layer)
Since Spatially Enabled DataFrame (SEDF) has a convenient method called to_featurelayer(), use it to save as feature layer.
# Save track of January 10, 2022 as FeatureLayer
lyr = query_tracks.spatial.to_featurelayer(title='tracks_20220110', tags='tracks')
As above, just saving as feature layer without writing code for 10 lines will suffice.
However, since that alone is not very interesting, here I introduce another approach using Pandas DataFrame for extraction, saving, and also trying graph display and map display code examples.
2)' Another approach; Extract conditions from Tracks layer with Pandas DataFrame
'Tracks layer' described in Help (tracks_layer) is FeatureLayer. Therefore first read data into DataFrame using Spatially Enabled DataFrame (SEDF)'s function 'from_layer()'.
# Read Tracks layer into Spatially Enabled DataFrame (SEDF)
import pandas as pd
sdf = pd.DataFrame.spatial.from_layer(tracks)
Next, specify date and time with location_timestamp field and read only tracking data targeted this time by Field Maps into another DataFrame.
# Read into DataFrame with corresponding BETWEEN condition of above Feature Layer
import datetime as dt
#2022-01-10
query_sdf = sdf[(sdf['location_timestamp'] >= dt.datetime(2022,1,10,6,30,0)) & (sdf['location_timestamp'] < dt.datetime(2022,1,10,10,0,0))]
2)'' Common operations with Pandas DataFrame; Graph display and map display
'Introduction to ArcGIS API for Python : Spatially Enabled DataFrame' edited by Sawatte Oboreru Asobi explains that Spatially Enabled DataFrame (SEDF) inserts spatial custom namespace into general Pandas DataFrame enabling spatial functions. Therefore common operations with Pandas DataFrame are also possible. The notebook published on GitHub (link planned) includes code tested for following operations but here only code is described due to length.
- Summary statistics calculation
#要約統計量の算出query_sdf.describe()
#特定の列を指定した統計値の算出query_sdf['speed'].mean()
#query 関数を使って平均速度以上のレコードの抽出over_sdf = query_sdf.query('speed >= 1.503300')over_sdf.describe()
その他にも、matplotlib でのグラフ表示（UTC時間）、マップにTracks レイヤーを表示も行っております。同様に、冗長になるため、ここでは実現するまでのコードのみ記載します。
- matplotlib でのグラフ表示（UTC時間）
# UTC から JST に変換してx:時間、y:スピード でグラフに表示してみます# 警告の回避のため明示的にDataFrameをcopy()query_sdf = query_sdf.copy()query_sdf['location_timestamp_jst'] = pd.to_datetime(query_sdf['location_timestamp'], utc=True).dt.tz_convert('Asia/Tokyo')# JST 時間に変換済みのx軸でグラフ表示query_sdf.plot(x='location_timestamp_jst', y='speed')
my_map = gis.map('埼玉県川越市')my_map #セルに地図を表示# Tracks レイヤーを追加my_map.add_layer(tracks)
3)' 別のアプローチ ; 計算したlocation_timestamp_jst を含むフィーチャ レイヤー（Feature Layer）として保存
3) と同様、Spatially Enabled DataFrame（SEDF）に、to_featurelayer() という便利なメソッドがあるので、それを利用して、フィーチャ レイヤーとして保存します。
※これはDataFrame でいろいろ操作した結果でも、フィーチャ レイヤーとして保存可能かを確認している操作になります。そのため、3) で保存したフィーチャ レイヤーと本質的に変わりありません。
# As a FeatureLayer, save the track from January 10, 2022 lyr = query_sdf.spatial.to_featurelayer(title='tracks_20220110', tags='tracks')
Finally
In this article,ArcGIS Developer Subscription is used to utilize the "ArcGIS Online Organization Plan (Development and Test Use Limited)" and try the next operation in ArcGIS Notebooks by explaining the points during the implementation of a sample notebook.
- Access to track data obtained with FieldMaps
- Query and confirm data for a specific date and time
- Try Spatially Enabled DataFrame
- Save queried data as a feature layer
For more advanced analysis of track data, see "Analyzing Tracking Data Obtained with ArcGIS Field Maps" on Learn ArcGIS. It introduces detailed analysis and visualization using ArcGIS Pro for local data storage, so please refer to it.
Also, for more advanced analysis with code, see "Tracker Scripts" on Esri's GitHub in the US. It introduces detailed analysis and visualization using ArcGIS API for Python, so please refer to it.
* When you need to retrieve your organization's track data and analyze it, don't forget to back up your data!
List of contents included in "Analyzing Tracking Data Obtained with ArcGIS Field Maps" (as of March 2022)
・Verify queries with tracking data(English article)
・Create line heart maps with tracking data(English article)
・Create animations using location tracking data(English article)
・Visualize aggregated tracking data(English article)
List of notebooks and scripts included in "Tracker Scripts" (as of March 2022)
Notebook;
・Quickstart Guide
・Basic Track Analysis
・Basic Track Analysis - PySpark
・Location Tracking Status
・Creating Track Lines
・Visualize Route Deviance
・Identify Inspected Buildings
・Find Dwell Times at Polygons
・Proximity Tracing
・Create an Aggregated Map Service
Script;
・Check Edit Location - README here
・Mirror LKL Layer - README here
・Polygon Cleanup Tracks - README here
・Generate Users Arcade Expression - README here
・Export Tracks From AGOL - README here
References
'Reference Articles' on Blog]
'Related Reference Information for This Article'
