Tracker for ArcGIS - tracks/time stamp configure to local time zone?

[kmsmikrud]

Hi,

We just got our first tracks recorded and I can view the tracks in the Track Viewer and see the corresponding point attributes along with the time shown in local time zone although this must be rendered based on my computer settings for the local time. However when I view the tracks feature layer I see the time must be recorded in UTC. I just went thru the process to configure and publish other feature layers using Pro to AGOL to have the local time zone. Is this possible to configure for the tracks as well? Otherwise it looks like we will have to create additional fields for the conversion to have the correct local date and time for the survey tracks.

Thanks,

Kathy

[kmsmikrud]

I also now see a field in the tracks feature layer view named 'location_timestamp' and I was hopeful this field would be in our local time zone. It has the right date compared to the created_date but the time is not correct for when the tracks were recorded.

How does the 'location_timestamp' field differ from the 'created_date' field?

For example one record in a tracks layer shows.

location_timestamp: 3/9/2020 11:21:14 PM  (correct date, but time is 8 hours off should be 3:21:14 PM)

created_date: 3/10/2020 12:46:50 AM

My hope is to create a field in a downloaded copy of the tracks to have values in our local time zone. I would also like to configure the pop-up for the feature layer service. I have a view of the tracks but it still does not allow me to create to a new field in AGOL which I thought was mentioned in another post as a work around.

Thanks,

Kathy

[Anonymous User]

Tracks are always recorded and stored in UTC as this is best practice for storing datetime information. This is not configurable. Can you explain why you would like to use local time as opposed to UTC? I believe Pro and other clients can display in UTC in local time or apply a specific offset.

The location tracking layers do not support additional fields since it's a managed schema and service.

The location_timestamp is the time when the location was recorded on the device, the created_date is when that feature was inserted into the feature service (when it was uploaded).

You can export the tracks directly from AGOL to a CSV file which can be used. You can also export to shapefile but the time is not preserved (limitation of the shapefile spec).

[kmsmikrud]

Thanks Aaron for the response.

I have other feature layers configured to the local time zone for use in the Collector app.  We then have scripts to query the feature layers post-survey to both calculate field values based on survey data and then download records based on the most recent survey data. Having the dates configured with local time zone is preferable as it just makes more sense to the surveyors in viewing their data than translating from UTC and for the local dates to be stored in the downloaded files. We have also noticed that with UTC during the workday we can values for 2 different days in UTC versus 1 survey day in local time which adds errors to the queries.

Adding the tracks layer to our current script post-survey process is the same, just makes more sense to look at the date/time in local time zone to be able to download that day's survey data. Since we can't add fields to the tracks layer view, we will have a translate UTC to local time zone in the script before querying the tracks layer to download the most recent survey data and then add in a field after the data is downloaded I guess. The feature class to feature class conversion tool does export the tracks layer as a feature class using Pro/arcpy so the fields are all there. I know you mentioned shapefiles but feature class downloads do work. Folks in our office have most commonly used handheld GPS to collect survey tracks and import into GIS. They are used to having the local date/time as the field not UTC or having to translate in between. We work in natural resources and this has been a very common workflow. I'm sure as more folks start using Collector and Tracker there will be similar requests.

I do have the tracks feature layer added into Pro currently and it shows the tracks recorded. It recognizes the Location Timestamp as the Time Field but nothing else is configurable as if its just locked even with an advanced pro license.

Also is there documentation in the Tracker for ArcGIS resources for all the fields created and what they are? I've seen some of your replies to other posts that some fields are only populated by Android devices and accuracy nuances but it would be great if it was just documented in one place.

Thanks for your time,
Kathy

[Anonymous User · Accepted solution]

Thanks for the additional context. 



One thing to be aware of regarding using the export tool, is that pagination is not supported in the AGOL implementation of the location tracking service (see bottom of this page). This means you can only get up to 28,800 features which is the maxRecordCount. Track Viewer uses a custom time-based paging algorithm to fetch users tracks. There are plans to support pagination in the near future.



I see what you mean about the time being locked in ArcGIS Pro.

Thanks for feedback, we have some documentation here about the location tracking service. I added an issue in our backlog to add more information specifically regarding the fields in the next couple months or so.

[kmsmikrud]

Thanks Aaron for the info info on the pagination and limit but hopefully we should be okay exporting after surveys. Thanks also for the documentation link!

[MelanieWawryk]

How are we supposed to extract the tracking layer if you only allow the first 28800 records? We used the tracking layer in Collector for a few years without issue. We were then forced to pay for tracker and now we can only extract the first 28800 records? This needs to be fixed right away. We need to read the features into our geodatabase to cover off legislative requirements for road and sidewalk patrols and use FME to extract the features each day. It was working fine until we hit the magic 28800 features limit. Stop making half baked products, your customers rely on actually using them for more than just product demos.

[Anonymous User]

Hi Melanie,

This was actually documented as a known limitation since location tracking in AGOL was supported. It has been addressed as of June 30th 2020. The most efficient way to export features is using the Export API. I'm not sure if FME supports it but this is how you can use it in the ArcGIS API for Python.

import arcgis

gis = arcgis.gis.GIS("https://arcgis.com", "admin", "password")

item = gis.admin.location_tracking.item

csv_item = item.export("tracks_today",

                       export_format='CSV',

                       parameters={

                           "layers": [

                               {

                                   "id": 0,

                                   "where": "created_date <= '2020-07-02 00:00:00' AND created_date >= '2020-07-01 00:00:00'"

                               }

                           ]

                       }

                       )

csv_item.download(save_path=".", file_name="test.csv")

You can also export using the item details page for a track view or the main service.

[MelanieWawryk]

Thanks that worked in FME as well. I caught up the missing days by adding in a where Statement created_date >= '2020-07-02 00:00:00' on each of the readers. I now have the readers with a where statement of created_date >= CURRENT_DATE() and run it each day to grab the tracks.

I get that it was a known limitation but why replace a working product with one with such a big limitation and only have it fixed a year later. I wouldn't have thought it to be an issue because AGOL only keeps 30 days worth of tracks soI thought that that was the limiting factor so as long as we processed within 30 days we should be fine. It would be nice to wait until the new product works fully so people who rely on the old product aren't left trying to figure out what no longer works and workarounds so patch up processes. Or at least allow them to run in parallel until the bugs are worked out of the new one. I appreciate the innovation but it is hard to constantly have to discover how things no longer work or don't work as expected.

Thank you for your quick reply and solution.

[NickWhite1]

Thanks for this clarification.

I'm using data from a different timezone. I just realized I should have saved in CSV format because I'm only getting the date and the timestamp shows as the previous day. I cannot do the conversion without time. Tracker data expires after 30 days so I can't go back for it now.

ESRI is doing excellent stuff with temporal data and analysis but I would like to see better recognition of users that may have non-local data from a different time zone. It is the future!