Strange date output from ArcGIS Online feature layer using ArcGIS API for Python

809
5
Jump to solution
05-01-2020 01:24 PM
TransCanadaTrailTCT
New Contributor II

I'm trying to extract field value from an ArcGIS Online feature layer using the ArcGIS API for Python.

In ArcGIS Online, the date is displayed like 10/9/2018 10:30 AM (Field type = DATE)

From the Spatially Enabled Panda DataFrame, the date looks like 1.5390954e+18.

When I try to copy the date in a File Geodatabase (InsertCursor) from the DataFrame into a DATE field, I'm getting this error:

RuntimeError: The value type is incompatible with the field type.

How can I convert the 1.5390954e+18 formatted date to a proper date format to be insert in a DATE field?

0 Kudos
1 Solution

Accepted Solutions
RandyBurton
MVP Regular Contributor

It appears to be a unix timestamp in nanoseconds.

Scientific notation
1.5390954e+18
+18 move decimal 18 places to right

Timestamp in nanoseconds
1539095400000000000

Converts to UTC as:
Tuesday, October 9, 2018 2:30:00 PM
‍‍‍‍‍‍‍‍‍

If you convert the UTC for your timezone, it should match.  AGOL usually stores the date as timestamp; but I would have expected milliseconds.

from datetime import datetime
from dateutil import tz
from pytz import timezone

ts = float("1.5390954e+18")

UTC = datetime.utcfromtimestamp(ts // 1000000000).replace(tzinfo=tz.tzutc())
print UTC
# 2018-10-09 14:30:00+00:00

east = UTC.astimezone(timezone('Canada/Eastern'))
print east
# 2018-10-09 10:30:00-04:00

View solution in original post

5 Replies
RandyBurton
MVP Regular Contributor

It appears to be a unix timestamp in nanoseconds.

Scientific notation
1.5390954e+18
+18 move decimal 18 places to right

Timestamp in nanoseconds
1539095400000000000

Converts to UTC as:
Tuesday, October 9, 2018 2:30:00 PM
‍‍‍‍‍‍‍‍‍

If you convert the UTC for your timezone, it should match.  AGOL usually stores the date as timestamp; but I would have expected milliseconds.

from datetime import datetime
from dateutil import tz
from pytz import timezone

ts = float("1.5390954e+18")

UTC = datetime.utcfromtimestamp(ts // 1000000000).replace(tzinfo=tz.tzutc())
print UTC
# 2018-10-09 14:30:00+00:00

east = UTC.astimezone(timezone('Canada/Eastern'))
print east
# 2018-10-09 10:30:00-04:00
TransCanadaTrailTCT
New Contributor II

Thanks Randy. You nailed it. It's working now. I never thought to get a date in this format.

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

TransCanadaTrail TCT‌, I just did some querying of feature layers using the ArcGIS API for Python, and I am not seeing the date behavior you are seeing.  It may be your issue is with the spatially-enabled data frame.  Can you show a code example of how you are creating the data frame?

TransCanadaTrailTCT
New Contributor II

I'm not sure if it's the way that I'm querying the feature layers, because I'm using the same process for other layers and it's working well. The only thing about this layer is the layer have not been created directly from ArcGIS Online, but it's an upload from ArcGIS Pro (SDE Geodatabase). Here is the code

from arcgis.gis import GIS
from arcgis.features import GeoAccessor, GeoSeriesAccessor
import pandas as pd


item = gis.content.get("id_item")
agol_layer = item.layers[0]

records = pd.DataFrame.spatial.from_layer(agol_layer)
0 Kudos
JoshuaBixby
MVP Esteemed Contributor

The code you are using is pretty boilerplate, so nothing seems odd there.  Plus, you mention other layers work fine.  At this point, it seems to be something in the translation/upload from Pro to AGOL.  It might be worth opening a case with Esri Support to dive deeper.

0 Kudos