|
POST
|
Hi Aaron -- hope all is well. Is there any update regarding the documentation on this issue? We're now seeing a much larger lag/delay in some instances and trying to determine if this is an app setting, device/OS type, connectivity issue or if there's something on the ESRI side of things that may cause this? Recently we noticed a specific instance with a 4 day delay between location_timestamp and created_date:
... View more
07-01-2020
11:58 AM
|
0
|
1
|
961
|
|
POST
|
EDIT: would a single user logged into ESRI Tracker app across multiple devices simultaneously be a source to any of this? I'm still skeptical even that would produce the same features with the exact same location_timestamp values but perhaps there's another process that is being affected by the simultaneous logins. Aaron, To reiterate, I'm not worried about the duplicates issue only because I'm handling them in an after-the-fact process that I can identify these instances. But wanted to provide more info to you and your team. Let me know if you'd like me to open a premium ticket to setup a more formal interaction outside of GeoNet, I would be able to remote session with you all if you need. I now see some triplicates. [
{
"attributes": {
"objectid": 886296,
"globalid": "cf208275-3d97-4f07-aa59-89dfe72d8f0a",
"activity": 0,
"altitude": -22,
"app_id": "tracker",
"battery_percentage": 87,
"battery_state": 1,
"course": 96.2351684570312,
"device_id": "04ba83a0-09b0-4b4a-b87e-2be331924053",
"floor": null,
"horizontal_accuracy": 3.90000009536743,
"location_source": "fused",
"location_timestamp": 1592434582000,
"session_id": "7aa745ba-42be-4b23-acc8-11a65d511740",
"signal_strength": null,
"speed": 26.8384494781494,
"vertical_accuracy": 100,
"created_user": "zzz",
"created_date": 1592434625398,
"last_edited_user": "zzz",
"last_edited_date": 1592434625398
},
"geometry": {
"x": -81.03567099999998,
"y": 25.857096300000023
}
},
{
"attributes": {
"objectid": 886297,
"globalid": "dbabadd9-9d46-43eb-b42b-05836c94e46e",
"activity": 0,
"altitude": -22,
"app_id": "tracker",
"battery_percentage": 87,
"battery_state": 1,
"course": 96.2351684570312,
"device_id": "04ba83a0-09b0-4b4a-b87e-2be331924053",
"floor": null,
"horizontal_accuracy": 3.90000009536743,
"location_source": "fused",
"location_timestamp": 1592434582000,
"session_id": "7aa745ba-42be-4b23-acc8-11a65d511740",
"signal_strength": null,
"speed": 26.8384494781494,
"vertical_accuracy": 100,
"created_user": "zzz",
"created_date": 1592434625398,
"last_edited_user": "zzz",
"last_edited_date": 1592434625398
},
"geometry": {
"x": -81.03567099999998,
"y": 25.857096300000023
}
},
{
"attributes": {
"objectid": 886298,
"globalid": "f7c56f5c-5d20-4772-9f28-2f5ba4e90640",
"activity": 0,
"altitude": -22,
"app_id": "tracker",
"battery_percentage": 87,
"battery_state": 1,
"course": 96.2351684570312,
"device_id": "04ba83a0-09b0-4b4a-b87e-2be331924053",
"floor": null,
"horizontal_accuracy": 3.90000009536743,
"location_source": "fused",
"location_timestamp": 1592434582000,
"session_id": "7aa745ba-42be-4b23-acc8-11a65d511740",
"signal_strength": null,
"speed": 26.8384494781494,
"vertical_accuracy": 100,
"created_user": "zzz",
"created_date": 1592434625398,
"last_edited_user": "zzz",
"last_edited_date": 1592434625398
},
"geometry": {
"x": -81.03567099999998,
"y": 25.857096300000023
}
}
]
... View more
06-18-2020
06:39 AM
|
0
|
0
|
961
|
|
POST
|
Like I mentioned, I don't have a sense of frequency just yet but I do have some monitoring tools setup. This duplicate issue has only happened about 7 or so times since I started monitoring about 50 ESRI Tracker users collecting roughly 30,000 tracks each 24hrs. I have a daily summary metrics that I am querying against the ESRI Tracker Service and the Hosted service I have moved points into from the tracker service. In the instances where there's both a "actualDiscrepancies" and "possibleDiscrepancies" values then these are cases where valid ESRI Tracker features were created much later after the copy process ran and these are true discrepancies that I recover those points. Whenever "actualDiscrepancies" = 0 and "possibleDiscrepancies" > 0 these are all instances where the ESRI track points were created later but already had created a point at the earlier time, hence it's a duplicate feature. I'm only interested in the actualDiscrepancies and have a process in place to move these into the destination feature service after-the-fact. (the -1 values are likely due to a missing ">=" somewhere in my programming that queries the feature services but I consider this quite minor and I simply ignore at the moment until I can find time to remedy it).
... View more
06-15-2020
09:31 AM
|
1
|
0
|
961
|
|
POST
|
Edit: I will have to determine OS. Also, I cannot find any significant pattern on when this occurs but it can be as much as a few hundred points. They do look like true duplicates but created at different datetime values, same location_timestamp values in milliseconds. This is from a different instance than in my screenshots above but all the same example in every way, "features" : [
{
"attributes" : {
"objectid" : 780399,
"location_timestamp" : 1592092364993,
"created_user" : "x",
"created_date" : 1592092528672,
"globalid" : "e8344433-d21e-469e-aa92-59c99df1ae56"
},
"geometry" :
{
"x" : -80.821906594999973,
"y" : 25.762405908000066
}
},
{
"attributes" : {
"objectid" : 781743,
"location_timestamp" : 1592092364993,
"created_user" : "x",
"created_date" : 1592093778509,
"globalid" : "e0e22585-ae08-4406-81cb-243e36fcb829"
},
"geometry" :
{
"x" : -80.821906594999973,
"y" : 25.762405908000066
}
}
] I also have a post-processing step that identifies discrepancies after I have moved points from the ESRI tracker service to another hosted feature service that creates local feature classes from the json result of queries against both services. Once they are FC's I run a selectByLocation for identical features as well as using Find Identical GP tool. When I run the Find Identical GP tool on the ESRI Tracker service results, they are definitely identical in every way except for the Created_date values.
... View more
06-15-2020
09:16 AM
|
0
|
0
|
961
|
|
POST
|
Hopefully this will do, I can get you the globalid's and service url if needed too. Bascially, you see there are 2 tracks on top of each other, identical location_timestamp and username, all of the same attributes altogether (except for OID and GlobalID). The difference is the created_date, which seems to show that they were created at later datetimes. The takeaway is that location_timestamp and username is exactly the same. I couldn't fit the full popup so just pay attention to the (1 of 2). Here are 2 images of point 1: Here's the second point:
... View more
06-12-2020
03:26 PM
|
0
|
3
|
2738
|
|
POST
|
Yep, will find some of those instances for asap. Thanks again! j
... View more
06-12-2020
01:14 PM
|
0
|
0
|
2738
|
|
POST
|
There is one other item that I cannot understand: Why are there some duplicate ESRI Track points? I can see that in those after-the-fact times created_date is after the location_timestamp, there may be instances of the same point even though it was created around the location_timestamp. The location_timestamp values are exactly the same. (edit: it's not a huge issue and I can identify/account for these pretty easily in my post processing tools, but would be good to fully understand.) j
... View more
06-12-2020
12:18 PM
|
0
|
7
|
2738
|
|
POST
|
Aaron -- one last question on this as I've got it worked out to post-process tracks that are created after the location_timestamp that my initial processing misses.... How much later in the future could a Created_date value actually be? I guess I really just need to know if that is more than a 24 hours / 1 calendar day. This would really help me to solidify recapturing those missed points. Thanks for any info! Take care, j
... View more
06-12-2020
11:26 AM
|
0
|
2
|
2739
|
|
POST
|
Thank you for the quick reply Aaron! It does sound like my assessment is somewhat correct and point features can be uploaded/added at a much later (hours) time, not necessarily in succession. We are processing the tracks each hour, enriching them with some other attributes and copying them into another destination feature service that has a similar schema to the tracker service. This is essentially what occurs at the top of each hour: To identify "new" track features, query the destination feature service to get the most recent feature location_timestamp value for a particular username. Query parameters include: orderByFields': 'location_timestamp DESC', 'resultRecordCount' : '1' With that location_timestamp value, query the ESRI Tracker service to see if there are any feature >= that value, if so then execute the process on those features. As you can see, if new point features were created in the ESRI tracker service after the process had executed then subsequent executions would not pickup those in the past. I'll have to implement a secondary post-processing that identifies these "missed" tracks that were created after the location_timestamp values and pull them into the destination feature service. Take care, j
... View more
06-03-2020
01:03 PM
|
0
|
0
|
2739
|
|
POST
|
Need some clarification on the two columns: location_timestamp vs. created_date If there is a large gap between these two values, does that mean the app user was stationary for that period of time and then at some point the ESRI Tracker service was updated with features later? For example, we can see in some point features the two date values are quite close (here they are only 2 mins apart): In other instances the two dates are hours apart: I'm asking because we are processing tracks at the top of each hour by querying the location_timestamp values from the last known location_timestamp value we maintain. However, if the point features were added at a much later time then they'll be missed if recent tracks have already been processed. That is, we don't look back in time for points added after the top of each hour. I'm pretty sure I'm assessing this correctly but hoped to get some confirmation so that I can implement something to process the missed features. Thanks!
... View more
06-03-2020
12:14 PM
|
0
|
15
|
3852
|
|
BLOG
|
We had been running this as a scheduled task on an 2012R windows box and is now failing. Here's the last print statement: {"replicaName":"","replicaID":"","layerServerGens":[{"id":0,"serverGen":0}],"submissionTime":1588943947580,"lastUpdatedTime":1588944574083,"status":"Completed","resultUrl":"https://services1.arcgis.com/sDAPyc2rGRn7vf9B/ArcGIS/rest/services/TrackerHistory/FeatureServer/replicafiles/e9c76fe590a14c74a640c67990c6070d.zip"} -Check 63: Completed -Temporary Directory: c:\users\jcrandal\appdata\local\temp\22\tmpz2orp_ ====================== FAIL: Downloading Survey exception: File is not a zip file ('File is not a zip file',) <class 'zipfile.BadZipfile'> 595 ----------------------
... View more
05-08-2020
06:42 AM
|
0
|
0
|
11001
|
|
POST
|
Thank you again, Stefan. I agree that's a bad example JSON (the "samples" array attribute is not useful for this question). Also, the "Results" attribute is default for any Geoprocessing service output I believe. In any event, it does sound like we would have the ability to setup the appropriate out-of-the-box connectors in Geoevent server using a REST source. We have some requirements to process OSIsoft/Pi REST api's using our own wrappers that enrich them with the geometry attributes from our Geospatial data services. In essence we have Geoprocessing services that consume both the REST api's and merge them together to create a new REST output. However we need a way to format that output into an actual Feature Service that can be consumed by AGOL map viewer and we're exploring options to accomplish this. We originally thought perhaps developing a Server Object Interceptor (SOI) for existing map/feature services could suffice, but I'm unsure if the volume of operational data would inhibit performance. Geoevent Server seemed like a much more robust option and researching to determine if it would meet our needs.
... View more
04-09-2020
05:24 AM
|
0
|
0
|
2371
|
|
POST
|
Hi Stefan, Thank you for the reply! Yes, that is an actual output from a GP service we have running on our ArcGIS Server 10.4 at the moment. every result has the attributes pointX and pointY Yes, correct. My question is: can this be used as an input to a GeoEvent server? My understanding is that GeoEvent can "poll" other REST services at a defined interval, which we would use to update an output of a GeoEvent server. Ultimately, we're thinking of using GeoEvent to update a Feature Service used in our other web products (AGOL and Portal). (sorry if I'm incorrectly defining things, still new to the GeoEvent Server terminology)
... View more
04-08-2020
08:45 AM
|
0
|
2
|
2371
|
|
POST
|
We have existing or in-development Geoprocessing Services that essentially output string JSON that we're simply enriching with location info (x/y or ring values). Since we're building these with Python I think we can format the output just about any way we want, but is there connectors in GeoEvent that can be configured to just specify the xy attributes of the json we want to use? In the sample output below, all of the attributes in the "result" array would be what we'd like to push to GeoEvent and output as a feature service. { "results": [ { "paramName": "resultJSON", "dataType": "GPString", "value": { "result": [ { "phosphorusUnit": "n/a", "samples": [ ], "phosphorus": "n/a", "pointX": -8938939.6672, "pointY": 3079872.605700001, "wmFeature": "STA-1E", "phosphorusTimestamp": "n/a", "_7dayFlowVolume": 0, "structureName": "S377" }, { "phosphorusUnit": "mg/L", "samples": [ { "qualityCode": "P", "collectMethod": "G", "collectDate": 1583936160000, "sampleId": "P114457-10", "value": 0.086, "testName": "PHOSPHATE, TOTAL AS P", "flag": null, "station": "G311", "testNumber": 25, "units": "mg/L" }, { "qualityCode": "P", "collectMethod": "G", "collectDate": 1584535260000, "sampleId": "P114450-12", "value": 0.065, "testName": "PHOSPHATE, TOTAL AS P", "flag": null, "station": "G311", "testNumber": 25, "units": "mg/L" }, { "phosphorusUnit": "n/a", "samples": [ ], "phosphorus": "n/a", "pointX": -8942434.6842, "pointY": 3083166.2346, "wmFeature": "STA-1E", "phosphorusTimestamp": "n/a", "_7dayFlowVolume": -99999.0, "structureName": "S375" } ] } } ], "messages": [ ] }
... View more
04-07-2020
09:50 AM
|
0
|
4
|
2511
|
| Title | Kudos | Posted |
|---|---|---|
| 1 | 02-17-2020 10:47 AM | |
| 1 | 10-25-2022 11:46 AM | |
| 1 | 08-08-2022 01:40 PM | |
| 1 | 02-15-2019 08:21 AM | |
| 2 | 08-14-2023 07:14 AM |
| Online Status |
Offline
|
| Date Last Visited |
01-22-2025
02:28 PM
|