Detect Incidents Outliers

[MuneerMajid]

I have a Geofencing analytic that detects vehicles inside Well Pad sites polygons. The feed for vehicles has a track ID and I am joining a few fields from the well pad sites into my output feature layer which is the result of detect incidents. I am currently using a spatial intersect as my open condition. The idea is to eventually report Well pads sites visited by operators and time on site. 

I have come across some outlier cases where a driver has visited multiple well pad sites over the course of a timespan recorded over few hours. However it seems that the all these events are recorded under a single Incident ID. This should never happen since each well pad site visited should be logged under a unique incident ID. Below is my analytic - 

Here is one example where 4 different Site IDs are logged under a single Incident ID, I would expect to see all these occurrence's under 4 different Incident IDs. 

This issue pretty much makes my analytic and detect incidents results unreliable and these outliers cases cannot be factored into my reports. Any thoughts or suggestions to overcome this issue would be really appreciated. 

[JeffSilberberg]

Once you open an incident I see no way it will be closed until 12 Hours expire.  So any subsequent transactions for this Track ID will have the same Incident ID.   Also, be sure you are looking at the Status field downstream of your Incident Detector. 

[MuneerMajid]

@JeffSilberberg Thanks for your response. I am quite certain that expiration is not the closing condition. That's for cases where the device/vehicle goes offline and we wanted to ensure the incident is not open in perpetuity. The multiple sites reported here are an outlier case which I can estimate is logged for 5% of the cases. 

Majority of incidents are closed when vehicle exits the geofence and a single Geofence ID is associated with the incident as expected. Here are several good examples below - 

If you have any additional inputs for me to tweak the incident configurations, please feel free to let me know. 

[KenGorton]

@MuneerMajid The first question that comes to mind is, "How fast are the vehicles reporting their locations?" The image shows a feature ingestion rate of 0.4 f/s which is petty slow. Thus if a vehicle reports its location while on a pad, Detect Incidents should observe an 'intersects' and open an incident. The implicit closing condition of 'intersects' is 'disjoint', so if the vehicle subsequently moves to a second pad before it has a chance to update its location and is on that 2nd pad when it updates its position, Detect Incidents once again will detect an 'intersects' with no intervening 'disjoint'. Therefore it will not close the incident but emit an OnGoing status. If you have control of the vehicle reporting interval, you might try to increase that. That way you will get more granular position updates and a better chance of detecting changes in spatial relationships. 

Furthermore, in order to eventually calculate time on site, you may want to use Enter/Exit as Open & Close conditions (actually, you would only need to specify Enter as the open condition, as Exit is the implicit close condition for Enter). Keep in mind, time on site would be based on the reporting time of the vehicle, so at best you would get an estimate of time on site. If the vehicle has already been on site for a few minutes when it reports its location, the Enter event would have a timestamp of the vehicle's update time rather than the time it actually entered the pad. Similarly, if it leaves the pad momentarily after reporting its location, and doesn't update its position until several minutes later, the close condition and subsequent estimation of time on site would include all of the time since the vehicle left the pad until it ultimately reported its latest position. Precision for all of this is best increased by increasing the reporting rate of each vehicle.

Hope this helps. If not not let us know.

Ken

Velocity Product Team

[MuneerMajid]

@KenGorton Your explanation makes a lot of sense. We are fetching location once every minute for our vehicles using a custom Dot Net app that we wrote which streams this data into Event Hub. 

We have this deployed across 250 devices currently and since we are also interested in studying historical tracks of our fleet of vehicles for upto 6 months, we were content with reporting the location once every minute. I am strongly going to consider increasing the frequency of my observations since I am also now interested in calculating motion statistics for these vehicles. 

As for the issue, I would still assume that since the tool is working in a stateful manner, it would prioritize the geofence for which the incident was opened. If a vehicle is detected inside another polygon at the next observation, the existing incident should have been closed and a new incident would be opened for the other polygon thats been breached. 

You are correct in stating that the other polygons were logged under the 'Ongoing' state with the same Incident ID. This table shows exactly that and you can see that the observation under this incident are quite muddied because of the 3 geofence polygons involved - 

I feel that the incident close condition could be better defined to handle these cases, specially because often times in field operations, pad sites are very densely clustered within a small area. The same concept could apply to other domains too. 

Do you think Enter & Exit condition will help me overcome this issue ? 

Also the fact that the Exit entry does not report the joined fields of the previous state, makes it very hard to report time on site with the accompanying Geofence ID attributes without doing complex SQL calculation's. 

 

[KenGorton]

@MuneerMajid  the Achilles heel is still going to be how frequently the vehicles report their locations. The more frequently they report, the greater your temporal precision will be, and therefore the more likely you will get timely detections of intersections or enter/exit events. As you've already noted, the scope of the spatial relationship is 'all' vs 'any' meaning it tests whether the spatial relationship is valid across the entire join dataset, not feature by feature. So the target feature intersects the join dataset if it intersects with any feature.

Enter/Exit work similarly but with a certain nuance: under the hood an Enter event is the result of two consecutive observations where the first is actually disjoint=true and the second is intersects=true. Exit works in the opposite manner. So if the target feature is first observed outside all join features and subsequently observed inside at least one, an Enter has occurred. As long as it continues to be observed inside that feature, or any feature, of the join dataset, no Exit event will be detected and no Ended status will be emitted. So it could Enter a join feature, then if it moves to and enters a different join feature before the next observation, Detect Incidents would simply emit an OnGoing status since no Exit was detected in between.

[MuneerMajid]

@KengKeng Thanks again for the detailed explanation. In our case, we are only getting this 'noise' for a small subset of the data (roughly 5%) since the driving speeds are very low out in the fields and the roads are mostly unpaved. There are not many cases where an operator will move between two pad sites within the span of a minute. 

Having said that, I still feel the tool leaves a lot to be desired here. I can think of many uses cases related to a fleet of vehicles where its possible for a vehicle on the move to be driving through several geofences through subsequent observations and yet never report outside a geofence. There could  also be cases of geofence polygons or even buffers around wells lets say which could be slightly overlapping and a vehicle could be driving through them. The subsequent 'Ongoing' observations where the vehicle is detected inside a different geofence polygon should be a condition enough to close the incident and log the interaction of the vehicle with the second polygon as a separate incident. That behavior would be true to the definition of an incident in my humble opinion. 

Do you think there is any way possible with a combination of close conditions where this result can be achieved as desired ? I appreciate your inputs and assistance thus far. 

[PeterNasuti]

@MuneerMajid Thanks for posting your use case and this scenario. What you are requesting regarding a point leaving one spatial feature and entering another without an observation triggering the closure of the old incident and the creation of a new incident is something we have heard requested from other users as well.

There is a planned tool parameter to be added in a future release for the Detect Incidents tool that will do exactly this. Basically it would take an observation and split it into two observations. One that "closes" the first incident detected, and another that "opens" the next incident with the new geometry. This does not yet have an anticipated release date.

While there is not a workaround for this at the moment with the Detect Incidents tool to do exactly what you are asking, a combination of the Join Features tool and the Calculate Field tool using track functions (TrackFieldWindow) and Filter by Expression could be used to filter an alerting pipeline whenever a worker changes from not-in-a-work-area to in-a-work-area OR from in-a-work-area to not-in-a-work-area OR from in-one-work-area to in-another-work-area.

Basically I would envision this flow as:

1. Use the Join Features tool to enrich point worker/vehicle observations with an attribute from your well pad polygons
   1. Keep all features in the join so the value is null if the worker/vehicle is not inside a well pad
   2. Lets say this field is called "InsideWellPadName"
2. Add a Calculate Fields tool and use the TrackFieldWindow function to obtain the prior value for "InsideWellPadName" for the feature in that Track that came before the current feature
   1. var priorWellPadName = TrackFieldWindow('InsideWellPadName', -1,0)
   2. Compare the prior value to the current value and then take your desired action using Arcade conditionals
      1. For example, if prior value is null and current value is populated, the worker/vehicle entered a well pad
      2. If prior value is populated and current value is null, the worker/vehicle exited a well pad
      3. If prior value is one well pad and current value is another well pad, the worker/vehicle moved from one well pad to another
3. Use the Filter by Expression tool (or multiple instances of it) to split analytic pipelines for desired handling