Real-Time Analytic Hosted Feature Layer Data Retention Policy

Jim_Wei

Hello Velocity Community,

I've encountered an issue while working with a spatiotemporal hosted feature layer in Real-Time Analytics. Despite the layer's data storage being configured to "Keep only the latest feature for each Track ID value", I am receiving an error message stating: "The existing feature layer was originally created without data retention specified. When ingesting data that will grow over time, you must use an output feature layer with data retention configured."

 

I understand that there is an article regarding this. This error message seems logical when applied to layers without data retention, where there is potential for data to grow infinitely. However, in a case like mine, where the configuration is set to "Keep only the latest features", it seems like there should be some allowance considering the data isn't expected to become unmanageably large.

 

Currently, my workflow includes the use of both development and production Real-Time Analytics. The challenge this error presents is significant as it makes it difficult to transition my in-production analytic to a new analytic.

I am curious to know if there are existing workarounds to this issue that the community could suggest. Additionally, I would like to suggest an enhancement in future releases of ArcGIS Velocity. This could potentially streamline workflows for those who operate in similar conditions as mine.

PeterNasuti

@Jim_Wei Thank you for your post! The Feature Layer (existing) output was designed to require any Velocity layers that it writes to have data retention enabled. This includes feature layers that have "Keep only the latest feature for each Track ID" configured.

Many real-time cases work with Track ID values that are not finite, but will continue to grow over time. Consider for example thunderstorm watch/warning polygons with a unique identifier for each polygon. The Track ID uniquely identifies each polygon and only keeps the latest value for each, but the number of thunderstorms will only continue to grow as time goes on. There are many other real-time cases where the values for unique ID/Track ID are not finite. 

For this reason, a data retention policy must be specified. Therefore, the workflow available is to recreate the layers with a data retention policy. You could use a big data analytic to populate the values from your "old" layer to this new layer as desired or reach out to Esri Support for assistance on this step as needed.