topic How can I remove duplicate events? in ArcGIS GeoEvent Server Questions https://community.esri.com/t5/arcgis-geoevent-server-questions/how-can-i-remove-duplicate-events/m-p/58080#M257 <HTML><HEAD></HEAD><BODY><P>We're experimenting with replacing a legacy system which uses Python scripts for processing and an Oracle DB for storage with GeoEvent Server for processing and the Spatiotemporal Big Data Store for storage.</P><P></P><P>I've been able to replicate our processing (filtering, data validation, geometry creation, etc) in GeoEvent 10.6 with one exception: detecting and removing duplicates. In Python we load data into a Pandas.Dataframe and removing duplicates is as simple as using the .drop_duplicates() method. I haven't found a way to do this in GeoEvent yet. Is it possible? Or can I somehow delete duplicate records from the Big Data Store after the events are pushed in?</P></BODY></HTML> Fri, 18 May 2018 13:30:08 GMT RyanClancy 2018-05-18T13:30:08Z How can I remove duplicate events? https://community.esri.com/t5/arcgis-geoevent-server-questions/how-can-i-remove-duplicate-events/m-p/58080#M257 <HTML><HEAD></HEAD><BODY><P>We're experimenting with replacing a legacy system which uses Python scripts for processing and an Oracle DB for storage with GeoEvent Server for processing and the Spatiotemporal Big Data Store for storage.</P><P></P><P>I've been able to replicate our processing (filtering, data validation, geometry creation, etc) in GeoEvent 10.6 with one exception: detecting and removing duplicates. In Python we load data into a Pandas.Dataframe and removing duplicates is as simple as using the .drop_duplicates() method. I haven't found a way to do this in GeoEvent yet. Is it possible? Or can I somehow delete duplicate records from the Big Data Store after the events are pushed in?</P></BODY></HTML> Fri, 18 May 2018 13:30:08 GMT https://community.esri.com/t5/arcgis-geoevent-server-questions/how-can-i-remove-duplicate-events/m-p/58080#M257 RyanClancy 2018-05-18T13:30:08Z