topic Re: Is there a way to delete records from a Feature Layer? in ArcGIS GeoEvent Server Questions

Is there a way to delete records from a Feature Layer?

AllenScully — Mon, 31 Aug 2015 21:24:19 GMT

We have a good set up with GEP 10.3 that takes table data w/ X/Y coordinates and pushes them to a Feature Class (published w/ feature access). Currently using an 'Update a Feature' output connector.

However I noticed recently that the target feature class is holding on to records we want deleted - in this case Police Department events that have been closed.

The input data is correct - only has active events. But because the closed/inactive event records are not in the input, they simply stay in the output Feature Class, with no way (that I know of) in GEP to get rid of them. We can have a simple status field that will allow us to use a definition query to keep 'closed' events from showing up, but I am wondering if there is a way to use GEP to see that a certain record is not in the newest input dataset, and therefore remove it from the output dataset.

Thanks -

Allen Scully

City of Tucson GIS Services

Re: Is there a way to delete records from a Feature Layer?

RJSunderman — Tue, 01 Sep 2015 02:23:13 GMT

Hey Allen -

Take a look at Ben Saunders's Blog: Orphan Tracks in GeoEvent Processor

The GeoEvent 'Add a Feature' and 'Update a Feature' output connectors can be configured to delete "old" features, if you have a field of type Date to which you can have the output refer. The output will periodically (every nn seconds) query the feature layer to which it is adding or updating features for any records older than mm minutes -- and delete them.

But you can't always assume that a record should be deleted just because a date/time stamp hasn't been updated for mm minutes. Nor do you always have a date/time stamp to which you can refer.

Ben proposes allowing real-time analytics to decide when a record is "closed" or "obsolete" and writing a value into a specific event attribute field. Then have a 'Poll an ArcGIS Server for Features' input periodically poll for features which are "closed" or "obsolete" ... and set the input's 'Delete Polled Features' property to Yes.

The beautiful thing is you don't event have to do anything with the records polled by the input. You could write them out to a confirmation log that these records are being deleted if you want ... but the input's only job is to identify records to be deleted, without requiring a Date field, and delete the identified records.

- RJ

Re: Is there a way to delete records from a Feature Layer?

AllenScully — Fri, 04 Sep 2015 22:52:52 GMT

Hi RJ -

Finally had a chance to work on this today. So far so good. In fact it is so simple and quick to set up I'm worried I have missed something.

What helped make this so easy is that our input data (table, published as feature service) already has a column that tags a record as NEW, UPDATE or CLOSED.

So our GeoEvent Service essentially takes a table w/ X/Y data, uses a poll website for JSON input connector, and writes records to a feature service - fairly simple set up.

All I did to get rid of closed records was set up a Poll Feature Service input connector, point it to our Feature Service mentioned above, and in the 'Advanced' section of the input connector, set the definiton query to 'RecordType='Closed', check 'yes' for Delete Polled Features, and put it in a new GeoEvent Service with a simple 'Write to JSON' output connector so we can see what was deleted.

Now, I have not yet had a chance to check on any performance impacts this might have, but we are not dealing with a ton of records so I'm optimistic.

Many thanks.

Re: Is there a way to delete records from a Feature Layer?

AllenScully — Thu, 07 Jan 2016 22:59:11 GMT

After being away from GEP for a few months, I'm back working with this and have a few updates and another question.

The process RJ referred to above in Ben Saunder's blog works very well. From what I can tell, it can be a perfect solution in some cases depending on your source data.

If your data feed updates a record for a feature from a status of 'Open' to 'Closed' (just for example) you are good to go.

However I have a situation where the source data will create a brand new row, for the same 'event' when that event changes status (our statuses are 'New', Updated' and Closed) - so we end up with a situation where the same event ends up in our output feature class many times, and the process mentioned above will delete 1 record, for the one where the status is closed, but the 'New' and 'Updated' versions of the event still are present, even though as a whole the event can be considered 'closed' and all records of it need to be removed from the output feature class.

So my question is - how to get around this. My thinking is that we need to use a date-time stamp, where only the most recent record for an event makes it's way into the output, and use this in conjunction with the Ben's method of deleting closed records.

I'm working on this now, but any suggestions are welcome.

Thanks -

Allen