Transactions

[JustinK]

Is geo-event processing in GeoEvent Server coordinated in an transaction (i.e. ACID transaction)? In other words, if the event was read by the adapter, translated into a GeoEvent and then passed into an GeoEvent Service, and at some point before leaving the service the power was to be switched off (or GeoEvent Server process is killed/dies), would the event be lost? If not, where is it persisted? And where does it continue processing from (i.e. from the last step in the Service? or at the beginning again? etc.)

Thanks.

[TimothyStiles]

Hi,

Thanks for your question. GeoEvent Processor processes events as they are received and makes every effort it can to ensure events are processed successfully. Since the product is designed to work with highly fluid data feeds and is very dynamic in the way events can be routed (ACID) transactions are not supported by design. So if the power goes off on the machine events could be lost. If you want to ensure events get from a feed to the GeoEvent Processor you may want to consider transports that have guaranteed delivery mechanisms built in such as JMS and AMQP.

Thanks,

Tim

[JustinK]

Thanks Tim.

Just to confirm, we are thinking of using ESRI GeoEvents Server with TIBCO EMS Queues (via JMS), but I believe you are saying that this also will not ensure messages are always processed?

In another words, are you saying that you pop the message off the incoming queue and don't process it in a transaction (i.e. you commit the pop of the queue before the workflow starts)?

[AdamMollenkopf]

If you setup TIBCO EMS Queues (via JMS) with the right JMS acknowledgement mode then a GeoEvent Processor input is configured to recieve off of the JMS queue will consistently receive the event.  If the input happened to fail in the process of receiving the JMS message it would be unacknowledged and put back on the Queue for it, or another GeoEvent Processor instance, to process again.  If your JMS broker supports it and you desire it you could also setup max # of reattempts and idle time between attempts.

GeoEvent Processor cannot support transactions after the point the input has recieved the event and submitted it for processing by GeoEvent Services.  This is due to the fact that there may be multiple GeoEvent Services listening to the input and those GeoEvent Services may have processing steps that change the event (i.e. field enricher, field calculator, etc...).  Also, once the processing is complete there may be multiple outputs that the events get routed to.  Defining the boundary of where a transaction would start and end in this type of environment is very challenging to do and would most likely have a negative impact on performance.  For these reasons, by design, at the 10.2 release GeoEvent Processor does not support transactions.  If this is an important capability for you please submit the idea on the Esri ideas site and we will consider it for a subsequent release if there is enough customer demand.

Hope this information is helpful,
Adam Mollenkopf
ArcGIS GeoEvent Processor for Server - Product Lead

[PatriceFREYDIERE]

Hi Adam,

Any updates about transaction support in 10.3 or a mechanisme permitting to route events that might have failed in a processor or output ?

Thank's for your reply

Patrice

[JustinK]

Thanks for the reply. This has helped to confirm our assumptions.

Cheers