Hello Munhwan -
Sorry that no one has gotten back in touch with you on your question. If you found a solution in the meantime and wanted to share it here, I'm sure the community would appreciate it.
If your concern were that features were not being added to a feature service, I would suggest that you set the DEBUG level on the com.esri.ges.transport.featureService.FeatureServiceOutboundTransport. From within GeoEvent Manager, navigate to Logs > Settings and specify the name of the component for which you want to increase the logging level.
A benefit of debug logging the Feature Service Outbound transport is that you'll see (in the karaf.log) all of the JSON transactions being made between GeoEvent and the ArcGIS Server REST interface used to add / update features.
One problem I've seen is that a feature service with a field of type esriFiedlTypeString specifies a length restriction of say 5 characters. Several events are received by GeoEvent, one of whose Name field (a String) is longer than the 5 character restriction. GeoEvent will successfully adapt the events received, batch them up and send out a REST request to add / update features once every second. ArcGIS Server receives the request, observes that one of the event records has a String whose length it cannot accept ... and rejects the transaction. The result is that none of the events in that transaction come though to update features.
You might find an indication of the rejection if you looked into the ArcGIS Server logs, but the only way you'd know from the GeoEvent karaf.log that there was a problem would be to look at DEBUG logging for the Feature Service Outbound transport and see the response received from the REST request GeoEvent made on the ArcGIS Server feature service's endpoint.
Now - you indicated in your message that you observed events missing from both the feature service and the CSV system file you were using to log the processed events. This I cannot explain. Obviously an issue with ArcGIS Server rejecting one or more events because they do not meet the requirements of the feature service's schema would have nothing to do with writing that same event data out to a system file. Completely different transports and adapters are being used to support these two outputs.
Were the event counts displayed in the GeoEvent Manager's Monitor page consistent? That is, did you receive 210 events In on an input, 420 events In/Out the GeoEvent Service and 210 events Out each your fs-out output and your text-csv-out output?
About all I can recommend - apart from attaching a debugger and stepping through the execution of your custom transport and adapter - is to incorporate a system file JSON or CSV output directly off of your input and a second system file JSON or CSV output (writing to a different file) downstream, after any filtering / processing you might be doing ... just before you output the event data to your fs-out to add/update features. Then you can compare the content of the two output files.
The data you see getting lost might be lost when an adapter fails to adapt a received payload of bytes, it might be discarded by a filter, it might be lost when a processor tries to process the data and throws an exception, or it might be lost when a REST transaction made by GeoEvent is rejected by the target server (as I describe above for adding / updating features).
- RJ
