GeoEvent Server Kafka logs

[EirikBuraas]

We are using GeoEvent server 10.6 and we are experiencing that the Kafka logs in default location: c:\ProgramData\Esri\GeoEvent-Gateway\kafka\logs grows over time and have now reached about 5GB. Is there a way to either delete the older logfiles without stopping the GeoEvent windows service or move the location of these logs to another folder location?

Best regards,

Eirik M. Buraas

Geodata AS

[EirikBuraas]

RJ Sunderman‌ do you have any insight on this issue?

Best regards,

Eirik M. Buraas

Geodata AS (Norwegian distributor)

[BrianLomas]

I wonder if your logs are getting so large is due to the "debug" log setting being selected. I've heard that having the debug setting selected can create large file outputs. However, I have not had any experience with this issue. You may want to check into this though. You can check it by going to the "logs" section in geoevent manager and click on "settings" and check the "log level". Next to the settings button is also a "delete logs" button. I am not sure the full extent of using this feature but it's something you could look into. Good luck.

[EricIronside]

The settings you describe in GeoEvent Manager > Logs are different from the Kafka log settings for GeoEvent Gateway that Eirik is asking about.

[EricIronside]

You can modify the settings for the logs but this is definitly an advanced configuration topic. The default settings provided by GeoEvent Gateway were intended to provide the best performance (at the expense of disk space), and you should put some thought (and then testing) into it before you start to change them. That being said, there are two properties you might consider adding to the kafka.properties file:

log.roll.ms=1200000
log.roll.jitter.ms=120000

Adding these properties will force your Kafka to roll over to a new log file approximately every 20 minutes (+- the log.roll.jitter.ms value of 2 minutes).  If you add these properties, you should rarely have more than 4 log files per partition for high-velocity topics, and no more than 1 for low-velocity topics.

Apache Kafka documentation for the other parameters included in the properties file can be found here.