GeoEvent 10.4.1 - ZooKeeper WARN messages

[ErikLanhammar]

Hello!

Im running GEE 10.4.1 and Im starting to recieve GEE log WARN messages like these:

org.apache.zookeeper.server.persistence.   FileTxnLog fsync-ing the write ahead log in SyncThread:1 took 1332ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide.

The ZooKeeper files are getting quite big, and I guess thats the reason for the messages...

Whats a good plan to automate a purge job on these files, keeping them small and GEE happy?!

Rgds,

/Erik

[LorenMueller]

Hi Erik. I am seeing the same thing and the ZooKeeper logs are up to about 30GB! How did you resolve this?

Thanks,

Loren

[ErikLanhammar]

Hey Loren,

I havent solved it yet.. I tried to delete the contents of the log files ”manually”. Dont.

I had to reinstall GEE..

So a good way to purge Zoo-keeper logfiles is something Im still looking for.. I will hopefully/eventually post the solution on the forum.

Rgds,

/Erik

[ChrisBeyett]

Hi Erik, 

The Zookeeper log files should be purging themselves once they reach 64mb in size. There is an auto-purge process built into zookeeper that ensures this occurs. Do you have more than 50 map/feature services published on the ArcGIS Server? I've seen this exact behavior happen when more than about 50 services are published and zookeeper simply keeps writing the same files over and over. 

- Chris

[ErikLanhammar]

Hi Chris,

No I only have 10-15 services published in total, including 2 steam services. Log files are +200 MB...

How configurable is ZK? Edit config files for purge file size, file location etc? Should this be done while GEE windows service is stopped?

Rgds,

/Erik

[ChrisBeyett]

Hi Erik, 

Can you confirm a behavior within the GeoEvent config files? 

1. Navigate to the following file location and open it in a text editor:

'C:\ProgramFiles\ArcGIS\Server\GeoEvent\etc\ArcGISGeoEvent.cfg'

2. Scoll down to the 16th variable location. It should read as follows:

wrapper.java.additional.16=-Djute.maxbuffer=0xA000003

3. Pay particular attention to the "b" in the maxbuffer variable. Is it upper or lower case?

- Chris 

[ErikLanhammar]

Mvh,

/Erik

[LorenMueller]

Hello,

I am following this thread as I am having the same issue (with only about 6 custom AGS map services running).

Should the 'b' be lower or upper case? It is lower case in my ArcGISGeoEvent.cfg file.

[ChrisBeyett]

In my testing I've found that switching the "b" to an upper case "B" results in the log messages purging themselves. Feel free to stop the GeoEvent Windows service and switch the case of the letter. 

- Chris

[ErikLanhammar]

Thanks Chris, will test this.

/Erik