GeoEvent 10.4.1 - ZooKeeper WARN messages

715
12
09-01-2017 07:45 AM
New Contributor III

Hello!

Im running GEE 10.4.1 and Im starting to recieve GEE log WARN messages like these:

org.apache.zookeeper.server.persistence.   FileTxnLog fsync-ing the write ahead log in SyncThread:1 took 1332ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide.

The ZooKeeper files are getting quite big, and I guess thats the reason for the messages...

ZooKeeper files

Whats a good plan to automate a purge job on these files, keeping them small and GEE happy?!

Rgds,

/Erik

Tags (1)
12 Replies
New Contributor III

Hi Erik. I am seeing the same thing and the ZooKeeper logs are up to about 30GB! How did you resolve this?

Thanks,

Loren

Reply
0 Kudos
New Contributor III

Hey Loren,

I havent solved it yet.. I tried to delete the contents of the log files ”manually”. Dont.

I had to reinstall GEE..

So a good way to purge Zoo-keeper logfiles is something Im still looking for.. I will hopefully/eventually post the solution on the forum.

Rgds,

/Erik

Reply
0 Kudos
Occasional Contributor

Hi Erik, 

The Zookeeper log files should be purging themselves once they reach 64mb in size. There is an auto-purge process built into zookeeper that ensures this occurs. Do you have more than 50 map/feature services published on the ArcGIS Server? I've seen this exact behavior happen when more than about 50 services are published and zookeeper simply keeps writing the same files over and over. 

- Chris

Reply
0 Kudos
New Contributor III

Hi Chris,

No I only have 10-15 services published in total, including 2 steam services. Log files are +200 MB...

How configurable is ZK? Edit config files for purge file size, file location etc? Should this be done while GEE windows service is stopped?

Rgds,

/Erik

Reply
0 Kudos
Occasional Contributor

Hi Erik, 

Can you confirm a behavior within the GeoEvent config files? 

1. Navigate to the following file location and open it in a text editor:

'C:\ProgramFiles\ArcGIS\Server\GeoEvent\etc\ArcGISGeoEvent.cfg'

2. Scoll down to the 16th variable location. It should read as follows:

wrapper.java.additional.16=-Djute.maxbuffer=0xA000003

3. Pay particular attention to the "b" in the maxbuffer variable. Is it upper or lower case?

- Chris 

Reply
0 Kudos
New Contributor III

Mvh,

/Erik

Reply
0 Kudos
New Contributor III

Hello,

I am following this thread as I am having the same issue (with only about 6 custom AGS map services running).

Should the 'b' be lower or upper case? It is lower case in my ArcGISGeoEvent.cfg file.

Reply
0 Kudos
Occasional Contributor

In my testing I've found that switching the "b" to an upper case "B" results in the log messages purging themselves. Feel free to stop the GeoEvent Windows service and switch the case of the letter. 

- Chris

Reply
0 Kudos
New Contributor III

Thanks Chris, will test this.

/Erik

Reply
0 Kudos