the design and implementation reference to kt and kafkatools
go get github.com/smallnest/kafka_monitor
# change the below options with your environments
[email protected]
smtpHost=xxxxxx
smtpPort=25
[email protected]
smtpPassword=xxxxxx
brokers=10.77.11.1:9092,10.77.11.2:9092,10.77.11.3:9092,10.77.11.4:9092
version=0.9.0.1
zkAddr=10.77.0.1:2181,10.77.0.2:2181,10.77.0.3:2181,10.77.0.4:2181,10.77.0.5:2181
./kafka_monitor -brokers ${brokers} -version ${version} -email=${informEmail} -smtp.host ${smtpHost} -smtp.port ${smtpPort) -smtp.user ${smtpUser} -smtp.password ${smtpPassword} -zkAddr ${zkAddr} -basePath "/kafka"
Once some partitions are not writable, down or some consumers don't work, an email will be sent to your email with details.
Normal kafka cluster info is output in console:
Brokers: 10.77.11.1:9092,10.77.11.2:9092,10.77.11.4:9092:10.77.11.3:9092
Version: 0.9.0.1
Topic: products, Group: default, Partitions: 32
+-----------+-------------------+--------+---------------+---------------+
| PARTITION | LEADER ADDRESS | LEADER | REPLICAS | ISR |
+-----------+-------------------+--------+---------------+---------------+
| 0 | 10.77.11.1:9092 | 1 | [1 2 3] | [1 2 3] |
| 1 | 10.77.11.2:9092 | 2 | [1 2 4] | [1 2 4] |
| 2 | 10.77.11.4:9092 | 4 | [2 4 3] | [2 4 3] |
| 3 | 10.77.11.3:9092 | 3 | [1 4 3] | [1 4 3] |
| 4 | 10.77.11.1:9092 | 1 | [1 2 4] | [1 2 4] |
| 5 | 10.77.11.2:9092 | 2 | [2 4 3] | [2 4 3] |
| 6 | 10.77.11.4:9092 | 4 | [1 4 3] | [1 4 3] |
| 7 | 10.77.11.3:9092 | 3 | [1 2 3] | [1 2 3] |
| 8 | 10.77.11.1:9092 | 1 | [1 4 3] | [1 4 3] |
| 9 | 10.77.11.2:9092 | 2 | [1 2 3] | [1 2 3] |
| 10 | 10.77.11.4:9092 | 4 | [1 2 4] | [1 2 4] |
| 11 | 10.77.11.3:9092 | 3 | [2 4 3] | [2 4 3] |
| 12 | 10.77.11.1:9092 | 1 | [1 2 3] | [1 2 3] |
| 13 | 10.77.11.2:9092 | 2 | [1 2 4] | [1 2 4] |
| 14 | 10.77.11.4:9092 | 4 | [2 4 3] | [2 4 3] |
| 15 | 10.77.11.3:9092 | 3 | [1 4 3] | [1 4 3] |
| 16 | 10.77.11.1:9092 | 1 | [1 2 4] | [1 2 4] |
| 17 | 10.77.11.2:9092 | 2 | [2 4 3] | [2 4 3] |
| 18 | 10.77.11.4:9092 | 4 | [1 4 3] | [1 4 3] |
| 19 | 10.77.11.3:9092 | 3 | [1 2 3] | [1 2 3] |
| 20 | 10.77.11.1:9092 | 1 | [1 4 3] | [1 4 3] |
| 21 | 10.77.11.2:9092 | 2 | [1 2 3] | [1 2 3] |
| 22 | 10.77.11.4:9092 | 4 | [1 2 4] | [1 2 4] |
| 23 | 10.77.11.3:9092 | 3 | [2 4 3] | [2 4 3] |
| 24 | 10.77.11.1:9092 | 1 | [1 2 3] | [1 2 3] |
| 25 | 10.77.11.2:9092 | 2 | [1 2 4] | [1 2 4] |
| 26 | 10.77.11.4:9092 | 4 | [2 4 3] | [2 4 3] |
| 27 | 10.77.11.3:9092 | 3 | [1 4 3] | [1 4 3] |
| 28 | 10.77.11.1:9092 | 1 | [1 2 4] | [1 2 4] |
| 29 | 10.77.11.2:9092 | 2 | [2 4 3] | [2 4 3] |
| 30 | 10.77.11.4:9092 | 4 | [1 4 3] | [1 4 3] |
| 31 | 10.77.11.3:9092 | 3 | [1 2 3] | [1 2 3] |
+-----------+-------------------+--------+---------------+---------------+
+-----------+------------+--------------+-----+
| PARTITION | END OF LOG | GROUP OFFSET | LAG |
+-----------+------------+--------------+-----+
| 0 | 1106509557 | 1106509542 | 15 |
| 1 | 1106517793 | 1106517779 | 14 |
| 2 | 1106573067 | 1106573047 | 20 |
| 3 | 1106531253 | 1106531253 | 0 |
| 4 | 1106529394 | 1106529385 | 9 |
| 5 | 11065504 | 1106550146 | 4 |
| 6 | 1106467253 | 1106467248 | 5 |
| 7 | 1106479964 | 1106479964 | 0 |
| 8 | 1106464607 | 1106464594 | 13 |
| 9 | 1106503601 | 1106503589 | 12 |
| 10 | 1106543568 | 1106543552 | 16 |
| 11 | 1106558780 | 1106558768 | 12 |
| 12 | 1106494251 | 1106494238 | 13 |
| 13 | 1106561496 | 1106561490 | 6 |
| 14 | 1106521678 | 1106521673 | 5 |
| 15 | 1106569720 | 1106569686 | 34 |
| 16 | 1106573067 | 1106573049 | 18 |
| 17 | 1106521317 | 1106521305 | 12 |
| 18 | 1106534494 | 1106534484 | 10 |
| 19 | 1106481325 | 1106481268 | 57 |
| 20 | 1106530486 | 1106530465 | 21 |
| 21 | 1106500650 | 1106500624 | 26 |
| 22 | 1106552135 | 1106552113 | 22 |
| 23 | 1106551799 | 1106551771 | 28 |
| 24 | 1106564761 | 1106564750 | 11 |
| 25 | 1106537078 | 1106537064 | 14 |
| 26 | 1106573795 | 1106573781 | 14 |
| 27 | 1106493569 | 1106493554 | 15 |
| 28 | 1106500029 | 1106500015 | 14 |
| 29 | 1106528095 | 1106528078 | 17 |
| 30 | 1106493651 | 1106493629 | 22 |
| 31 | 1106562315 | 1106562312 | 3 |
+-----------+------------+--------------+-----+
| TOTAL | 482 |
+-----------+------------+--------------+-----+