Kafka Shell Operate

文章目录
  1. 1. Create a topic
  2. 2. Start some messages
  3. 3. Start a consumer
  4. 4. Understand random characters in the log file of kafka
  5. 5. Inspect kafka group
  6. 6. Lookup for assigned of partition

Create a topic

Let’s create a topic named “test” with a single partition and only one replica:

1
> bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test

We can now see that topic if we run the list topic command:

1
2
3
> bin/kafka-topics.sh --list --bootstrap-server localhost:9092

test

Start some messages

Kafka comes with a command line client that will take input from a file or from standard input and send it out as messages to the Kafka cluster. By default, each line will be sent as a separate message.
Run the producer and then type a few messages into the console to send to the server.

1
2
3
4
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

This is a message
This is another message

Start a consumer

Kafka also has a command line consumer that will dump out messages to standard output.

1
2
3
4
> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

This is a message
This is another message

Understand random characters in the log file of kafka

1
> bin/kafka-run-class.sh kafka.tools.DumpLogSegments --deep-iteration --print-data-log --files topic/*.log|index|timeindex

Inspect kafka group

1
> bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group hiway.group.projection --describe -- state
TOPICPARTITIONCURRENT-OFFSETLOG-END-OFFSETLAGCONSUMER-IDHOSTCLIENT-ID
hiway.topic.order0660consumer-21-d8cb9666-2d54-408b-82a6-0480a0b5a10b/172.168.5.9consumer-21
hiway.topic.aps-billing010100consumer-21-fbea0085-d5fa-4645-a2cf-c3b4e9f2837b/172.168.0.7consumer-21
hiway.topic.account023230consumer-15-24e95c62-03f9-4051-8cd0-8b40d691b0c9/172.168.0.7consumer-15

Lookup for assigned of partition

1
> bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --members --group hiway.group.projection
CONSUMER-IDHOSTCLIENT-ID#PARTITIONS
consumer-27-d56cb0ba-90a8-498a-b019-9f71c6046e03/172.168.0.7consumer-270
consumer-6-cacd38dc-dffb-456c-b0d2-5badbb06f108/172.168.0.7consumer-60
consumer-24-ba6ef6b7-d0fe-4104-a0e9-5bcda354cda8/172.168.0.7consumer-241
consumer-21-d8cb9666-2d54-408b-82a6-0480a0b5a10b/172.168.5.9consumer-211
consumer-21-fbea0085-d5fa-4645-a2cf-c3b4e9f2837b/172.168.0.7consumer-211