Publish Data to Kafka Topic using KafkaProducer Api

In order to publish data on Kafka topics. First, you need to run the zookeeper and Kafka cluster in your local environment.

If you haven't set up your local environment of Kafka and zookeeper cluster so please have a look at this youtube tutorial it will surely help you.

Run the zookeeper and Kafka Service

To run the zookeeper enter the below command in your terminal

$ cd kafka/bin
$ zookeeper-server-start.sh config/zookeeper.properties
$ kafka-server-start.sh config/server.properties

Now both the single-node clusters are running successfully. Now you need to create the topic on which we are going to publish our data.

$ kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 3 --topic sample-topic

Also, you can list all the different topics that you currently have in your Kafka cluster or bootstrap server by using the below command.

$ kafka-topics.sh --list --bootstrap-server localhost:9092

Concept Visualization

Implementation

In order to publish data on the topic, you need to do 5 necessary things.

• Create properties object for producer
• Create the Producer
• Format the message in order to publish
• Send data
• Flush and Close the producer

Once you are done with the coding part. Now it's time to run the consumer which reads the data from the Kafka topic(sample-topic) and writes it into the terminal.

Run the below command to run the Kafka console consumer.

$ kafka-console-consumer.sh --bootstrap-server localhost:90092 --topic sample-topic --group java-group