{{theTime}}

Search This Blog

Total Pageviews

Kafka

Kafka Beginner's Tutorial

Kafka Beginner's Tutorial

Table of Contents


Installation

To get started with Apache Kafka, you can follow these steps:

  1. Download Apache Kafka from the official website.
  2. Extract the downloaded archive to a suitable location on your system.
  3. Set up ZooKeeper (required for Kafka) by starting ZooKeeper server: bin/zookeeper-server-start.sh config/zookeeper.properties
  4. Start Kafka broker by running: bin/kafka-server-start.sh config/server.properties

Writing Producers

To write a producer in Kafka, follow these steps:

  1. Create a new Java project in your preferred IDE.
  2. Add Kafka dependencies to your project, such as kafka-clients.
  3. Write code to configure Kafka producer properties:
import org.apache.kafka.clients.producer.*;

import java.util.Properties;

public class KafkaProducerExample {

    public static void main(String[] args) {
        // Set up producer properties
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        // Create Kafka producer
        Producer producer = new KafkaProducer<>(props);

        // Send messages to Kafka topic
        String topic = "my-topic";
        for (int i = 0; i < 10; i++) {
            String key = "key-" + i;
            String value = "Message " + i;
            ProducerRecord record = new ProducerRecord<>(topic, key, value);
            producer.send(record, new Callback() {
                @Override
                public void onCompletion(RecordMetadata metadata, Exception exception) {
                    if (exception == null) {
                        System.out.println("Message sent successfully: " + metadata);
                    } else {
                        System.err.println("Error sending message: " + exception.getMessage());
                    }
                }
            });
        }

        // Close the producer
        producer.close();
    }
}



Writing Consumers

To write a consumer in Kafka, follow these steps:

  1. Create a new Java project in your preferred IDE.
  2. Add Kafka dependencies to your project, such as kafka-clients.
  3. Write code to configure Kafka consumer properties:
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "my-group");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
  1. Write code to subscribe to Kafka topics:
String topic = "my-topic";
consumer.subscribe(Collections.singletonList(topic));
  1. Write code to consume messages from Kafka topics:
while (true) {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
    for (ConsumerRecord<String, String> record : records) {
        System.out.printf("Received message: key=%s, value=%s%n", record.key(), record.value());
    }
}

Make sure to handle exceptions and gracefully close the consumer when done.

No comments:

MongoDB Tutorial MongoDB Tutorial Table of Contents Introduction to MongoDB Installation Creating a Database Creati...