Getting Started with various Tools

21 / 44

Kafka

Purpose: Apache Kafka is an open-source distributed event streaming platform used for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

Home Page: https://kafka.apache.org/

Documentation: https://kafka.apache.org/documentation/

Related resources to learn:

How to get started:

  1. In the web console tab on the right side of the screen, Include Kafka binaries in the path. HDP includes the kafka and installs at /usr/hdp/current/kafka-broker

    export PATH=$PATH:/usr/hdp/current/kafka-broker/bin
    
  2. Create the topic. Replace localhost with the hostname of node where zookeeper server is running. Generally, zk runs on all hosts on the cluster. Replace test with your topic name

    kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic sandeepgiri9034_test
    
  3. Check if topic is created

    kafka-topics.sh  --list --zookeeper localhost:2181
    
  4. Push messages to topic, type "my first kafka topic"

    kafka-console-producer.sh --broker-list cxln4.c.thelab-240901.internal:6667 --topic sandeepgiri9034_test
    
  5. Test if producer is working by consuming messages in another terminal. Replace localhost with the hostname of broker

    export PATH=$PATH:/usr/hdp/current/kafka-broker/bin
    kafka-console-consumer.sh --zookeeper localhost:2181 --topic sandeepgiri9034_test --from-beginning
    

No hints are availble for this assesment

Answer is not availble for this assesment

Loading comments...