Getting Started with various Tools

33 / 44


Purpose: Apache Flume is a distributed, reliable, and available software for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows.

Home Page:


Related resources to learn:

  1. In the web console tab on the right side of the screen, to get a copy of sample flume conf from common data:

    hadoop fs -copyToLocal /data/flume/conf
  2. Change the port if needed and location in HDFS

    nano conf/
  3. Launch the flume agent

     flume-ng agent --conf conf --conf-file conf/ --name a1 Dflume.root.logger=INFO,console
  4. Open a new console and Connect to the same port that you defined in config

     nc localhost 44443
  5. Generate some data by typing something in the console

  6. Open a new console and Check in hdfs using

     hadoop fs -ls flume_webdata
     hadoop fs -cat 'flume_webdata/FlumeData*'

No hints are availble for this assesment

Answer is not availble for this assesment

Loading comments...