Purpose: Apache Flume is a distributed, reliable, and available software for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows.
Home Page: https://flume.apache.org/
Documentation: https://flume.apache.org/documentation.html
Related resources to learn: https://cloudxlab.com/assessment/playlist-intro/14/flume?course_id=1&playlist_id=14
In the web console tab on the right side of the screen, to get a copy of sample flume conf from common data:
hadoop fs -copyToLocal /data/flume/conf
Change the port if needed and location in HDFS
nano conf/flume.properties
Launch the flume agent
flume-ng agent --conf conf --conf-file conf/flume.properties --name a1 Dflume.root.logger=INFO,console
Open a new console and Connect to the same port that you defined in config
nc localhost 44443
Generate some data by typing something in the console
Open a new console and Check in hdfs using
hadoop fs -ls flume_webdata
hadoop fs -cat 'flume_webdata/FlumeData*'
Taking you to the next exercise in seconds...
No hints are availble for this assesment
Answer is not availble for this assesment
Loading comments...