Purpose: Apache Flume is a distributed, reliable, and available software for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows.
Home Page: https://flume.apache.org/
Documentation: https://flume.apache.org/documentation.html
Related resources to learn: https://cloudxlab.com/assessment/playlist-intro/14/flume?course_id=1&playlist_id=14
In the web console tab on the right side of the screen, to get a copy of sample flume conf from common data:
hadoop fs -copyToLocal /data/flume/conf
Change the port if needed and location in HDFS
nano conf/flume.properties
Launch the flume agent
flume-ng agent --conf conf --conf-file conf/flume.properties --name a1 Dflume.root.logger=INFO,console
Open a new console and Connect to the same port that you defined in config
nc localhost 44443
Generate some data by typing something in the console
Open a new console and Check in hdfs using
hadoop fs -ls flume_webdata
hadoop fs -cat 'flume_webdata/FlumeData*'
Taking you to the next exercise in seconds...
Want to create exercises like this yourself? Click here.
No hints are availble for this assesment
Answer is not availble for this assesment
Loading comments...