10 Things to Look for When Choosing a Big Data course / Institute

Every now and then, I keep seeing a new company coming up with Hadoop classes/courses. Also, my friends keep asking me which of these courses is good to take. I gave them a few tips to choose the best course suitable for them. Here are the few tips to decide which course you should attend to:

1. Does the instructor have domain expertise?

Know your instructor. You must know about the instructor’s background. Has (s)he done any big data related work? I have seen a lot of instructors who just attend a course somewhere and become instructors.

If the instructor never worked in the domain, do not take such classes. Also, avoid training institutes that do not tell you details about the instructor.

2. Is the instructor hands on? When did she/he code last time?

In the domain of technology, there is a humongous difference between one instructor who is hands-on in coding and another who is delivering based on theoretical knowledge. Also, know when the instructor worked on codes the last time. If instructor never coded, do not attend the class.

3. Does the instructor encourage & answer your questions?

There are many recorded free videos available across the internet. The only reason you would go for live classes would be to get your questions answered and doubts cleared immediately.

If the instructor does not encourage questions and answers, such classes are fairly useless.

Continue reading “10 Things to Look for When Choosing a Big Data course / Institute”

Introduction to Apache Flume in 30 minutes

What is Apache Flume?

Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating & moving large data from many different sources to a centralized data store.

Flume supports a large variety of sources Including:

  • tail (like unix tail -f),
  • syslog,
  • log4j – allowing Java applications to write logs to HDFS via flume

Flume Nodes

Flume nodes can be arranged in arbitrary topologies.Typically there is a node running on each source machine, with tiers of aggregating nodes that the data flows through on its way to HDFS.

Topics Covered

  • What is Flume
  • Flume: Use Case
  • Flume: Agents
  • Flume: Use Case – Agents
  • Flume: Multiple Agents
  • Flume: Sources
  • Flume: Delivery Reliability
  • Flume: Hands-on

Introduction to Flume Presentation

 

Please feel free to leave your comments in the comment box so that we can improve the guide and serve you better. Also, Follow CloudxLab on Twitter to get updates on new blogs and videos.

If you wish to learn Hadoop and Spark technologies such as MapReduce, Hive, HBase, Sqoop, Flume, Oozie, Spark RDD, Spark Streaming, Kafka, Data frames, SparkSQL, SparkR, MLlib, GraphX and build a career in BigData and Spark domain then check out our signature course on Big Data with Apache Spark and Hadoop which comes with

  • Online instructor-led training by professionals having years of experience in building world-class BigData products
  • High-quality learning content including videos and quizzes
  • Automated hands-on assessments
  • 90 days of lab access so that you can learn by doing
  • 24×7 support and forum access to answer all your queries throughout your learning journey
  • Real-world projects
  • A certificate which you can share on LinkedIn

Machine Learning with Mahout

[This blog is from KnowBigData.com. It is pretty old. Many things have changed since then. People have moved to MLLib. We have also moved to CloudxLab.com.]

What is Machine Learning?

Machine Learning is programming computers to optimize a Performance using example data or past experience, it is a branch of Artificial Intelligence.

Types of Machine Learning

Machine learning is broadly categorized into three buckets:

  • Supervised Learning – Using Labeled training data, to create a classifier that can predict the output for unseen inputs.
  • Unsupervised Learning – Using Unlabeled training data to create a function that can predict the output.
  • Semi-Supervised Learning – Make use of unlabeled data for training – typically a small amount of labeled data with a large amount of unlabeled data.

Machine Learning Applications

  • Recommend Friends, Dates, Products to end-user.
  • Classify content into pre-defined groups.
  • Find Similar content based on Object Properties.
  • Identify key topics in large Collections of Text.
  • Detect Anomalies within given data.
  • Ranking Search Results with User Feedback Learning.
  • Classifying DNA sequences.
  • Sentiment Analysis/ Opinion Mining
  • Computer Vision.
  • Natural Language Processing,
  • BioInformatics.
  • Speech and HandWriting Recognition.

Mahout

Mahout – Keeper/Driver of Elephants. Mahout is a Scalable Machine Learning Library built on Hadoop, written in Java and its Driven by Ng et al.’s paper “MapReduce for Machine Learning on Multicore”. Development of Mahout Started as a Lucene sub-project and it became Apache TLP in Apr’10.

Topics Covered

  • Introduction to Machine Learning and Mahout
  • Machine Learning- Types
  • Machine Learning- Applications
  • Machine Learning- Tools
  • Mahout – Recommendation Example
  • Mahout – Use Cases
  • Mahout Live Example
  • Mahout – Other Recommender Algos

Machine Learning with Mahout Presentation

Machine Learning with Mahout Video

6 Reasons Why Big Data Career is a Smart Choice

Confused whether to take up a career in Big Data or not? Planning to invest your time in getting certified and to acquire expertise in related frameworks like Hadoop, Spark etc. and worried whether you are making a huge mistake? Just spend a few minutes reading this blog and you will get six reasons on why you are making a smart choice by selecting a career in big data.

Why Big Data?

There are several people out there who believe that Big Data is the next big thing which would help companies to spring up above others and help them position themselves as the best in class in their respective sectors.

Companies these days generate a gigantic amount of information irrespective of which industry they belong to and there is a need to store these data which are being generated so that they can be processed and not miss out on important information which could lead to a new breakthrough in their respective sector.  Atul Butte, of Stanford School of Medicine, has stressed the importance of data by saying “Hiding within those mounds of data is the knowledge that could change the life of a patient, or change the world”. And this is where Big Data analytics play a very crucial role.

With the use of Big Data platforms, a gigantic amount of data can be brought together and be processed to develop patterns which would help the company in making better decisions which would help them to grow, increase their productivity and to help create value to their products and services.

Continue reading “6 Reasons Why Big Data Career is a Smart Choice”

One Day Machine Learning Bootcamp at IITB – CloudxLab

Our past two Bootcamp on Machine Learning at National Singapore University and RV College of Engineering were very interesting and all the attendees found it very useful. These feedbacks prompted us to have more Bootcamps like these.

Thanks to Prof. Alankar, who invited us to conduct yet another Machine Learning Bootcamp at Indian Institute of Technology, Bombay. Before we move on to the details of Bootcamp, let us give you a brief introduction to Prof. Alankar. He is an Assistant Professor at IIT Bombay in Mechanical Engineering Department and works in the area of Multiscale Modeling of Deformation. He is a graduate of IIT Roorkee, holds a masters degree from University of British Columbia (Canada) and doctoral degree from Washington State University (USA). He has previously worked at Max-Planck Institute (Germany), Los Alamos National Laboratory (USA) and Modumetal, Inc (USA).

Machine Learning Bootcamp

So it all happened on Mar 17 where Machine Learning enthusiasts, which includes professors and students from every branch of IIT, gathered to attend the one day workshop on Machine Learning. The presenter was none other than Mr. Sandeep Giri, who has over 15 years of experience in the domain of Machine learning and Big Data technologies. He has worked in companies like Amazon, InMobi, and D. E. Shaw.

Continue reading “One Day Machine Learning Bootcamp at IITB – CloudxLab”

A Successful Machine Learning Bootcamp by CloudxLab – Singapore

Cloudxlab has conducted many successful online events on Machine Learning and Big Data, for it is relatively easy to attend so many attendees simultaneously. Furthermore, it eliminates the need for a tiring visit to the event location. One can simply log in from the comfort of one’s house and start learning.

Sure, online events have their own perks. But it hasn’t stopped us to conduct offline events. Our Machine Learning session at R.V College of engineering was one such successful example.

This time, we wanted to conduct a little bigger event, therefore Cloudxlab joined hands together with IOTSG and National Singapore university, enterprise to organize another successful Machine Learning Bootcamp.

The Venue

CloudxLab was organizing the Machine learning Bootcamp for the first time in Singapore. To be frank, we were a little nervous as we did not know how welcoming the country is. But all our minor doubts were cleared once we experienced the warm welcome from everyone there. So much so that we would like to do one more Bootcamp in Singapore in near future.

National Singapore University was very cooperative in helping us to organize the Bootcamp in their campus.

Continue reading “A Successful Machine Learning Bootcamp by CloudxLab – Singapore”

Machine Learning & IoT Bootcamp – Singapore

Have you ever wondered how you can apply various Machine Learning and IOT techniques for everyday business problems? Or, are you someone who has heard of Machine Learning but couldn’t get a chance to dig a little deeper?  If your answer is Yes, then you’ve come to the right place.

Cloudxlab is conducting a Machine Learning & IoT Bootcamp in Singapore.

  • Date: Saturday, Feb 10, 2018
  • Place: NUS Enterprise, #02-01, 71 Ayer Rajah Crescent, Singapore
  • Time: 9:30 AM to 5:00 PM

What will be covered?

An exposure to Machine Learning using Python to analyze, draw intelligence and build powerful models using real-world datasets. You’ll also gain the insights to apply data processing and Machine Learning techniques in real time.

After completing this workshop, you will be able to build and optimize your own automated classifier to extract insights from real-world data sets.

Continue reading “Machine Learning & IoT Bootcamp – Singapore”

The Pursuit of Education – A Story of Strength

Today, we will not talk tech or discuss our regular tutorials. Instead, we will take you on a different journey – a journey about strength, a journey about hope, and a journey on life.

It was a regular working day for us when an email caught our attention. It was from an individual who faced unimaginable hardships in his life but still hopes for a better future by executing his passion for learning.

His message was rather long and it clearly showed that he was in desperate need of a higher education. We thought he was a student, and we offered him the student’s discount on one of our self-paced courses on Big Data. But much to our surprise, he was not in a state to pay even the discounted price.

We were not clear on why he would be requesting for a free course. However, we came to know about the kind of hardship that he had recently gone through, and about his real mission to move back to his native place and help poor and needy students by providing free education.

He was a Rohingya refugee and had lost his entire family in the recent clashes of Myanmar. He managed to survive the traumatic ordeal but thinking of a new life was more of an impossible dream for him. However, he stepped up and decided to move on with his life.

He wanted to continue his education, and therefore, started to look out for a Big Data course that he could do for free because of his terrible financial crisis. He came across CloudxLab and got in touch with us explaining his situation. He also mentioned that he wanted to help the needy back in his country for which he needed to go through the course.

We were much in awe of this person’s strength of mind. He came across as an epitome of strength who is ruthlessly following his dream despite all odds and ordeals.

We offered him our course at no cost, but we did not know how much it meant to him until he sent his reply:

I can’t explain my feelings in words how much happy I am now. You are an angel for me who help me to stand on my feet. Sir thank you for believing in me and giving me a chance to continue my dream. I promise I will do my best and complete the course as fast as I can. Thank you.

This is probably our biggest achievement as a team.

We salute this individual for his unthinkable strength in facing such a catastrophe in his life while nurturing a selfless desire to help others. We wish him good days ahead and hope that he completes his education and embarks on the journey to help his people.

Streaming Twitter Data using Flume

In this blog post, we will learn how to stream Twitter data using Flume on CloudxLab

For downloading tweets from Twitter, we have to configure Twitter App first.

Create Twitter App

Step 1

Navigate to Twitter app URL and sign in with your Twitter account

Step 2

Click on “Create New App”

Create New App

Continue reading “Streaming Twitter Data using Flume”

A Simple Tutorial on Scala – Part – 2

Welcome back to the Scala tutorial.

This post is the continuation of A Simple Tutorial on Scala – Part – 1

In the Part-1 we learned the following topics on Scala

  • Scala Features
  • Variables and Methods
  • Condition and Loops
  • Variables and Type Inference
  • Classes and Objects

Keeping up the same pace, we will learn the following topics in the 2nd part of the Scala series.

  • Functions Representation
  • Collections
  • Sequence and Sets
  • Tuples and Maps
  • Higher Order Functions
  • Build Tool – SBT

Functions Representation

We have already discussed functions. We can write a function in different styles in Scala. The first style is the usual way of defining a function.

Please note that the return type is specified as Int.

In the second style, please note that the return type is omitted, also there is no “return” keyword. The Scala compiler will infer the return type of the function in this case.

If the function body has just one statement, then the curly braces are optional. In the third style, please note that there are no curly braces.

Continue reading “A Simple Tutorial on Scala – Part – 2”