Apache Spark Basics

You are currently auditing this course.
69 / 89

Apache Spark - Problem Solving - Compute Average




Not able to play video? Try with youtube

INSTRUCTIONS
  • How to compute average?

    Approach 1:

    var rdd = sc.parallelize(Array(1.0,2,3, 4, 5 , 6, 7), 3);
    var avg = rdd.reduce(_ + _) / rdd.count();
    

    Approach 2:

    var rdd = sc.parallelize(Array(1.0,2,3, 4, 5 , 6, 7), 3);
    var rdd_count = rdd.map((_, 1))
    var (sum, count) = rdd_count.reduce((x, y) => (x._1 + y._1, x._2 + y._2))
    var avg = sum / count
    

Loading comments...