Apache Spark Basics

64 / 89

Apache Spark - Reduce, Commutative & Associative




Not able to play video? Try with vimeo

INSTRUCTIONS
  • More Actions - Reduce()

    var seq = sc.parallelize(1 to 100)
    def sum(x: Int, y:Int):Int = {return x+y}
    var total = seq.reduce(sum);
    

    and

    sum(1,3)
    
  • Using reduce for avg()

    var seq = sc.parallelize(Array(3.0, 7, 13, 16, 19))
    def avg(x: Double, y:Double):Double = {return (x+y)/2}
    var total = seq.reduce(avg);
    

Loading comments...