Apache Spark - Reduce, Commutative & Associative

More Actions - Reduce()

var seq = sc.parallelize(1 to 100)
def sum(x: Int, y:Int):Int = {return x+y}
var total = seq.reduce(sum);

and

sum(1,3)

Using reduce for avg()

var seq = sc.parallelize(Array(3.0, 7, 13, 16, 19))
def avg(x: Double, y:Double):Double = {return (x+y)/2}
var total = seq.reduce(avg);