Apache Spark Basics

39 / 89

Apache Spark - Transformations - map & filter




Not able to play video? Try with youtube

INSTRUCTIONS
  • Map Transformation code in Scala

    val arr = 1 to 10000
    val nums = sc.parallelize(arr)
    def multiplyByTwo(x:Int):Int = x*2
    

    Write the following command in a new cell:

    multiplyByTwo(5)
    

    Write the following command in a new cell:

    var dbls = nums.map(multiplyByTwo);
    dbls.take(5)
    
  • Transformations - filter() code in Scala

    var arr = 1 to 1000
    var nums = sc.parallelize(arr)
    def isEven(x:Int):Boolean = x%2 == 0
    

    Write the following commands in a new cell:

    var evens = nums.filter(isEven)
    evens.take(3)
    

Loading comments...