Pyspark - Good way to filter | Automated hands-on| CloudxLab

Apache Spark Basics with Python

Home
Assessment

45 / 86

Is this a good way to find elements starting with "cloudxlab" inside an rdd having strings as records:

filter(lambda x:x.startswith("cloudxlab"), myrdd.collect())

Yes, it will be very efficient even if the data is big
No, The collect() on RDD brings data locally and then filters using local scala library fuction. Collect() will overflow the memory if data is big

XP

Taking you to the next exercise in seconds...

Want to create exercises like this yourself? Click here.

Note - Having trouble with the assessment engine? Follow the steps listed here

Result

Checking Please wait.

Success

Error

No hints are availble for this assesment

Answer is not availble for this assesment

Previous Index Next

Loading comments...

Apache Spark Basics with Python

1 Apache Spark with Python - Apache Spark Ecosystem

2 Apache Spark - What is not true about Apache spark?...

3 Apache Spark - What are advantages of using spark?...

4 Apache Spark - If you already have Apache Hadoop cluster setup, you can...

5 Apache Spark - If you need to process data continuously, which library you...

6 Apache Spark - If you need to provide a graph NoSQL storage, you...

7 Apache Spark - Spark does not provide API in which language?...

8 Apache Spark - Which of the following tasks are not possible using Apache...

9 Apache Spark with Python - Why Spark?

10 Apache Spark - Why is Spark faster than Hadoop?...

11 Apache Spark - Which of the following list is in increasing order in...

12 Getting Started with Spark using CloudxLab

13 Apache Spark with Python - Accessing Spark on CloudxLab

14 Apache Spark with Python - Cluster Installation (Optional)

15 Apache Spark - The command for running scala spark interactive shell is:...

16 Apache Spark - The command for talking to Spark using R with the interactive shell on CloudxLab is

17 Apache Spark - Which of the following is not a valid spark shell?...

18 Apache Spark - In Spark 1.x, which objects are provided by the spark-shell...

19 Apache Spark - In Spark 2.x, which objects are provided by the spark-shell...

20 Apache Spark - Whatever command you run on spark-shell or pyspark, they are...

21 Apache Spark - Which of the following is python code thats reads data...

22 Apache Spark - For running something unattended, which command would you use?...

23 Apache Spark with Python - What is an RDD?

24 Apache Spark - The fullform of RDD is...

25 Apache Spark - Which of the following is not true about RDD?...

26 Apache Spark - In case we are creating RDD using sc.textFile(), what does...

27 Apache Spark - In case, we are creating RDD using sc.parallelize(Array(1,2,3,4)), what would...

28 Apache Spark - An RDD is not analogous to:...

29 Apache Spark - Each partition is maintained and processed by:...

30 Apache Spark - The work interacting with Spark Applications is done by:...

31 Apache Spark - When we launch an spark-shell or pyspark, it also launches:...

32 Apache Spark - We can modify an RDD?...

33 Apache Spark with Python - Preparing the environment

34 Apache Spark with Python - Creating RDD

35 Question: How to create RDD using Python?

36 Is this method of creating rdd correct: val myrdd = sc.parallelize(scala.io.Source.fromFile("./myfile").getLines.toList) ?

37 Apache Spark - To get first 10 elements of an rdd myrdd, which...

38 Apache Spark with Python - Counting Word Frequencies

39 Apache Spark with Python - Transformations - map & filter

40 Apache Spark - The operations provided on RDD are classified as:...

41 Apache Spark - What is not true about transformations?...

42 Apache Spark - The argument to map and filter is a function....

43 Apache Spark - What is not true about map transformations?...

44 Apache Spark - We can not implement the following with map:...

45 Pyspark - Good way to filter

46 Apache Spark - We can not implement the following with filter:...

47 Apache Spark - If the following code returned true, someop was fiter or...

48 Apache Spark with Python - Actions - take & saveTextFile

49 Apache Spark - What is not true about action?...

50 Apache Spark - What is not true about saveAsTextFile(arg)?...

51 Apache Spark with Python - Lazy Evaluation & Lineage Graph

52 Apache Spark - Which of the following has lazy evaluation?...

53 Apache Spark - Which of the following gets executed immediately?...

54 Apache Spark - Which of the following gets executed lazily?...

55 Pyspark - Lineage graph - Order of statements

56 Apache Spark - What is not true about lazy evaluation?...

57 Apache Spark with Python - More Operations - Transformations & Actions

58 Apache Spark - Which one is equivalent of Hadoop's map phase?...

59 Apache Spark - Which one can be emulated with flatMap()?...

60 Apache Spark - The number of records in various transformations- M&FM

61 Apache Spark - To concatenate two RDDs, we use:...

62 Apache Spark - Which one of these are not executed in distributed fashion?...

63 Apache Spark with Python - Reduce, Commutative & Associative

64 Apache Spark - The reduce function can not be used to compute which...

65 Apache Spark - Which function is not commutative?...

66 Apache Spark - Which function is not associative?...

67 Apache Spark - Which statement about the function passed to reduce in case...

68 Apache Spark with Python - Problem Solving - Compute Average

69 Apache Spark with Python - More RDD Operations

70 More RDD Ops - Does sample transformation involve sorting?...

71 More RDD Ops - If you want to process the whole partition, which function...

72 More RDD Ops - To order data in an RDD, which function do we...

73 More RDD Ops - On what basis does sortBy transformation orders the data?...

74 More RDD Ops - What is the role of the third argument numPartitions of...

75 More RDD Ops - Can we sort key-value RDD or PairRDD using sortBy function?...

76 More RDD Ops - Is every RDD a set by default?...

77 More RDD Ops - What does union transformation do?...

78 More RDD Ops - What transformation to use to find common elements between two...

79 More RDD Ops - If we need have two RDDs, adjectives = ["good", "bad"]...

80 More RDD Ops - If we need to reduce an RDD into a value...

81 Pyspark - function to sum up values

82 PySpark - Computing Average efficiently

83 Pyspark Word Count

84 More RDD Ops - If you want to process each record of an RDD...

85 More RDD Ops - Say, we want to persist records of an RDD into...

86 More RDD Ops - Does top(n) action involve shuffling?...