Spark Programming - Spark reserves 20% of memory to shuffle and agregation buffers....

You are currently auditing this course.

47 / 53

Spark reserves 20% of memory to shuffle and agregation buffers. What are these buffer?

These are the temporary data structures required while we invoking operations such as reduceByKey, sort etc.
The libraries that your program depends-on is loaded into this buffer

Loading comments...