This is a simple test that has a lot of hidden complexity. Let's start by creating a list of the record, as shown in the following code block:
class InheritanceRdd extends FunSuite {
val spark: SparkContext = SparkSession
.builder().master("local[2]").getOrCreate().sparkContext
test("use extended RDD") {
//given
val rdd = spark.makeRDD(List(Record(1, "d1")))
The Record is just a case class that has an amount and description, so the amount is 1 and d1 is the description.
We then created MultipledRDD and passed rdd to it, and then set the multiplier equal to 10, as shown in the following code:
val extendedRdd = new MultipliedRDD(rdd, 10)
We are passing the parent RDD because it has data that was loaded in another RDD. In this way, we build the inheritance chain of two RDD's.