Creating couple relationships

In this section, we will create couple relationships and edges between our vertices. Here, we'll have a relationship that is an Edge. An Edge is a case class from the org.apache.spark.graphx package. It is a bit more involved because we need to specify the source vertex ID and destination vertex ID. We want to specify that vertex ID 1 and 2 have a relationship, so let's make a label for this relationship. In the following code, we will specify vertex ID 1 and ID 2 as a friend, then we will specify vertex ID 1 and ID 3 as a friend as well. Lastly, vertex ID 2 and ID 4 will be a wife:

    val relationships =
spark.parallelize(Array(
Edge(1L, 2L, "friend"),
Edge(1L, 3L, "friend"),
Edge(2L, 4L, "wife")
))

Also, a label could be of any type—it doesn't need to be a String type; we can type what we want and pass it. Once we have our vertices, users, and edge relationships, we can create a graph. We are using the Graph class' apply method to construct our Spark GraphX graph. We need to pass users, VertexId, and relationships, as follows:

Returning graph is an RDD, but it's a special RDD:

    val graph = Graph(users, relationships)

When we go to the Graph class, we will see that the Graph class has an RDD of vertices and an RDD of edges, so the Graph class is a companion object for two RDDs, as shown in the following screenshot:

We can get the underlying RDD of vertices and edges by issuing some methods. For example, if you want to get all the vertices, we can map all vertices and we will get the attribute and VertexId. Here, we are only interested in the attribute and we will convert it into uppercase, as follows:

    val res = graph.mapVertices((_, att) => att.toUpperCase())

The following are the attributes:

    val users: RDD[(VertexId, (String))] =
spark.parallelize(Array(
(1L, "a"),
(2L, "b"),
(3L, "c"),
(4L, "d")
))

Once we convert it into uppercase, we can just collect all the vertices and perform toList(), as follows:

    println(res.vertices.collect().toList)
}

}

We can see that after applying the transformation to the values, our graph has the following vertices:

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset