For an example of a transformer (and more) check this blog post: https://blog.insightdatascience.com/spark-pipelines-elegant-yet-powerful-7be93afcdd42