Tuning an algorithm or machine learning application can be thought of as simply a process through which one goes when they optimise the parameters that impact the model in order to enable the algorithm to perform the best (in terms of run-time and memory usages). In this chapter, we have shown how to perform ML model tuning using train-validation split and cross-validation techniques of Spark ML.

We also want to mention that the tuning related support and algorithms are still not well enriched until the date of (14th October 2016) the current release of Spark. Interested readers are encouraged to visit the Spark tuning page at http://spark.apache.org/docs/latest/ml-tuning.html for more updates since we believe that more features will be added to the Spark website and they will certainly provide enough documentation.

In the next chapter, we will discuss how to make your machine learning algorithm or models adaptable for new datasets. This chapter covers advanced machine learning techniques to be able to make algorithms adaptable to new data. It will mainly focus on batch/streaming architectures and on online learning algorithms by using Spark streaming.

The ultimate target is to bring dynamism to the static machine learning models. Readers will also see how machine learning algorithms learn incrementally over the data; that is to say the models are updated each time they see a new training instance.