So now that you understand a little about the parameters and the model that we want to run, it's time to go ahead and train and inspect our network:

val dl = new DeepLearning(dlParams) 
val dlModel = dl.trainModel.get 

The code created the DeepLearning model builder and launched it. By default, the launch of trainModel is asynchronous (that is, it never blocks, but returns a job), but it is possible to wait until the end of computation by calling the method get. You can also explore the job progress in UI or even explore the unfinished model by typing getJobs into the Flow UI (see Figure 18).

The result of the computation is a DeepLearning model - we can directly explore the model and its details from the Spark shell:

println(s"DL Model: ${dlModel}") 

We can also obtain a frame of predictions for the test data directly by calling the score method of the model:

val testPredictions = dlModel.score(testHF) 
 
testPredictions: water.fvec.Frame = 
Frame _95829d4e695316377f96db3edf0441ee (19912 rows and 3 cols): 
         predict                   p0                    p1 
    min           0.11323123896925524  0.017864442175851737 
   mean            0.4856033079851807    0.5143966920148184 
 stddev            0.1404849885490033   0.14048498854900326 
    max            0.9821355578241482    0.8867687610307448 
missing                           0.0                   0.0 
      0        1   0.3908680007591152    0.6091319992408847 
      1        1   0.3339873797352686    0.6660126202647314 
      2        1   0.2958578897481016    0.7041421102518984 
      3        1   0.2952981947808155    0.7047018052191846 
      4        0   0.7523906949762337   0.24760930502376632 
      5        1   0.53559438105240... 

The table contains three columns:

• predict: Predicted value based on default threshold
• p0: Probability of selecting class 0
• p1: Probability of selecting class 1

We can also get model metrics for the test data:

import water.app.ModelMetricsSupport._ 
val dlMetrics = binomialMM(dlModel, testHF) 
 

The output directly shows the AUC and accuracy (respective error rate). Please note that the model is really good at predicting Higgs-Boson; on the other hand, it has a high False Positive rate!

Finally, let's see how we can build a similar model using the GUI, only this time, we are going to exclude the physicist-hand-derived features from our model and use more neurons for inner layers:

1. Select the model to use for TrainingHF.

As you can see, H2O and MLlib share many of the same algorithms with differing levels of functionality. Here we are going to select Deep Learning and then de-select the last eight hand-derived features.

2. Build DNN and exclude hand-derived features.

Here we are manually choosing to ignore features 21-27, which represent physicist-derived features in the hope that our network will learn them. Note also the ability to perform k-folds cross - validation should you choose to go this route as well.

3. Specify the network topology.

As you can see, we are going to build a three-layer DNN using the rectifier activation function, where each layer will have 1,024 hidden neurons and this will run for 100 epochs.

4. Explore the model results.

After running this model, which takes some time, we can click on the View button to inspect the AUC for both the training and testing set:

If you click your mouse and drag-and-drop on a section of the AUC curve, you can actually zoom in on that particular part of the curve and H2O gives summary statistics about the accuracy and precision at the various thresholds of the selected area.

Also, there is a little button labeled Preview Plain Old Java Object (POJO), which we will explore in the latter chapters, which is how you will deploy your model into a production setting.

OK so we've built a few dozen models; it's time now to begin inspecting our results and figuring which one gives us the best results given the overall error and AUC metric. Interestingly, when we host the many meetups at our office and talk with top kagglers, these types of tables showing results are frequently constructed and it is a good way to keep track of a) what works and what doesn't and b) look back at what you have tried as a form of documentation.

So, which one do we go with? In this case, we like the the GBM model since it provides the second highest AUC value with the lowest accuracy. But always this decision is driven by the modeling goal - in this example, we were strictly motivated by accuracy of the model in finding Higgs-Bosons; however, in other cases, selection of the right model or models can be influenced by various aspects - for example, the time to find and build the best model.