The instantiation of the MLPNetwork class requires a minimum set of two parameters with an instance of the model as an optional third argument (line 2):

The code is as follows:

class MLPNetwork(config: MLPConfig, 
     topology: Array[Int], 
     model: Option[MLPModel] = None)
     (implicit mode: MLPMode){ //2

  val layers = topology.zipWithIndex.map { case(t, n) => 
    if(topology.size != n+1) 
       MLPLayer(n, t+1, config.activation) 
   else MLPOutLayer(n, t) 
  }  //3
  val connections = zipWithShift1(layers,1).map{case(src,dst) => 
     new MLPConnection(config, src, dst,  model)} //4

  def trainEpoch(x: DblArray, y: DblArray): Double //5
  def getModel: MLPModel  //6
  def predict(x: DblArray): DblArray  //7
}

A MLP network has the following components, which are derived from the topology array:

The topology is defined as an array of number of nodes per layer, starting with the input nodes. The array indices follow the forward path within the network. The size of the input layer is automatically generated from the observations as the size of the features vector. The size of the output layer is automatically extracted from the size of the output vector (line 3).

The constructor for MLPNetwork creates a sequence of layers by assigning and ordering an MLPLayer instance to each entry in the topology (line 3). The constructor creates number of layers – 1 interlayer connections of the MLPConnection type (line 4). The zipWithShift1 method of the XTSeries object zips a time series with its duplicated shift by one element.

The trainEpoch method (line 5) implements the training of this network for a single pass of the entire set of observations (refer to the Putting it all together section under The training epoch). The getModel method retrieves the model (synapses) generated through training of the MLP (line 6). The predict method computes the output value generated from the network using the forward propagation algorithm (line 7).

The following diagram visualizes the interaction between the different components of a model: MLPLayer, MLPConnection, and MLPSynapse:

Core components of the MLP Network

First, let's start with the definition of the MLPLayer layer class, which is completely specified by its position (or rank) id in the network and the number of nodes, numNodes, it contains:

class MLPLayer(val id: Int, val numNodes: Int, 
    val activation: Double => Double)  //8
    (implicit mode: MLPMode){  //9
  
  val output = Array.fill(numNodes)(1.0)  //10

  def setOutput(xt: DblArray): Unit =
      xt.copyToArray(output, 1) //11
  def activate(x: Double): Double = activation(x) .//12
  def delta(loss: DblArray, srcOut: DblArray, 
      synapses: MLPConnSynapses): Delta //13
  def setInput(_x: DblArray): Unit  //14
}

The id parameter is the order of the layer (0 for input, 1 for the first hidden layer, and n – 1 for the output layer) in the network. The numNodes value is the number of elements or nodes, including the bias element, in this layer. The activation function is the last argument of the layer given a user-defined mode or objective (line 8). The operating mode has to be provided implicitly prior to the instantiation of a layer (line 9).

The output vector for the layer is an uninitialized array of values updated during the forward propagation. It initializes the bias value with the value 1.0 (line 9). The matrix of difference of weights, deltaMatrix, associated with the output vector (line 10) is updated using the error backpropagation algorithm, as described in the Step 2 – error back propagation section under The training epoch. The setOutput method initializes the output values for the output and hidden layers during the backpropagation of the error on the output of the network (expected – predicted) values (line 11).

The activate method invokes the activation method (tanh, sigmoid, …) defined in the configuration (line 12).

The delta method computes the correction to be applied to each weight or synapses, as described in the Step 2 – error back propagation section under The training epoch (line 13).

The setInput method initializes the output values for the nodes of the input and hidden layers, except the bias element, with the value x (line 14). The method is invoked during the forward propagation of input values:

def setInput(x: DblVector): Unit = 
  x.copyToArray(output, output.length -x.length)

The methods of the MLPLayer class for the input and hidden layers are overridden for the output layer of the MLPOutLayer type.

Contrary to the hidden layers, the output layer does not have either an activation function or a bias element. The MLPOutLayer class has the following arguments: the order id in the network (as the last layer of the network) and the number, numNodes, of the output or nodes (line 15):

class MLPOutLayer(id: Int, numNodes: Int) 
    (implicit mode: MLP.MLPMode)  //15
  extends MLPLayer(id, numNodes, (x: Double) => x) {

  override def numNonBias: Int = numNodes
  override def setOutput(xt: DblArray): Unit = 
    obj(xt).copyToArray(output)
  override def delta(loss: DblArray, srcOut: DblArray, 
     synapses: MLPConnSynapses): Delta 
 …
}

The numNonBias method returns the actual number of output values from the network. The implementation of the delta method is described in the Step 2 – error back propagation section under The training epoch.

A synapse is defined as a pair of real (a floating point) values:

Its type is defined as MLPSynapse, as shown here:

type MLPSynapse = (Double, Double)

The connections are instantiated by selecting two consecutive layers of an index n (with respect to n + 1) as a source (with respect to destination). A connection between two consecutive layers implements the matrix of synapses as the (w_ij , ∆w_ij) pairs. The MLPConnection instance is created with the following parameters (line 16):

The MLPConnection class is defined as follows:

type MLPConnSynapses = Array[Array[MLPSynapse]]

class MLPConnection(config: MLPConfig, 
    src: MLPLayer, 
    dst: MLPLayer,
    model: Option[MLPModel]) //16
    (implicit mode: MLP.MLPMode) {

  var synapses: MLPConnSynapses  //17
  def connectionForwardPropagation: Unit //18
  def connectionBackpropagation(delta: Delta): Delta  //19
    …
}

The last step in the initialization of the MLP algorithm is the selection of the initial (usually random) values of the weights (synapse) (line 17).

The MLPConnection methods implement the forward propagation of weights' computation for this connectionForwardPropagation connection (line 18) and the backward propagation of the delta error during training connectionBackpropagation (line 19). These methods are described in the next section related to the training of the MLP model.

The initialization values for the weights depends is domain specific. Some problems require a very small range, less than 1e-3, while others use the probability space [0, 1]. The initial values have an impact on the number of epochs required to converge toward an optimal set of weights [9:6].

Our implementation relies on the sigmoid activation function and uses the range [0, BETA/sqrt(numOutputs + 1)] (line 20). However, the user can select a different range for random values, such as [-r, +r] for the tanh activation function. The weight of the bias is obviously defined as w₀ =+1, and its weight adjustment is initialized as ∆w₀ = 0, as shown here (line 20):

var synapses: MLPConnSynapses = if(model == None) {
  val max = BETA/Math.sqrt(src.output.length+1.0) //20
  Array.fill(dst.numNonBias)(
    Array.fill(src.numNodes)((Random.nextDouble*max,0.00))
  )
} else model.get.synapses(src.id)  //21

The connection derives its weights or synapses from a model (line 21) if it has already been created through training.

As mentioned earlier, the output values of a hidden layer are computed as the sigmoid or hyperbolic tangent of the dot product of the weights w_ij and the input values x_i.

In the following diagram, the MLP algorithm computes the linear product of the weights w_ij and input x_i for the hidden layer. The product is then processed by the activation function σ (the sigmoid or hyperbolic tangent). The output values z_j are then combined with the weights v_ij of the output layer that doesn't have an activation function:

The distribution of weights in MLP hidden and output layers

The mathematical formulation of the output of a neuron j is defined as a composition of the activation function and the dot product of the weights w_ij and input values x_i.

Note

M3: The computation (or prediction) of the output layer from the output values z_j of the preceding hidden layer and the weights vkj is defined as:

M4: The estimation of the output values for a binary classification with an activation function σ is defined as:

As seen in the network architecture section, the output values for the multinomial (or multiclass) classification with more than two classes are normalized using an exponential function, as described in the following Softmax section.

The computational flow

The computation of the output values y from the input x is known as the input forward propagation. For the sake of simplicity, we represent the forward propagation between layers with the following block diagram:

A computation model of the input forward propagation

The preceding diagram conveniently illustrates a computational model for the input forward propagation, as the programmatic relation between the source and destination layers and their connectivity. The input x is propagated forward through each connection.

The connectionForwardPropagation method computes the dot product of the weights and the input values and applies the activation function, in the case of hidden layers, for each connection. Therefore, it is a member of the MLPConnection class.

The forward propagation of input values across the entire network is managed by the MLP algorithm itself. The forward propagation of the input value is used in the classification or prediction y = f(x). It depends on the value weights w_ij and v_ij that need to be estimated through training. As you may have guessed, the weights define the model of a neural network similar to the regression models. Let's take a look at the connectionForwardPropagation method of the MLPConnection class:

def connectionForwardPropagation: Unit = {
  val _output = synapses.map(x => {
    val dot = inner(src.output, x.map(_._1) ) //27
    dst.activate(dot)  //28
  })
  dst.setOutput(_output) //29
}

The first step is to compute the linear inner (or dot) product (refer to the Time series in Scala section in Chapter 3, Data Preprocessing) of the output, _output, of the current source layer for this connection and the synapses (weights) (line 27). The activation function is computed by applying the activate method of the destination layer to the dot product (line 28). Finally, the computed value, _output, is used to initialize the output for the destination layer (line 29).

Error functions

As mentioned in the Problem types (modes) section, there are two approaches to compute the error or loss on the output values:

• The sum of the squared errors between expected and predicted output values, as defined in the M5 mathematical expression
• Cross-entropy of expected and predicted values described in the M6 and M7 mathematical formulas

Note

M5: The sum of the squared errors ε and mean square error for predicted values ~y and expected values y are defined as:

M6: Cross entropy for a single output value y is defined as:

M7: Cross entropy for a multivariable output vector y is defined as:

The sum of squared errors and mean squared error functions have been described in the Time series in Scala section in Chapter 3, Data Preprocessing.

The crossEntropy method of the XTSeries object for a single variable is implemented as follows:

def crossEntropy(x: Double, y: Double): Double = 
  -(x*Math.log(y) + (1.0 - x)*Math.log(1.0 - y))

The computation of the cross entropy for multiple variable features as a signature is similar to the single variable case:

def crossEntropy(xt: DblArray, yt: DblArray): Double = 
  yt.zip(xt).aggregate(0.0)({ case (s, (y, x)) => 
    s - y*Math.log(x)}, _ + _)

trait MLPMode { 
  def apply(output: DblArray): DblArray   //30
  def error(labels: DblArray, output: DblArray): Double = 
    mse(labels, output)  //31
}

class MLPBinClassifier extends MLPMode {   
  override def apply(output: DblArray): DblArray = 
    output.map(sigmoid(_))  //32
  override def error(labels: DblArray,  
      output: DblArray): Double = 
    crossEntropy(labels.head, output.head)  //33
}

class MLPRegression extends MLPMode  {
  override def apply(output: DblArray): DblArray = output
}

class MLPMultiClassifier extends MLPMode {
  override def apply(output: DblArray):DblArray = softmax(output)
}

Softmax

In the case of a classification problem with K classes (K > 2), the output has to be converted into a probability [0, 1]. For problems that require a large number of classes, there is a need to boost the output y_k with the highest value (or probability). This process is known as exponential normalization or softmax [9:8].

Note

M8: The softmax formula for the multinomial (K > 2) classification is as follows:

Here is the simple implementation of the softmax method of the MLPMultiClassifier class:

def softmax(y: DblArray): DblArray = {
  val softmaxValues = new DblArray(y.size)
  val expY = y.map( Math.exp(_))  //34
  val expYSum = expY.sum  //35
  
  expY.map( _ /expYSum).copyToArray(softmaxValues, 1) //36
  softmaxValues
}

The softmax method implements the M8 mathematical expression. First, the method computes the expY exponential values of the output values (line 34). The exponentially transformed outputs are then normalized by their sum, expYSum, (line 35) to generate the array of the softmaxValues output (line 36). Once again, there is no need to update the bias element y(0).

The second step in the training phase is to define and initialize the matrix of delta error values to be back propagated between layers from the output layer back to the input layer.

The error backpropagation is an algorithm that estimates the error for the hidden layer in order to compute the change in weights of the network. It takes the sum of squared errors of the output as the input.

Weights' adjustment

The connection weights ∆v and ∆w are adjusted by computing the sum of the derivatives of the error, over the weights scaled with a learning factor. The gradient of weights are then used to compute the error of the output of the source layer [9:9].

The simplest algorithm to update the weights is the gradient descent [9:10]. The batch gradient descent was introduced in Let's kick the tires in Chapter 1, Getting Started.

The gradient descent is a very simple and robust algorithm. However, it can be slower in converging toward a global minimum than the conjugate gradient or the quasi-Newton method (refer to the Summary of optimization techniques section in the Appendix A, Basic Concepts).

There are several methods available to speed up the convergence of the gradient descent toward a minimum, such as the momentum factor and adaptive learning coefficient [9:11].

Large variations of the weights during training increase the number of epochs required for the model (connection weights) to converge. This is particularly true for a training strategy known as online training. The training strategies are discussed in the next section. The momentum factor α is used for the remaining section of the chapter.

Note

M9: The learning rate

The computation of neural network weights using the gradient descent is as follows:

M10: The learning rate and momentum factor

The computation of neural network weights using the gradient descent method with the momentum coefficient α is as follows:

The simplest version of the gradient descent algorithm (M9) is selected by simply setting the momentum factor α to zero in the generic (M10) mathematical expression.

The error propagation

The objective of the training of a perceptron is to minimize the loss or cumulative error for all the input observations as either the sum of squared errors or the cross entropy as computed at the output layer. The error ε_k for each output neuron y_k is computed as the difference between a predicted output value and label output value. The error cannot be computed on output values of the hidden layers z_j because the label values for those layers are unknown:

An illustration of the back-propagation algorithm

In the case of the sum of squared errors, the partial derivative of the cumulative error over each weight of the output layer is computed as the composition of the derivative of the square function and the derivative of the dot product of weights and the input z.

As mentioned earlier, the computation of the partial derivative of the error over the weights of the hidden layer is a bit tricky. Fortunately, the mathematical expression for the partial derivative can be written as the product of three partial derivatives:

• The derivative of the cumulative error ε over the output value y_k
• The derivative of the output value yk over the hidden value z_j, knowing that the derivative of a sigmoid σ is σ(1 - σ)
• The derivative of the output of the hidden layer z_j over the weights w_ij

The decomposition of the partial derivative produces the following formulas for updating the synapses' weights for the output and hidden neurons by propagating the error (or loss) ε.

Note

Output weights' adjustment

M11: The computation of delta δ and weight adjustment ∆v for the output layer with the predicted value ~y and expected value y, and output z of the hidden layer is as follows:

Hidden weights' adjustment

M12: The computation of delta δ and weight adjustment ∆w for the hidden layer with the predicted value ~y and expected value y, output z of the hidden layer, and the input value x is as follows:

The matrix δ_ij is defined by the delta matrix in the Delta class. It contains the basic parameters to be passed between layers, traversing the network from the output layer back to the input layer. The parameters are as follows:

• Initial loss or error computed at the output layer
• Matrix of the delta values from the current connection
• Weights or synapses of the downstream connection (or connection between the destination layer and the following layer)

The code will be as follows:

case class Delta(val loss: DblArray, 
  val delta: DblMatrix = Array.empty[DblArray],
  val synapses: MLPConnSynapses = Array.empty[Array[MLPSynapse]] )

The first instance of the Delta class is generated for the output layer using the expected values y, then propagated to the preceding hidden layer in the MLPNetwork.trainEpoch method (line 24):

val diff = (x: Double, y: Double) => x - y
Delta(zipToArray(y, layers.last.output)(diff))

The M11 mathematical expression is implemented by the delta method of the MLPOutLayer class:

def delta(error: DblArray, srcOut: DblArray, 
     synapses: MLPConnSynapses): Delta = {

  val deltaMatrix = new ArrayBuffer[DblArray] //34
  val deltaValues = error./:(deltaMatrix)( (m, l) => {
    m.append( srcOut.map( _*l) )
    m
  })   //35
  new Delta(error, deltaValues.toArray, synapses)  //36
}

The method generates the matrix of delta values associated with the output layer (line 34). The M11 formula is actually implemented by the fold over the srcOut output value (line 35). The new delta instances are returned to the trainEpoch method of MLPNetwork and backpropagated to the preceding hidden layer (line 36).

The delta method of the MLPLayer class implements the M12 mathematical expression:

def delta(oldDelta: DblArray, srcOut: DblArray, 
     synapses: MLPConnSynapses): Delta = {
  
  val deltaMatrix = new ArrayBuffer[(Double, DblArray)]
  val weights = synapses.map(_.map(_._1))
       .transpose.drop(1) //37

  val deltaValues = output.drop(1)
   .zipWithIndex./:(deltaMatrix){  // 38
     case (m, (zh, n)) => {
       val newDelta = inner(oldDelta, weights(n))*zh*(1.0 - zh)
       m.append((newDelta, srcOut.map( _ * newdelta) )
       m
     } 
  }.unzip
  new Delta(deltaValues._1.toArray, deltaValues._2.toArray)//39
}

The implementation of the delta method is similar to the MLPOutLayer.delta method. It extracts the weights v from the output layer through transposition (line 37). The values of the delta matrix in the hidden connection is computed by applying the M12 formula (line 38). The new delta instance is returned to the trainEpoch method (line 39) to be propagated to the preceding hidden layer if one exists.

The computational model

The computational model for the error backpropagation algorithm is very similar to the forward propagation of the input. The main difference is that the propagation of δ (delta) is performed from the output layer to the input layer. The following diagram illustrates the computational model of the backpropagation in the case of two hidden layers z_s and z_t:

An illustration of the backpropagation of the delta error

The connectionBackPropagation method propagates the error back from the output layer or one of the hidden layers to the preceding layer. It is a member of the MLPConnection class. The backpropagation of the output error across the entire network is managed by the MLP class.

It implements the two set of equations where synapses(j)(i)._1 are the weights w_ji, dst.delta is the vector of the error derivative in the destination layer, and src.delta is the error derivative of the output in the source layer, as shown here:

def connectionBackpropagation(delta: Delta): Delta = {  //40
  val inSynapses =  //41
    if( delta.synapses.length > 0) delta.synapses 
    else synapses 
  
  val delta = dst.delta(delta.loss, src.output,inSynapses) //42
  synapse = synapses.zipWithIndex.map{ //43
    case (synapsesj, j) => synapsesj.zipWithIndex.map{
      case ((w, dw), i) => { 
        val ndw = config.eta*connectionDelta.delta(j)(i)
        (w + ndw - config.alpha*dw, ndw)
      } 
    }
  }
  new Delta(connectionDelta.loss, 
       connectionDelta.delta, synapses)
}

The connectionBackPropagation method takes delta associated with the destination (output) layer as an argument (line 40). The output layer is the last layer of the network, and therefore, the synapses for the following connection is defined as an empty matrix of length zero (line 41). The method computes the new delta matrix for the hidden layer using the delta.loss error and output from the source layer, src.output (line 42). The weights (synapses) are updated using the gradient descent with the momentum factor as in the M10 mathematical expression (line 43).

Note

The adjustable learning rate

The computation of the new weights of a connection for each new epoch can be further improved by making the learning adjustable.

The convergence criterion consists of evaluating the cumulative error (or loss) relevant to the operating mode (or problem) against a predefined eps convergence. The cumulative error is computed using either the sum of squares error formula (M5) or the cross-entropy formula (M6 and M7). An alternative approach is to compute the difference of the cumulative error between two consecutive epochs and apply the eps convergence criteria as the exit condition.

The MLP class is defined as a data transformation of the ITransform type using a model implicitly generated from a training set, xt, as described in the Monadic data transformation section in Chapter 2, Hello World! (line 44).

The MLP algorithm takes the following parameters:

The V type of the output of the prediction or classification method |> of this implicit transform is DblArray (line 45):

class MLP[T <: AnyVal](config: MLPConfig, 
    hidden: Array[Int] = Array.empty[Int],
    xt: XVSeries[T], 
    expected: XVSeries[T])
    (implicit mode: MLPMode, f: T => Double) 
  extends ITransform[Array[T]](xt) with Monitor[Double] {  //44

  type V = DblArray  //45

  lazy val topology = if(hidden.length ==0) 
    Array[Int](xt.head.size, expected.head.size) 
  else  Array[Int](xt.head.size) ++ hidden ++ 
         Array[Int](expected.head.size)  //46

  val model: Option[MLPModel] = train
  def train: Option[MLPModel]   //47
  override def |> : PartialFunction[Array[T], Try[V]] 
}

The topology is created from the xt input variables, the expected values, and the configuration of hidden layers, if any (line 46). The generation of the topology from parameters of the MLPNetwork class is illustrated in the following diagram:

Topology encoding for multi-layer perceptrons

For instance, the topology of a neural network with three input variables: one output variable and two hidden layers of three neurons each is specified as Array[Int](4, 3, 3, 1). The model is generated through training by invoking the train method (line 47). Finally, the |> operator of the ITransform trait is used for classification, prediction, or regression, depending on the selected operating mode (line 48).

There are two approaches to find the most appropriate network architecture for a given classification or regression problem, which are follows:

The destructive tuning strategy removes the synapses by zeroing out their weights. This is commonly accomplished using regularization.

You have seen that regularization is a powerful technique to address overfitting in the case of the linear and logistic regression in the Ridge regression section in Chapter 6, Regression and Regularization. Neural networks can benefit from adding a regularization term to the sum of squared errors. The larger the regularization factor is, the more likely some weights will be reduced to zero, thus reducing the scale of the network [9:13].

The MLPModel instance is created (trained) during the instantiation of the multilayer perceptron. The constructor iterates through the training cycles (or epochs) over all the data points of the xt time series, until the cumulative is smaller than the eps convergence criteria, as shown in the following code:

def train: Option[MLPModel] = {
  val network = new MLPNetwork(config, topology) //48
  val zi =  xt.toVector.zip(expected.view)   // 49

  Range(0, config.numEpochs).find( n => {  //50
    val cumulErr = fisherYates(xt.size)
       .map(zi(_))
       .map{ case(x, e) => network.trainEpoch(x, e)}
       .sum/st.size   //51
     cumulErr  < config.eps  //52
  }).map(_ => network.getModel)
}

The train method instantiates an MLP network using the configuration and topology as the input (line 48). The method executes multiple epochs until either the gradient descent with a momentum converges or the maximum number of allowed iterations is reached (line 50). At each epoch, the method shuffles the input values and labels using the Fisher-Yates algorithm, invokes the MLPNetwork.trainEpoch method, and computes the cumulErr cumulative error (line 51). This particular implementation compares the value of the cumulative error against the eps convergence criteria as the exit condition (line 52).

Lazy views are used to reduce the unnecessary creation of objects (line 49).

Once the model is created during the instantiation of the multilayer perceptron, it is available to predict or classify the class of a new observation.

The Step 5 – implementing the classifier section under Let's kick the tires in Chapter 1, Getting Started, describes a home grown shuffling algorithm as an alternative to the scala.util.Random.shuffle method of the Scala standard library. This section describes an alternative shuffling mechanism known as the Fisher-Yates shuffling algorithm:

def fisherYates(n: Int): IndexedSeq[Int] = {

   def fisherYates(seq: Seq[Int]): IndexedSeq[Int] = {
     Random.setSeed(System.currentTimeMillis)
    (0 until seq.size).map(i => {
       var randomIdx: Int = i + Random.nextInt(seq.size-i) //53
       seq(randomIdx) ^= seq(i)    //54
       seq(i) = seq(randomIdx) ^ seq(i) 
       seq(randomIdx) ^= (seq(i))
       seq(i)
    })
  }

  if( n <= 0)  Array.empty[Int]
  else 
    fisherYates(ArrayBuffer.tabulate(n)(n => n)) //55
}

The Fisher-Yates algorithm creates an ordered sequence of integers (line 55), and swaps each integer with another integer, randomly selected from the remaining of the initial sequences (line 52). This implementation is particularly fast because the integers are swapped in place using the bit operator, also known as bitwise swap (line 54).

The |> data transformation implements the runtime classification/prediction. It returns the predicted value that is normalized as a probability if the model was successfully trained and None otherwise. The methods invoke the forward prediction function of MLPNetwork (line 53):

override def |> : PartialFunction[Array[T],Try[V]] ={
  case x: Array[T] if(isModel && x.size == dimension(xt)) => 
   Try(MLPNetwork(config, topology, model).predict(x)) //56
}

The predict method of MLPNetwork computes the output values from an input x using the forward propagation as follows:

def predict(x: DblArray): DblArray = {
   layers.head.set(x)
   connections.foreach( _.connectionForwardPropagation)
   layers.last.output
}

The fitness of a model measures how well the model fits the training set. A model with a high-degree of fitness will likely overfit. The fit fitness method computes the mean squared errors of the predicted values against the labels (or expected values) of the training set. The method returns the percentage of observations for which the prediction value is correct, using the higher order count method:

def fit(threshold: Double): Option[Double] = model.map(m => 
  xt.map( MLPNetwork(config, topology, Some(m)).predict(_) )
    .zip(expected)
    .count{case (y, e) =>mse(y, e.map(_.toDouble))< threshold }
    /xt.size.toDouble
)

Our MLP class is now ready to tackle some classification challenges.