How humans think and attain consciousness is certainly one of the universe's open questions. There is scarce knowledge on how human beings achieve rational thought or on how physicians make complex clinical decisions. However, perhaps the closest we have come to mimicking human brain performance in common cognitive tasks, as of this writing, is through neural networks and deep learning.

A neural network is modeled after the nervous system of mammals, in which predictor variables are connected to sequential layers of artificial "neurons” that aggregate and sum weighted inputs before sending their nonlinearly transformed outputs to the next layer. In this fashion, the data may pass through several layers before ultimately producing an outcome variable that indicates the likelihood of the target value is positive. The weights are usually trained by using the backpropagation technique, in which the negative difference between the correct output and predicted output is added to the weights at each iteration.

The neural network and the backpropagation technique was first reported in the 1980s in a famous paper published by Nature journal, as was discussed in Chapter 1, Introduction to Healthcare Analytics (Rumelhart et al., 1986); in the 2010s, modern computing power along with vast amounts of data led to the rebranding of neural networks as "deep learning." Along with the increases in computing power and data availability, there have been state-of-the-art performance gains in machine learning tasks, such as speech recognition, image and object identification, and digit recognition.

The fundamental advantage of neural networks is that they are built to handle nonlinearities and complex interactions between predictor variables in the data. This is because each layer in a neural network is essentially performing a linear regression on the output of the previous layer, not simply on the input data itself. The more layers one has in a network, the more complex functions the network can model. The presence of nonlinear transformations in the neurons also contributes to this ability.

Neural networks also easily lend themselves to multiclass problems, in which there are more than two possible outcomes. Recognizing digits 0 through 9 is just one example of this.

Neural networks also have disadvantages. First of all, they have low interpretability and can be difficult to explain to nontechnical stakeholders on a project. Understanding neural networks requires knowledge of college-level calculus and linear algebra.

Second of all, neural networks can be difficult to tune. There are often many parameters involved (for example, how to initialize weights, the number, and size of hidden layers, what activation functions to use, connectivity patterns, regularization, and learning rates) and tuning all of them systematically is close to impossible.

Finally, neural networks are prone to overfitting. Overfitting is when the model has “memorized” the training data and cannot generalize well to previously unseen data. This can happen if there are too many parameters/layers and/or if the data is iterated over too many times.

We will work with neural networks in Chapter 7, Making Predictive Models in Healthcare.