Continuing the same project of stock price prediction from the last chapter, in this chapter I will introduce and explain neural network models in depth. We will start by building the simplest neural network and go deeper by adding more layers to it. We will cover neural network building blocks and other important concepts, including activation functions, feedforward, and backpropagation. We will also implement neural networks from scratch with scikit-learn and TensorFlow. We will pay attention to how to learn with neural networks efficiently without overfitting, utilizing dropout and early stopping techniques. Finally, we will train a neural network to predict stock prices and see whether it can beat what we achieved with the three regression algorithms in the previous chapter.
We will cover the following topics in this chapter:
Here comes probably the most frequently mentioned model in the media, artificial neural networks (ANNs); more often we just call them neural networks. Interestingly, the neural network has been (falsely) considered equivalent to machine learning or artificial intelligence by the general public.
The ANN is just one type of algorithm among many in machine learning. And machine learning is a branch of artificial intelligence. It is one of the ways we achieve general artificial intelligence.
Regardless, it is one of the most important machine learning models and has been rapidly evolving along with the revolution of deep learning (DL). Let's first understand how neural networks work.
I will first talk about different layers in a network, then the activation function, and finally training a network with backpropagation.
A simple neural network is composed of three layers: the input layer, hidden layer, and output layer, as shown in the following diagram: