10.2.1 Problem

We want to create an artificial stock market that replicates real stocks.

10.2.2 Solution

Implement Geometric Brownian Motion. This was invented by Paul Samuelson, Nobel Laureate [36].

10.2.3 How It Works

Paul Samuelson [11] created a stock model based on Geometric Brownian Motion. This approach produces realistic numbers and will not go negative. This is effectively a random walk in log space. The stochastic differential equation is

(10.1)

S is the stock price. W(t) is a Brownian, random walk, process. t is the time, and dt is the time differential. r is the drift, and σ is the volatility of the stock market. Both range from zero to one. It could also be written in differential equation form as

(10.2)

The solution is

(10.3)

The following shows the code used to generate the stock trends. We use cumsum to sum the random numbers for the random walk. We use a Gaussian or normal distribution produced by randn to create the random numbers. The function can create multiple stocks.

The demo is based on the Wilshire 5000 statistics. It is an index of all US stocks. If you run it, you will get different values since the input is random.

The results are shown in Figure 10.1. They look like a real stock. Changing the drift or volatility will change the overall shape. For example, if you set the volatility, σ = 0, you get the very nice stock shown with the red line. Increasing r makes the stock grow faster. This gives us the general rule that we want high r and low σ. See Figure 10.2.

Our model is based on two coefficients. We could make a stock-picking algorithm by just fitting stock price curves and computing r and σ. However, we want to see how well deep learning does. Remember, this is a simple model of stock prices. Both σ and r could also be functions of time or random variables by themselves. Of course, there are other stock models too! The idea here is that deep learning creates its internal model without a need to be told about the model underlying the observed data.

The function PlotStock.m plots the stock price. Notice that we format the y tick labels ourselves to get rid of the exponential format that MATLAB would normally employ. gca returns the current axes handle.

The built-in demo is the same as in StockPrice.

10.3.1 Problem

We want to create a stock market.

10.3.2 Solution

Use the stock price function to create 100 stocks with randomly chosen parameters.

10.3.3 How It Works

We write a function that randomly picks stock starting prices, volatility, and drift. It also creates random three-letter stock names. We use a half-normal distribution for the stock prices. This code generates the random market. We limit the drift to between 0 and 0.5. This creates more stocks (for small markets) that go down.

The following code plots all of the stocks in one plot. We create a legend and make the y labels integers (using PlotStock).

The demo is shown as follows:

Two runs are shown in Figure 10.3.

A stock market with a hundred stocks is shown in Figure 10.4.

10.4.1 Problem

We want to build a deep learning system to predict the performance of a stock. This can be applied to the stock market created earlier to predict the performance of a portfolio.

10.4.2 Solution

The history of a stock is a time series. We will use a Long Short-Term Memory (LSTM) network to predict the future performance of the stock based on past data. Past performance is not necessarily indicative of future results. All investments carry some amount of risk. You are encouraged to consult with a certified financial planner before making any investment decisions. This utilizes the Deep Learning Toolbox’s lstmLayer layer. We will use part of the time series to test the results.

10.4.3 How It Works

An LSTM layer learns long-term dependencies between time steps in a time series. It automatically deweights past data. LSTMs have replaced recursive neural nets (RNNs) in many applications.

The script StockMarketNeuralNet.m implements the neural network. The first part creates a market with a single stock. We set the random number generator to its default value, rng(’default’), so that every time you run the script, you will get the same result. If you remove this line, you will get different results each time. The neural network training data is the time sequence, and the time sequence is shifted by a one-time step.

The stock price is shown in Figure 10.5.

We divide the outputs into training and testing data, using the first 80% of the data for training. We normalize the data which can produce better results when data has a large range. For simplicity in this example, we use the same data for validation and testing, although in a production system these should be different.

The next part of the script sets up and trains the network. We use the “Adam” method [20]. Adam is a first-order gradient-based optimization of stochastic objective functions. It is computationally efficient and works well with problems with noisy or sparse gradients. See the reference for more details. We have a four-layer network including an LSTM layer.

The minimum number of layers when using the LSTM neural net is four, as shown in the following:

The layer structure is shown by analyzeNetwork in Figure 10.6. analyzeNetwork isn’t too interesting for such a simple structure. It is more interesting when you have dozens or hundreds of layers. The script also provides the option to try a bilstm layer and two lstm layers:

1. 1.

   sequenceInputLayer(inputSize) defines a sequence input layer. inputSize is the size of the input sequence at each time step. In our problem, the sequence is just the last value in the time sequence, so inputSize is 1. You could have longer sequences.

    
2. 2.

   lstmLayer(numHiddenUnits) creates a Long Short-Term Memory layer. numHiddenUnits is the number of hidden units in the layer. The number of hidden units is the number of neurons in the layer.

    
3. 3.

   fullyConnectedLayer(outputSize) creates a fully connected layer with specified output size.

    
4. 4.

   regressionLayer creates a regression output layer for a neural network. Regression is data fitting.

    

The learning rate starts at 0.005. It is decreased by a factor of 0.2 every 125 epochs in a piecewise manner using these options:

We let “patience” be inf. This means the learning will continue to the last epoch even if no progress is made. The training window is shown in Figure 10.7. The top plot shows the root-mean-squared error (RMSE) calculated from the data and the bottom plot the loss. We are also using the test data for validation. Note that the validation data needs to be normalized with its own mean and standard deviation.

The final part tests the network using predictAndUpdateState. We need to unnormalize the output for plotting.

Compare Figure 10.8 with Figure 10.5. The red is the prediction. The prediction reproduces the trend of the stock. It gives you an idea of how it might perform. The neural network cannot predict the exact stock history but does recreate the overall performance that is expected.

Results for the bilstm layer and two lstm layers are shown in Figure 10.9 and Figure 10.10. All produce acceptable models. The bilstm looks like it predicts the trends better.

This chapter demonstrated that an LSTM can produce an internal model that replicates the behavior of a system from just observations of the process. In this case, we had a model, but in many systems, a model does not exist or has considerable uncertainty in its form. For this reason, neural nets can be a powerful tool when working with dynamical systems. We haven’t tried this with real stocks.