This section covers the how and why of time series VAEs and gives a couple of examples where they have been used. Time series are such a big topic in finance that Chapter 4, Understanding Time Series, is heavily focused to it.

Autoencoders have found applications in connection to time series as they are able to encode a long time series into a single, descriptive vector. This vector can then, for example, be used to efficiently compare one time series to another time series, based on specific and complex patterns that cannot be captured with a simple correlation, for instance.

Consider the 2010 "Flash Crash." On May 6, 2010, starting at 02:32, US markets saw a major loss of value. The Dow Jones Industrial Average lost about 9%, which equates to about a trillion dollars' worth of value being wiped out in a couple of minutes. 36 minutes later, the crash was over, most of the lost value was regained, and people started wondering what on earth had just happened.

Five years later, a man named Navinder Singh Sarao was arrested for having in part caused the flash crash and having made $40 million in the process. Sarao engaged in a practice called "spoofing" in which he used an automated bot to place large sell orders that could not be filled in the market but would drive prices down.

The bot would leave the orders in the order books of the stock exchange for only a short period of time before canceling them. In the mean time, Sarao would buy the stock at the new lower prices and then profit when the stocks started rebounding after the canceled sales orders. While Sarao was certainly not the only one responsible for the flash crash, practices such as spoofing are now illegal, and exchanges, such as the NASDAQ (US), Tokyo (Japan), and Bombay (India) Stock Exchanges, now have to monitor and flag such cases.

If you dig back into old blog posts about high-frequency trading, such as Bloomberg's Spoofers Keep Markets Honest, which you can view at https://www.bloomberg.com/opinion/articles/2015-01-23/high-frequency-trading-spoofers-and-front-running, then you will find that some traders working at large firms openly recommend spoofing or front-running large orders, but that is a story for another time.

How would we detect when someone engages in spoofing? One way is to use an autoencoder. By using a large amount of order book information, we can train an autoencoder to reconstruct "normal" trading behavior. For traders whose trading patterns deviate a lot from normal trading, the reconstruction loss of the trained autoencoder for the transaction will be quite high.

Another option is to train the autoencoder on different kinds of patterns, whether these are illegal or not, and then cluster the patterns in the latent space, just as we did for the fraudulent credit card transactions.

Recurrent neural networks (RNNs), by default, take in a time series and output a single vector. They can also output sequences if Keras' return_sequences argument is set to True. Using recurrent neural networks such as LSTMs, building an autoencoder for time series can be done using the following code:

from keras.models import Sequential
from keras.layers import LSTM, RepeatVector

model = Sequential()                                            #1
model.add(LSTM(latent_dim, input_shape=(maxlen, nb_features)))  #2
model.add(RepeatVector(maxlen))                                 #3
model.add(LSTM(nb_features, return_sequences=True))             #4

Let's pause for a second and break down what we've just coded. As you can see, there are four key elements to this code:

VAEs can also find their way into trading. They can be used to augment backtesting by generating new, unseen data for testing. Likewise, we can use VAEs to generate data about contracts where data is missing.

It is reasonable to assume that just because two market days look a bit different, the same forces might be at work. Mathematically, we can assume that market data is sampled from a probability distribution, p(x), with a small number of latent variables, h. Using an autoencoder, we can then approximate p(h|x), the distribution of h given x. This will allow us to analyze the driving forces, h, in a market.

This solves the problem that a standard maximum likelihood model for this kind of problem is computationally intractable. Two other methods performing the same feat are the Markov Chain Monte Carlo and Hamilton Monte Carlo methods. While neither will be covered in depth here, though they will be featured in later chapters, it's worth understanding that VAEs address long-standing problems in mathematical finance in a computationally tractable way.

Generative models can also be used to solve problems beyond the scope of traditional methods. Financial markets are fundamentally adversarial environments in which investors are trying to achieve something that is impossible in aggregate: above-average returns. Knowing that a company is doing well is not enough: if everyone knows the company is doing well, then the stock price will be high and returns will be low. The key is knowing that a company is doing well while everyone else believes it is doing poorly. Markets are a zero-sum game-theoretic environment. GANs make use of these dynamics to generate realistic data.