About this book

The goal of this book is to provide the definitive guide for anyone interested in learning about Generative Adversarial Networks (GANs) from the ground up. Starting from the simplest examples, we advance to some of the most innovative GAN implementations and techniques. We make cutting-edge research accessible by providing the intuition behind these advances while sparing you all but the essential math and theory.

Ultimately, our goal is to give you the knowledge and tools necessary not only to understand what has been accomplished in GANs to date, but also to empower you to find new applications of your choosing. The generative adversarial paradigm is full of potential to be unraveled by enterprising individuals like you who can make an impact through academic and real-world applications alike. We are thrilled to have you join us on this journey.

Who should read this book

This book is intended for readers who already have some experience with machine learning and neural networks. The following list indicates what you should ideally know. Although we try our best to explain most things as we go, you should be confident about at least 70% of this list:

• We expect you to be able to run intermediate Python programs. You do not need to be a Python master, but you should have at least two years of Python experience (ideally as a full-time data scientist or software engineer).
• You should understand object-oriented programming, how to work with objects, and how to figure out their attributes and methods. You need to be able to understand reasonably typical Python objects (for example, Pandas Data-Frames) as well as atypical ones (for example, Keras layers).
• You should understand the basics of machine learning theory, such as train/test split, overfitting, weights, and hyperparameters, as well as the basics of supervised, unsupervised, and reinforcement learning. You should also be familiar with metrics such as accuracy and mean squared error.
• You should understand basic statistics and calculus, such as probability, density functions, probability distributions, differentiation, and simple optimization.
• You should understand elementary linear algebra, such as matrices, high-dimensional spaces, and, ideally, principal component analysis.
• You should understand the basics of deep learning—things such as feed-forward networks, weights and biases, activation functions, regularization, stochastic gradient descent, and backpropagation.
• You should also have elementary familiarity with, or willingness to independently learn, the Python-based machine learning library Keras.

We are not trying to scare you, but rather ensure that you will get the most out of this book. You may try to take a stab at it anyway, but the less you know, the more you should expect to search online on your own. However, if this list does not seem scary to you, you should be good to go.

About the code

This book contains many examples of source code, both in numbered listings and inline with normal text. In both cases, the source code is formatted in a fixed-width font like this to separate it from ordinary text. Sometimes the code is also in bold to highlight code that has changed from previous steps in the chapter, such as when a new feature adds to an existing line of code.

In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In rare cases, even this was not enough, and listings include line-continuation markers (). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts. The code for the examples in this book is available for download from the Manning website at www.manning.com/books/gans-in-action and from GitHub at https://github.com/GANs-in-Action/gans-in-action.

Throughout this book, we will be using Jupyter notebooks, as it the standard for data science education. Using Jupyter is also a prerequisite, but for intermediate Pythonistas, this should be easy to pick up. We are aware that sometimes it may be difficult to access GPUs or get everything working, especially on Windows. So for some chapters, we also provide Google Colaboratory notebooks (Colab for short), which are Google’s free platform (available at https://colab.research.google.com) and come prepackaged with all the essential data science tools as well as a free GPU for a limited time. You can run all of these lessons straight from your browser! For the other chapters, feel free to upload them to Colab, as the two formats are made to be compatible.

liveBook discussion forum

Purchase of GANs in Action includes free access to a private web forum run by Manning Publications, where you can make comments about the book, ask technical questions, and receive help from the authors and from other users. To access the forum, go to https://livebook.manning.com/#!/book/gans-in-action/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook.manning.com/#!/discussion.

Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the authors can take place. It is not a commitment to any specific amount of participation on the part of the authors, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the authors some challenging questions lest their interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.

Other online resources

GANs are an active field with excellent (albeit fragmented) resources only a Google search away. Those with an academic bent can find the latest papers in arXiv (https://arxiv.org), an online repository of academic e-prints owned and operated by Cornell University. We hope that this book will equip you with all that is needed to keep up-to-date on the latest developments in this ever-changing field.

Both Jakub and Vladimir are active contributors to Medium (particularly the tech-focused publications Towards Data Science and Hacker Noon), where you can find the most recent content from the authors.

How this book is organized: a roadmap

GANs in Action strives to provide a balance of theory and practice. The book is organized into three parts:

Part 1, “Introduction to GANs and generative modeling”

Here, we introduce the foundational concepts behind generative learning and GANs and implement the most canonical GAN variants:

• Chapter 1, “Introduction to GANs”—We introduce Generative Adversarial Networks (GANs) and provide a high-level explanation of how they work. You will learn that GANs consist of two separate neural networks (the Generator and the Discriminator), and the networks are trained through a competitive dynamic. The knowledge you will acquire in this chapter will provide the foundation for the remainder of this book.
• Chapter 2, “Intro to generative modeling with autoencoders”—We discuss autoencoders, which can be seen as precursors to GANs in many ways. Given the relative novelty of generative learning, we decided to include a chapter that helps set GANs in a broader context. This chapter also contains the first code tutorial, where we will build a variational autoencoder to generate handwritten digits—the same task we will be exploring in our GAN tutorials in later chapters. However, if you are already familiar with autoencoders or want to dive straight into GANs, feel free to skip this chapter.
• Chapter 3, “Your first GAN: Generating handwritten digits”—We dive deeper into the theory behind GANs and adversarial learning. We explore the key differences between GANs and traditional neural networks: namely, we discuss the differences in their cost functions and training processes. In a coding tutorial at the end of the chapter, you will apply what you’ve learned to implement a GAN in Keras and train it to generate handwritten digits.
• Chapter 4, “Deep Convolutional GAN”—We introduce convolutional neural networks and batch normalization. We then implement Deep Convolutional GAN (DCGAN), an advanced GAN architecture that uses convolutional networks as its Generator and Discriminator and takes advantage of batch normalization to stabilize the training process.

Part 2, “Advanced topics in GANs”

Building on the foundations, we dive deeper into the theory underlying GANs and implement a selection of advanced GAN architectures:

• Chapter 5, “Training and common challenges: GANing for success”—We discuss many of the theoretical and practical hurdles to training GANs and how to overcome them. We provide a comprehensive overview of the best practices for training a GAN based on relevant academic papers and presentations. We also cover options for evaluating GAN performance and why we need to worry about that.
• Chapter 6, “Progressive growing of GANs”—We explore the Progressive GAN (PGGAN, or ProGAN), a cutting-edge training methodology for the Generator and Discriminator. By adding new layers during the training process, the PGGAN achieves superior image quality and resolution. We explain how it all works in theory as well as in practice through hands-on code samples and by using the TensorFlow Hub (TFHub).
• Chapter 7, “Semi-Supervised GAN”—We continue to explore innovations based on the core GAN model. You will learn about the enormous practical importance of improving classification accuracy with only a small subset of labeled training examples through semi-supervised learning. Then, we implement the Semi-Supervised GAN (SGAN) and explain how it uses labels to turn the Discriminator into a robust multiclass classifier.
• Chapter 8, “Conditional GAN”—We present another GAN architecture that uses labels in training: Conditional GAN (CGAN). Conditional GAN addresses one of the main shortcomings of generative modeling—the inability to specify explicitly what example to synthesize—by using labels or other conditioning information while training its Generator and Discriminator. At the end of the chapter, we implement a CGAN to see targeted data generation firsthand.
• Chapter 9, “CycleGAN”—We discuss one of the most interesting GAN architectures: Cycle-Consistent Adversarial Networks (CycleGANs). This technique can be used to translate one image into another, such as turning a photo of a horse into a photo of a zebra. We walk through the CycleGAN architecture and explain its main components and innovations. As a coding tutorial, we then implement a CycleGAN to convert apples into oranges, and vice versa.

Part 3, “Where to go from here”

We discuss how and where we can apply our knowledge of GANs and adversarial learning:

• Chapter 10, “Adversarial examples”—We look at adversarial examples, a set of techniques to intentionally deceive a machine learning model into making a mistake. We discuss their significance through theory and practical examples and explore their connection to GANs.
• Chapter 11, “Practical applications of GANs”—We cover practical applications of GANs. We explore how to use techniques covered in earlier chapters for real-world use cases in medicine and fashion. In medicine, we look at how GANs can be used to augment a small dataset to improve classification accuracy. In fashion, we show how GANs can drive personalization.
• Chapter 12, “Looking ahead”—We wrap up our learning journey by summarizing the key takeaways and discussing the ethical considerations of GANs. We also mention emerging GAN techniques for those interested in continuing to explore this field beyond this book.

About the authors

Jakub Langr is a cofounder of a startup that uses GANs for creative and advertising applications. Jakub has worked in data science since 2013, most recently as a data science tech lead at Filtered.com and as an R&D data scientist at Mudano. He also designed and teaches data science courses at the University of Birmingham (UK) and at numerous private companies, and is a guest lecturer at the University of Oxford. He was an Entrepreneur in Residence at the seventh cohort of deep technology talent investor Entrepreneur First. Jakub is also a fellow at the Royal Statistical Society and an invited speaker at various international conferences. He graduated from the University of Oxford. Jakub is donating all of his proceeds from this publication to the nonprofit British Heart Foundation.

Vladimir Bok recognized the immense potential of GANs while pursuing an independent research project in musical style transfer at Microsoft Research. His work experience ranges from applied data science at a Y Combinator-backed startup to leading cross-functional initiatives at Microsoft. Most recently, Vladimir has been managing data science projects at a New York-based startup that provides machine learning services to online travel and e-commerce brands, including Fortune 500 companies. Vladimir graduated cum laude with a bachelor’s degree in computer science from Harvard University. He is donating all of his proceeds from this book to the nonprofit organization Girls Who Code.