About this Book

Algorithms of the Intelligent Web was written to provide you with a roadmap to the design and creation of intelligent algorithms. It draws on many areas of computer science, including machine learning and artificial intelligence, but has been written with the practitioner in mind. It’s a cookbook of sorts and provides the relative newcomer in this field with several real-world, worked examples that can be modified for your own purpose.

Who should read this book

This book is firmly aimed at those who are new to writing intelligent algorithms but who have a firm grasp of programming and basic math and statistics. Wherever possible, we’ve tried to write the book so the mathematical rigor can be glossed over, leaving you with an overall impression of the applicability of an approach. Of course, if you’re more mathematically inclined, we encourage you to follow the detail. Ideally, readers of this book should have had at least some exposure to a programming language and have attended a first-year undergraduate mathematics course.

Roadmap

This book is organized into eight chapters and one appendix:

• Chapter 1 provides an overview of intelligent algorithms and outlines some key characteristics. It also provides you with guiding principles for the rest of the book.
• Chapter 2 discusses the concept of structure within your data. In particular, it introduces the concept of the feature space. In this chapter, we introduce expectation maximization and eigenvectors.
• Chapter 3 introduces recommender systems. We introduce collaborative filtering techniques and discuss the Netflix Prize.
• Chapter 4 outlines classification techniques and introduces logistic regression. We use logistic regression to perform fraud detection.
• Chapter 5 presents a case study on click prediction for online advertising. We outline how online advertising works behind the scenes and provide you with a worked example of click prediction using a publicly available web-click dataset.
• Chapter 6 is about deep learning and neural networks. We provide a whistle-stop tour of neural networks from their humble beginnings and discuss recent advances in learning with deep networks.
• Chapter 7 outlines how to make the right choice. We discuss statistical significance for A/B testing and several approaches for online learning using multi-armed bandits.
• Chapter 8 presents a forward-looking summary of the intelligent web.
• The appendix discusses how we might process high-velocity event streams and build intelligent algorithms that use them. We discuss several design patterns for web-log processing and outline some key pitfalls to avoid.

For the most part, the chapters can be read independently. But note that chapter 5 is a case study and relies on knowledge of logistic regression that’s introduced in chapter 4.

Downloads

All the code and data required to run the examples in this book are available to download from the publisher’s website (www.manning.com/books/algorithms-of-the-intelligent-web-second-edition) and from GitHub (https://github.com/dougmcilwraith/aiw-second-edition). The only notable exception is the Criteo Display Challenge Dataset, which must be downloaded directly from Criteo due to its size. Instructions are provided in chapter 5.

All code has been tested in Ubuntu 14.04.2 using Python 2.7.10. Various dependencies exist; these can be found in the requirements file provided with the downloads. In this file we also include instructions to ensure that your environment is compatible with the code samples in this book.

Code conventions

This book provides copious examples. Source code in listings and code terms in text are in a fixed-width font like this to separate them from ordinary text. In some places, we’ve added line breaks and reworked indentation to accommodate the available page space in the book. When even this was not enough, listings include line-continuation markers (). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany some of the source code listings, highlighting important concepts.

Math conventions

In the text, you’ll find many equations that support the code and concepts introduced. We follow a standard convention for the notation throughout. Matrices are represented by upright, capital, bold characters, such as M. Upright, lowercase, bold characters represent vectors: v. Scalar values are expressed using lowercase italic, as in the case of λ.

About the authors

DR. DOUGLAS MCILWRAITH earned his first degree at Cambridge in computer science before completing a PhD at Imperial College in London. He is a machine learning expert currently working as a data scientist for a London-based advertising network. He has made research contributions to the fields of distributed systems, ubiquitous computing, pervasive sensing, and robotics and security—and he gets excited when technology has a positive impact on people’s lives.

DR. HARALAMBOS MARMANIS is a pioneer in the adoption of machine learning techniques for industrial solutions. He has 25 years of experience in developing professional software.

DMITRY BABENKO has designed and built a wide variety of applications and infrastructure frameworks for banking, insurance, supply-chain management, and business intelligence companies. He received a MS degree in computer science from Belarussian State University of Informatics and Radioelectronics.

Author Online

Purchase of Algorithms of the Intelligent Web includes free access to a private web forum run by Manning Publications where you can make comments about the book, ask technical questions, and receive help from the authors and from other users. To access the forum and subscribe to it, point your web browser to www.manning.com/books/algorithms-of-the-intelligent-web-second-edition. This page provides information on how to get on the forum once you’re registered, what kind of help is available, and the rules of conduct on the forum.

Manning’s commitment to our readers is to provide a venue where a meaningful dialog between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the authors, whose contribution to Author Online remains voluntary (and unpaid). We suggest you try asking the authors some challenging questions, lest their interest stray! The Author Online forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.

About the cover illustration

The illustration on the cover of Algorithms of the Intelligent Web, Second Edition, is taken from a French book of dress customs, Encyclopedie des Voyages by J. G. St. Saveur, published in 1706. Travel for pleasures was a relatively new phenomenon at the time, and illustrated guides such as this one were popular, introducing both the tourist and the armchair traveler to the inhabitants of other far-off regions of the world, as well as to the more familiar regional costumes of France and Europe.

The diversity of the drawings in the Encyclopedie des Voyages speaks vividly of the uniqueness and individuality of the world’s countries and peoples just 200 years ago. This was a time when the dress codes of two regions separated by a few dozen miles identified people uniquely as belonging to one or the other, and when members of a social class or a trade or a tribe could be easily distinguished by what they were wearing. This was also a time when people were fascinated by foreign lands and faraway places, even though they couldn’t travel to these exotic destinations themselves.

Dress codes have changed since then, and the diversity by region, so rich at the time, has faded away. It’s now often hard to tell the inhabitants of one continent from another. Perhaps, trying to view it optimistically, we’ve traded a world of cultural and visual diversity for a more varied personal life, or a more varied and interesting intellectual and technical life.

We at Manning celebrate the inventiveness, the initiative, and the fun of the computer business with book covers based on native and tribal costumes from two centuries ago, brought back to life by pictures like those from this travel guide.