Preface

In 2015, David was working as a core developer for Docker, the company that made containers popular. His day-to-day work was divided between helping the community and growing the project. Part of his job was reviewing the firehose of pull requests that members of the community sent us; he also had to ensure that Docker worked for all kinds of scenarios, including high-performance workloads that were running and provisioning thousands of containers at any point of time.

To diagnose performance issues at Docker, we used flame graphs, which are advanced visualizations to help you navigate that data easily. The Go programming language makes it really easy to measure and extract application performance data using an embedded HTTP endpoint and generate graphs based on that data. David wrote an article about Go’s profiler capabilities and how you can use its data to generate flame graphs. A big pitfall about the way that Docker collects performance data is that the profiler is disabled by default, so if you’re trying to debug a performance issue, the first action to take is to restart Docker. The main issue with this strategy is that by restarting the service, you’ll probably lose the relevant data that you’re trying to collect, and then you need to wait until the event you’re trying to trace happens again. In David’s article about Docker flame graphs, he mentioned this as a necessary step to measure Docker’s performance, but that it didn’t need to be this way. This realization made him start researching different technologies to collect and analyze any application’s performance, which led him to discover BPF.

In the meantime, but far from David, Lorenzo was looking for a reason to better study the Linux kernel internals, and he discovered that it was easy to get to know many kernel subsystems by approaching them while learning about BPF. A couple of years later, he was able to apply BPF at his job at InfluxData to understand ways to make the data ingestion faster in InfluxCloud. Now Lorenzo is involved in the BPF community and IOVisor, and he works at Sysdig on Falco, a tool that uses BPF to do runtime security for containers and Linux.

Over the past few years, we’ve used BPF in multiple scenarios, from gathering utilization data from Kubernetes clusters to managing network traffic policies. We’ve learned its ins and outs by using it and by reading many blog posts from technology leaders like Brendan Gregg and Alexei Starovoitov and companies like Cilium and Facebook. Their articles and publications helped us tremendously in the past, and they have also been a huge reference for the development of this book.

After reading many of those resources, we realized that every time we needed to learn something about BPF, we needed to jump between many blog posts, man pages, and other places on the internet. This book is our attempt to put the knowledge scattered across the web in a central location for the next generation of BPF enthusiasts to learn about this fantastic technology.

We’ve divided our work between nine different chapters to show you what you can accomplish by using BPF. You can read some chapters in isolation as reference guides, but if you’re new to BPF, we recommend you to read them in order. That will give you an overview of the core concepts in BPF and guide you through the possibilities ahead of you.

Whether you’re already an expert in observability and performance analysis or you’re researching new possibilities to answer questions about your production systems that you couldn’t solve before, we hope you find new knowledge in this book.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This element signifies a tip or suggestion.

Note

This element signifies a general note.

Warning

This element indicates a warning or caution.

Using Code Examples

Supplemental material (code examples, exercises, etc.) is available for download at https://oreil.ly/lbpf-repo.

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Linux Observability with BPF by David Calavera and Lorenzo Fontana (O’Reilly). Copyright 2020 David Calavera and Lorenzo Fontana, 978-1-492-05020-9.”

If you feel your use of code examples falls outside fair use or the permission given here, feel free to contact us at [email protected].

O’Reilly Online Learning

Note

For over 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.

Our unique network of experts and innovators share their knowledge and expertise through books, articles, conferences, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, please visit http://oreilly.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

• O’Reilly Media, Inc.
• 1005 Gravenstein Highway North
• Sebastopol, CA 95472
• 800-998-9938 (in the United States or Canada)
• 707-829-0515 (international or local)
• 707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/linux-bpf.

Email [email protected] to comment or ask technical questions about this book.

For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

Writing a book is more difficult than we thought, but it was probably one of the most rewarding activities we have done in our lives. Many days and nights went into this book, and it would have not been possible without the help of our partners, families, friends, and dogs. We’d like to thank Debora Pace, Lorenzo’s girlfriend, and his son, Riccardo, for all the patience they had waiting for him during long writing sessions. Thanks also to Lorenzo’s friend Leonardo Di Donato for all the advice provided and in particular for writing about XDP and testing.

We’re eternally grateful to Robin Means, David’s wife, for proofreading early drafts of several chapters and the initial overview that started this book and for helping him write many articles over the years and laughing at his made-up English words that sounded cuter than they really were.

We both want to say a big thank you to all those who made eBPF and BPF possible. To David Miller and Alexei Starovoitov for their continuous contributions to improve the Linux kernel and ultimately eBPF and the community around it. To Brendan Gregg for his willing to share, his enthusiasm, and his work on tooling that make eBPF more accessible to everyone. To the IOVisor group for their vibes, emails, and all the work they put into bpftrace, gobpf, kubectl-trace, and BCC. To Daniel Borkmann for all his inspirational work, in particular on libbpf and the tools infrastructure. To Jessie Frazelle for writing the foreword and being inspirational to both of us and to thousands of developers out there. To Jérôme Petazzoni for being the best technical reviewer we could want; his questions made us rethink many parts of this book and our approach to the code examples in it.

And thank you to all the thousands of Linux kernel contributors, in particular the ones active in the BPF mailing list, for their questions/answers, their patches, and their initiatives. Finally, to all those who have been part of publishing this book at O’Reilly, including our editors, John Devins and Melissa Potter, and all the people behind the scenes who made the cover, reviewed the pages, and made this book be more professional than anything else we have done in our careers.