Preface
Enterprises are becoming increasingly data driven, and a key component of any enterprise’s data strategy is a data warehouse—a central repository of integrated data from all across the company. Traditionally, the data warehouse was used by data analysts to create analytical reports. But now it is also increasingly used to populate real-time dashboards, to make ad hoc queries, and to provide decision-making guidance through predictive analytics. Because of these business requirements for advanced analytics and a trend toward cost control, agility, and self-service data access, many organizations are moving to cloud-based data warehouses such as Google BigQuery.
In this book, we provide a thorough tour of BigQuery, a serverless, highly scalable, low-cost enterprise data warehouse that is available on Google Cloud. Because there is no infrastructure to manage, enterprises can focus on analyzing data to find meaningful insights using familiar SQL.
Our goal with BigQuery has been to build a data platform that provides leading-edge capabilities, takes advantage of the many great technologies that are now available in cloud environments, and supports tried-and-true data technologies that are still relevant today. For example, on the leading edge, Google’s BigQuery is a serverless compute architecture that decouples compute and storage. This enables diverse layers of the architecture to perform and scale independently, and it gives data developers flexibility in design and deployment. Somewhat uniquely, BigQuery supports native machine learning and geospatial analysis. With Cloud Pub/Sub, Cloud Dataflow, Cloud Bigtable, Cloud AI Platform, and many third-party integrations, BigQuery interoperates with both traditional and modern systems, at a wide range of desired throughput and latency. And on the tried-and-true front, BigQuery supports ANSI-standard SQL, columnar optimization, and federated queries, which are key to the self-service ad hoc data exploration that many users demand.
Who Is This Book For?
This book is for data analysts, data engineers, and data scientists who want to use BigQuery to derive insights from large datasets. Data analysts can interact with BigQuery through SQL and via dashboarding tools like Looker, Data Studio, and Tableau. Data engineers can integrate BigQuery with data pipelines written in Python or Java and using frameworks such as Apache Spark and Apache Beam. Data scientists can build machine learning models in BigQuery, run TensorFlow models on data in BigQuery, and delegate distributed, large-scale operations to BigQuery from within a Jupyter notebook.
Conventions Used in This Book
The following typographical conventions are used in this book:
- Italic
-
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width-
Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
Constant width bold-
Shows commands or other text that should be typed literally by the user.
Constant width italic-
Shows text that should be replaced with user-supplied values or values determined by context.
Tip
This element signifies a tip or suggestion.
Note
This element signifies a general note.
Warning
This element indicates a warning or caution.
Using Code Examples
Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/GoogleCloudPlatform/bigquery-oreilly-book.
If you have a technical question or a problem using the code examples, please send email to [email protected].
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Google BigQuery: The Definitive Guide by Valliappa Lakshmanan and Jordan Tigani (O’Reilly). Copyright 2020 Valliappa Lakshmanan and Jordan Tigani, 978-1-492-04446-8.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [email protected].
O’Reilly Online Learning
Note
For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.
Our unique network of experts and innovators share their knowledge and expertise through books, articles, conferences, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, please visit http://oreilly.com.
How to Contact Us
Please address comments and questions concerning this book to the publisher:
- O’Reilly Media, Inc.
- 1005 Gravenstein Highway North
- Sebastopol, CA 95472
- 800-998-9938 (in the United States or Canada)
- 707-829-0515 (international or local)
- 707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/google_bigquery_tdg.
To comment or ask technical questions about this book, send email to [email protected].
For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Follow the authors on Twitter: https://twitter.com/lak_gcp and https://twitter.com/jrdntgn
Watch us on YouTube: http://www.youtube.com/oreillymedia
Acknowledgments
We (Lak and Jordan) were extremely fortunate in our reviewers—Elliott Brossard, Evan Jones, Graham Polley, Rebecca Ward, and Tegan Tigani reviewed every chapter of this book and made numerous suggestions for improvement. Elliott kept our SQL queries lean and clean. We benefited from Evan’s experience using BigQuery in Google Finance. Graham brought a valuable customer perspective to many of our discussions involving cost and regionalization. Rebecca kept us factual, and Tegan made sure our language was simple and straightforward. Besides these five, many Googlers (Chad Jennings, Haris Khan, Misha Brukman, Daniel Gundrum, Mosha Pashumansky, Amir Hormati, and Mingge Deng) reviewed parts of the manuscript in their areas of expertise. Any errors that remain are ours, of course.
Thanks also to our respective families, teammates, and managers (Rochana Golani and Sudhir Hasbe) for their support. Nicole Taché and Kristen Brown, our editors at O’Reilly, were a pleasure to work with. The text is immeasurably better because of the eagle-eyed work of Bob Russell, our copyeditor. This book was Saptarshi Mukherjee’s idea, and it was he who pushed the two of us to collaborate on a new BigQuery book. Finally, we would like to thank BigQuery users (and competitors!) for pushing us to make BigQuery better, and the BigQuery engineering team for making magic happen.
We are donating 100% of the royalties from this book to United Way of King County, where we both live. We strongly encourage you to get involved with a local charity to give, volunteer, and take action to help solve your community’s toughest challenges.