Chapter 2
1. Stephen Hawking, Stuart Russell, Max Tegmark, and Frank Wilczek, “Stephen Hawking: “Transcendence Looks at the Implications of Artificial Intelligence—But Are We Taking AI Seriously Enough?” The Independent, May 1, 2014, http://www.independent.co.uk/news/science/stephen-hawkingtranscendence-looks-at-the-implications-of-artificial-intelligence-but-are-we-taking-9313474.html.
2. Paul Mozur, “Beijing Wants A.I. to Be Made in China by 2030,” New York Times, July 20, 2017, https://www.nytimes.com/2017/07/20/business/china-artificial-intelligence.html?mcubz=0&_r=0.
3. Steve Jurvetson, “Intelligence Inside,” Medium, August 9, 2016, https://medium.com/@DFJvc/intelligence-inside-54dcad8c4a3e.
4. William D. Nordhaus, “Do Real-Output and Real-Wage Measures Capture Reality? The History of Lighting Suggests Not,” Cowles Foundation for Research in Economics, Yale University, 1998, https://lucept.files.wordpress.com/2014/11/william-nordhaus-the-cost-of-light.pdf.
5. This was part of a long trend in reducing the general cost of computation. See William D. Nordhaus, “Two Centuries of Productivity Growth in Computing,” Journal of Economic History 67, no. 1 (2007): 128–159.
6. Lovelace, quoted in Walter Isaacson, The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution (New York: Simon & Schuster, 2014), 27.
7. Lovelace, The Innovators, 29.
8. Amazon already is working on potential security issues with such a plan. In 2017, it launched Amazon Key, a system that allowed its delivery people to unlock your door and deposit packages inside all under the watchful eye of a camera to record that everything went smoothly.
9. Interestingly, some startups are already thinking this way. Stitch Fix uses machine learning to predict what clothes its customers will want and ships a package to them. The customer then returns the clothes they do not want. In 2017, Stitch Fix had a successful IPO based on this model—perhaps the first “AI-first” startup to do so.
10. See US Patent Number 8,615,473 B2. Also, Praveen Kopalle, “Why Amazon’s Anticipatory Shipping Is Pure Genius,” Forbes, January 28, 2014, https://www.forbes.com/sites/onmarketing/2014/01/28/why-amazons-anticipatory-shipping-is-pure-genius/#2a3284174605.
Chapter 3
1. As a reminder about the importance of careful interpretation of predictions, we note that the oracle at Delphi predicted that a great empire would be destroyed if he attacked. Emboldened, the king attacked Persia, and to his surprise, his own Lydian empire was destroyed. The prediction was technically correct, but misunderstood.
2. “Mastercard Rolls Out Artificial Intelligence across Its Global Network,” Mastercard press release, November 30, 2016, https://newsroom.mastercard.com/press-releases/mastercard-rolls-out-artificial-intelligence-across-its-global-network/.
3. Adam Geitgey, “Machine Learning Is Fun, Part 5: Language Translation with Deep Learning and the Magic of Sequences,” Medium, August 21, 2016, https://medium.com/@ageitgey/machine-learning-is-fun-part-5-language-translation-with-deep-learning-and-the-magic-of-sequences-2ace0acca0aa.
4. Yiting Sun, “Why 500 Million People in China Are Talking to This AI,” MIT Technology Review, September 14, 2017, https://www.technologyreview.com/s/608841/why-500-million-people-in-china-are-talking-to-this-ai/.
5. Salvatore J. Stolfo, David W. Fan, Wenke Lee, and Andreas L. Prodromidis, “Credit Card Fraud Detection Using Meta-Learning: Issues and Initial Results,” AAAI Technical Report, WS-97-07, 1997, http://www.aaai.org/Papers/Workshops/1997/WS-97-07/WS97-07-015.pdf, with a false positive rate around 15 percent to 20 percent. Another example is E. Aleskerov, B. Freisleben, and B. Rao, “CARDWATCH: A Neural Network Based Database Mining System for Credit Card Fraud Detection,” Computational Intelligence for Financial Engineering, 1997, http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=618940. Note that these comparisons are not entirely equal, because they use different training data sets. Still, the broad point on accuracy holds.
6. Abhinav Srivastava, Amlan Kundu, Shamik Sural, and Arun Majumdar, “Credit Card Fraud Detection Using Hidden Markov Model,” IEEE Transactions on Dependable and Secure Computing 5, no. 1 (January–March 2008): 37–48, http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4358713. See also Jarrod West and Maumita Bhattacharya, “Intelligent Financial Fraud Detection: A Comprehensive Review, Computers & Security 57 (2016): 47–66, http://www.sciencedirect.com/science/article/pii/S0167404815001261.
7. Andrej Karpathy, “What I Learned from Competing against a ConvNet on ImageNet,” Andrej Karpathy (blog), September 2, 2014, http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/; ImageNet, Large Scale Visual Recognition Challenge 2016, http://image-net.org/challenges/LSVRC/2016/results; Andrej Karpathy, LISVRC 2014, http://cs.stanford.edu/people/karpathy/ilsvrc/.
8. Aaron Tilley, “China’s Rise in the Global AI Race Emerges as It Takes Over the Final ImageNet Competition,” Forbes, July 31, 2017, https://www.forbes.com/sites/aarontilley/2017/07/31/china-ai-imagenet/#dafa182170a8.
9. Dave Gershgorn, “The Data That Transformed AI Research—and Possibly the World,” Quartz, July 26, 2017, https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/.
10. Definitions from the Oxford English Dictionary.
Chapter 4
1. Of course, there are many people who buy insurance and also lottery tickets. Economists don’t really know how to explain that behavior, so we leave their study to psychologists. You can’t do everything.
2. D. Karlan et al., “Agricultural Decisions after Relaxing Credit and Risk Constraints,” Quarterly Journal of Economics 129, no. 2 (2014): 597–652, https://dx.doi.org/10.1093/qje/qju002.
3. The reasons for this are still a puzzle, but the researchers speculated that there was still a lack of trust that payouts may be forthcoming in what might otherwise be a rare event.
4. Michela Rosano, “Throwback Thursday: Nazi Weather Station in Labrador,” Canadian Geographic, September 9, 2015, https://canadiangeographic.ca/articles/throwback-thursday-nazi-weather-station-in-labrador/.
5. Michael Lewis, The Fifth Risk (New York: Norton, 2018), 150.
6. Andrew Blum, The Weather Machine (New York: Ecco, 2019), 12.
7. Ajay Agrawal, Joshua S. Gans, and Avi Goldfarb, “Exploring the Impact of Artificial Intelligence: Prediction versus Judgment,” Information Economics and Policy 47 (2019): 1–6.
8. Isaac Ehrlich and Gary S. Becker, “Market Insurance, Self-Insurance, and Self-Protection,” Journal of Political Economy 80, no. 4 (1972): 623–648.
9. Eric Lamarre, “AI Strategy Implementation—Transforming the Enterprise,” Market for Machine Intelligence Conference, Toronto, October 2019, https://www.youtube.com/watch?v=8b82n8LDmvU.
Chapter 5
1. J. McCarthy, Marvin L. Minsky, N. Rochester, and Claude E. Shannon, “A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence,” August 31, 1955, http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html.
2. Jeff Hawkins and Sandra Blakeslee, On Intelligence (New York: Times Books, 2004), 89.
3. McCarthy et al., “A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence.”
4. Ian Hacking, The Taming of Chance (Cambridge, UK: Cambridge University Press, 1990).
Chapter 6
1. Hal Varian, “Beyond Big Data,” lecture, National Association of Business Economists, San Francisco, September 10, 2013.
2. Ngai-yin Chan and Chi-chung Choy, “Screening for Atrial Fibrillation in 13,122 Hong Kong Citizens with Smartphone Electrocardiogram,” BMJ 103, no. 1 (January 2017), http://heart.bmj.com/content/103/1/24; Sarah Buhr, “Apple’s Watch Can Detect an Abnormal Heart Rhythm with 97% Accuracy, UCSF Study Says,” Techcrunch, May 11, 2017, https://techcrunch.com/2017/05/11/apples-watch-can-detect-an-abnormal-heart-rhythm-with-97-accuracy-ucsf-study-says/; Alive-Cor, “AliveCor and Mayo Clinic Announce Collaboration to Identify Hidden Health Signals in Humans,” Cision PR newswire, October 24, 2016, http://www.prnewswire.com/news-releases/alivecor-and-mayo-clinic-announce-collaboration-to-identify-hidden-health-signals-in-humans-300349847.html.
3. Buhr, “Apple’s Watch Can Detect an Abnormal Heart Rhythm with 97% Accuracy, UCSF Study Says”; and Avesh Singh, “Applying Artificial Intelligence in Medicine: Our Early Results,” Cardiogram (blog), May 11, https://blog.cardiogr.am/applying-artificial-intelligence-in-medicine-our-early-results-78bfe7605d32.
4. We don’t know if Cardiogram in particular will succeed. We are, however, confident that smartphones and other sensors will be used for medical diagnosis going forward.
5. Six thousand is a relatively small number of units for this kind of study, which is a main reason why the study was listed as “preliminary.” This data was enough for Cardiogram’s initial purpose because it was a preliminary study to show proof of concept. No lives were put at risk. For the results to be clinically useful, it will likely need much more data.
6. Dave Heiner, “Competition Authorities and Search,” Microsoft Technet (blog), February 26, 2010, https://blogs.technet.microsoft.com/microsoft_on_the_issues/2010/02/26/competition-authorities-and-search/. Google has argued that Bing is big enough to reap the benefits of scale in search.
Chapter 7
1. Sixty percent of the time you choose X and are correct 60 percent of the time, while 40 percent of the time you choose O and are correct only 40 percent of the time. On average, this is 0.6^2 + 0.4^2 = 0.52.
2. Amos Tversky and Daniel Kahneman, “Judgment under Uncertainty: Heuristics and Biases,” Science 185, no. 4157 (1974): 1124–1131, https://people.hss.caltech.edu/~camerer/Ec101/JudgementUncertainty.pdf.
3. See Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Strauss and Giroux, 2011); and Dan Ariely, Predictably Irrational (New York: HarperCollins, 2009).
4. Michael Lewis, Moneyball (New York: Norton, 2003).
5. Of course, while Moneyball was based on the use of traditional statistics, it should be no surprise that teams are now looking to machine-learning methods to perform that function, gathering far more data in the process. See Takashi Sugimoto, “AI May Help Japan’s Baseball Champs Rewrite ‘Moneyball,’” Nikkei Asian Review, May 2, 2016, http://asia.nikkei.com/Business/Companies/AI-may-help-Japan-s-baseball-champs-rewrite-Moneyball.
6. Jon Kleinberg et al., “Human Decisions and Machine Predictions,” working paper 23180, National Bureau of Economic Research, 2017.
7. The research also shows that the algorithm would likely reduce racial disparities.
8. Mitchell Hoffman, Lisa B. Kahn, and Danielle Li, “Discretion in Hiring,” Quarterly Journal of Economics 133, no. 2 (2018): 765–800.
9. Donald Rumsfeld, news briefing, US Department of Defense, February 12, 2002, https://en.wikipedia.org/wiki/There_are_known_knowns.
10. Bertrand Rouet-Leduc et al., “Machine Learning Predicts Laboratory Earthquakes,” Cornell University, 2017, http://arxiv.org/abs/1702.05774.
11. Dedre Gentner and Albert L. Stevens, Mental Models (New York: Psychology Press, 1983); Dedre Gentner, “Structure Mapping: A Theoretical Model for Analogy,” Cognitive Science 7 (1983): 15–170.
12. Even as machines get better at such situations, the laws of probability mean that in small samples, there will always be some uncertainty. Thus, when data is sparse, machine predictions will be imprecise in a known way. The machine can provide a sense of how imprecise its predictions are. As we discuss in chapter 9, this creates a human role for judging how to act on imprecise predictions.
13. Nassim Nicholas Taleb, The Black Swan (New York: Random House, 2007).
14. In Isaac Asimov’s Foundation series, prediction becomes powerful enough that it could foresee the destruction of the Galactic Empire and the various growing pains of the society that is the focus of the story. Important to the plot line, however, is that these predictions could not foresee the rise of “the mutant.” Predictions did not foresee the unexpected event.
15. Joel Waldfogel, “Copyright Protection, Technological Change, and the Quality of New Products: Evidence from Recorded Music since Napster,” Journal of Law and Economics 55, no. 4 (2012): 715–740.
16. Donald Rubin, “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies,” Journal of Educational Psychology 66, no. 5 (1974): 688–701; Jerzy Neyman, “Sur les applications de la theorie des probabilites aux experiences agricoles: Essai des principes,” master’s thesis, 1923, excerpts reprinted in English, D. M. Dabrowska, and T. P. Speed, translators, Statistical Science 5 (1923): 463–472.
17. Garry Kasparov, Deep Thinking (New York: Perseus Books, 2017), 99–100.
18. Google Panda, Wikipedia, https://en.wikipedia.org/wiki/Google_Panda, accessed July 26, 2017. Most notably as described in Google webmasters, “What’s It Like to Fight Webspam at Google?” YouTube, February 12, 2014, https://www.youtube.com/watch?v=rr-Cye_mFiQ.
19. For example, publicized overhauls in September 2016: Ashitha Nagesh, “Now You Can Finally Get Rid of All Those Instagram Spammers and Trolls,” Metro, September 13, 2016, http://metro.co.uk/2016/09/13/now-you-can-finally-get-rid-of-all-those-instagram-spammers-and-trolls-6125645/. Then, again, in June 2017: Jonathan Vanian, “Instagram Turns to Artificial Intelligence to Fight Spam and Offensive Comments,” Fortune, June 29, 2017, http://fortune.com/2017/06/29/instagram-artificial-intelligence-offensive-comments/. The challenge of using prediction machines in the face of strategic actors is a problem with a long history. In 1976, economist Robert Lucas made this point with respect to macroeconomic policy on inflation and other economic indicators. If people will be better off changing behavior after the policy change, they will do so. Lucas emphasized that even though employment tended to be high when inflation was high, if the central bank changed to a policy of increasing inflation, people would anticipate that inflation and the relationship would break down. So, instead of policy based on extrapolating from past data, he argued that policy should be made based on understanding the underlying drivers of human behavior. This became known as the “Lucas Critique.” See Robert Lucas, “Econometric Policy Evaluation: A Critique,” Carnegie-Rochester Conference Series in Public Policy 1, no. 1 (1976): 19–46, https://ideas.repec.org/a/eee/crcspp/v1y1976ip19-46.html. Economist Tim Harford described this differently: Fort Knox has never been robbed. How much should be spent on protecting Fort Knox? Because it has never been robbed, spending on security does not predict a reduction in robberies. A prediction machine might then recommend spending nothing. Why bother spending money when security doesn’t reduce robberies? Tim Harford, The Undercover Economist Strikes Back: How to Run—or Ruin—an Economy (New York: Riverhead Books, 2014).
20. Dean Karlan and Michael Luca, “How to Use Correlation to Make Predictions,” Harvard Business Review, April 6, 2022, https://hbr.org/2022/04/how-to-use-correlation-to-make-predictions.
21. Dayong Wang et al., “Deep Learning for Identifying Metastatic Breast Cancer,” Camelyon Grand Challenge, June 18, 2016, https://arxiv.org/pdf/1606.05718.pdf.
22. Charles Babbage, On the Economy of Machinery and Manufactures (London: Charles Knight Pall Mall East, 1832), 162.
23. Daniel Paravisini and Antoinette Schoar, “The Incentive Effect of IT: Randomized Evidence from Credit Committees,” working paper 19303, National Bureau of Economic Research, August 2013.
Chapter 8
1. Jody Rosen, “The Knowledge, London’s Legendary Taxi-Driver Test, Puts Up a Fight in the Age of GPS,” New York Times, November 10, 2014, https://www.nytimes.com/2014/11/10/t-magazine/london-taxi-test-knowledge.html?_r=0.
2. For a textbook treatment, see Joshua S. Gans, Core Economics for Managers (Australia: Cengage, 2005).
3. To see why:
Average “Take” Payoff = (3/4)(Dry with Umbrella) + (1/4)(Dry with Umbrella) = (3/4)8 + (1/4)8 = 8
Average “Leave” Payoff = (3/4)(Dry without Umbrella) + (1/4)(Wet) = (3/4)10 + (1/4)0 = 7.5
Chapter 9
1. Andrew McAfee and Erik Brynjolfsson, Machine, Platform, Crowd: Harnessing Our Digital Future (New York: Norton, 2017), 72.
2. This example is taken from Jean-Pierre Dubé and Sanjog Misra, “Scalable Price Targeting,” working paper, Booth School of Business, University of Chicago, 2017, http://conference.nber.org/confer//2017/SI2017/PRIT/Dube_Misra.pdf.
Chapter 10
1. Daisuke Wakabayashi, “Meet the People Who Train the Robots (to Do Their Own Jobs),” New York Times, April 28, 2017, https://www.nytimes.com/2017/04/28/technology/meet-the-people-who-train-the-robots-to-do-their-own-jobs.html?_r=1.
2. Ibid.
3. Marc Mangel and Francisco J. Samaniego, “Abraham Wald’s Work on Aircraft Survivability,” Journal of the American Statistical Association 79, no. 386 (1984): 259–267.
4. Bart J. Bronnenberg, Peter E. Rossi, and Naufel J. Vilcassim, “Structural Modeling and Policy Simulation,” Journal of Marketing Research 42, no. 1 (2005): 22–26, http://journals.ama.org/doi/abs/10.1509/jmkr.42.1.22.56887.
5. Jean Pierre Dubé et al., “Recent Advances in Structural Econometric Modeling,” Marketing Letters 16, no. 3–4 (2005): 209–224, https://link.springer.com/article/10.1007%2Fs11002-005-5886-0?LI=true.
Chapter 11
1. “Robot Mailman Rolls on a Tight Schedule,” Popular Science, October 1976, https://books.google.ca/books?id=HwEAAAAAMBAJ&pg=PA76&lpg=PA76&dq=mailmobile+robot&source=bl&ots=SHkkOiDv8K&sig=sYFXzvvZ8_GvOV8Gt30hoGrFhpk&hl=en&sa=X&ei=B3kLVYr7N8meNoLsg_AD&redir_esc=y#v=onepage&q=mailmobile%20robot&f=false.
2. George Stigler as communicated by Nathan Rosenberg to the authors in 1991.
3. Nobel citation: “Studies of Decision Making Lead to Prize in Economics,” Royal Swedish Academy of Sciences, press release, October 16, 1978, https://www.nobelprize.org/nobel_prizes/economic-sciences/laureates/1978/press.html. Turing award citation: Herbert Alexander Simon, A.M. Turing Award, 1975, http://amturing.acm.org/award_winners/simon_1031467.cfm. See Herbert A. Simon, “Rationality as Process and as Product of Thought,” American Economic Review 68, no. 2 (1978): 1–16; Allen Nevell and Herbert A. Simon, “Computer Science as Empirical Inquiry,” Communications of the ACM 19, no. 3 (1976): 120.
4. Frederick Jelinek quoted in Roger K. Moore, “Results from a Survey of Attendees at ASRU 1997 and 2003,” INTERSPEECH-2005, Lisbon, September 4–8, 2005.
Chapter 12
1. Jmdavis, “Autopilot Worked for Me Today and Saved an Accident,” Tesla Motors Club (blog), December 12, 2016, https://teslamotorsclub.com/tmc/threads/autopilot-worked-for-me-today-and-saved-an-accident.82268/.
2. A few weeks later, another driver’s dash cam caught the system in operation: Fred Lambert, “Tesla Autopilot’s New Radar Technology Predicts an Accident Caught on Dashcamera a Second Later,” Electrek, December 27, 2016, https://electrek.co/2016/12/27/tesla-autopilot-radar-technology-predict-accident-dashcam/.
3. NHTSA, “U.S. DOT and IIHS Announce Historic Commitment of 20 Auto-makers to Make Automatic Emergency Braking Standard on New Vehicles,” March 17, 2016, https://www.nhtsa.gov/press-releases/us-dot-and-iihs-announce-historic-commitment-20-automakers-make-automatic-emergency. This was supposed to have a deadline of 2022, but as of the time of writing, this has been pushed back to 2023; NHTSA, “NHTA Announces 2020 Update on AEB Installation by 20 Automakers,” US Department of Transportation, December 17, 2020, https://www.nhtsa.gov/press-releases/nhtsa-announces-2020-update-aeb-installation-20-automakers.
4. Kathryn Diss, “Driverless Trucks Move All Iron Ore at Rio Tinto’s Pilbara Mines, in World First,” ABC News, October 18, 2015, http://www.abc.net.au/news/2015-10-18/rio-tinto-opens-worlds-first-automated-mine/6863814.
5. Tim Simonite, “Mining 24 Hours a Day with Robots,” MIT Technology Review, December 28, 2016, https://www.technologyreview.com/s/603170/mining-24-hours-a-day-with-robots/.
6. Samantha Murphy Kelly, “Stunning Underwater Olympics Shots Are Now Taken by Robots,” CNN, August 9, 2016, http://money.cnn.com/2016/08/08/technology/olympics-underwater-robots-getty/.
7. Hoang Le, Andrew Kang, and Yisong Yue, “Smooth Imitation Learning for Online Sequence Prediction,” International Conference on Machine Learning, June 19, 2016, https://www.disneyresearch.com/publication/smooth-imitation-learning/.
8. The laws were (1) A robot may not injure a human being or, through inaction, allow a human being to come to harm; (2) A robot must obey orders given it by human beings except where such orders would conflict with the First Law; (3) A robot must protect its own existence as long as such protection does not conflict with the First or Second Law. See Isaac Asimov, “Runaround,” I, Robot (The Isaac Asimov Collection ed.) (New York: Doubleday, 1950), 40.
9. Department of Defense Directive 3000.09: Autonomy in Weapon Systems, November 21, 2012, https://www.hsdl.org/?abstract&did=726163.
10. For instance, there are various clauses allowing alternatives when there is time pressure in battle. Mark Guburd, “Why Should We Ban Autonomous Weapons? To Survive,” IEEE Spectrum, June 1, 2016, http://spectrum.ieee.org/automaton/robotics/military-robots/why-should-we-ban-autonomous-weapons-to-survive.
Chapter 13
1. This is a point discussed in detail by Timothy Bresnahan, “Artificial Intelligence Technologies and Aggregate Growth Prospects,” working paper, Stanford University Department of Economics, 2019).
2. Based on research conducted on August 9, 2021, at 8:48 a.m., https://www.amazon.ca/Peggy-11-Non-Skid-Stainless-2-Pack/dp/B07XXN9CTD.
3. Based on research conducted on August 9, 2021 at 8:50 a.m., https://www.amazon.ca/Bone-Dry-Ceramic-Collection-Small/dp/B08H3Z8ZDJ.
4. Amazon review, accessed April 25, 2022, https://www.amazon.com/review/R15MVV6O3YMY69/ref=cm_cr_srp_d_rdp_perm?ie=UTF8&ASIN=B01MRXMX41.
5. We know this because our deep forensic analysis identified a review of paw protectors in 2020 just a few months before the dog bowl fiasco. Amazon review, accessed April 25, 2022, https://www.amazon.com/gp/customer-reviews/RNK1QA46ZP3UM?ref=pf_vv_at_pdctrvw_srp.
6. And believe us, truly random.
7. Guy Rosen, “Facebook Publishes Enforcement Numbers for the First Time,” Meta, May 15, 2018, https://about.fb.com/news/2018/05/enforcement-numbers/.
8. Rosen, “Facebook Publishes Enforcement Numbers.”
9. Charlotte Jee, “Facebook Needs 30,000 of Its Own Content Moderators, Says a New Report,” MIT Technology Review, June 8, 2020, https://www.technologyreview.com/2020/06/08/1002894/facebook-needs-30000-of-its-own-content-moderators-says-a-new-report/.
10. Many content moderators are employed by other firms that sell those services to Facebook, so it is hard to get a correct percentage.
11. Casey Newton, “The Trauma Floor,” The Verge, February 25, 2019, https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona; and Paul M. Barrett, “Who Moderates the Social Media Giants,” NYU Stern Center for Business and Human Rights, June 2020, https://issuu.com/nyusterncenterforbusinessandhumanri/docs/nyu_content_moderation_report_final_version?fr=sZWZmZjI1NjI1Ng.
12. Bresnahan, “Artificial Intelligence Technologies and Aggregate Growth Prospects.”
13. “Human vs Machine,” Spotify: A Product Story podcast, episode 4, March 2021, https://open.spotify.com/episode/0T3nb0PcpvqA4o1BbbQWpp?si=LB5Mg7-OTxOaQYqCf8Ja6w&nd=1.
Chapter 14
1. Robert Solow, “We’d Better Watch Out,” New York Times Book Review, July 12, 1987, 36.
2. Michael Hammer, “Reengineering Work: Don’t Automate, Obliterate,” Harvard Business Review, July–August 1990, https://hbr.org/1990/07/reengineering-work-dont-automate-obliterate.
3. Art Kleiner, “Revisiting Reengineering,” Strategy + Business, July 2000, https://www.strategy-business.com/article/19570?gko=e05ea.
4. Nanette Byrnes, “As Goldman Embraces Automation, Even the Masters of the Universe Are Threatened,” MIT Technology Review, February 7, 2017, https://www.technologyreview.com/s/603431/as-goldman-embraces-automation-even-the-masters-of-the-universe-are-threatened/.
5. “Google Has More Than 1,000 Artificial Intelligence Projects in the Works,” The Week, October 18, 2016, http://theweek.com/speedreads/654463/google-more-than-1000-artificial-intelligence-projects-works.
6. Scott Forstall, quoted in “How the iPhone Was Born,” Wall Street Journal video, June 25, 2017, http://www.wsj.com/video/how-the-iphone-was-born-inside-stories-of-missteps-and-triumphs/302CFE23-392D-4020-B1BD-B4B9CEF7D9A8.html.
Chapter 15
1. Steve Jobs in Memory and Imagination: New Pathways to the Library of Congress, Michael Lawrence Films, 2006, https://www.youtube.com/watch?v=ob_GX50Za6c.
1. Steven Levy, “A Spreadsheet Way of Knowledge,” Wired, October 24, 2014, https://backchannel.com/a-spreadsheet-way-of-knowledge-8de60af7146e.
2. Nick Statt, “The Next Big Leap in AI Could Come from Warehouse Robots,” The Verge, June 1, 2017, https://www.theverge.com/2017/6/1/15703146/kindredorb-robot-ai-startup-warehouse-automation.
3. L. B. Lusted, “Logical Analysis in Roentgen Diagnosis,” Radiology 74 (1960): 178–193.
4. Siddhartha Mukherjee, “A.I. versus M.D.,” New Yorker, April 3, 2017, http://www.newyorker.com/magazine/2017/04/03/ai-versus-md.
5. S. Jha and E. J. Topol, “Adapting to Artificial Intelligence: Radiologists and Pathologists as Information Specialists,” Journal of the American Medical Association 316, no. 22 (2016): 2353–2354.
6. Many of these ideas are related to Frank Levy’s discussion in “Computers and the Supply of Radiology Services,” Journal of the American College of Radiology 5, no. 10 (2008): 1067–1072.
7. See Verdict Hospital (http://www.hospitalmanagement.net/features/feature51500/) for an interview with the 2009 president of the American College of Radiology. Or, for a more academic reference, see Leonard Berlin, “The Radiologist: Doctor’s Doctor or Patient’s Doctor,” American Journal of Roentgenology 128, no. 4 (1977), http://www.ajronline.org/doi/pdf/10.2214/ajr.128.4.702.
8. Levy, “Computers and the Supply of Radiology Services.”
9. Jha and Topol, “Adapting to Artificial Intelligence”; S. Jha, “Will Computers Replace Radiologists?” Medscape 30 (December 2016), http://www.medscape.com/viewarticle/863127#vp_1.
10. Carl Benedikt Frey and Michael A. Osborne, “The Future of Employment: How Susceptible Are Jobs to Computerisation?” Oxford Martin School, University of Oxford, September 2013, http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf.
11. Truck makers are already embedding convoy capabilities in their newest vehicles. Volvo has deployed this in several tests, and Telsa’s new semi has these capabilities built in from the start.
Chapter 17
1. “How Germany’s Otto Uses Artificial Intelligence,” The Economist, April 12, 2017, https://www.economist.com/news/business/21720675-firm-using-algorithm-designed-cern-laboratory-how-germanys-otto-uses.
2. Zvi Griliches, “Hybrid Corn and the Economics of Innovation,” Science 29 (July 1960): 275–280.
3. Bryce Ryan and N. Gross, “The Diffusion of Hybrid Seed Corn,” Rural Sociology 8 (1943): 15–24; and Bryce Ryan and N. Gross, “Acceptance and Diffusion of Hybrid Corn Seed in Two Iowa Communities,” Iowa Agriculture Experiment Station Research Bulletin, no. 372 (January 1950).
4. Kelly Gonsalves, “Google Has More Than 1,000 Artificial Intelligence Projects in the Works,” The Week, October 18, 2016, http://theweek.com/speedreads/654463/google-more-than-1000-artificial-intelligence-projects-works.
5. A rich, entertaining, and ultimately useless debate rages about whether these sabermetric analysts are better or worse than the scouts. As Nate Silver highlights, both the Moneyball types and the scouts have important roles to play. Nate Silver, The Signal and the Noise (New York: Penguin Books, 2015), chapter 3.
6. You may counter and say that surely, in order to improve, the prediction machine needs that past repository of data? This is a subtle issue. Prediction works best when adding new data does not change algorithms too much—that stability is an outcome of good statistical practice. That means when you use feedback data to improve the algorithm, it is of most value precisely when what is being predicted is itself evolving. So if yogurt demand was suddenly shifting with demographics or some other fad, new data will help you improve the algorithm. However, it does this precisely when those changes mean that “old data” is less useful for prediction.
7. Daniel Ren, “Tencent Joins the Fray with Baidu in Providing Artificial Intelligence Applications for Self-Driving Cars,” South China Morning Post, August 27, 2017, http://www.scmp.com/business/companies/article/2108489/tencent-forms-alliance-push-ai-applications-self-driving.
8. Ren, “Tencent Joins the Fray with Baidu in Providing Artificial Intelligence Applications for Self-Driving Cars.”
Chapter 18
1. The theory of adaptation and incentives outlined here comes from Steven Tadelis, “Complexity, Flexibility, and the Make-or-Buy Decision,” American Economic Review 92, no. 2 (May 2002): 433–437.
2. Silke Januszewski Forbes and Mara Lederman, “Adaptation and Vertical Integration in the Airline Industry,” American Economic Review 99, no. 5 (December 2009): 1831–1849.
3. Sharon Novak and Scott Stern, “How Does Outsourcing Affect Performance Dynamics? Evidence from the Automobile Industry,” Management Science 54, no. 12 (December 2008): 1963–1979.
4. Jim Bessen, Learning by Doing (New Haven, CT: Yale University Press, 2106).
5. In 2016, Wells Fargo faced massive fraud claims as a result of the actions of account managers who faced incentives to open costly accounts for customers and charge them fees for doing so.
6. This discussion is based on Dirk Bergemann and Alessandro Bonatti, “Selling Cookies,” American Economic Journal: Microeconomics 7, no. 2 (2015): 259–294.
7. One example is Mastercard Advisors consulting services, which use Mastercard’s vast quantity of data to provide a variety of predictions, ranging from consumer fraud to retention rates. See http://www.mastercardadvisors.com/consulting.html.
Chapter 19
1. As told to Steven Levy. See Will Smith, “Stop Calling Google Cardboard’s 360-Degree Videos ‘VR,’” Wired, November 16, 2015, https://www.wired.com/2015/11/360-video-isnt-virtual-reality/.
2. Jessir Hempel, “Inside Microsoft’s AI Comeback,” Wired, June 21, 2017, https://www.wired.com/story/inside-microsofts-ai-comeback/.
3. “What Does It Mean for Google to Become an ‘AI-First’ (Quoting Sundar) Company?” Quora, April 2016, https://www.quora.com/What-does-it-mean-for-Google-to-become-an-AI-first-company.
4. Clayton M. Christensen, The Innovator’s Dilemma (Boston: Harvard Business Review Press, 1997).
5. For more on these disruption dilemmas, see Joshua S. Gans, The Disruption Dilemma (Cambridge, MA: MIT Press, 2016).
6. Nathan Rosenberg, “Learning by Using: Inside the Black Box: Technology and Economics,” paper, University of Illinois at Champaign-Urbana, 1982, 120–140.
7. In the case of video games, because the goal (maximizing score) is closely related to prediction (will this move increase or decrease the score?), the automated process does not separately need judgment. The judgment is the simple recognition that the objective is to score the most points. Teaching a machine to play a sandbox game like Minecraft or a collection game like Pokemon Go would require more judgment, since different people enjoy different aspects of the games. It isn’t clear what the goal should be.
8. Chesley “Sully” Sullenberger quoted in Katy Couric, “Capt. Sully Worried about Airline Industry,” CBS News, February 10, 2009; https://www.cbsnews.com/news/capt-sully-worried-about-airline-industry/.
9. Mark Harris, “Tesla Drivers Are Paying Big Bucks to Test Flawed Self-Driving Software,” Wired, March 4, 2017, https://backchannel.com/tesla-drivers-are-guinea-pigs-for-flawed-self-driving-software-c2cc80b483a#.s0u7lsv4f.
10. It turns out that many of these reinforcement learning models have their antecdents in econometric techniques invented decades ago. See Mitsuru Igami, “Artificial Intelligence as Structural Estimation: Deep Blue, Bonanza, and AlphaGo,” Econometrics Journal 23, no. 3 (2020): S1–S24.
11. Nikolai Yakovenko, “GANS Will Change the World,” Medium, January 3, 2017, https://medium.com/@Moscow25/gans-will-change-the-world-7ed6ae8515ca; Sebastian Anthony, “Google Teaches ‘AIs’ to Invent Their Own Crypto and Avoid Eavesdropping,” Ars Technica, October 28, 2016, https://arstechnica.com/information-technology/2016/10/google-ai-neural-network-cryptography/.
12. Francesco Bova, Avi Goldfarb, and Roger G. Melko, “Quantum Economic Advantage,” No. w29724, National Bureau of Economic Research, 2022.
13. S. N. Genin et al., “Estimating Phosphorescent Emission Energies in IrIII Complexes Using Large-Scale Quantum Computing Simulations,” Angewandte Chemie International Edition 2022, e202116175.
14. Apple, “Privacy,” https://www.apple.com/ca/privacy/.
15. Ibid.
16. The bet is possible because of technological advances in privacy-protecting data analysis, especially Cynthia Dwork’s invention of differential privacy: Cynthia Dwork, “Differential Privacy: A Survey of Results,” in M. Agrawal, D. Du, Z. Duan, and A. Li (eds), Theory and Applications of Models of Computation. TAMC 2008. Lecture Notes in Computer Science, vol. 4978 (Berlin: Springer, 2008), https://doi.org/10.1007/978-3-540-79228-4_1.
17. William Langewiesche, “The Human Factor,” Vanity Fair, October 2014, http://www.vanityfair.com/news/business/2014/10/air-france-flight-447-crash.
18. Tim Harford, “How Computers Are Setting Us Up for Disaster,” The Guardian, October 11, 2016, https://www.theguardian.com/technology/2016/oct/11/crash-how-computers-are-setting-us-up-disaster.
1. L. Sweeney, “Discrimination in Online Ad Delivery,” Communications of the ACM 56, no. 5 (2013): 44–54, https://dataprivacylab.org/projects/onlineads/.
2. Sweeney, “Discrimination in Online Ad Delivery.”
3. “Racism Is Poisoning Online Ad Delivery, Says Harvard Professor,” MIT Technology Review, February 4, 2013, https://www.technologyreview.com/s/510646/racism-is-poisoning-online-ad-delivery-says-harvard-professor/.
4. Anja Lambrecht and Catherine Tucker, “Algorithmic Bias? An Empirical Study into Apparent Gender-Based Discrimination in the Display of STEM Career Ads,” Management Science 65, no. 7 (2019): 2966–2981.
5. Diane Cardwell and Libby Nelson, “The Fire Dept. Tests That Were Found to Discriminate,” New York Times, July 23, 2009, https://cityroom.blogs.nytimes.com/2009/07/23/the-fire-dept-tests-that-were-found-to-discriminate/?mcubz=0&_r=0; US v. City of New York (FDNY), https://www.justice.gov/archives/crt-fdny/overview.
6. Paul Voosen, “How AI Detectives Are Cracking Open the Black Box of Deep Learning,” Science, July 6, 2017, http://www.sciencemag.org/news/2017/07/howai-detectives-are-cracking-open-black-box-deep-learning.
7. T. Blake, C. Nosko, and S. Tadelis, “Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment,” Econometrica 83 (2015): 155–174.
8. Hossein Hosseini, Baicen Xiao, and Radha Poovendran, “Deceiving Google’s Cloud Video Intelligence API Built for Summarizing Videos” (paper presented at CVPR Workshops, March 31, 2017), https://arxiv.org/pdf/1703.09793.pdf; see also “Artificial Intelligence Used by Google to Scan Videos Could Easily Be Tricked by a Picture of Noodles,” Quartz, April 4, 2017, https://qz.com/948870/the-ai-used-bygoogle-to-scan-videos-could-easily-be-tricked-by-a-picture-of-noodles/.
9. See, for example, the thousands of citations to C. S. Elton, The Ecology of Invasions by Animals and Plants (New York: John Wiley, 1958).
10. Based on discussions with University of Waterloo dean Pearl Sullivan, professor Alexander Wong, and other Waterloo professors on November 20, 2016.
11. There is a fourth benefit to prediction on the ground: sometimes it is necessary for practical purposes. For instance, Google Glass needed to be able to determine whether an eyelid movement was a blink (nonintentional) or a wink (intentional), with the latter being a means by which the device could be controlled. Because of the speed with which that determination needed to be made, sending the data to the cloud and waiting for an answer was impractical. The prediction machine needed to be hosted in the device.
12. Ryan Singel, “Google Catches Bing Copying; Microsoft Says ‘So What?’” Wired, February 1, 2011, https://www.wired.com/2011/02/bing-copies-google/.
13. See Shane Greenstein for a discussion of why it was unacceptable; “Bing Imitates Google: Their Conduct Crosses a Line,” Virulent Word of Mouse (blog), February 2, 2011, https://virulentwordofmouse.wordpress.com/2011/02/02/bing-imitates-google-their-conduct-crosses-a-line/; and Ben Edelman for a counterpoint, “In Accusing Microsoft, Google Doth Protest Too Much,” hbr.org, February 3, 2011, https://hbr.org/2011/02/in-accusing-microsoft-google.html.
14. It is also interesting that Google’s attempt to manipulate Microsoft’s machine learning did not work very well. Of the one hundred experiments it conducted, only seven to nine actually appeared in Bing search results. See Joshua Gans, “The Consequences of Hiybbprqag’ing,” Digitopoly, February 8, 2011; https://digitopoly.org/2011/02/08/the-consequences-of-hiybbprqaging/.
15. Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart, “Stealing Machine Learning Models via Prediction APIs” (paper presented at the Proceedings of the 25th USENIX Security Symposium, Austin, TX, August 10–12, 2016), https://regmedia.co.uk/2016/09/30/sec16_paper_tramer.pdf.
16. James Vincent, “Twitter Taught Microsoft’s AI Chatbot to Be a Racist Asshole in Less Than a Day,” The Verge, March 24, 2016, https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist.
17. Rob Price, “Microsoft Is Deleting Its Chatbot’s Incredibly Racist Tweets,” Business Insider, March 24, 2016, http://www.businessinsider.com/microsoft-deletes-racist-genocidal-tweets-from-ai-chatbot-tay-2016-3?r=UK&IR=T.
Chapter 21
1. James Vincent, “Elon Musk Says We Need to Regulate AI Before It Becomes a Danger to Humanity,” The Verge, July 17, 2017, https://www.theverge.com/2017/7/17/15980954/elon-musk-ai-regulation-existential-threat.
2. Chris Weller, “One of the Biggest VCs in Silicon Valley Is Launching an Experiment That Will Give 3000 People Free Money Until 2022,” Business Insider, September 21, 2017, http://www.businessinsider.com/y-combinator-basic-income-test-2017-9.
3. Stephen Hawking, “This Is the Most Dangerous Time for Our Planet,” The Guardian, December 1, 2016, https://www.theguardian.com/commentisfree/2016/dec/01/stephen-hawking-dangerous-time-planet-inequality.
4. “The Onrushing Wave,” The Economist, January 18, 2014, https://www.economist.com/news/briefing/21594264-previous-technological-innovation-has-always-delivered-more-long-run-employment-not-less.
5. For more, see John Markoff, Machines of Loving Grace: The Quest for Common Ground between Humans and Robots (New York: Harper Collins, 2015); Martin Ford, Rise of the Robots: Technology and the Threat of a Jobless Future (New York: Basic Books, 2016); Ryan Avent, The Wealth of Humans: Work, Power, and Status in the Twenty-First Century (London: St. Martin’s Press, 2016); Daniel Susskind, A World without Work: Technology, Automation and How We Should Respond (London: Penguin UK, 2020).
6. For an overview of the arguments regarding innovation (including AI) and its impact on inequality, see Joshua Gans and Andrew Leigh, Innovation + Equality: How to Create a Future That Is More Star Trek Than Terminator (Cambridge, MA: MIT Press, 2019).
7. Jason Furman, “Is This Time Different? The Opportunities and Challenges of AI,” https://obamawhitehouse.archives.gov/sites/default/files/page/files/20160707_cea_ai_furman.pdf.
8. Claudia Dale Goldin and Lawrence F. Katz, The Race between Education and Technology (Cambridge, MA: Harvard University Press, 2009), 90.
9. Lesley Chiou and Catherine Tucker, “Search Engines and Data Retention: Implications for Privacy and Antitrust,” working paper no. 23815, National Bureau of Economic Research, http://www.nber.org/papers/w23815.
10. Google AdWords, “Reach more customers with broad match,” 2008.
11. For a review of antitrust and other implications around algorithms, data, and AI, see Ariel Ezrachi and Maurice Stucke, Virtual Competition: The Promise and Perils of the Algorithm-Driven Economy (Cambridge, MA: Harvard University Press, 2016). For a view that perhaps algorithms themselves will be concentrated into a single algorithm, see Pedro Domingos, The Master Algorithm (New York: Basic Books, 2015). Finally, Steve Lohr provides an overview of how businesses are preemptively investing in data for strategic advantage; see Steve Lohr, Dataism (New York: Harper Business, 2015).
12. James Vincent, “Putin Says the Nation That Leads in AI ‘Will Be the Ruler of the World,’” The Verge, September 4, 2017, https://www.theverge.com/2017/9/4/16251226/russia-ai-putin-rule-the-world.
13. The reports are: (1) Jason Furman, “Is This Time Different? The Opportunities and Challenges of Artificial Intelligence” (remarks at AI Now, New York University, July 7, 2016), https://obamawhitehouse.archives.gov/sites/default/files/page/files/20160707_cea_ai_furman.pdf; (2) Executive Office of the President, “Artificial Intelligence, Automation, and the Economy,” December 2016, https://obamawhitehouse.archives.gov/sites/whitehouse.gov/files/documents/Artificial-Intelligence-Automation-Economy.PDF; (3) Executive Office of the President, National Science and Technology Council, and Committee on Technology, “Preparing for the Future of Artificial Intelligence,” October 2016, https://obamawhitehouse.archives.gov/sites/default/files/whitehouse_files/microsites/ostp/NSTC/preparing_for_the_future_of_ai.pdf; (4) National Science and Technology Council and Networking and Information Technology Research and Development Subcommittee, “The National Artificial Intelligence Research and Development Strategic Plan,” October 2016, https://obamawhitehouse.archives.gov/sites/default/files/whitehouse_files/microsites/ostp/NSTC/national_ai_rd_strategic_plan.pdf.
14. Dan Trefler and Avi Goldfarb, “AI and Trade,” in Ajay Agrawal, Joshua Gans, and Avi Goldfarb, eds., The Economics of Artificial Intelligence: An Agenda (Chicago: University of Chicago Press, 2019), 463–492.
15. Paul Mozur, “Beijing Wants AI to Be Made in China by 2030,” New York Times, July 20, 2017, https://www.nytimes.com/2017/07/20/business/china-artificial-intelligence.html?_r=0.
16. “Why China’s AI Push Is Worrying,” The Economist, July 27, 2017, https://www.economist.com/news/leaders/21725561-state-controlled-corporations-are-developing-powerful-artificial-intelligence-why-chinas-ai-push?frsc=dg%7Ce.
17. Paul Mozur, “Beijing Wants AI to Be Made in China by 2030,” New York Times, July 20, 2017, https://www.nytimes.com/2017/07/20/business/china-artificial-intelligence.html?_r=0.
18. Image 37 of Impact of Basic Research on Technological Innovation and National Prosperity: Hearing before the Subcommittee on Basic Research of the Committee on Science, House of Representatives, One Hundred Sixth Congress, first session, September 28, 1999, 27.
19. “Why China’s AI Push Is Worrying.”
20. Will Knight, “China’s AI Awakening,” MIT Technology Review, November 2017.
21. Jessi Hempel, “How Baidu Will Win China’s AI Race—and Maybe the World’s,” Wired, August 9, 2017, https://www.wired.com/story/how-baidu-will-win-chinas-ai-raceand-maybe-the-worlds/.
22. Will Knight, “10 Breakthrough Technologies—2017: Paying with Your Face,” MIT Technology Review, March–April 2017, https://www.technologyreview.com/s/603494/10-breakthrough-technologies-2017-paying-with-your-face/.
23. Oren Etzioni, “How to Regulate Artificial Intelligence,” New York Times, September 1, 2017, https://www.nytimes.com/2017/09/01/opinion/artificial-intelligence-regulations-rules.html?_r=0.
24. Aleecia M. McDonald and Lorrie Faith Cranor, “The Cost of Reading Privacy Policies,” I/S 4, no. 3 (2008): 543–568, http://heinonline.org/HOL/Page?handle=hein.journals/isjlpsoc4&div=27&g_sent=1&casa_token=&collection=journals.
25. Christian Catalini and Joshua S. Gans, “Some Simple Economics of the Blockchain,” Communications of the ACM 63, no. 7 (2020): 80–90.
26. Nick Bostrom, Superintelligence (Oxford, UK: Oxford University Press, 2016).
27. For an excellent recent discussion of this debate, see Max Tegmark, Life 3.0: Being Human in the Age of Artificial Intelligence (New York: Knopf, 2017).
28. For an economic argument as to why the paper-clip AI is unlikely to destroy the world, see Joshua S. Gans, “Self-Regulating Artificial General Intelligence,” working paper no. w24352, National Bureau of Economic Research, 2018. For a treatment that deals with current machine learning, see Brian Christian, The Alignment Problem: Machine Learning and Human Values (New York, WW Norton & Company, 2020).
29. “Prepare for the Future of Artificial Intelligence,” Executive Office of the President, National Science and Technology Council, Committee on Technology, October 2016.