The Formula How Algorithms Solve All Our Problems-And Create More

281 Pages·2014·2.19 MB·English

by  Luke Dormehl

2
A PERIGEE BOOK
Published by the Penguin Group
Penguin Group (USA) LLC
375 Hudson Street, New York, New York 10014
USA • Canada • UK • Ireland • Australia • New Zealand • India • South Africa • China
penguin.com
A Penguin Random House Company
THE FORMULA
Copyright © 2014 by Luke Dormehl
Penguin supports copyright. Copyright fuels creativity, encourages diverse voices, promotes free
speech, and creates a vibrant culture. Thank you for buying an authorized edition of this book and
for complying with copyright laws by not reproducing, scanning, or distributing any part of it in any
form without permission. You are supporting writers and allowing Penguin to continue to publish
books for every reader.
PERIGEE is a registered trademark of Penguin Group (USA) LLC.
The “P” design is a trademark belonging to Penguin Group (USA) LLC.
ISBN: 978-0-698-15884-9
First American edition: November 2014
Previously published in the UK in 2014 by Virgin Books, an imprint of Ebury Publishing.
While the author has made every effort to provide accurate telephone numbers, Internet addresses,
and other contact information at the time of publication, neither the publisher nor the author
assumes any responsibility for errors, or for changes that occur after publication. Further, the
publisher does not have any control over and does not assume any responsibility for author or
third-party websites or their content.
Most Perigee books are available at special quantity discounts for bulk purchases for sales
promotions, premiums, fund-raising, or educational use. Special books, or book excerpts, can also
be created to fit specific needs. For details, write: [email protected].
3
To my friend, Tim Plester
4
Contents
Title Page
Copyright
Dedication
Acknowledgments
An Explanation of the Title, and Other Cyberbole
1 The Quantified Selves
2 The Match & the Spark
3 Do Algorithms Dream of Electric Laws?
4 The Machine That Made Art

Conclusion: Predicting the Future
A Note on Author Interviews
Notes
Index
5
Acknowledgments
Wri(cid:415)ng a book is almost always a solitary ac(cid:415)vity, but I was fortunate enough
to be surrounded by a group of people whose love and/or support made The
Formula a pleasure to work on. Thanks first of all to Clara, Tim and Celia Lunt,
as well as members of my family. I could not have completed this project
without the invaluable aid of Ed Faulkner, while it would never have got
through the door in the first place were it not for my agent Maggie Hanbury,
Henry de Rougemont, Simon Garfield and Jake Lingwood. Many thanks to
Marian Lizzi, my U.S. editor. Apprecia(cid:415)ve nods also go in the direc(cid:415)on of all
those who spent countless hours speaking with me as part of my research (a
full list of their names is printed on page 243), in addi(cid:415)on to my FastCo.Labs
editor Chris Dannen, Cult of Mac’s Leander Kahney, the excellent Nicole
Mar(cid:415)nelli, Karl French, tech comms guru Alice Bonasio-Atkinson, Tim Ma(cid:425)s,
Alex Millington, Michael Grothaus, Tom Atkinson, Simon Callow, and my
brothers-from-other-mothers, Andre and Nathan Trantraal. All helped this
book along in one way or another. All I can take full credit for are the (hopefully
few) mistakes.
6
An Explanation of the Title, and Other
Cyberbole
At their root, algorithms are li(cid:425)le more than a series of step-by-
step instruc(cid:415)ons, usually carried out by a computer. However, if
their descrip(cid:415)on is straigh(cid:414)orward, their inner workings and
impact on our lives are anything but.
Algorithms sort, filter and select the informa(cid:415)on that is
presented to us on a daily basis. They are responsible for the
search results shown to us by Google, the informa(cid:415)on about our
friends that is highlighted by Facebook, and the type of products
Amazon predicts we will be most likely to buy. Increasingly, they
will also be responsible for what movies, music and other
entertainment look like, which people we are partnered with in
predic(cid:415)ve rela(cid:415)onships, and even the ways in which laws are
enforced and police operate. An algorithm can scan through your
metadata and recommend that you will likely make a
hardworking employee, just as one could accuse you of a crime, or
determine that you are unfit to drive a car. In the process,
algorithms are profoundly changing the way that we view (to
quote Douglas Adams) life, the universe and everything.
One of my favorite observa(cid:415)ons about technology is the one
o(cid:332)en a(cid:425)ributed to the cultural theorist Paul Virilio: “The
inven(cid:415)on of the ship was also the inven(cid:415)on of the shipwreck.”
One could, of course, turn this around and say that the inventor of
the shipwreck was also the person that invented the ship.
Algorithms have had their fair share of shipwrecks (which I will
discuss during the course of this book), but they also perform
incredibly useful func(cid:415)ons: allowing us to navigate through the
2.5 quintillion bytes of data that are generated each day (a million
(cid:415)mes more informa(cid:415)on than the human brain is capable of
holding) and draw actionable conclusions from it.
7
As with the old adage about how to carve an elephant statue
(you chip away everything that isn’t an elephant), I will start out
by explaining what this book is not. It is not, for one thing, a
computer science textbook about algorithms. There are far
be(cid:425)er books (and, indeed, far more qualified writers) to achieve
this task.
Neither is it a history of the algorithm as a concept. While I
considered a(cid:425)emp(cid:415)ng such a thing, I was put off by both the
sheer scale of the project and the fact that its end result—while
no doubt fascina(cid:415)ng under the stewardship of the right author—
would be not en(cid:415)rely dissimilar to the textbook I also shied away
from. By this I do not mean that a history book and a textbook are
necessarily the same thing, but rather that a history of a once-
niche mathema(cid:415)cal concept would likely appeal only to those
mathematicians or computer scientists already familiar with it.
Instead, I want to tell the story of the myriad ways (some
subtle, others less so) that algorithms affect all of our lives: from
the entertainment we enjoy to the way we think about human
rela(cid:415)onships. What do scoring hot dates, shoo(cid:415)ng Hollywood
turkeys, bagging your own poo, and cu(cid:427)ng opportuni(cid:415)es for
lawyers’ fees have in common? This is a book about the
algorithmization of life as we know it.
In my day job, wri(cid:415)ng about a field known as the “digital
humani(cid:415)es” for Fast Company, I’m constantly considering the
implica(cid:415)ons of “algorithmic” culture and the idea (not always a
misguided one) that no ma(cid:425)er what the problem, it can be solved
with the right algorithm.
A typical illustra(cid:415)on of what I mean can be seen in Bill Tancer’s
2009 book Click: What We Do Online and Why It Ma(cid:425)er.s Tancer
—described in at least one place as “the world’s preeminent
expert on online [behavior]”—begins his book by describing a
radio interview he listened to in the car one day. Being
interviewed was a Bri(cid:415)sh psychologist referring to a
mathema(cid:415)cal formula he had developed to determine the most
8
objec(cid:415)vely depressing week of the year. A(cid:332)er much work, he had
discovered that this was the third week of January, a feat brought
about by the convergence of failed New Year’s resolu(cid:415)ons, credit-
card debt accumulated over the holiday season, and the usual
dismal weather pa(cid:425)erns. Tancer notes that he remained
unconvinced: a perspec(cid:415)ve that was later backed up when the
formula was severely cri(cid:415)cized for its lack of scien(cid:415)fic rigor.
However, his lack of convic(cid:415)on has nothing to do with the
sugges(cid:415)on that a reduc(cid:415)ve formula could possibly provide
answers on a topic as complex and mul(cid:415)faceted as depression,
but rather because he believes that he had come up with a be(cid:425)er
formula.1
In other words, his problem wasn’t with the existence of the
sum, but rather with its working.
This book was spurred by years of hearing similar observa(cid:415)ons,
all claiming that there is no problem technology cannot reduce to
its most formulaic level and thereby determine objec(cid:415)ve answers
in response to. This thinking is the reason “The Formula” is in
upper case rather than simply exis(cid:415)ng as a catchall for the various
technical processes I describe. It implies an ideological element,
and that ideology is evidenced by the more expansive view I take
of algorithms and their associated technological apparatus:
conceiving of them as the embodiment of a par(cid:415)cular form of
techno-ra(cid:415)onality, symptoma(cid:415)c of a type of social ordering built
around the promise of objec(cid:415)vity. In this way I use The Formula
much as the late American poli(cid:415)cal scien(cid:415)st and communica(cid:415)ons
theorist Harold Lasswell used the word “technique”: referring, in
Lasswell’s words, to “the ensemble of prac(cid:415)ces by which one uses
available resources to achieve values.” It is about both the
applica(cid:415)on and the scope of applica(cid:415)on, as well as the existence
of objec(cid:415)ve truths lurking just beneath the surface—to be teased
out with the right data-mining tools.
Writers on technology tend, with a few notable excep(cid:415)ons, to
be overwhelmingly utopian in their outlook. To them, all progress
9
is posi(cid:415)ve. As a result, there is a tendency among technology
writers to christen each new inven(cid:415)on as the totemic figurehead
of its own “era”—something that has led to the disdainful term
“cyberbole.” While this book could very well join the number of
volumes about algorithms and big data already lining the shelves,
what I am interested in goes back much further than simply the
birth of the Internet or the age of the personal computer.
Wri(cid:415)ng during the first half of the 1960s, the French sociologist
(and Chris(cid:415)an anarchist!) Jacques Ellul described a creature
known as the Technical Man, an individual “fascinated by results,
by the immediate consequences of se(cid:427)ng standardized devices
into mo(cid:415)on . . . commi(cid:425)ed to the never-ending search for ‘the
one best way’ to achieve any designated objective.” This objective
could occasionally be clouded (or else speeded up) by a naive
enthusiasm for the means of ge(cid:427)ng there: not by anything so
unquan(cid:415)fiable as ethical concerns, but rather by an enthusiasm
for the ingenuity, elegance and “spectacular effec(cid:415)veness” of
man’s ability to dream up solutions.
As Ellul’s observa(cid:415)on proves, this approach is not therefore a
new one, and the founders of Google and the heads of the various
high-tech companies I discuss are not the first people to display
what the late American sociologist Lewis Mumford called the
“will-to-order”—meaning the desire to make formulaic sense of
the world. Wri(cid:415)ng in the 1930s, long before the birth of the
modern computer, Mumford noted that automa(cid:415)on was
simultaneously for “enlarging the mechanical or sensory
capaci(cid:415)es of the human body” and “for reducing to a measurable
order and regularity the processes of life.” To make sense of a big
picture, we reduce it, he suggested. To take an abstract concept
such as human intelligence and turn it into something
quan(cid:415)fiable, we abstract it further, stripping away complexity
and assigning it a seemingly arbitrary number, which becomes a
person’s IQ.
What is new is the scale that this idea is now being enacted
10