How to Measure Anything: Finding the Value of Intangibles in Business, Second Edition

323 Pages·2010·2.77 MB·English

by  Douglas W. Hubbard

(continued from front fl ap) Praise for the SECOND EDITION of H $49.95 USA/$59.95 CAN
H
H M A U
OW TO EASURE NYTHING H M
• B OW TO E ASURE
How not to measure risk B OW TO
FINDING THE VALUE OF “INTANGIBLES” IN BUSINESS A
• R
Methods for measuring “soft” things like D A N Y TH I NG
happiness, satisfaction, quality, and more
“How to Measure Anything was already my favorite book (just ahead of Hubbard’s second book, The Failure of Risk
•
How to fi ne-tune human judges to be Management) and one I actively promote to my students and colleagues. But the Second Edition, improving on
M
powerful, calibrated measurement the already exquisite fi rst edition, is an achievement of its own. As a physicist and economist, I applied these
SECOND EDITION
instruments techniques in several fi elds for several years. For the fi rst time, somebody wrote together all these concerns on E A S U R E
• one canvas that is at the same time accessible to a broad audience and applicable by specialists. This book is a
How you can use the Internet as an must for students and experts in the fi eld of analysis (in general) and decision-making.” A nything can be measured. This bold
instrument of measurement
—Dr. Johan Braet, University of Antwerp, Faculty of Applied Economics, H assertion is the key to solving many
Risk Management and Innovation
problems in business and life in general.
A complete resource with case studies and a robust A
accompanying Web site providing downloadable “Doug Hubbard’s book is a marvelous tutorial on how to defi ne sound metrics to justify and manage complex O The myth that certain things can’t be measured
spreadsheet-based examples, How to Measure programs. It is a must-read for anyone concerned about mitigating the risks involved with capital planning, W N Y T H I NG is a signifi cant drain on our nation’s economy,
investment decisions, and program management.” public welfare, the environment, and even national
Anything, Second Edition illustrates how author
—Jim Flyzik, former Government CIO, White House Technology Advisor and security. In fact, the chances are good that some part
Douglas Hubbard—creator of Applied Information
CIO magazine Hall of Fame Inductee T
of your life or your professional responsibilities is
Economics—has used his approach across various
O greatly harmed by a lack of measurement—by you,
industries. You’ll learn how any problem, no
Praise for the first edition—The bestselling Business Math book two years in a row!
your fi rm, or even your government.
matter how diffi cult, ill-defi ned, or uncertain,
M
can lend itself to measurement using proven
Building up from simple concepts to illustrate
methods. Straightforward and easy-to-follow, “I love this book. Douglas Hubbard helps us create a path to know the answer to almost any question, in
this is the resource you’ll turn to again and business, in science, or in life . . . How to Measure Anything provides just the tools most of us need to measure E FINDING THE VALUE OF the hands-on yet intuitively easy application of
advanced statistical techniques, How to Measure
again—beyond measure. anything better, to gain that insight, to make progress, and to succeed.”
—Peter Tippett, PhD, MD, Chief Technology Offi cer, A “INTANGIBLES” IN BUSINESS Anything, Second Edition reveals the power of
measurement in our understanding of business
CyberTrust, and inventor, fi rst antivirus software S
DOUGLAS W. HUBBARD is and the world at large. This insightful and engaging
U
“Interestingly written and full of case studies and rich examples, Hubbard’s book is a valuable resource for those
the inventor of Applied Information 2nd book shows you how to measure those things in
who routinely make decisions involving uncertainty. This book is readable and quite entertaining, and even R
Economics (AIE), a measurement your business that, until now, you may have
those who consider themselves averse to statistics may fi nd it highly approachable.”
methodology that has been used in E Edition considered “immeasurable,” including technology
—Strategic Finance
IT portfolios, entertainment media, ROI, organizational fl exibility, customer satis-
military logistics, R&D portfolios, and many more “This book is remarkable in its range of measurement applications and its clarity of style. A must-read for every A REVISED, faction, and technology risk. Offering examples
professional who has ever exclaimed, ‘Sure, that concept is important, but can we measure it?’” EXPANDED &
areas where big decisions are based on factors that will get you to attempt measurements—even
—Dr. Jack Stenner, cofounder and N SIMPLIFIED
that seem diffi cult or impossible to measure. He when it seems impossible—this book provides you
CEO of MetaMetrics, Inc.
is an internationally recognized expert in metrics, Y with the substantive steps for measuring anything,
decision analysis, and risk management, and “Hubbard has made a career of fi nding ways to measure things that other folks thought were immeasurable. especially uncertainty and risk.
Quality? The value of telecommuting? The benefi ts of greater IT security? Public image? He says it can be done— T
is a popular speaker at numerous conferences.
and without breaking the bank . . . If you’d like to fare better in the project-approval wars, take a look at this book.” H Don’t wait—take a look inside and fi nd out:
He has written articles for InformationWeek, CIO
—ComputerWorld
Enterprise, Architecture Boston, Analytics, and OR/MS •
I The three reasons why things may seem
Today, and is also the author of The Failure of Risk N
“I use this book as a primary reference for my measurement class at MIT. The students love it because it provides immeasurable but are not
Management: Why It’s Broken and How to Fix It.
practical advice that can be applied to a variety of scenarios, from aerospace and defense, healthcare, politics, etc.” G •
Inspirational examples of where seemingly
—Ricardo Valerdi, PhD, Lecturer, MIT
impossible measurements were resolved
with suprisingly simple methods
SECOND
•
EDITION How computing the value of information
will show that you probably have been
measuring all the wrong things
DOUGLAS W. HUBBARD
(continued on back fl ap)
ISBN: 978-0-470-53939-2 4-COLOR GLOSSY
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
iv
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
How to Measure Anything
Finding the Value of
“Intangibles” in Business
Second Edition
DOUGLAS W. HUBBARD
JohnWiley&Sons,Inc.
i
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
Copyright(cid:1)C 2010byDouglasW.Hubbard.Allrightsreserved.
PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey.
PublishedsimultaneouslyinCanada.
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,or
transmittedinanyformorbyanymeans,electronic,mechanical,photocopying,
recording,scanning,orotherwise,exceptaspermittedunderSection107or108ofthe
1976UnitedStatesCopyrightAct,withouteitherthepriorwrittenpermissionofthe
Publisher,orauthorizationthroughpaymentoftheappropriateper-copyfeetothe
CopyrightClearanceCenter,Inc.,222RosewoodDrive,Danvers,MA01923,
[phone],fax[phone],oronthewebatwww.copyright.com.Requeststo
thePublisherforpermissionshouldbeaddressedtothePermissionsDepartment,John
Wiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,[phone],
fax[phone],oronlineathttp://www.wiley.com/go/permissions.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheir
besteffortsinpreparingthisbook,theymakenorepresentationsorwarrantieswith
respecttotheaccuracyorcompletenessofthecontentsofthisbookandspecifically
disclaimanyimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose.No
warrantymaybecreatedorextendedbysalesrepresentativesorwrittensalesmaterials.
Theadviceandstrategiescontainedhereinmaynotbesuitableforyoursituation.You
shouldconsultwithaprofessionalwhereappropriate.Neitherthepublishernorauthor
shallbeliableforanylossofprofitoranyothercommercialdamages,includingbutnot
limitedtospecial,incidental,consequential,orotherdamages.
Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport,
pleasecontactourCustomerCareDepartmentwithintheUnitedStatesat
[phone],outsidetheUnitedStatesat(317)572-3993orfax(317)572-4002.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthat
appearsinprintmaynotbeavailableinelectronicbooks.Formoreinformationabout
Wileyproducts,visitourwebsiteatwww.wiley.com.
LibraryofCongressCataloging-in-PublicationData:
Hubbard,DouglasW.,1962-
Howtomeasureanything:findingthevalueof“intangibles”inbusiness/
DouglasW.Hubbard.–2nded.
p. cm.
Includesindex.
ISBN978-0-470-53939-2(cloth)
1.Intangibleproperty–Valuation. I.Title.
HF5681.I55H832010
657(cid:2).7–dc22 2009051051
PrintedintheUnitedStatesofAmerica.
10 9 8 7 6 5 4 3 2 1
ii
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
I dedicate this book to the people who are my inspirations
for so many things: to my wife, Janet, and to our children,
Evan, Madeleine, and Steven, who show every potential for
being Renaissance people.
I also would like to dedicate this book to the military men
and women of the United States, so many of whom I know
personally. I’ve been out of the Army National Guard for
many years, but I hope my efforts at improving battlefield
logistics for the U.S. Marines by using better measurements
have improved their effectiveness and safety.
iii
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
iv
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
Contents
Preface xi
Acknowledgments xv
SECTION I MEASUREMENT: THE SOLUTION EXISTS 1
CHAPTER1 Intangibles and the Challenge 3
Yes, I Mean Anything 5
The Proposal 6
CHAPTER2 An Intuitive Measurement Habit: Eratosthenes,
Enrico, and Emily 9
How an Ancient Greek Measured the Size of Earth 10
Estimating: Be Like Fermi 11
Experiments: Not Just for Adults 13
Notes on What to Learn from Eratosthenes,
Enrico, and Emily 18
CHAPTER3 The Illusion of Intangibles:
Why Immeasurables Aren’t 21
The Concept of Measurement 22
The Object of Measurement 26
The Methods of Measurement 28
Economic Objections to Measurement 35
The Broader Objection to the Usefulness
of “Statistics” 37
v
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
vi Contents
Ethical Objections to Measurement 39
Toward a Universal Approach to Measurement 41
SECTION II BEFORE YOU MEASURE 45
CHAPTER4 Clarifying the Measurement Problem 47
Getting the Language Right: What “Uncertainty”
and “Risk” Really Mean 49
Examples of Clarification: Lessons for Business
from, of All Places, Government 51
CHAPTER5 Calibrated Estimates: How Much Do You Know Now? 57
Calibration Exercise 59
Further Improvements on Calibration 64
Conceptual Obstacles to Calibration 65
The Effects of Calibration 71
CHAPTER6 Measuring Risk through Modeling 79
How Not to Measure Risk 79
Real Risk Analysis: The Monte Carlo 81
An Example of the Monte Carlo Method and Risk 82
Tools and Other Resources for Monte
Carlo Simulations 91
The Risk Paradox and the Need for Better
Risk Analysis 93
CHAPTER7 Measuring the Value of Information 99
The Chance of Being Wrong and the Cost of
Being Wrong: Expected Opportunity Loss 100
The Value of Information for Ranges 103
The Imperfect World: The Value of Partial
Uncertainty Reduction 107
The Epiphany Equation: How the Value of
Information Changes Everything 110
Summarizing Uncertainty, Risk, and Information
Value: The First Measurements 114
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
Contents vii
SECTION III MEASUREMENT METHODS 117
CHAPTER8 The Transition: From What to Measure to
How to Measure 119
Tools of Observation: Introduction to the
Instrument of Measurement 120
Decomposition 124
Secondary Research: Assuming You Weren’t the
First to Measure It 127
The Basic Methods of Observation: If One
Doesn’t Work, Try the Next 128
Measure Just Enough 131
Consider the Error 132
Choose and Design the Instrument 136
CHAPTER9 Sampling Reality: How Observing Some Things
Tells Us about All Things 139
Building an Intuition for Random Sampling: The
Jelly Bean Example 141
A Little about Little Samples: A Beer Brewer’s
Approach 142
Statistical Significance: A Matter of Degree 145
When Outliers Matter Most 148
The Easiest Sample Statistics Ever 150
A Biased Sample of Sampling Methods 153
Measure to the Threshold 162
Experiment 165
Seeing Relationships in the Data: An
Introduction to Regression Modeling 169
One Thing We Haven’t Discussed—and Why 174
CHAPTER10 Bayes: Adding to What You Know Now 177
Simple Bayesian Statistics 178
Using Your Natural Bayesian Instinct 181
Heterogeneous Benchmarking: A “Brand
Damage” Application 187
Bayesian Inversion for Ranges: An Overview 190
P1:OTA/XYZ P2:ABC
FM JWBT262-Hubbard March11,2010 17:25 PrinterName:tba
viii Contents
Bayesian Inversion for Ranges: The Details 193
The Lessons of Bayes 196
SECTION IV BEYOND THE BASICS 201
CHAPTER11 Preference and Attitudes: The Softer Side
of Measurement 203
Observing Opinions, Values, and the Pursuit
of Happiness 203
A Willingness to Pay: Measuring Value
via Trade-offs 207
Putting It All on the Line: Quantifying
Risk Tolerance 211
Quantifying Subjective Trade-offs: Dealing with
Multiple Conflicting Preferences 214
Keeping the Big Picture in Mind: Profit
Maximization versus Purely Subjective Trade-offs 218
CHAPTER12 The Ultimate Measurement Instrument:
Human Judges 221
Homoabsurdus: The Weird Reasons behind
Our Decisions 222
Getting Organized: A Performance
Evaluation Example 227
Surprisingly Simple Linear Models 228
How to Standardize Any Evaluation: Rasch Models 230
Removing Human Inconsistency: The Lens Model 234
Panacea or Placebo?: Questionable Methods
of Measurement 238
Comparing the Methods 246
CHAPTER13 New Measurement Instruments for Management 251
The Twenty-First-Century Tracker: Keeping Tabs
with Technology 251
Measuring the World: The Internet as an Instrument 254
Prediction Markets: A Dynamic Aggregation
of Opinions 257