How to Measure Anything: Finding the Value of Intangibles in Business

484 Pages·2016·6.43 MB·English

by  Douglas W. Hubbard

How to Measure
Anything
Finding the Value of “Intangibles in Business”
Third Edition
DOUGLAS W. HUBBARD
Cover design: Wiley
Cover image: © iStockphoto.com (clockwise from the top); © graphxarts,
© elly99, © derrrek, © procurator, © Olena_T, © miru5
Copyright © 2014 by Douglas W. Hubbard. All rights reserved.
Published by John Wiley & Sons, Inc., Hoboken, New Jersey.
First edition published by John Wiley & Sons, Inc., in 2007.
Second edition published by John Wiley & Sons, Inc., in 2010.
Published simultaneously in Canada.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by
any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted
under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written
permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the
Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax (978)
646-8600, or on the Web at www.copyright.com. Requests to the Publisher for permission should be
addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030,
[phone], fax [phone], or online at http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or
completeness of the contents of this book and specifically disclaim any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be created or extended by sales
representatives or written sales materials. The advice and strategies contained herein may not be suitable for
your situation. You should consult with a professional where appropriate. Neither the publisher nor author
shall be liable for any loss of profit or any other commercial damages, including but not limited to special,
incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our
Customer Care Department within the United States at [phone], outside the United States at (317)
572-3993, or fax [phone].
Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material
included with standard print versions of this book may not be included in e-books or in print-on-demand. If
this book refers to media such as a CD or DVD that is not included in the version you purchased, you may
download this material at http://booksupport.wiley.com. For more information about Wiley products, visit
www.wiley.com.
Library of Congress Cataloging-in-Publication Data
Hubbard, Douglas W., 1962–
How to measure anything : finding the value of intangibles in business /
Douglas W. Hubbard.—Third edition.
pages cm
Includes bibliographical references and index.
ISBN 978-1-118-53927-9 (cloth); ISBN 978-1-118-83644-6 (ebk);
ISBN 978-1-118-83649-1 (ebk)
1. Intangible property—Valuation. I. Title.
HF5681.I55H83 2014
657'.7—dc23
2013044540
I dedicate this book to the people who are my inspirations
for so many things: to my wife, Janet, and to our children,
Evan, Madeleine, and Steven, who show every potential
for being Renaissance people.
I also would like to dedicate this book to the military men
and women of the United States, so many of whom I know
personally. I've been out of the Army National Guard for
many years, but I hope my efforts at improving battlefield
logistics for the U.S. Marines by using better measurements
have improved their effectiveness and safety.
CONTENTS
Preface to the Third Edition
Acknowledgments
About the Author
PART I: The Measurement Solution Exists
CHAPTER 1: The Challenge of Intangibles
The Alleged Intangibles
Yes, I Mean Anything
The Proposal: It’s about Decisions
A “Power Tools” Approach to Measurement
A Guide to the Rest of the Book
CHAPTER 2: An Intuitive Measurement Habit: Eratosthenes, Enrico,
and Emily
How an Ancient Greek Measured the Size of Earth
Estimating: Be Like Fermi
Experiments: Not Just for Adults
Notes on What to Learn from Eratosthenes, Enrico, and Emily
Notes
CHAPTER 3: The Illusion of Intangibles: Why Immeasurables Aren’t
The Concept of Measurement
The Object of Measurement
The Methods of Measurement
Economic Objections to Measurement
The Broader Objection to the Usefulness of “Statistics”
Ethical Objections to Measurement
Reversing Old Assumptions
Notes
Note
PART II: Before You Measure
CHAPTER 4: Clarifying the Measurement Problem
Toward a Universal Approach to Measurement
The Unexpected Challenge of Defining a Decision
If You Understand It, You Can Model It
Getting the Language Right: What “Uncertainty” and “Risk” Really
Mean
An Example of a Clarified Decision
Notes
Notes
CHAPTER 5: Calibrated Estimates: How Much Do You Know Now?
Calibration Exercise
Calibration Trick: Bet Money (or Even Just Pretend To)
Further Improvements on Calibration
Conceptual Obstacles to Calibration
The Effects of Calibration Training
Notes
Notes
CHAPTER 6: Quantifying Risk through Modeling
How Not to Quantify Risk
Real Risk Analysis: The Monte Carlo
An Example of the Monte Carlo Method and Risk
Tools and Other Resources for Monte Carlo Simulations
The Risk Paradox and the Need for Better Risk Analysis
Notes
CHAPTER 7: Quantifying the Value of Information
The Chance of Being Wrong and the Cost of Being Wrong: Expected
Opportunity Loss
The Value of Information for Ranges
Beyond Yes/No: Decisions on a Continuum
The Imperfect World: The Value of Partial Uncertainty Reduction
The Epiphany Equation: How the Value of Information Changes
Everything
Summarizing Uncertainty, Risk, and Information Value: The Pre-
Measurements
Notes
PART III: Measurement Methods
CHAPTER 8: The Transition: From What to Measure to How to
Measure
Tools of Observation: Introduction to the Instrument of Measurement
Decomposition
Secondary Research: Assuming You Weren’t the First to Measure It
The Basic Methods of Observation: If One Doesn’t Work, Try the
Next
Measure Just Enough
Consider the Error
Choose and Design the Instrument
Note
CHAPTER 9: Sampling Reality: How Observing Some Things Tells Us
about All Things
Building an Intuition for Random Sampling: The Jelly Bean Example
A Little about Little Samples: A Beer Brewer’s Approach
Are Small Samples Really “Statistically Significant”?
When Outliers Matter Most
The Easiest Sample Statistic Ever
A Biased Sample of Sampling Methods
Notes
Notes
CHAPTER 10: Bayes: Adding to What You Know Now
The Basics and Bayes
Using Your Natural Bayesian Instinct
Heterogeneous Benchmarking: A “Brand Damage” Application
Bayesian Inversion for Ranges: An Overview
The Lessons of Bayes
Notes
PART IV: Beyond the Basics
CHAPTER 11: Preference and Attitudes: The Softer Side of
Measurement
Observing Opinions, Values, and the Pursuit of Happiness
A Willingness to Pay: Measuring Value via Trade-Offs
Putting It All on the Line: Quantifying Risk Tolerance
Quantifying Subjective Trade-Offs: Dealing with Multiple
Conflicting Preferences
Keeping the Big Picture in Mind: Profit Maximization versus Purely
Subjective Trade-Offs
Notes
CHAPTER 12: The Ultimate Measurement Instrument: Human Judges
Homo Absurdus: The Weird Reasons behind Our Decisions
Getting Organized: A Performance Evaluation Example
Surprisingly Simple Linear Models
How to Standardize Any Evaluation: Rasch Models
Removing Human Inconsistency: The Lens Model
Panacea or Placebo?: Questionable Methods of Measurement
Comparing the Methods
Example: A Scientist Measures the Performance of a Decision Model
Notes
CHAPTER 13 : New Measurement Instruments for Management
The Twenty-First-Century Tracker: Keeping Tabs with Technology
Prediction Markets: A Dynamic Aggregation of Opinions
Notes
CHAPTER 14: A Universal Measurement Method: Applied Information
Economics
Bringing the Pieces Together
Case: The Value of the System That Monitors Your Drinking Water
Case: Forecasting Fuel for the Marine Corps
Case: Measuring the Value of ACORD Standards
Ideas for Getting Started: A Few Final Examples
Summarizing the Philosophy
Notes
APPENDIX: Calibration Tests (and Their Answers)
Index
List of Tables
Appendix
Calibration Survey for Ranges: A
Answers for Calibration Survey for Ranges: A
Calibration Survey for Ranges: B
Answers to Calibration Survey for Ranges: B
Calibration Survey for Binary: A
Answers for Calibration Survey for Binary: A
Calibration Survey for Binary: B
Answers to Calibration Survey for Binary: B
List of Illustrations
Chapter 4
Exhibit 4.1 IT Security for the Department of Veterans Affairs
Exhibit 4.2 Department of Veterans Affairs Estimates for the Effects of
Virus Attacks
Chapter 5
Exhibit 5.1 Sample Calibration Test
Exhibit 5.2 Actual versus Ideal Scores: Initial 10 Question 90% CI Test
Exhibit 5.3 Spin to Win!
Exhibit 5.4 Methods to Improve Your Probability Calibration
Exhibit 5.5 Aggregate Group Performance
Exhibit 5.6 90% Confidence Interval Test Score Distribution after
Training (Final 20-Question Test)21
Exhibit 5.7 Calibration Experiment Results for 20 IT Industry Predictions
in 1997
Chapter 6
Exhibit 6.1 The Normal Distribution
Exhibit 6.2 Simple Monte Carlo Layout in Excel
Exhibit 6.3 Histogram
Exhibit 6.4 The Binary (a.k.a. Bernoulli) Distribution
Exhibit 6.5 The Uniform Distribution
Exhibit 6.6 Optional: Additional Monte Carlo Concepts for the More
Ambitious Student
Exhibit 6.7 A Few Monte Carlo Tools
Chapter 7
Exhibit 7.1 Extremely Simple Expected Opportunity Loss Example
Exhibit 7.2 EOL “Slices” for Range Estimates
Exhibit 7.3 Example EVPI Calculation for Segments in a Range (total
number of rows in actual table would be 20)
Exhibit 7.4 Example of the Relative Threshold
Exhibit 7.5 Expected Opportunity Loss Factor Chart
Exhibit 7.6 Loss Functions for Decisions on a Continuum
Exhibit 7.7 The Value verses Cost of Partial Information
Exhibit 7.8 The Effect of Time Sensitivity on EVPI and EVI
Exhibit 7.9 Measurement Inversion
Chapter 9
Exhibit 9.1 Simplified t-Statistic. Pick the nearest sample size (or
interpolate if you prefer more precision).