Table Of ContentA Practical Approach
to Quantitative Validation
of Patient-R eported Outcomes
A Practical Approach to Quantitative
Validation of Patient- Reported Outcomes
A Simulation- Based Guide Using SAS
Andrew G. Bushmakin and Joseph C. Cappelleri
Statistical Research and Data Science Center
Pfizer Inc.
This edition first published 2023
© 2023 John Wiley & Sons, Inc.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted,
in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as
permitted by law. Advice on how to obtain permission to reuse material from this title is available at
http://www.wiley.com/go/permissions.
The right of Andrew G. Bushmakin and Joseph C. Cappelleri to be identified as the authors of this work has
been asserted in accordance with law.
Registered Office
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA
For details of our global editorial offices, customer services, and more information about Wiley products visit
us at www.wiley.com.
Wiley also publishes its books in a variety of electronic formats and by print- on- demand. Some content that
appears in standard print versions of this book may not be available in other formats.
Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/
or its affiliates in the United States and other countries and may not be used without written permission. All
other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any
product or vendor mentioned in this book.
Limit of Liability/Disclaimer of Warranty
In view of ongoing research, equipment modifications, changes in governmental regulations, and the
constant flow of information relating to the use of experimental reagents, equipment, and devices, the reader
is urged to review and evaluate the information provided in the package insert or instructions for each
chemical, piece of equipment, reagent, or device for, among other things, any changes in the instructions or
indication of usage and for added warnings and precautions. While the publisher and authors have used their
best efforts in preparing this work, they make no representations or warranties with respect to the accuracy
or completeness of the contents of this work and specifically disclaim all warranties, including without
limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be
created or extended by sales representatives, written sales materials or promotional statements for this work.
The fact that an organization, website, or product is referred to in this work as a citation and/or potential
source of further information does not mean that the publisher and authors endorse the information or
services the organization, website, or product may provide or recommendations it may make. This work is
sold with the understanding that the publisher is not engaged in rendering professional services. The advice
and strategies contained herein may not be suitable for your situation. You should consult with a specialist
where appropriate. Further, readers should be aware that websites listed in this work may have changed or
disappeared between when this work was written and when it is read. Neither the publisher nor authors
shall be liable for any loss of profit or any other commercial damages, including but not limited to special,
incidental, consequential, or other damages.
Library of Congress Cataloging- in- Publication Data
Names: Bushmakin, Andrew G., author. | Cappelleri, Joseph C., author.
Title: A practical approach to quantitative validation of patient-reported
outcomes : a simulation-based guide using SAS / Andrew G. Bushmakin and
Joseph C. Cappelleri.
Description: Hoboken, NJ : Wiley, 2023. | Includes bibliographical
references and index.
Identifiers: LCCN 2022024236 (print) | LCCN 2022024237 (ebook) | ISBN
9781119376378 (cloth) | ISBN 9781119376316 (adobe pdf) | ISBN
9781119376309 (epub)
Subjects: MESH: Patient Reported Outcome Measures | Patient Outcome
Assessment | Reproducibility of Results | Computer Simulation | Clinical Outcome Assessment
Classification: LCC R853.Q34 (print) | LCC R853.Q34 (ebook) | NLM W 84.41
| DDC 610.72/1–dc23/eng/20220706
LC record available at https://lccn.loc.gov/2022024236
LC ebook record available at https://lccn.loc.gov/2022024237
Cover Design: Wiley
Cover Image: Courtesy of Andrew Bushmakin
Set in 9.5/12.5pt STIXTwoText by Straive, Pondicherry, India
Disclosure
Andrew G. Bushmakin and Joseph C. Cappelleri are employees of Pfizer Inc.
This book is written for educational and instructional purposes, with emphasis on
the methodology of quantitative validation of patient-r eported outcomes. Views and
opinions expressed in this book are the authors’ own and do not necessarily reflect
those of Pfizer Inc.
vii
Contents
Preface xi
About the Authors xv
1 Introduction 1
1.1 What Is a PRO Measure? 1
1.2 Development of a PRO Measure 4
1.2.1 Concept Identification 4
1.2.1.1 Literature and Instrument Review 5
1.2.1.2 Patient- Centered Input 6
1.2.2 Item Development 9
1.2.3 Cognitive Interviews 11
1.2.4 Additional Considerations 12
1.2.5 Documentation of Development Process with Conceptual
Framework 13
1.3 Psychometric Validation 15
1.3.1 Psychometric Evaluation Data 16
1.3.2 Psychometric Properties 17
1.3.2.1 Distributional Characteristics 19
1.3.2.2 Measurement Model Structure 20
1.3.2.3 Reliability 22
1.3.2.4 Construct Validity 23
1.3.2.5 Ability to Detect Change 24
1.3.2.6 Interpretation 25
1.4 Learning Through Simulations 26
1.5 Summary 27
References 28
viii Contents
2 Validation Workflow 35
2.1 Clinical Trials as a Data Source for Validation 35
2.2 Validation Workflow for Single- Item Scales 39
2.3 Confirmatory Validation Workflow for Multi-i tem Multi- domain
Scales 43
2.4 Validation Flow for a New Multi- item Multi- domain Scale 45
2.4.1 New Scale with Known Conceptual Framework 45
2.4.2 New Scale with Unknown Measurement Structure 47
2.5 Cross- Sectional Studies and Field Tests 48
2.6 Summary 49
References 49
3 An Assessment of Classical Test Theory and Item Response Theory 51
3.1 Overview of Classical Test Theory 52
3.1.1 Basics 52
3.1.2 Illustration 52
3.1.3 Another Look 53
3.2 Person- Item Maps 55
3.2.1 CTT Revisited 55
3.2.2 Note on IRT 56
3.2.3 Implementation of Person- Item Maps 58
3.2.4 CTT- Based Scoring vs. IRT-B ased Scoring 69
3.3 Summary 78
References 80
4 Reliability 83
4.1 Reproducibility/Test–Retest 85
4.1.1 Measurement Error Model 85
4.1.2 Two Time Points 87
4.1.3 Random- Effects Model for ICC Estimation 90
4.1.4 Test–Retest Reliability Assessment in the Context of Clinical
Studies 95
4.1.4.1 Pre- Treatment/Pre- Baseline Data 95
4.1.4.2 Post- Baseline Data 97
4.1.4.3 Time Period Between Observations 101
4.1.5 Spearman- Brown Prophecy Formula 104
4.1.6 Domain Score Test–Retest vs. Item Test–Retest 109
4.1.7 Observer- Based and Interviewer-B ased Scales 111
4.1.8 Uncovering True Relationship Between Measurements 113
4.1.8.1 Accounting for Measurement Error 113
4.1.8.2 Measurement Error Model with Two Observations 122
