automated detection of surgical adverse events from retrospective clinical data

113 Pages·2017·2.49 MB·English

AUTOMATED DETECTION OF SURGICAL ADVERSE EVENTS
FROM RETROSPECTIVE CLINICAL DATA

A DISSERTATION
SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL
OF THE UNIVERSITY OF MINNESOTA

BY

ZHEN HU

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

ADVISOR: GENEVIEVE B. MELTON-MEAUX, MD, PHD
CO-ADVISOR: GYORGY J. SIMON, PHD

AUGUST 2017
© Zhen Hu 2017
Acknowledgements

I would like to express the deepest appreciation to my advisor, Dr. Genevieve
Melton-Meaux, and my co-advisor, Dr. Gyorgy Simon. It has been an honor and pleasure
to work with them.
I would like to thank Genevieve, for providing me the opportunity to work on this
challenging and exciting project. Her exceptional guidance and continuous support has
kept me on track, and I have always been inspired by her numerous ideas and insights into
the research in health informatics.
I would like to thank Gyorgy, for his great guidance in data analytics. Whenever I
have a problem in project, he could always help me figure it out by his versatile knowledge
and skills in machine learning and statistics. I have benefitted a lot from working closely
with him.
Thanks are also due to Dr. David Pieczkiewicz, Dr. Terrence Adam, and Dr.
Weihua Guan for agreeing to serve on my dissertation committee and for sparing their
invaluable time reviewing this dissertation. Also, I would like to express my gratitude to
Dr. Chih-Lin Chi, Dr. Lael Gatewood, Dr. Stuart Speedie, and Dr. Rui Zhang, at the
Institute for Health Informatics for their help and support.
My collaborators and colleagues at the Institute of Health Informatics and the
Department of Surgery have always been very supportive. Many thanks to:
• IHI: Dr. Yan Wang, Dr. Zohara Cohen, Reed McEwan, Jessica Whitcomb-Trance,
and Lindsay Bork.
• Surgery: Dr. Elliot Arsoniadis, Dr. Steven Skube, Dr. Mary Kwaan, Dr. Eric Jensen,
Alexandra Broek, and Carol Bigalke.
I would like to thank other students at the Institute for Health Informatics for their
support and friendship. They are Era Kim, Jin Wang, Wonsuk Oh, and Ranyah Aldekhyyel.
Finally, I would like to acknowledge Fairview Health Services and the University
of Minnesota Academic Health Center for support of the research.
i
Dedication

This dissertation is dedicated to the memory of my beloved mother, Guo Xiu-Rong.
Her love and encouragement have sustained me throughout my life.
ii
Abstract

The Detection of surgical adverse events has become increasingly important with
the growing demand for quality improvement and public health surveillance with surgery.
Event reporting is one of the key steps in determining the impact of postoperative
complications from a variety of perspectives and is an integral component of improving
transparency around surgical care and ultimately around addressing complications. Manual
chart review is the most commonly used method in identification of adverse events.
Though the manual chart review is the most commonly used method that is considered the
“gold-standard” for detecting adverse events for many patient safety studies (research
setting), it could be very labor-intensive and time-consuming and thus many hospitals have
found it too expensive to routinely use.
In this dissertation, aiming to accelerate the process of extracting postoperative
outcomes from medical charts, an automated postoperative adverse events detection
application has been developed by using structured electronic health record (EHR) data
and unstructured clinical notes. First, pilot studies are conducted to test the feasibility by
using only completed EHR data and focusing on three types of surgical site infection (SSI).
The built models have high specificity as well as very high negative predictive values,
reliably eliminating the vast majority of patients without SSI, thereby significantly
reducing the chart reviewers’ burden. Practical missing data treatments have also been
explored and compared. To address modeling challenges, such as high-dimensional dataset,
and imbalanced distribution, several machine learning methods haven been applied.
Particularly, one single-task and five multi-task learning methods are developed and
iii
compared for their detection performance. The models demonstrated high detection
performance, which ensures the feasibility of accelerating the manual process of extracting
postoperative outcomes from medical chart. Finally, the use of structured EHR data,
clinical notes and the combination of these data types have been separately investigated.
Models using different types of data were compared on their detection performance.
Models developed with very high AUC score have demonstrated that supervised machine
learning methods can be effective for automated detection of surgical adverse events.

iv
Table of Contents
CHAPTER 1 INTRODUCTION ..................................................................................... 1
1.1 PROBLEM AND SIGNIFICANCE .................................................................................... 1
1.2 RELATED WORK ......................................................................................................... 3
1.3 AIMS AND HYPOTHESES ............................................................................................. 3
1.4 OUTLINE OF THESIS .................................................................................................... 5
CHAPTER 2 BACKGROUND ........................................................................................ 7
2.1 NSQIP ....................................................................................................................... 7
2.2 HOSPITAL ACQUIRED INFECTIONS ............................................................................ 11
2.3 STRUCTURED EHR DATA ......................................................................................... 13
2.4 UNSTRUCTURED CLINICAL NOTES ............................................................................ 14
CHAPTER 3 PRELIMINARY STUDIES AND PILOT TESTING .......................... 16
3.1 INTRODUCTION ......................................................................................................... 16
3.2 METHODS AND MATERIALS ..................................................................................... 17
3.2.1 Data collection and Patient cohort identification ........................................... 18
3.2.2 Data preprocessing .......................................................................................... 19
3.2.3 Model development .......................................................................................... 22
3.2.4 Model evaluation ............................................................................................. 23
3.3 RESULTS .................................................................................................................. 23
3.3.1 Significant variables selected .......................................................................... 23
3.3.2 Model performance .......................................................................................... 26
3.4 DISCUSSION ............................................................................................................. 27
3.5 CONCLUSION ............................................................................................................ 28
CHAPTER 4 MISSING DATA IN ELECTRONIC HEALTH RECORDS ............. 29
4.1 CAPTURING THE CONTEXT OF “MISSING DATA” ........................................................ 29
4.2 RELATED WORK ON HANDLING MISSING DATA ......................................................... 32
4.3 DATA COLLECTION AND PREPROCESSING ................................................................. 34
4.4 MISSING DATA IMPUTATION OF THE INCOMPLETE DATASET ..................................... 36
v
4.5 MODELING METHOD ................................................................................................. 38
4.6 MODEL EVALUATION ............................................................................................... 39
4.7 RESULTS .................................................................................................................. 40
4.8 DISCUSSION ............................................................................................................. 46
4.9 LIMITATIONS ............................................................................................................ 49
4.10 CONCLUSIONS ........................................................................................................ 51
CHAPTER 5 DETECTION OF POSTOPERATIVE COMPLICATIONS USING
MULTI-TASK LEARNING METHODS ..................................................................... 53
5.1 INTRODUCTION ......................................................................................................... 53
5.2 DATA COLLECTION AND PREPROCESSING ................................................................. 56
5.3 METHODS ................................................................................................................. 57
5.3.1 Hierarchical Classification .............................................................................. 58
5.3.2 Offset Method ................................................................................................... 59
5.3.3 Propensity weighted observation ..................................................................... 61
5.3.4 Multi-task learning with penalties ................................................................... 64
5.3.5 Partial least squares regression ...................................................................... 65
5.4 EVALUATION ............................................................................................................ 66
5.5 RESULTS .................................................................................................................. 67
5.5.1 Evaluation results of six detection methods ..................................................... 67
5.5.2 Significant variables selected .......................................................................... 71
5.6 DISCUSSION ............................................................................................................. 73
5.7 CONCLUSION ............................................................................................................ 77
CHAPTER 6 USING EHR DATA AND CLINICAL NOTES TO
AUTOMATICALLY DETECT SURGICAL ADVERSE EVENTS ......................... 79
6.1 INTRODUCTION ......................................................................................................... 79
6.2 MATERIALS AND METHODS ..................................................................................... 79
6.2.1 Data collection ................................................................................................. 79
6.2.2 Data analysis ................................................................................................... 82
6.3 RESULTS .................................................................................................................. 82
vi
6.4 DISCUSSION AND CONCLUSION ................................................................................ 84
CHAPTER 7 SUMMARY AND FUTURE DIRECTIONS ........................................ 87
7. 1 SUMMARY ............................................................................................................... 87
7. 2 FUTURE DIRECTIONS ................................................................................................ 88
BIBLIOGRAPHY ........................................................................................................... 91

vii
List of Tables
TABLE 2-1. THE STANDARD DEFINITION OF SEPSIS USED BY THE NSQIP
REGISTRY ......................................................................................................................... 8
TABLE 2-2. 21 DEFINED POSTOPERATIVE SURGICAL ADVERSE EVENTS IN
NSQIP REGISTRY .......................................................................................................... 12
TABLE 2-3. RELEVANT KEYWORDS AND CONCEPTS IN CLINICAL NOTES .. 15
TABLE 3-1. SIGNIFICANT INDICATORS FOR DETECTING SUPERFICIAL SSI .. 24
TABLE 3-2. SIGNIFICANT INDICATORS FOR DETECTING DEEP SSI ................. 25
TABLE 3-3. SIGNIFICANT INDICATORS FOR DETECTING ORGAN SPACE SSI 25
TABLE 3-4. SIGNIFICANT INDICATORS FOR DETECTING OVERAL SSI ........... 26
TABLE 3-5. NEGATIVE PREDICTIVE VALUE AND SPECIFICITY FOR FOUR SSI
MODELS .......................................................................................................................... 27
TABLE 4-1. IMPUTED DATASETS WITH EIGHT IMPUTATION METHODS. ...... 37
TABLE 4-2. TRAINING AND TEST SET PATIENT AND SURGICAL SITE
INFECTION (SSI) CHARACTERISTICS ...................................................................... 41
TABLE 4-3. BIAS ANALYSIS FOR EIGHT IMPUTED MODELS AS WELL AS THE
REFERENCE MODEL WHEN DETECTING POSTOPERATIVE SSI. ....................... 45
TABLE 5-1. POSTOPERATIVE COMPLICATIONS DISTRIBUTION IN TRAINING
AND TEST SET ............................................................................................................... 56
TABLE 5.2. PLS ALGORITHM ...................................................................................... 66
TABLE 5-3. PAIRED T-TEST RESULTS TO COMPARE DIFFERENT METHODS 70
TABLE 5-4. SELECTED IMPORTANT VARIABLES FOR ALL COMPLICATIONS
AND THEIR DESCRIPTIONS ........................................................................................ 72
viii