Design of an Interactive Crowdsourcing Platform to Facilitate User-Centered Information Needs ...

148 Pages·2016·2.47 MB·English

Design of an Interactive Crowdsourcing Platform to Facilitate
User-Centered Information Needs Evaluation

By

Kevin Reid Dufendach

Thesis
Submitted to the Faculty of the
Graduate School of Vanderbilt University
in partial fulfillment of the requirements
for the degree of

MASTER OF SCIENCE
in

Biomedical Informatics
August, 2016
Nashville, Tennessee

Approved:
Christoph U. Lehmann, M.D., F.A.A.P., F.A.C.M.I.
Kim M. Unertl, Ph.D.
Cynthia S. Gadd, Ph.D., M.B.A.
Copyright © 2016 by Kevin Reid Dufendach
All Rights Reserved
With the exception of:
Dufendach KR, Eichenberger JA, McPheeters ML, et al. Core Functionality in Pediatric
Electronic Health Records. Rockville (MD): Agency for Healthcare Research and Quality (US)
2015. http://www.ncbi.nlm.nih.gov/books/NBK293626.
(Public Domain)

Dufendach KR, Lehmann CU. Topics in Neonatal Informatics: Essential Functionalities of the
Neonatal Electronic Health Record. NeoReviews 2015;16:e668–73. doi:10.1542/neo.16-12-e668
Copyright © 2015 by the American Academy of Pediatrics
(Used by permission)
Background
Effective medical software is designed to fit the needs of the end users, translating their work into
action. User-centered design seeks to involve users at all stages of the design process, but the
process itself can be tedious, leading to variable degrees of implementation amongst vendors. This
research seeks to create a new method of involving multiple end users remotely in the user-
centered design process in order to establish the features and design required for clinicians need to
perform effectively.
Objectives
The objectives of this research are to summarize currently identified necessary pediatric-specific
EHR functionalities and create an online software platform to delineate further needs and
functionalities, contributing to remote user-centered design of electronic medical record software.
Methods
We created Vanderbilt Active Interface Design (VandAID), a novel web-based software platform
for crowdsourcing user interface design. The platform provides immediate real-time feedback on
user interface design and layout decisions using example patient scenarios. The scenarios can pull
information from a variety of sources using standards such as a Fast Health Interoperability
Resource (FHIR). The design platform allows the selected options to be sent to a REDCap project
for statistical analysis or viewed directly in the VandAID platform. We performed a randomized
controlled trial to test the usability and utility of this software platform for the design of a neonatal
handoff tool.
Conclusions
This research advances scientific approaches to user-centered design of health information
technology by creating a means of collecting remote feedback from multiple users. Results from
the randomized controlled trial in the first use case demonstrate this software platform to be a
highly usable and effective means of performing cooperative user-centered design.

Approved: Date:

_______________________________________________ __________________
Christoph U. Lehmann, M.D., F.A.A.P, F.A.C.M.I.
iii
TABLE OF CONTENTS
LIST OF FIGURES ....................................................................................................................... vi
Chapter
I. Introduction ..............................................................................................................................1
Background.............................................................................................................................. 1
Objectives ................................................................................................................................ 1
Specific Aims .......................................................................................................................... 1
II. Core Functionality in Pediatric Electronic Health Records .....................................................3
III. Topics in Neonatal Informatics: Essential Functionalities of the Neonatal Electronic Health
Record.......................................................................................................................................5
IV. An Interactive Crowdsourcing Platform to Facilitate User-Centered Design ..........................6
Abstract.................................................................................................................................... 6
Background.............................................................................................................................. 7
Methods ................................................................................................................................... 8
Canvas ................................................................................................................................. 8
Preferences pane ................................................................................................................. 9
Integration with REDCap ................................................................................................... 9
Results ..................................................................................................................................... 9
Conclusions ........................................................................................................................... 10
Funding .................................................................................................................................. 10
V. NICU Handoff Information Needs .........................................................................................11
Abstract.................................................................................................................................. 11
Background............................................................................................................................ 11
Methods ................................................................................................................................. 12
Results ................................................................................................................................... 14
Discussion.............................................................................................................................. 16
Role of the Clinical Informatician .................................................................................... 17
Limitations ........................................................................................................................ 17
Next Steps ......................................................................................................................... 17
Acknowledgements ............................................................................................................... 18
VI. Summary.................................................................................................................................19
Appendix
A. Role of the student in the manuscripts ...................................................................................22
B. Core Functionality in Pediatric Electronic Health Records ...................................................23
iv
C. Topics in Neonatal Informatics: Essential Functionalities of the Neonatal Electronic Health
Record.....................................................................................................................................24
REFERENCES ............................................................................................................................. 25

v
LIST OF FIGURES

Figure Page
2 - VandAID interface. On the left hand, the user selects content and formatting design decisions
from the toolbox. On the right the user experiences the results of all selections in real time on the
dynamic canvas. .............................................................................................................................. 8
3 - VandAID interface. On the left hand, the user selects content and formatting design decisions
from the toolbox. On the right the user experiences the results of all selections in real time on the
dynamic canvas. ............................................................................................................................ 13
4 - Percentage of individuals who desired a particular item on the handoff list, separated by role.
PD participants are illustrated with the lighter bars, while the data from the VandAID software is
shown with dark bars. ................................................................................................................... 14
5 - Histogram indicating minutes each user would be willing to spend updating a handoff
document. A time of zero implies the user would like the entire tool to be created automatically
with information pulled from the EHR. ........................................................................................ 15
6 - Histograms from each clinician group indicating user preferences on the types of patients to
include on the handoff document and the general amount of detail requested for the patients, with
the option to include more detail on relatively critical, sick, or active patients. The least amount of
data is represented by the top bars, and the most by the bottom bars. .......................................... 16

vi
CHAPTER I
Introduction
Background
Classical methods of software design focus on implementing specific functions and requirements
so that the software can perform a specific purpose. Optimal software design will support the user
by translating their work into actions and outcomes.[1] User-centered design seeks to involve end
users in all stages of the design process in order to help build software that is more congruent with
the work that needs to be accomplished.[2] Current electronic health record (EHR) certification
requirements from the Office of the National Coordinator for Health Information Technology
(ONC) require EHR vendors to attest to implementing user-centered design principles, but
compliance remains variable.[3–5]
Effective pediatric care requires specific EHR functionality which may not be emphasized in other
areas of care. It is neither effective nor safe simply to take an EHR designed for adult patients and
apply it to the pediatric population.[6,7] One of the main reasons for this is the child’s evolving
physiology and maturity associated with growth and development. Some key problem areas
include weight-based medication dosing, age-based vital sign reference ranges, key developmental
transitions, and well-child care functionalities. In many of these areas, best practices and the key
functionalities have not been rigorously evaluated in the medical literature.
This research seeks to identify the needs and requirements for effective care of pediatric patients
and to develop new approaches to understand and collect end user technology needs. The research
begins with a broad overview of the needed functionalities for general pediatrics with a technical
brief examining the current literature, then it shifts on those functionalities that are needed
specifically for neonatal medicine. The major body of work in this research focuses on a novel
technique to elucidate the needs and functionalities of a specific electronic health record
component, which in the initial use case was a neonatal handoff tool.
Objectives
The objectives of this research are to summarize currently identified necessary pediatric-specific
EHR functionalities and create an online software platform to delineate further needs and
functionalities, contributing to remote user-centered design of electronic medical record software.
Specific Aims
Aim 1: Specific Aim 1 of the research seeks to establish a baseline of current needs and
requirements of a pediatric electronic health record. The methods for this portion consist of a
Technical Brief for pediatrics in general and a review of the literature for neonatology. A Technical
Brief includes a limited systematic literature review and discussions with key informants. Its goals
are to provide an objective description of the state of the science, identify a framework for
analyzing interventions, and summarize ongoing research and identify research gaps.
1
Dufendach KR, Eichenberger JA, McPheeters ML, et al. Core Functionality in Pediatric
Electronic Health Records. Rockville (MD): : Agency for Healthcare Research and Quality (US)
2015. http://www.ncbi.nlm.nih.gov/books/NBK293626/ (accessed 11 Jul2016).
Dufendach KR, Lehmann CU. Topics in Neonatal Informatics: Essential Functionalities of the
Neonatal Electronic Health Record. NeoReviews 2015;16:e668–73. doi:10.1542/neo.16-12-e668
Aim 2: Specific Aim 2 of the research seeks to develop and evaluate a software platform to
facilitate user-centered medical software design. The methods for this portion are largely
technical and include web-based software development using JavaScript and the AngularJS
development framework. Evaluation includes a randomized controlled trial comparing use of the
new design platform with classical user-centered design techniques. The first use case of this
software is to evaluate the information needs for an EHR-integrated handoff tool for neonatal
intensive care unit (NICU) clinicians.
Dufendach KR, Koch S, Unertl KM, et al. An Interactive Crowdsourcing Platform to Facilitate
User-Centered Design. (in progress)
Aim 3: Specific Aim 3 of the research seeks to use the data from the randomized controlled
trial in Aim 2 to evaluate the information needs and features necessary for creating a NICU
handoff tool. The methods for this analysis will include a variety of ethnographic techniques,
including both statistical analysis and visual analysis.Dufendach KR, Unertl KM, Gadd CS, et al.
NICU Handoff Information Needs. (in progress)

2
CHAPTER II
Core Functionality in Pediatric Electronic Health Records
Dufendach KR, Eichenberger JA, McPheeters ML, et al. Core Functionality in Pediatric
Electronic Health Records. Rockville (MD): Agency for Healthcare Research and Quality (US)
2015. http://www.ncbi.nlm.nih.gov/books/NBK293626/ (accessed 11 Jul2016).
Background
Clinicians, informaticians, policy makers, and professional organizations such as the American
Academy of Pediatrics have described the need for electronic health record (EHR) systems and
information technology tools that better support pediatric health care through the availability of
pediatric functionalities. The Children’s EHR Format created almost 700 requirements pertaining
to pediatric functionality. While the report included multiple desired functions, the large number
of requirements as well as the lack of prioritization may have had a paralyzing effect on most
vendors, who, confronted with Meaningful Use requirements, did not leverage the format to
improve their products.
Purpose
A Technical Brief is a report of an emerging intervention for which there are limited published
data and too few completed research studies to support definitive conclusions. The goals of the
Technical Brief are to provide an objective description of the state of the science, identify a
potential framework for assessing the applications and implications of the intervention, summarize
ongoing research, and present research gaps. We developed a technical brief on the state of practice
and the current literature around core functionalities for pediatric electronic health records to
describe current practice and to provide a framework for future research.
Methods
We had conversations with Key Informants representing clinicians, policy experts, and
researchers. We searched online sources for information about currently available programs and
resources. We conducted a literature search to identify currently available research on the
effectiveness of individual functionalities.
Findings
There is expert consensus in the literature that EHRs used in the care of children require specific
functionalities to support the work of child health care providers and assure the delivery of quality
care to pediatric patients. These functionalities relate to a child’s evolving physiology and maturity
and associated conditions. Key areas include vaccination, child development, physiologic
medication dosing, pediatric disease management, pediatric norms, and the relationship between
pediatric patients and their caregivers, including adolescent privacy. Empirical evidence for health
outcomes associated with the introduction of a pediatric EHR or for implementation of systems
such as clinical decision support is largely limited to pre-post studies on a subset of important
functionalities. Key Informants indicated that if these functionalities are implemented well, the
EHR will also better support the care of all patients.
Summary and implications
While many of the key functionalities identified in this brief are not purely pediatric, their key role
in the care of children in contrast to their minimal role for adults could mean they can get omitted
in an EHR designed primarily for adult care. Incentives for developing pediatric functionalities for
3
EHRs are currently driven by (1) meaningful use requirements and the patient-centered medical
home; (2) a desire to support and maintain patient safety; and (3) the increasing presence of
pediatric-specific clinical quality measures. Introducing a new pediatric functionality to an EHR
should, therefore, be done thoughtfully and ideally is done in consideration of utility, testability,
and usability principles. Understanding the importance of computability and specificity of
guidelines as well as motivations for development of pediatric-specific functionalities provides
further insight into how dissemination and development will be driven in the future.
FOR FULL TEXT, SEE APPENDIX B

4