The Inter- and Intra-Rater Reliability of Athletic Trainers at Identifying ACL Injury Risk Factors Using

95 Pages·2017·1.25 MB·English

by  Kelly Buckholz Smallbeck

University of Connecticut
OpenCommons@UConn
Master's Theses University of Connecticut Graduate School
12-16-2016
The Inter- and Intra-Rater Reliability of Athletic
Trainers at Identifying ACL Injury Risk Factors
Using the Cutting Screening Tool
Kelly Buckholz Smallbeck
University of Connecticut - Storrs, [email protected]
Recommended Citation
Buckholz Smallbeck, Kelly, "The Inter- and Intra-Rater Reliability of Athletic Trainers at Identifying ACL Injury Risk Factors Using
the Cutting Screening Tool" (2016).Master's Theses. 1038.
https://opencommons.uconn.edu/gs_theses/1038
This work is brought to you for free and open access by the University of Connecticut Graduate School at OpenCommons@UConn. It has been
accepted for inclusion in Master's Theses by an authorized administrator of OpenCommons@UConn. For more information, please contact
[email protected].
The Inter- and Intra-Rater Reliability of Athletic Trainers at Identifying ACL Injury Risk Factors
Using the Cutting Screening Tool

Kelly Michelle Buckholz Smallbeck

B.S. in Exercise and Sport Science, Oregon State University, 2010

A Thesis
Submitted in Partial Fulfillment of the
Requirements for the Degree of
Master of Science
At the
University of Connecticut
2016

i
COPYRIGHT

© 2016
Kelly Michelle Buckholz Smallbeck
ALL RIGHTS RESERVED
ii
APPROVAL PAGE

Master of Science Thesis

The Inter- and Intra-Rater Reliability of Athletic Trainers at Identifying ACL Injury Risk Factors
Using the Cutting Screening Tool

Presented By
Kelly Michelle Buckholz Smallbeck, B.S.

Major Advisor______________________________________________________
Lindsay J. DiStefano

Associate Advisor____________________________________________________
Douglas J. Casa

Associate Advisor____________________________________________________
Thomas Trojian

Associate Advisor_____________________________________________________
Carl M. Maresh

University of Connecticut
2016
iii
ACKNOWLEDGEMENTS

Dr. DiStefano – Words cannot describe how grateful I am for your guidance and support.
I am extremely blessed to have an adviser like you, who always pushed me to continue to
work and fight. You believed in me when I had almost stopped believing in myself, and
that support has led me to accomplish things I never thought I could.

Dr. Casa – Thank you so much for taking a chance on me and welcoming me into such a
wonderful program at the University of Connecticut. You made it easy to believe that
moving across the country would be a wonderful chance, and I wouldn’t trade my time
here for anything.

Dr. Maresh – Thank you for being a part of my thesis committee. Your dedication to
furthering the department and the university is truly inspiring. Thank you for helping me
believe that a West Coast transplant could succeed in New England.

Dr. Trojian – Thank you for being a part of my thesis committee. Your standards for
educating others pushed me to want to be better. I appreciate all the work you have done
for this department and for your time and dedication to my thesis.

This would not have been possible without the patient (and sometimes impatient) support
of my family and friends. Your love and support has guided me through the last two
years. And to Scott Smallbeck, you are more than I asked for and just what I need. Thank
you for always knowing what I’m thinking, both good and bad, and handling it anyway.

iv
TABLE OF CONTENTS

I. Review of Literature……………………………………………………………… 1
a. Epidemiology………………………………………………………………. 1
b. Costs and Consequences of Injury…………………………………………. 2
c. Gender Differences………………………………………………………… 5
d. Risk Factors…………………………………………………………………7
e. Recognition of Risk Factors ……………………………………………….18
f. Statement of Purpose ………………………………………………………21

II. Introduction…………………………………………………………… …………22

III. Methods…………………………………………………………………………..24

IV. Results……………………………………………………………… ……………34

V. Discussion and Conclusions……………………………………………………...65

VI. Future Research…………………………………………………………………..74

VII. References………………………………………………………………………...76

VII. Appendix A……………………………………………………………………….85

v
TABLES AND FIGURES

Tables
1. Operational Definitions of the Cutting Screening Tool…………………………… ...28
2. Inter-rater reliability based on total scores…………………………………………...35
3. Kappa statistic agreement ………………………………………………………….38
4. Inter-rater reliability of foot placement……………………………………………....38
5. Inter-rater reliability of foot rotation…………………………………………………38
6. Inter-rater reliability of hip drop…………………………………………… ………..39
7. Inter-rater reliability of lateral trunk displacement…………………………….. .…39
8. Inter-rater reliability of medial knee displacement……………………………. .….39
9. Inter-rater reliability of plantar flexion at initial contact……………………… ..….40
10. Inter-rater reliability of knee flexion displacement………………………….. ….…40
11. Inter-rater reliability of small trunk flexion displacement………………………... .40
12. Inter-rater reliability of excessive trunk flexion displacement………………… …..41
13. Inter-rater reliability of joint displacement stiff………………………………… …...41
14. Inter-rater reliability of joint displacement soft…………………………………...…41
15. Inter-rater reliability of overall impression poor…………………………………. .42
16. Inter-rater reliability of overall impression excellent……………………………...... 42

Figures
1. An example of Anterior Tibial Shear Force and its effects on the ACL………… ….10
2. Visual of cutting screening task ……………………………………………………...26
3. Comprehensive inter-rater reliability by item…………………………………… …..43
4. Inter-rater reliability of foot placement………………………………………………44
5. Inter-rater reliability of foot rotation…………………………………………………45
6. Inter-rater reliability of hip drop………………………………………………… …..46
7. Inter-rater reliability of lateral trunk displacement……………………………… …..47
8. Inter-rater reliability of medial knee displacement……………………………… …..48
9. Inter-rater reliability of plantar flexion at initial contact………………………… ….49
10. Inter-rater reliability of knee flexion displacement…………………………… …….50
11. Inter-rater reliability of small trunk flexion displacement………………………. …..51
12. Inter-rater reliability of excessive trunk flexion displacement…………………… …52
13. Inter-rater reliability of joint displacement stiff………………………………… …..53
14. Inter-rater reliability of joint displacement soft………………………………… …..54
15. Inter-rater reliability of overall impression poor……………………………………..55
16. Inter-rater reliability of overall impression excellent………………………………...56
17. Comprehensive intra-rater reliability…………………………………………… .......57
18. Comprehensive intra-rater reliability, V3 removed…………………………… …..58
vi
19. Intra-rater reliability of S05…………………………………………………… …….58
20. Intra-rater reliability of S06…………………………………………………… …….59
21. Intra-rater reliability of S17……………………………………………………… ….59
22. Intra-rater reliability of S18……………………………………………………… ….60
23. Intra-rater reliability of S22……………………………………………………… ….60
24. Comprehensive intra-rater reliability divided by experience level……………… …..61
25. Comprehensive intra-rater reliability divided by experience level, V3 removed… …61
26. Intra-rater reliability of S05 divided by experience level………………………… ....62
27. Intra-rater reliability of S06 divided by experience level………………………… …62
28. Intra-rater reliability of S17 divided by experience level………………………… …63
29. Intra-rater reliability of S18 divided by experience level………………………… …63
30. Intra-rater reliability of S22 divided by experience level………………………… …64

vii
ABSTRACT

Background: ACL injuries are a frequently talked about injury in sports and have a multitude of
negative side effects. A majority of injuries occur during single-leg cutting motions. Clinical
screening tools are an effective way to identify those demonstrating more ACL injury risk
factors. Recent research has created a variety of clinical screening tools to identify ACL injury
risk factors, however none screen athletes during a single-leg cutting motion.
Hypothesis: A clinical screening tool using a single-leg cutting motion (the Cutting Screening
Tool, or CST) will reliably identify ACL injury risk factors when used by athletic trainers.
Methods: Six currently certified, licensed, and practicing athletic trainers (4 Female, 2 Male; 3
“Novice” with less than five years experience, 3 “Veteran” with more than 10 years experience)
in the state of Connecticut were recruited to rate 20 subjects performing three trials of a single-
leg cutting task from the frontal and sagittal planes. This data was used to determine inter-rater
reliability. After a minimum waiting period of three days, the raters re-scored five subjects a
second time to assess intra-rater reliability.
Results: The CST had overall poor inter-rater reliability (ICC = 0.32, SEM = 1.42). Inter-
(2,1)
rater reliability of specific items on the CST varied from poor to excellent. One system of data
reduction (Cohen’s kappa statistic) occasionally had low inter-rater reliability because of high
percentage of chance agreement. Intra-rater reliability was moderate when comparing Novice vs.
Veteran raters (ICC = 0.64, SEM = 0.69) Novice raters had excellent intra-rater reliability
(2,1)
(ICC = 0.99, SEM = .10) while Veteran raters had poor intra-rater reliability (ICC = 0.38,
(2,1) (2,1)
viii
SEM = 1.72). One Veteran rater was removed from certain calculations due to discrepancies in
re-scoring subjects.
Conclusions: The CST is not currently ready for use as a clinical screening tool, however it will
be a valuable method of identifying those at greater risk for ACL injury following further
research.
Keywords: ACL injury; clinical screening tool; single-leg; cutting motion; injury risk

ix