Table Of ContentCalhoun: The NPS Institutional Archive
DSpace Repository
Theses and Dissertations 1. Thesis and Dissertation Collection, all items
2013-09
Performance assessment of military teams in
simulator and live exercises
Mjelde, Frode V.
Monterey, California: Naval Postgraduate School
http://hdl.handle.net/10945/37677
Downloaded from NPS Archive: Calhoun
NAVAL
POSTGRADUATE
SCHOOL
MONTEREY, CALIFORNIA
THESIS
PERFORMANCE ASSESSMENT OF MILITARY TEAMS
IN SIMULATOR AND LIVE EXERCISES
by
Frode V. Mjelde
September 2013
Thesis Advisor: Christian (Kip) Smith
Second Reader: Michael McCauley
Approved for public release; distribution is unlimited
THIS PAGE INTENTIONALLY LEFT BLANK
REPORT DOCUMENTATION PAGE
Form Approved OMB No. 0704–0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction,
searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send
comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to
Washington headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA
22202–4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704–0188) Washington DC 20503.
1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED
September 2013 Master’s Thesis
4. TITLE AND SUBTITLE 5. FUNDING NUMBERS
PERFORMANCE ASSESSMENT OF MILITARY TEAMS IN SIMULATOR AND
LIVE EXERCISES
6. AUTHOR(S) Frode V. Mjelde
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
Naval Postgraduate School REPORT NUMBER
Monterey, CA 93943–5000
9. SPONSORING /MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING
The Royal Norwegian Naval Academy AGENCY REPORT NUMBER
PO Box 1, Haakonsvern
5886 BERGEN, NORWAY
11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy
or position of the Department of Defense or the U.S. Government. IRB Protocol number ____N/A____.
12a. DISTRIBUTION / AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE
Approved for public release; distribution is unlimited A
13. ABSTRACT
The purpose of this paper is to present and evaluate a tool designed to assess the performance of military teams
participating in complex military training exercises and to investigate the effectiveness of simulator training and live
training from the matching of inherent stressors. Specifically, this study evaluates a tool that has been used by
Norwegian military subject matter experts (SMEs) to assess the performance of eight cadet teams at the Royal
Norwegian Naval Academy (RNoNA) during two separate 4-hour simulator exercises and a 48-hour live exercise.
The resulting positive Spearman rank correlation coefficients between team performance assessments in the simulator
exercises and the live exercise were strongest when the simulator scenario emphasized the stressors inherent in the
live exercise and weakest when the simulator scenario did not facilitate the task demands in the live exercise. The
study showed that (1) team performance measured in simulator training exercises can predict performance in a
subsequent live training exercise, and (2) that scenario-based simulator training can realistically and effectively
represent training demands for live operations. Our findings show the RNoNA tool can be easily applied to team
training exercises and provide a meaningful evaluation of a team's future performance.
14. SUBJECT TERMS Human Systems Integration, Manpower, Personnel, Training, Human Factors 15. NUMBER OF
Engineering, Military teams, Team training, Team performance, Team performance assessment, PAGES
Teamwork, Taskwork, Norwegian Naval Academy, Simulator systems, Virtual environment, Live 129
environment, Reduced cost, Improved schedule, Improved performance 16. PRICE CODE
17. SECURITY 18. SECURITY 19. SECURITY 20. LIMITATION OF
CLASSIFICATION OF CLASSIFICATION OF THIS CLASSIFICATION OF ABSTRACT
REPORT PAGE ABSTRACT
Unclassified Unclassified Unclassified UU
NSN 7540–01–280–5500 Standard Form 298 (Rev. 2–89)
Prescribed by ANSI Std. 239–18
i
THIS PAGE INTENTIONALLY LEFT BLANK
ii
Approved for public release; distribution is unlimited
PERFORMANCE ASSESSMENT OF MILITARY TEAMS IN SIMULATOR AND
LIVE EXERCISES
Frode V. Mjelde
Lieutenant Commander, The Royal Norwegian Nav y
B.S., The Royal Norwegian Naval Academy, 199 5
Submitted in partial fulfillment of th e
requirements for the degree o f
MASTER OF SCIENCE IN HUMAN SYSTEMS INTEGRATION
from the
NAVAL POSTGRADUATE SCHOOL
September 2013
Author: Frode V. Mjelde
Approved by: Christian (Kip) Smith
Thesis Advisor
Michael McCauley
Second Reader
Robert F. Dell
Chair, Department ofO perations Research
ii i
THIS PAGE INTENTIONALLY LEFT BLANK
iv
ABSTRACT
The purpose of this paper is to present and evaluate a tool designed to assess the
performance of military teams participating in complex military training exercises and to
investigate the effectiveness of simulator training and live training from the matching of
inherent stressors. Specifically, this study evaluates a tool that has been used by
Norwegian military subject matter experts (SMEs) to assess the performance of eight
cadet teams at the Royal Norwegian Naval Academy (RNoNA) during two separate 4-
hour simulator exercises and a 48-hour live exercise. The resulting positive Spearman
rank correlation coefficients between team performance assessments in the simulator
exercises and the live exercise were strongest when the simulator scenario emphasized
the stressors inherent in the live exercise and weakest when the simulator scenario did not
facilitate the task demands in the live exercise. The study showed that (1) team
performance measured in simulator training exercises can predict performance in a
subsequent live training exercise, and (2) that scenario-based simulator training can
realistically and effectively represent training demands for live operations. Our findings
show the RNoNA tool can be easily applied to team training exercises and provide a
meaningful evaluation of a team's future performance.
v
THIS PAGE INTENTIONALLY LEFT BLANK
v i
TABLE OF CONTENTS
I. INTRODUCTION .......................................................................................................1
A. OVERVIEW .....................................................................................................1
B. BACKGROUND ..............................................................................................1
C. OBJECTIVE ....................................................................................................3
D. PROBLEM STATEMENT .............................................................................3
E. RESEARCH QUESTIONS & HYPOTHESES ............................................4
1. Assessment Tool ...................................................................................4
2. Stressors ................................................................................................4
F. SCOPE AND LIMITATIONS ........................................................................5
G. HUMAN SYSTEMS INTEGRATION (HSI) ................................................6
1. Human Factors Engineering ...............................................................6
2. Manpower .............................................................................................7
3. Personnel ...............................................................................................7
4. Training ................................................................................................8
H. SUMMARY ......................................................................................................8
I. THESIS ORGANIZATION ............................................................................9
II. LITERATURE REVIEW .........................................................................................11
A. OVERVIEW ...................................................................................................11
B. STRESSORS ..................................................................................................11
C. TEAM COGNITION AND DECISION MAKING ....................................12
1. Team cognition ...................................................................................12
2. Team decision making .......................................................................16
a. Creative ....................................................................................19
b. Analytical .................................................................................20
c. Rule-based ...............................................................................20
d. Recognition-primed .................................................................20
D. TEAM PERFORMANCE .............................................................................21
E. TEAMWORK ................................................................................................24
1. Team orientation ................................................................................24
2. Backup behavior ................................................................................25
3. Mutual performance monitoring and mutual trust ........................26
4. Closed-loop communication ..............................................................27
5. Team leadership .................................................................................27
6. Shared mental models and interdependence ...................................28
7. Adaptability ........................................................................................29
8. Agility ..................................................................................................31
F. TASKWORK .................................................................................................32
1. Creative action ...................................................................................32
2. Speed ...................................................................................................33
3. Thoroughness .....................................................................................33
4. Success .................................................................................................34
vi i
