Calibration of Armed Services Vocational Aptitude Battery Forms 8

19 Pages·2007·0.39 MB·English

DOCUMENT RESUME
TM 820 683
ED 222 532
Ree, Malcolm James; And Others
AUTHOR
Calibration of Armed Services Vocational Aptitude
TITLE
9, and 10.
Battery Forms 8,
Air ForceiHuman Resources Lab., Brooks AFB, Texas.
INSTITUTION
AFiTL-.TR81-49
REPORT NO
PUB DATE
Feb -S-2.
NOTE
19p.
MF01/PC01 Plus Postage.
EDRS PRICE
*Aptitude Tests; *Armed Forces; Enlisted Personnel;
DESCi'IPTORS
*Equated Scores; Test Reliability
*Armed Forces Qualification Test; *Armed Services
IDENTIFIERS
Vocational Aptitude Battery; Calibration
ABSTRACT
A calibration of the Armed Forces Qualification Test
(AFQT) composite of the Armed Services Vocational Aptitude Battery
(ASVAB) Forms 8a, 8b, 9a, 9b, 10a, and 10b to the metric of the AFQT
Form 7a (AFQT-7a) and a comparison of these outcomes to the
operational calibration tables implemented 1 October 1980 are
presented. A sample of applicants for military enlistment was
administered one form of ASVAB and the AFQT-7a in counterbalanced
order. For analytic purposes, an edited sample (15,115 males) was
separated into 6 samples based on the 6 forms of ASVAB administered.
Data were collected at 20 geographicallSr dispersed Armed Forces
Examining and Entrance Stations on the 6 forms of ASVAB and the
AFQT-7a. Each of the six samples was edited and scored, and
descriptive statistics were computed. The root-mean-square and
investigating the similarity of
average absolute deviation measures,
differences
the equated scores across the forms, showed only small
during this study.
among the operational table and tables developed
9, and 10 of ASVAB were found to be parallel when equated to
Forms 8,
AFQT-7a, and a single conversion table was deemed appropriate for
operational enlistment processing. (Author/PN)
***********************************************************************
Reproductions supplied by EDRS are the best that can be made
from the original document.
***********************************************************************
U.S. DEPARTMENT OF EDUCATION
PERMISSION TO REPRODUCE THIS
NATIONAL INSTITUTE OF EDUCATION
MATERIAL HAS BEEN GRANTED BY
EDUCATIONAL RESOURCES INFORMATION
CENTER (ERIC)
This do,ument lids been reproduced as
received from the person Or organization
AFHRL-TR-81-49
orioin3hnq it
PL2.4-e,Wrc.)
thlen made to improve
%nor chanqes tidy.
reproduction quality
TO THE EDUCATIONAL RESOURCES
Pnts of view or opinrons stated In this docu
INFORMATION CENTER (ERIC)"
rhent do not neces,drilv represent official NIE
pUltt y
por;ilwr
%IR FORCE
CALIBRATION OF ARMED SERVICES VOCATIONAL
APTITUDE BATTERY FORMS 8, 9, AND 10
U.
By
Malcolm James Ree
John J. Mathews
Cecil J. Mullins
A
Randy H. Massey, Capt, USAF
MANPOWER AND PERSONNEL DIVISION
Brooks Air Force Base, Texas 78235
February 1982
July 1981
Interim Report for Period October 1980
0
public release: distribution unlimited.
Approved for
LABORATORY
COMMAND
AIR FORCE SYSTEMS
78235
BROOKS AIR FORCE BASE,TEXAS
2
NOTICE
When Government drawings, specifications, or other data are used for any purpose other than in
connection with a definitely Government-related procurement, the United States Government incurs
that the Government may have formulated or
no responsibility or any obligation whatsoever. The fact
in any way supplied the said drawings, specifications, or other data, is not to be regarded by
implication, or otherwise in any manner construed, as licensing the holder, or any other person or
corporation; or as conveying any rights or permission to manufacture, use, or sell any patented
invention that may in any way be related thereto.
is releasable to the National Technical
The Public Affairs Office has reviewed this report, and it
Information Service, where it will be available to the general public, including foreign nationals.
This report has been reviewed and is approved for publication.
NANCY GUINN, Technical Director
Manpower and Personnel Division
RONALD W. TERRY. Colonel, USAF
Corn ma nder
3
m
SECURITY CLASSIFICATION oF THIS PA'GE (When Data It:Wet-ea',
READ INsTRucTioNs
REPORT DOCUMENTATION PAGE
BEFOP.E CONIPLETING FORM
RECIPIENT's cATALOG NUMBER
REPORT NUMBER
12. GOV'T A-CES5ION NO
3
1
\ I IMI - r11-81. Pi
,
TYPE OF REPORT ti PERIO:) (-ovERE.O
TITLE land Subirtle,
5
4
Ill106111
\ " \lift\ ki
[Ilk \ Ti( )\ 1 11
\ 1011.1) ,FIl \ l(:I:
\ I
(leinluir 1980 - Juls, 1981
I
\ NI) III
\ l' 1.1.11 Id. II \ III- lik II)11 \IS 8. 9.
PERFORMING 3-20, REPORT NUMBER
6
CONTRA:7T 7)R GRANT NUMBER, s,
k.:TmoR, ,
8
7
\folio!,
uieil .1.
uI111 .Idnio, HIN,
\latheli.
Hands,
.11(111,
.1
FROGRAM ELEMENT, PROJECT. T
pERcoRmIN-.; oRGANIZATION NAME AND ADDREss
13
.3
AREA a WCRK utUT NUMBERS
111(1 1)(4...I'M" III""1""
uritFI
62.71131'
I .1'4, IIIIIII,III liemlu rrr,
dit
IN
km
I
:7191801
I uric !him-.
I;tnuls.
I.,^ \J.- 7823.;
\ II
12. REPORT DATE
coNTROLLING :)FFI.CE NAME AND ADDRESS
I
February 1982
\ ht.)
1 ir 1.-nre, Human 81--nurri-- kallorilINF\
I 11
(
NUMBER OF PAGES
kir
1 3
1 c.d.. 7821.,
1 loll. lid...e.
Brook
18
this rep,ri
c,ntroilrn
(of
SECURITY CLASS.
I5
1(1,-e)
MON n- GRIN -.3 AGEN ---Y N AME a ADDRESSof different fro
4
I (lila-11)rd
Is.,
DECLASSIFICATION DOWNGRADING
SCHEDULE
it rills Report)
DISTRIBHTION sTATEMEN
6
111-trilonlon unliontrd.
\ pprio,eil tor !whin wird,
DISTRIBUTION STATEMENT 'of the abstract entered in Block 20, it different horn Report,
7
SUPPLEMENTARY NOTES
IS
,\I
11d\ Numhtir 7728
KEY WORDS (Continue on reverse side if n cessary and Went!! y by block number)
19
Ir,IIIIV
clIrs, Y. ,III0BIIIIIIg
i1111105
ii(platillg '
\ 1:1,Y1
rquipercimillIr
dploiolii to..d.-
\s\ \II
IIIII 110111I,II rPgre"IIIII
,dIihralion
ABSTRACT 'Cont(nue on reverse side If necessary and identify by block number)
20
\ I'VF) compoSilli 4 Ow \ ruled ,-jers, ice,
'no' ohpieliSti IIii, In ealihrdh, [Ile
\ Filled 1011.1, QIIiIhriCililull Te.t
I
( \S\ \ In Form. 8a. 8b. Oa, Oh. Ma. and IDli lo Ole imilrie nf I lie
\ 0,ational \ plitodo ILIIIvrs,
\ 1:QT Form Ta
\ ,ailiple of
()ember I980.
7,0 dm! In cullipan, !hemn uilIculne, lo the nperatinual valihrdlinn table, implementrd
1
dpplicont . 1nr iiiiIitdo hildisImmit sAa.- admini,11ire1i nor form of k-; \ \ It and 1111, kryT-Ta ill vntiotcrhalaileed order.
wide.- (Ink- 511(19)1r ul 1.1 15 11i1- (II'S ehyell IIhrmigh data eidillog techniques
I row this 1drgel ...,ouphi Id 22.11M, a
ile..iglipil to ovitide Itimah-- and l'a,V- SA lill 1111'0111111111' PI' IIIIIPabie data. For ;mak tit purpo,cs. Illi, edited ...ample sAa,
fp,Irilit'd
\ 8 adillini,lered. Data mere collected at 20 wingrapliit all \
1,,,,I BB Olt' "IS fOriII, of
\
11110 "IS si111111;1`,
1
\ S \ .\ 11 and Ille
\ ruled Imre,'" E \ atilining and EIIIranrh. SISIION. ( \ 1:1-4;;) (ill Ow ..iS form. of
\ 1.1,1T-Ta.
111,1er-lid
EDITION OF I NOV 65 IS OBSOLETE
1473
DO
FOR"
1 JAN 73
1
SECURITY CLASSIFICATION OF THIS PAGE (When Dta Enter.d)
t nelassified
SECURITY CLASSIFICATION OF THIS PAGE(Whon Dat Entorad
lontimi(d):
Item 20
males onl) .amples Mill% edited and scored. and descriptive statistics \very computed. Vereentiles for
Kach of the
smoothed b) a pol.)notilial regression procedure, F..ach sample
both the 1S1 113 and the 11V1-1a mere equated
mas plit in half. and the equating and smoothing mere repeated on each half sample. Since results mere consistent
among the large sample and the tmo half samples. the) were accepted. In order to in estigate the similarit) Of the
equated scores aeross the forms. root-mean-square (liNtS) and average absolute d('viation (1A1)) measures were
the)
1 comparison of the forms found them to be equivalent N1
computed betmeen the various equating table,.
mere equate(i to 1i,QT-7a. Th, 1iN1S and 1 11) measures showed onl) small differences tinning the operational table
and table, de\ eloped during this stud). Forms R Q. and 10 of 1S) 113 Nvere found to be parallel when equated to
rcry-Ta. and a .ingle conversion table Mil:, deemed appropriate for operational enlistment processing.
I. nclassified
SECURITY CL ASSIFICATION OF T..` P AGE(Whon Data Entorad)
PREFACE
This study was completed under the auspices of Personnel Qualification Systems which is part of a
larger effort in Force Acquisition and Distribution. It was substinwd under project 77191801. Maintenance
and Improvement of Enlisted Selection and Classification Tests.' and executed as part of the responsilnlity of
the Air Force Human Resources Laboratory (AEHRL) as lead laboratory under the executive agent (Air
Force) for Armed Services Vocational Aptitude Battery research and development.
An effort such as this. ahhough under the direction of an individual. can IM` accomplished only through
Jenny
a team effort. The authors wish to express their appreciation to Roy Cho Ilman. James Ear les. Al C
Hodge, and A 1C Gerald Yates. A debt of gratitude is owed to Doris Black, who served to condense and
translate the analytic requests into operational procedure for the Technical Services Division. Henry Clark
wrought minor magic by convituing the computer to produce analyses prior to established due dates.
The authors also wish to express their appreciation to Jacohina Skinner and the other nwmbers of
Pubhcation Review Panel 2 for helpful comments on an earlier draft of this manuscript.
TABLE OF CONTENTS
Page
Introduction
5
I.
Calibration of Tests
5
6
Method
II.
6
The Tests
Administration of Tests to Subjects
7
8
Data Editing
8
Sample
Equipercentile Equating and Calibrating
8
9
Table Generation
Results and Discussion
10
III.
10
Data Editing
11
Analysis Samples
12
Descriptive Statistics
12
Equating
Tables for the AFQT Forms
12
16
Conclusions
IV.
16
References
LIST OF ILLUSTRATIONS
Page
Figure
Scatter Plot of Arithmetic Reasoning and Numerical Operations Test Scores
11
I
LIST OF TABLES
Page
Table
6
Name and Number of Items for Power and Speeded ASVAB Subtests in Forms 8, 9. and 10
i
AFEES Sites and Target Sample at Sites
7
2
9
Example of Smoothing by Polynomial
3
Number of Subjects Flagged by Key Verification by Test Form
10
.1
10
Example Cases from Key Verification Procedures
5
12
6 Number of Subjects by ASVAB Form
Descriptive Statistics for ASVAB 8, 9, and 10 and AFQT-7a
12
7
Conversion Tables for Each Form
13
8
9
Deviation Measures Comparing Use of One Versus Six Conversion Tables
15
Classification by Mental Category Based on One Versus Six Tables
15
I()
16
Deviation of Percentile Scores across Category Lines
11
7
3
CALIBRATION OF ARMED SERVICES VOCATIONAL APTITUDE
BATTERY FORMS 8, 9, AND 10
I. INTRODUCTION
The measurement of human characteristics has been a necessary part of selection and classification for mili-
tary occupations for over 60 years. Like measurement of physical characteristics, such as length, weight, or density,
no natural units of measure exist for psychological characteristics; rather, artificial units are established by consen-
sus. One of the most frequently used units of measurement for human characteristics is the percentile equivalent.
The percentile is reported in reference to some standard population or group. Ability tests used for military selec-
tion and classification are usually referenced to the 1944 mobilization base, and this is usually accomplished by
equating new tests to old tests. Equating is the conversion of score units of one test tu the score units of another test.
The current study describes the referencing of Forms 8a, 8b, 9a, 9b, 10a, and 10b of the Armed Services Voca-
tional Aptitude Battery (ASVAB) to the mobilization base metric, through the use of an anchor test.
There are two important reasons why current tests are equated to past tests. The first is to enable the testing
agency to report on the relat;ve distribution of scores on a year-to-year basis in a common metric. For example, the
various military services lice to be able to compare current accessions to past accessions on the same scale. The
second reason is to provide a consistent meaning for cutting scores for selection and classification tests. In theory, a
the past
score for the new test at the 80th percentile can be said to be equivalent to a score at the 80th percentile on
tests, and this equivalence becomes the definition of consistency.
ly, it is an advantage if they are parallel, which
When several forms of a test are to be operational simultanec
allows the use of a single equating table. Gulliksen (1950) offers a definition of parallel tests which includes same-
with an
ness of factor structure, equality of means, equality of variances, and equality of non-zero correlations
external criterion. It also seems reasonable to include equivalence of skew anti kurtosis (Ree, 1977), the third Paid
fourth moments of the distribution, although little research exists in the firo'n
Parallel tests may be constructed by assigning items randomly to forms. This method is usually called "Ran-
domly Parallel Forms." Or items may be matched on difficulty and/or discrimination, stratified, and then assigned
random:), to one of a set of multiple forms. This procedure is called "Stratified Parallel." Analytic methods of con-
structing parallel forms also exist (Ree, 1976), but they tend to be intensive of computer time.
UsinOhe Stratified Parallel method, Forms 8, 9, and 10 of the ASVAB were constructed to be parallel in
terms of raw scores so that a single table might be used to convert raw scores on any of the six forms to percentile
equivalents. The objective of this study was to determine if a single table were appropriate.
Calibration of Tests
Because two or more forms of a test can never be made precisely equivalent in range and level, it is necessary
(Flanapn, 1951;
to render the forms interchangeable by equating. The equating procedure may be defined
Angoff, 1971) as converting the scoring units of one test to the scoring units of another.
In general, two procedures have been in common use; linear and equipercentile equating. Linear equating
requires that equivalent Z-score transformations of the two tests represent the same cumulative proportion. Said
differently, the shapes of score distributions should differ only trivially. Equipercentile equating, on the other
hand, makes no such assumption of Z-scoke equivalence. The linear method offers the advantage of dealing with
analytic statistics (means, standard deviations, etc.) which are verifiable. Equipercentile equating is preferable
when the distributions differ and is often offered as the definition of equating (Jaeger, 1981). It should be noted
that the linear and equipercentile approach coincide when both the distributions to which they are applied have
the same shape.
8
5
example,
Angoff (1971) uses the term "calibration" to describe the equating of tests of differing abilities. For
it is
the equating of a test of Word Knowledge to a test of Reading would be called "calibration." Therefore,
Angoff is
appropriate to say that military selection and classification tests have beell calibrated rather than equated.
calibration. It is
somewhat critical of the calibration technique because a problem arises from the nature of
lead to
repeatedly stated in the literature (Angoff, 1971; Flanagan, 1951; Jaeger, 1981) that calibrating does not
the
sample-unique solutions, as does equating, although empirical evidence is not offered. The non-uniqueness of
of the test.
solution makes difficult the, interpretation of several calibrations of the same test, or parallel forms
Form 8a of the
Military selection and classification tests have frequently been 'alibrated, rather than equated.
high school
ASVAB was linked via calibration to an anchor test using several differing subject groups ranging from
study in order
students to new military recruits. The effects of calibrating, as opposed to equating, require further
to understand ful!.y the consequences of the technique.
1980) were conducted which
Three previous studies (Boldt, 1980; Maier & Grafton, 1981; Sims & Truss,
10
calibrated Form 8a to Armed Forces Qualification Test Form 7a (AFQT-7a). Because ASVAB Forms 8, 9, and
previously as "Stratified Parallel Forms," it was reasoned
were constructed to be parallel by the method described
the
that calibrating one form was tantamount to calibrating all forms. That is, because calibration sets raw scores of
forms were
calibrated test equivalent to raw scores on an anchor or target test, and because the raw scores of the six
be applicable to all
constructed to be equivalent, then any one form may be calibrated, and the results should then
calibrations are
the other forms. The crucial requirement is that the forms be parallel. If they are not, separate
produced the tables
required. The present study seeks to verify the results of the earlier calibration studies which
implemented 1 October 1980. These are referred to as the operational tables.
ASVAB-8a and thereby
In order to determine if the assumptions underlying the procedures for calibrating
Evaluation (10T&E) was
Forms 8b, 9a, 9b, 10a, and 10b were acceptable, an Initial Operational Test and
selection and classification of
undertaken. The IOT&E was begun shortly after the test was put into operation for
candidates for military enlistment.
H. METHOD
The Tests
subtests. Eight of the
Forms 8, 9, and 10 of the ASVAB are multiple aptitude batteries comprised of 10
the number of items, and
subtests are power subtests, while two are speeded subtests. Table 1 shows the name,
ASVAB forms by the inclusion of
whether the subtest is power or speeded. These forms differ from the previous
of Automotive Information
Paragraph Comprehension (PC) and Coding Speed (CS) subtests, by the combination
measuring Space Perception,
and Shop Information into a single subtest (AS), and by the deletion of subtests
of the forms is about 180
Attention to Detail, and General Information. The overall administration time for any
minutes, and in operation, the test is answered on a machine scannable answer sheet.
Table 1. Name and Number of Items for Power and Speeded ASVAII
Subtests in Forms 8, 9, and 10
Power/Speed
Number of Items
Name
Power
25
General Science (GS)
Power
35
Word Knowledge (WK)
Power
30
Arithmetic Reasoning (AR)
Power
15
Paragraph Comprehension (PC)
Speed
50
Numerical Operations (NO)
Speed
84
Coding Speed (CS)
Power
25
Auto-Shop Information (AS)
Power
25
Mathematics Knowledge (MK)
Power
25
Mechanical Comprehension (MC)
Power
20
Electronics Information (EI)
6
9
The Armed Forces Qualification Test (AFQT) composite is used for military enlistment qualification and is
comprised of PC, Word Knowledge (WK), Arithmetic Reasoning (AR), and Numerical Operations (No) subtests.
All subtests are unit weighted except for NO, which is weighted by one-half.
The AFQT-7a served as the anchor test. This test
was previously used for enlistment qualification but has
been inactive for several years. It was chosen as the anchor test because its content is close to that of the
test used in
the 1944 mobilization base development testing. It is not believed to be compromised, and
an earlier form (Form 3)
of the ASVAB was calibrated against it.
The AFQT-7a has 100 items evenly distributed in the ability
areas of WK, AR, Boxes (B), and Tool
Knowledge (TK). The first two. WK and AR. are similar to the like-named subtests in tlw
current ANT portion of
the ASVAB. The latter two. B and TK, are not found in the current AFQT portions of the ASVAB. It is the
disparity'
in the ability
nwasured which leads to labeling the equating effort a calibration and which leads
to the
problem of non-unique solutions.
Administration of Tests to Subjects
.% sample of subjects was drawn to provide for equal geographical representation. Data collection took place in
20 Armed Forces Examining and Entrance Stations (AFEESs). Table 2 shows the locations of the AFEESs and the
number of subjects at each. Each subject took the AFQT-7a and
one form of the ASVAB, which was used for
qualification for military enlistment. The AFQT-7a was adinin isten.d on a separate
answer sheet. The ASVAB arid
ANT-7a tests were administered in counterbalanced order by reversing order of their administration each day
from that employed the previous day. Tests', were also administered at locations affiliated with the AFEES, called
Mobile Examining Team (MET) sites and Office of Personnel Management (OPM) sites.
Table 2. AFEES Sites and Sample at Sites'
Subjects
AFEES
1,500
Chicago
1,300
Cleveland
800
Atlanta
1,600
Baltimore
1,300
Boston
1,400
Jacksonville
2,600
Los Angeles
900
Montgomery
1,400
Newark
1,400
Philadelphia
1,200
Richmond
1,400
St. Louis
500
Spokane
600
Denver
600
Houston
Phoenix
500
400
Portland
600
San Diego
1,200
Minneapolis
1,200
Omaha
22,400
Total
3Sites included AFEES, MET, and OPM locations for test administration.
7