Table Of ContentInnovative Uses of Assessments
for Teaching and Research
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
1182
ACS SYMPOSIUM SERIES
Innovative Uses of Assessments
for Teaching and Research
Lisa K. Kendhammer, Editor
University of Georgia
Athens, Georgia
Kristen L. Murphy, Editor
University of Wisconsin-Milwaukee
Milwaukee, Wisconsin
Sponsored by the
ACS Division of Chemical Education
AmericanChemicalSociety,Washington,DC
DistributedinprintbyOxfordUniversityPress
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
LibraryofCongressCataloging-in-PublicationData
Innovativeusesofassessmentsforteachingandresearch/LisaK.Kendhammer,editor,
UniversityofGeorgia,Athens,Georgia,KristenL.Murphy,editor,Universityof
Wisconsin-Milwaukee,Milwaukee,Wisconsin;sponsoredbytheACSDivisionof
ChemicalEducation.
pagescm.-- (ACSsymposiumseries;1182)
Includesbibliographicalreferencesandindex.
ISBN978-0-8412-2986-0(alk.paper)
1. Chemistry--Studyandteaching--Evaluation.2. Curriculumplanning.
I.Kendhammer,LisaK.,editor.II.Murphy,KristenL.,editor.III.American
ChemicalSociety.DivisionofChemicalEducation.
QD42.I562014
540.71--dc23
2014042388
ThepaperusedinthispublicationmeetstheminimumrequirementsofAmericanNational
Standard for Information Sciences—Permanence of Paper for Printed Library Materials,
ANSIZ39.48n1984.
Copyright©2014AmericanChemicalSociety
DistributedinprintbyOxfordUniversityPress
AllRightsReserved. ReprographiccopyingbeyondthatpermittedbySections107or108
oftheU.S.CopyrightActisallowedforinternaluseonly,providedthataper-chapterfeeof
$40.25plus$0.75perpageispaidtotheCopyrightClearanceCenter,Inc.,222Rosewood
Drive,Danvers,MA01923,USA.Republicationorreproductionforsaleofpagesinthis
bookispermittedonlyunderlicensefromACS.Directtheseandotherpermissionrequests
toACSCopyrightOffice,PublicationsDivision,115516thStreet,N.W.,Washington,DC
20036.
Thecitationoftradenamesand/ornamesofmanufacturersinthispublicationisnottobe
construedasanendorsementorasapprovalbyACSofthecommercialproductsorservices
referenced herein; nor should the mere reference herein to any drawing, specification,
chemicalprocess, orotherdataberegardedasalicenseorasaconveyanceofanyright
or permission to the holder, reader, or any other person or corporation, to manufacture,
reproduce,use,orsellanypatentedinventionorcopyrightedworkthatmayinanywaybe
relatedthereto. Registerednames,trademarks,etc.,usedinthispublication,evenwithout
specificindicationthereof,arenottobeconsideredunprotectedbylaw.
PRINTEDINTHEUNITEDSTATESOFAMERICA
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
Foreword
The ACS Symposium Series was first published in 1974 to provide a
mechanism for publishing symposia quickly in book form. The purpose of
the series is to publish timely, comprehensive books developed from the ACS
sponsoredsymposiabasedoncurrentscientificresearch. Occasionally,booksare
developed from symposia sponsored by other organizations when the topic is of
keeninteresttothechemistryaudience.
Beforeagreeingtopublishabook,theproposedtableofcontentsisreviewed
forappropriateandcomprehensivecoverageandforinteresttotheaudience. Some
papersmaybeexcludedtobetterfocusthebook;othersmaybeaddedtoprovide
comprehensiveness. When appropriate, overview or introductory chapters are
added. Draftsofchaptersarepeer-reviewedpriortofinalacceptanceorrejection,
andmanuscriptsarepreparedincamera-readyformat.
As a rule, only original research papers and original review papers are
included in the volumes. Verbatim reproductions of previous published papers
arenotaccepted.
ACSBooksDepartment
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
Editors’ Biographies
Lisa K. Kendhammer
LisaK.KendhammerreceivedherPh.D.in2013,inChemicalEducationand
AnalyticalChemistry,attheUniversityofWisconsin-Milwaukee. Herdissertation
project in Chemical Education under the direction of Kristen Murphy included
identifying items that exhibited differential item functioning (DIF) on multiple-
choiceassessmentsandexploringthepossiblereasonswhyDIFoccurs. Currently,
she is a post-doctoral research associate at the University of Georgia under the
directionofNorbertPienta,wherehermainresearchprojectinvolvesexamining
students’difficultywithgeneralchemistrylaboratoryskillsandtechniques. Since
2011,shehashadonepublicationandpresented3invitedtalksonDIF.
Kristen L. Murphy
Kristen L. Murphy received her Ph.D. in Physical Inorganic Chemistry at
theUniversityofWisconsin-Milwaukee(UWM).ShehasservedastheAssociate
Director for the American Chemical Society Examinations Institute since 2005
andasafacultymemberinchemicaleducationresearchatUWMsince2008. Her
researchprojectsincludetheinvestigationofastudent’sscaleliteracyandusing
differentialitemfunctioningtoexaminemultiple-choicegeneralchemistryexams
for differential item performance by gender subgroups. She has been dedicated
to the undergraduate program at UWM, teaching many large general chemistry
courses, and received the UWM Faculty Distinguished Undergraduate Teaching
Awardin2012.
©2014AmericanChemicalSociety
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
Chapter 1
Innovative Uses of Assessments for Teaching
and Research
LisaK.Kendhammer1andKristenL.Murphy*,2
1DepartmentofChemistry,UniversityofGeorgia,140CedarStreet,
Athens,Georgia30602
2DepartmentofChemistryandBiochemistry,
UniversityofWisconsin-Milwaukee,3210N.CramerStreet,
Milwaukee,Wisconsin53201
*E-mail: [email protected].
Instruction and assessment are so common to teaching and
learning that for many readers this may be second nature.
There are certainly many kinds of instruction and assessment
available to instructors, and these are chosen based on
many factors. Where instruction may be more commonly
discussed, assessments may be more guarded. Some may
view assessments as any resource into understanding more
about student learning, while others may view assessments in
a narrower sense of hourly exams or final, summative exams.
While these tests certainly do qualify as assessments and may
have the necessity for being kept guarded (as some reuse tests
or some use standardized tests), there are also other kinds of
assessmentsthatproviderichinformationabouttheefficacyof
theinstruction. Further,theresultsofassessmentscanbeused
tomakedecisions,suchascoursegrades,andcommonlymaybe
associatedwithfulfillingthatneed. However,assessmentscan
provide information to instructors and researchers about many
other factors including students’ prior knowledge, conceptual
understanding, longitudinal progression of knowledge, and
misconceptions. Finally, classroom assessments are valuable
tools to reflect locally on instruction and globally to consider
student content knowledge when reflected on the longitudinal
performanceofstudentsandtheimplicationsfortheprogram.
©2014AmericanChemicalSociety
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
Introduction
Whatdostudentsknow? Howdotheirpriorknowledgeandexperiencesshape
this? Whataretheirmotivationsforlearningortheirconfidenceinlearning? How
dowefindthisout?
Asinstructorsandresearchers, thesequestionsmaybefundamental, butthe
manner in which these are answered is diverse and exciting. One method by
which to answer these questions involves the use of assessments. Assessment
maybeviewednarrowlybysomeascourseteststhataresummativeandformal.
These classroom assessment techniques are important as they are used to judge
what students know in terms of content knowledge and can contribute to the
decisions of course grades. However, these assessments can be formative as
they provide feedback to students about what they know and what they do
not. Beyond this, content tests can be used to examine prior knowledge of
students, and the feedback to instructors can guide future instruction and target
the needs of the students. Other types of classroom assessments could include
informalassessmentsforformativefeedbacktostudents,assessmentsspecifically
examining for student misconceptions, assessments built to examine aspects
of the affective domain including self-efficacy or motivation, and assessments
examining students’ metacognition. These assessments can take many forms,
from forced-response tests (multiple-choice tests) to open-ended questionnaires
orevenstudentinterviews.
Where classroom assessment can provide implications for classroom
instruction,programmaticassessmentcanprovideimplicationsforthecollection
of courses that constitute a program. Therefore, programmatic assessment may
build on the same classroom assessment techniques, but these techniques are
now considered in the context of the program. This could be considered for a
single student or a cohort of students longitudinally, for a single course over
an extended period of time, or commonly a collection of courses that build a
program. Regardless, many different assessments can be used to reflect on the
efficacyofaprogram,extendingbeyondsummativefinalexams.
What Information Assessments Can Provide
Logically,weexpecttolearnwhatstudentsdoordonotknowaboutspecific
contentareasfromtypicalcourseassessments. Welearnthisthroughwhatstudents
can or cannot do correctly on a test. This information can be valuable when
assigning grades and providing more specific feedback to students about areas
ofstrengthorweakness. However,thiscaninforminstructorswhenconsidering
methodsofinstructionusedforthosespecificcontentareas. Perhaps,themethod
ofinstructionwasalteredormoreresourceswereprovided: howdidthisaffectthe
students’contentknowledge? Contenttestsmayprovidesomeinformationabout
this.
Thinking beyond standard content tests, considering other, innovative
assessments,wemayenjoyaricherpictureofwhatstudentsknoworunderstand
by investigating prior knowledge, misconceptions, motivations, or self-concept.
This can inform us as instructors so we can provide a better instructional
2
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
environment that can target areas of weakness. This can also inform us as
researchers on different areas such as investigating how students learn and how
priorexperiencesorperceptionsaffectlearning.
Awordofcautionaboutdrawingconclusionsbasedoninformationprovided
from assessments. The words “valid” and “reliable” are used commonly to
describeassessmentresultsandmanyresearchersarecarefulintheirconclusions
based on limitations associated with establishing validity and reliability (1).
As with any assessment or instrument, we must carefully consider what we
are measuring and how we measure it. Validity checks should be considered
routinely, particularly when developing new assessments (2). Innovation in
developingandusingassessmentsshouldnotbehamperedbyaddingvalidityand
reliabilitychecks,butratherstrengthenedbecauseofthis.
Purpose of This Book
Thepurposeofthisbookistoprovideasmallcollectionofinnovativeways
thatassessmentshavebeenusedforclassroomorprogrammaticassessmentorfor
researchinvestigations. Thisisbynomeanscomprehensive, butratherameans
to encourage innovation in other classrooms or in investigating other research.
Therefore,thisselectionofferssamplesofassessmentsthathavebeendeveloped
or adapted, new assessment methods or techniques, new methods of providing
feedback,orcomparisonsofmethodsforestablishingtestfairness. Wehopethis
providesasparkorideaforinnovativeusesofassessmentinotherareasofteaching
or research.
How To Use This Book
There is no prescribed method for using this book. If an instructor or
researcher has similar goals as described, then it is reasonable that the same
or similar assessment may be used in the manner described. Even when using
the same or similar population, one would expect that instructors or researchers
wouldincorporatevaliditychecks(2)inordertoestablishtrustintheresultsand
correspondingjudgmentsmadebasedontheassessments. Itisalsopossiblethat
instructorsorresearcherswillconsiderthedifferenttypesandusesofassessments
presented and adapt these for different populations or testing environments or
conditions. Thisnewresearchwouldthenaddtotheinnovationinitiallypresented
heretofurtherourcollectiveknowledgeofwhatstudentsknow. Finally,wealso
couldexpectthataninstructororresearcherconsideringtheworkpresentedhere
could be invigorated to investigate new assessment pathways that would lead to
newandinnovativeassessmentsforresearchandteaching.
The book is organized into four general sections as shown in Table 1. The
first section describes the processes by which assessments are constructed and
used. The second section focuses on what is learned from assessments in an
informalenvironment,includingtheuseofpracticeexamsandfeedbackprovided
to help students reflect on their own learning. Formal classroom assessments
andthedecisionsassociatedwithdifferentassessmentsandtechniquescomprises
3
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
the third section. The final section focuses on assessment goals and innovative
investigationsofstudentlearningwithdescriptionsofnewassessmentsandnew
onlinetoolsformeasuringstudentunderstanding.
Table1. OrganizationoftheChapters
SectionI Chapters2,3,and4 TheProcessByWhichAssessmentsAre
DevelopedAndEvaluatedAndHowThis
CanFacilitateResearch
SectionIi Chapters5and6 InformalClassroomAssessments–Helping
StudentsReflectOnTheirLearning
SectionIii Chapters7and8 FormalClassroomAssessments–Gauging
StudentLearning
SectionIv Chapters9,10,and11 AssessmentGoalsAndInnovativeMethods
ForInvestigatingStudentLearning
Wehopethatinwhateverformyoufindthisworkuseful,youareencouraged
toinvestigatestudentlearningthatpushesusalltothinkaboutansweringimportant
questionssuchas,“Whatdomystudentsknow?”
References
1. Barbera, J., VandenPlas, J. R. All Assessment Materials Are Not Created
Equal: TheMythsaboutInstrumentDevelopment,Validity,andReliability.
In Investigating Classroom Myths through Research on Teaching and
Learning; Bunce, D. M., VandenPlas, J. R., Eds.; ACS Symposium Series
1074;AmericanChemicalSociety: Washington,DC,2011;pp. 177−194.
2. Arjoon, J. A.; Xu, X. Y.; Lewis, J. E. Understanding the State of the
Art for Measurement in Chemistry Education Research: Examining the
PsychometricEvidence. J.Chem. Educ. 2013,90,536–545.
4
In Innovative Uses of Assessments for Teaching and Research; Kendhammer, et al.;
