Table Of ContentSocial Indicators Research Series 68
Tineke de Jonge
Ruut Veenhoven
Wim Kalmijn
Diversity in Survey
Questions on the
Same Topic
Techniques for Improving Comparability
Social Indicators Research Series
Volume 68
SeriesEditor
AlexC.Michalos,FacultyofArtsOffice,BrandonUniversity,Brandon,Manitoba,
Canada
Editors
EdDiener,UniversityofIllinois,Champaign,USA
WolfgangGlatzer,J.W.GoetheUniversity,FrankfurtamMain,Germany
TorbjornMoum,UniversityofOslo,Norway
MirjamA.G.Sprangers,UniversityofAmsterdam,TheNetherlands
JoachimVogel,CentralBureauofStatistics,Stockholm,Sweden
RuutVeenhoven,ErasmusUniversity,Rotterdam,TheNetherlands
This series aims to provide a public forum for single treatises and collections of
papersonsocialindicatorsresearchthataretoolongtobepublishedinourjournal
Social IndicatorsResearch.Like the journal, the book series deals with statistical
assessmentsofthequalityoflifefromabroadperspective.Itwelcomestheresearch
onawidevarietyofsubstantiveareas,includinghealth,crime,housing,education,
family life, leisure activities, transportation, mobility, economics, work, religion
andenvironmentalissues.Theseareasofresearchwillfocusontheimpactofkey
issuessuchashealthontheoverallqualityoflifeandviceversa.Aninternational
reviewboard,consistingofRuutVeenhoven,JoachimVogel,EdDiener,Torbjorn
Moum,MirjamA.G.SprangersandWolfgangGlatzer,willensurethehighquality
oftheseriesasawhole.
More information about this series at http://www.springer.com/series/6548
Tineke de Jonge • Ruut Veenhoven
Wim Kalmijn
Diversity in Survey
Questions on the Same Topic
Techniques for Improving Comparability
TinekedeJonge RuutVeenhoven
ErasmusHappinessEconomics ErasmusHappinessEconomics
ResearchOrganisation ResearchOrganisation
ErasmusUniversityRotterdam ErasmusUniversityRotterdam
Rotterdam,TheNetherlands Rotterdam,TheNetherlands
WimKalmijn(deceased)
ErasmusHappinessEconomics
ResearchOrganisation
ErasmusUniversityRotterdam
Rotterdam,TheNetherlands
ISSN1387-6570 ISSN2215-0099 (electronic)
SocialIndicatorsResearchSeries
ISBN978-3-319-53260-8 ISBN978-3-319-53261-5 (eBook)
DOI10.1007/978-3-319-53261-5
LibraryofCongressControlNumber:2017933832
©SpringerInternationalPublishingAG2017
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilarmethodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthis
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained
hereinor for anyerrors oromissionsthat may havebeenmade. Thepublisher remainsneutralwith
regardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Today, we are investing more in scientific research than ever before in human
history, and a search in Google Scholar suggests that in 2015 some eight million
scientificpublications were produced. Allthis research is assumed toresultinthe
‘accumulationofknowledge’,thoughttotakeplacethroughanexchangeofinfor-
mationinthis‘academicforum’.Realityismorecomplicatedthanthis.Thereisnot
one academic forum, but rather many different market places, and exchange of
knowledgeinthesebazaarsisconsiderablyhinderedbythedifferentlanguagesand
currenciesused.Asaresult,muchofthegatheredinformationgetslostinthedust
of libraries. Obstacles to accumulation of knowledge differ across academic set-
tings. In this book, we address one particular problem in one particular research
field.
This book is about a problem in survey research: in this case, large-scale
questionnaire studies among samples of the general population in countries. We
focuson‘socialsurveys’thatareaboutthequalityoflifeinnations.Mostmodern
countries conduct such surveys periodically, for example, the ‘Understanding
Society’ study in the UK, the Level of Living survey in Sweden and the Public
Opinion Survey on the Life of the Nation in Japan. Then, there are international
survey programs, such as the Eurobarometer, the European Social Survey, the
GallupWorldPollandtheWorldValuesSurvey.Worldwide,somemillionpeople
takepartinsuchsurveyseveryyear.
Social surveys address similar themes, such as social position and opinions.
Manyofthetopicsareidentical;allthesurveyshavequestionsaboutage,sexand
education,andmostalsoaskaboutincomeandethnicity.Yet,thequestionsasked
onthesemattersareoftenformulateddifferently,forexample,questionsaboutthe
education level of the respondent sometimes ask the respondent to give a number
for years of education, while in other surveys, a list of education types is given,
from which the respondent mustselect the final type achieved. In this case, many
differentclassificationsofschoollevelcanbeusedwhichvarywithinandbetween
countries.Inthismixtureofwaysofaskingthesamequestionofeducationlevels
v
vi Preface
acrossnationsandtime,typicallylessthanhalfoftheavailablesurveydatacanbe
usedforsuchpurpose.
Oneofthecommontopicsinsocialsurveysis‘happiness’,thatis,thesatisfac-
tionwithone’slifeasawhole.Thismatteristypicallymeasuredusingsingledirect
questions, such as: ‘Taking all together, how happy would you say you are these
days?Wouldyousayyouareveryhappy,prettyhappyornottoohappy?’Todate
(2016), such questions have figured in some 10,000 survey studies and have been
answered by some 200,000 respondents. The observed distributions of these
responses are gathered in the collection ‘Happiness in Nations’ in the World
Database of Happiness (WDH) (Veenhoven 2016). Viewing this collection of
data, we can see that differently formulated questions have been used and that
there are also many differences in the response options presented to respondents.
Again, this means that only part of the data is available for comparing happiness
across nations and time. These problems are not confined to comparing levels of
educationandhappinessacrosspopulationsandnations.Theyarefoundformany
othertopicsinsurveyresearch,suchashealthcareorcustomersatisfaction,
In this book, we present methods for dealing with this diversity in survey
questions on the same subject; we review existing methods used to homogenize
data and propose new ones. The book is a spin-off from the World Database of
Happiness,themainaimofwhichistocollateandmakeavailableresearchfindings
onthesubjectiveenjoymentoflifeandtopreparethesedataforresearchsynthesis.
The first methods we discuss were proposed in the book Happiness in Nations
(Veenhoven1993,Chap.7‘HowtheDataAreHomogenized’)whichwereusedat
the inception of the World Database of Happiness. Some 10 years later, a new
method was introduced: the International Happiness Scale Interval Study (HSIS)
(Veenhoven2008).TakingtheHSISasabasis,WimKalmijn(2010)developedthe
ContinuumApproach.Then,buildingonKalmijn’swork,TinekeDeJonge(2015)
developedtheReferenceDistributionMethod.
Inthisbook,wedescribetheevolutionoftheseinnovationsandprovideaview
ofwherewestandnow.Wegoontosuggestwaystoevolvethislineofresearch.
The book is based on the doctoral dissertation of Tineke DeJonge (2015); most
chapters are based on articles authored in most cases by the three of us and Lidia
ArendsoftheErasmusUniversityRotterdam.
WimKalmijnpassedawayinNovember2015aged81;hecontributedmuchto
the preparation of this book. We thank Willem Saris of the University Pompeu
Fabra in Barcelona for his comments on the draft version and Miranda Aldham-
Breary,seniorvolunteeroftheWDH,forimprovingtheEnglish.
Rotterdam,TheNetherlands RuutVeenhoven
Preface vii
References
De Jonge, T. (2015). Different survey questions on the same topic. How to make responses
comparable? Dissertation, Erasmus University Rotterdam, The Netherlands. Enschede:
IpskampDrukkers.Availableathttp://repub.eur.nl/pub/78339
Kalmijn,W.M.(2010).Quantificationofhappinessinequality.Dissertation,ErasmusUniversity
Rotterdam, The Netherlands. Enschede: Ipskamp Drukkers. Available at http://repub.eur.nl/
pub/21777
Veenhoven, R. (1993). Happiness in nations, subjective appreciation of life in 56 nations,
1946–1992(Studiesinsocial-culturaltransformation,2).Risbo,ErasmusUniversityRotter-
dam,Netherlands.Availableat:https://personal.eur.nl/veenhoven/Pub1990s/93b-con.html
Veenhoven,R.(2008).TheInternationalScaleIntervalStudy.InV.Møller&D.Huschka(Eds.),
Quality of Life in the new millennium: ‘Advances in quality-of-life studies, theory and
research’,Part2:Refiningconceptsandmeasurementtoassesscross-culturalquality-of-life
(SocialIndicatorResearchSeries,Vol.35,pp.45–58).Dordrecht:SpringerPress.
Veenhoven,R.(2016).Happinessinnations.WorldDatabaseofHappiness,ErasmusUniversity
Rotterdam.Availableat:http://worlddatabaseofhappiness.eur.nl/hap_nat/nat_fp.php?mode¼1
Contents
PartI ComparingResponsestoDifferentSurveyQuestionsonthe
SameTopic:ProblemsandConventionalSolutions
1 DiversityinSurveyItemsandtheComparabilityProblem. . . . . . . 3
Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
AnImpressionoftheDiversityinSurveyItems. . . . . . . . . . . . . . . . . 4
TheIncomparabilityofTimeSeriesfromDifferentSurveys
andDifferentTimePeriods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
ConventionalScaleTransformationMethodsforItemsfrom
DifferentSurveys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
LinearStretch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
SemanticJudgmentofFixedWordValue. . . . . . .. . . . . . . . . . . .. 12
SomeWordsonMethodstoCorrectforResponseHeterogeneity
attheIndividualLevel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
TheNeedforFurtherInnovation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
PartII Innovation1:TheHappinessScaleIntervalStudy
2 TheHappinessScaleIntervalStudy. . . . . . . . . . . . . . . . . . . . . . . . 19
IntroductiontotheHappinessScaleIntervalStudy. . . . . . . . . . . . . . . 19
TheScaleIntervalRecorder. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
DifferencewithConventionalMethodsforScaleTransformation. . 21
ThreeScaleTransformationMethodsAppliedto
EmpiricalData. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
TheThreeTransformationMethodsandthe
ComparabilityProblem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
ix
x Contents
3 UseofHappinessScaleIntervalStudiesinThisBook. . . . . . . . . . . 27
ResearchQuestionsAddressedinThisBookUsing
HSIS-Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
HSIS-StudiesUsedandtheSelectionofSurveyItems. . . . . . . . . . . . 28
TheUSA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
TheNetherlands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Spain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
RepresentativenessofJudges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4 EquivalenceofRatingScalesUsingDifferentKeywords. . . . . . . . . 31
SurveyswithEquivalentItemsfor‘Happiness’and
‘SatisfactionwithLife’. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
TheProblem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
TheScaleIntervalRecorderasanInstrumenttoCompare
theIntervalsRepresentedbyEquivalentResponseOptions. . . . . . . . . 33
DifferencesinValueRangesRepresentedbyResponseOptions
LabeledinDutch,SpanishandEnglish. . . . . . . . . . . . . . . . . . . . . . . 34
ResponseOptionsLabeledinDutch. . . . . . . . . . . . . . . . . . . . . . . . 34
ResponseOptionsLabeledinSpanish. . . . . . . . . . . . . . . . . . . . . . 38
ResponseOptionsLabeledinEnglish. . . . . . . . . . . . . . . . . . . . . . 41
Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
MethodologicalConsideration. . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
ImplicationoftheMethod. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
AdviceforFurtherResearch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Conclusion.. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . . .. . . 45
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5 ‘VeryHappy’IsNotAlwaysEquallyHappy. . . . . . . . . . . . . . . .. . 47
EvaluationoftheLabelsofResponseOptionsinthe
ContextoftheScale. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
TheSemanticJudgmentofFixedWordValue
MethodRevisited. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
TheContextoftheScaleandComparability. . . . . . . . . . . . . . . . . . 48
InterpretationofResponseOptionsoftheSameItemin
DifferentLanguages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
TranslationandComparability. . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
DifferenceinInterpretationBetweenDifferentLanguages. . . . . . . 51
DifferentQuestionsAbouttheSameTopic,ButwithIdentical
ResponseOptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
TheLeadingQuestionandComparability. . . .. . . . .. . . . .. . . . .. 54
DifferenceinInterpretationGivenaDifferenceinLeading
Question. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . 55
