Community and Thread Methods for Identifying Best Answers in Online Question Answering ...

340 Pages·2016·6.18 MB·English

Open Research Online
The Open University’s repository of research publications
and other research outputs
Community and Thread Methods for Identifying Best
Answers in Online Question Answering Communities
Thesis
How to cite:
Burel, Gregoire (2016). Community and Thread Methods for Identifying Best Answers in Online Question
Answering Communities. PhD thesis The Open University.
For guidance on citations see FAQs.
(cid:13)c 2016 Gregoire Burel
https://creativecommons.org/licenses/by-nc-nd/4.0/
Version: Version of Record
Link(s) to article on publisher’s website:
http://dx.doi.org/doi:10.21954/ou.ro.0000b440
Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyright
owners. For more information on Open Research Online’s data policy on reuse of materials please consult the policies
page.
oro.open.ac.uk
COMMUNITY AND THREAD METHODS FOR IDENTIFYING
BEST ANSWERS IN ONLINE QUESTION ANSWERING
COMMUNITIES
GRÉGOIRE BUREL
MSc,UniversityofCaenNormandy,2008
BSc,ParisDescartesUniversity,2006
ThesissubmittedinpartialfulfilmentoftherequirementsforthedegreeofDoctorofPhilosophyin
ComputerSciences
KnowledgeMediaInstitute
TheOpenUniversity
2015
Grégoire Burel: Community and Thread Methods for Identifying
Best Answers in Online Question Answering Communities, Thesis
submittedinpartialfulfilmentoftherequirementsforthedegreeof
DoctorofPhilosophyinComputerSciences,©2015
SUPERVISORS:
HarithAlani
YulanHe
PaulMulholland
EXAM PANEL CHAIR:
TrevorCollins
EXAMINERS:
StefanRueger
JohnBreslin
Dedicatedtomymother,mywifeandmydaughter.

A B S T R AC T
Muchresearchhasrecentlyinvestigatedthemeasurementofquality
answers in QUESTION ANSWERING (Q&A) communities in the
form ofautomaticbestanswer identification. Previousapproaches
havefocusedonmanualuserannotationsanddiversefeaturesbased
onintuitionforidentifyingbestanswersandprovedrelativelysuc-
cessful despite considering best answer identification as a general
classificationproblem.
Bestanswermodellingisgenerallydistancedfromcommunitystud-
iesaboutwhatusersregardasimportantforidentifyingqualitycon-
tent(i.e.userstudies).Inparticular,previousresearchtendstoonly
focusontheautomaticaspectsofbestanswersidentificationmodel
byapplyinggenericlearningalgorithms.
This thesis introduces the concepts of qualitative and structural
design in order to investigate if features derived from community
questionnaires can enrich the understanding of best answer iden-
tification in Q&A communities and if the thread-like structure of
Q&A communities can be exploited for better results. Two differ-
ent approaches for exploiting the thread structure of Q&A com-
munitiesareproposedandtwonew,previouslyunstudied,features
v
are introduced. First, a measure of question complexity is intro-
duced as a proxy measure of answerer knowledge. Second, differ-
entmodelsofcontributioneffortareproposedforrepresentingthe
answeringreactivityofcontributors.
The experiments are systematically conducted on datasets issued
from three different communities that vary in size, content and
structure. The results show that the newly proposed features allow
for better understanding of what constitute best answers. The find-
ings also reveal that the thread-wise algorithms and optimisation
techniques created from the structural design methodology correl-
ate with best answers. In general both structural and qualitative
design appear to improve best answer identification meaning that
structuralandqualitativemethodsmayimproveunrelatedclassific-
ationtasks.
vi
AC K N OW L E D G M E N T S
Firstandforemost,Iwouldliketothankmysupervisors,DrHarith
Alani,DrYulanHeandDr.PaulMulhollandfortheirpatienceand
constant supervision throughout my research. Whenever I had an
issue or a question they always had time for discussion and feed-
back. I would like to particularly thank Dr. Harith Alani for his
though challenging questions that allowed me to focus on the fun-
damentals whenever I was diverting from my research. His abil-
itytotransformcomplexideasintoclearconceptsshowedmethat
anythingcanbetackledwiththerightformulationandcoherentex-
planations. Thanks to Dr. Yulan He’s invaluable technical help, I
was able to frame and focus the most complex part of my thesis.
Her critical help taught me how to design robust experiments and
to discuss results critically. Finally, Dr. Paul Mulholland guidance
showed me the importance of good writing in research. Hopefully,
his ability to spot the smallest typos and grammatical errors made
meabetterwriter.
I would also like to thank my friends and colleagues at the Know-
ledge Media Institute (KMi) and beyond who have supported me
onewayoranotherduringthecourseofmystudies.
Finally, I would like to thank my family and particularly my wife
forsupportingmeoveralltheseyears.Theirunconditionalsupport
kept me going throughout my studies. Last but not least, I would
vii
liketothankmydaughterforhersupervisingabilitiesandenthusi-
asm.Hersmilesandcooingofencouragementcheeredmeupwhen
Iwasfinalisingmydissertation.
viii
C O N T E N T S
1 INTRODUCTION 1
1.1 Motivation 4
1.1.1 Sustainability of Question Answering Com-
munities 5
1.1.2 User Studies, Derived Features and Thread-
wiseBestAnswerIdentification 7
1.2 ResearchQuestions,HypothesesandContributions 8
1.3 ThesisMethodologyandStructure 17
1.3.1 Part I — Background and Exploratory Sur-
vey 19
1.3.2 PartII—StructuralDesignOptimisation 19
1.3.3 PartIII—QualitativeDesignOptimisation 20
1.3.4 PartIV—ConclusionsandFutureWork 21
1.4 Publications 21
i BACKGROUND AND EXPLORATORY SURVEY 23
2 STRUCTURAL AND QUALITATIVE ANALYSIS 25
2.1 Introduction 26
2.2 Structural/QualitativeDesignandBestAnswers 28
2.3 DesignandStructureofQ&ACommunities 30
2.3.1 TypesofOnlineCommunities 32
2.3.2 TypesofQ&ACommunities 34
2.3.3 StructureofQ&APlatforms 41
2.3.4 ReputationandAnswerMetadata 43
ix