Detection and Analysis of 2016 US Presidential Election Related Rumors on Twitter

0.86 MB·

by  Zhiwei Jin

Rumor Detection on Twitter Pertaining to the 2016 U.S. Presidential Election
ZhiweiJin1,2,JuanCao1,HanGuo1,2,YongdongZhang1,YuWang3 and JieboLuo3
1KeyLaboratoryofIntelligentInformationProcessing,
InstituteofComputingTechnology,CAS,Beijing100190,China
2UniversityofChineseAcademyofSciences,Beijing100049,China
3UniversityofRochester,Rochester,NY14627,USA
{jinzhiwei,caojuan,guohan,zhyd}@ict.ac.cn;[email protected];[email protected]
Abstract
7
1 The 2016 U.S. presidential election has witnessed the
0 major role of Twitter in the year’s most important po-
2 liticalevent.Candidatesusedthissocialmediaplatform
n extensivelyforonlinecampaigns.Millionsofvotersex-
a pressedtheirviewsandvotingpreferencesthroughfol-
J lowingandtweeting.Meanwhile,socialmediahasbeen
filledwithfakenewsandrumors,whichcouldhavehad
3
huge impacts on voters’ decisions. In this paper, we
2
present a thorough analysis of rumor tweets from the

followers of two presidential candidates: Hillary Clin-
]
I ton and Donald Trump. To overcome the difficulty of Figure 1: The number of rumors for popular presidential
S labeling a large amount of tweets as training data, we candidatesonSnopes.com.
. first detect rumor tweets by matching them with veri-
s
c fiedrumorarticles.Toensureahighaccuracy,wecon-
[ ductacomparativestudyoffiverumordetectionmeth-

ods. Based on the most effective method which has a
1 rumordetectionprecisionof94.7%,weanalyzeover8
v milliontweetscollectedfromthefollowersofthetwo
0 candidates.Ourresultsprovideanswerstoseveralpri-
5 maryconcernsaboutrumorsinthiselection,including:
2 which side of the followers posted the most rumors,
6 whopostedtheserumors,whatrumorstheyposted,and
0 whentheypostedtheserumors.Theinsightsofthispa-
1. percanhelpusunderstandtheonlinerumorbehaviors
0 inAmericanpolitics.
7
1
Introduction
:
v
i Online social media services, such as Twitter, play an im- Figure2:VarioustypesofrumorsrelatedtoHillaryClinton.
X portant role in various political events in recent years. Es-
r pecially in the 2016 U.S. presidential election, Twitter be-
a
cameaprimarybattleground:candidatesandtheirsupport-
find that many rumors coming from social media are re-
erswereactivelyinvolvedtodocampaignsandexpresstheir
lated to the presidential candidates. Figure 1 illustrates the
opinions by tweeting. For example, the Twitter account of
numberofrumorsaboutfourpopularcandidates.Amongall
HillaryClintonhas12millionfollowersandhermostpopu-
the1,723checkedrumorsfromSnopes.com,303rumorsare
lartweetreceivedmorethan600,000retweetsandonemil-
about Donald Trump and 226 rumors are about his Demo-
lionlikes.
cratic opponent Hillary Clinton. There are 681 rumors re-
Meanwhile, the fact that various fake news and rumors
latedtothesefourcandidatesintotal,whichconsistute40%
werespreadingonsocialmediaduringtheelectionbecame
ofallcheckedrumors.InFigure2,welisttherumorsabout
a serious concern. Many news articles criticized that fake
Hillary Clinton in categories. These rumors cover various
news on social media may have influenced the election.
topicstoquestionhercompetenceasapresident.Undoubt-
FromthepopularrumordebunkingwebsiteSnopes.com,we
edly, spreading them would have negative impacts on her
candidacy.
Copyright(cid:13)c 2015,AssociationfortheAdvancementofArtificial
Intelligence(www.aaai.org).Allrightsreserved. In this paper, we aim to understand the rumor spreading
rumorsduringpoliticaleventsinthefuture.
Insummary,ourworkmakesthreemaincontributions:
1. WeformulatetherumordetectionproblemonTwitteras
an unsupervised matching task. We use correctly veri-
fiedrumorsonSnopes.comasgoldstandardsamplesfor
rumor matching. Compared with other existing machine
learningapproaches,thisschemeismoreefficientandin-
terpretable.
2. We conduct an empirical study of several competing
matchingalgorithms.Aftercomparingtheperformanceof
fivedifferentunsupervisedmatchingalgorithmsonarea-
sonably sized labeled set, we select the BM-25 method
Figure3:Rumordetectionasatextmatchingtask.
forrumortweetdetection,whichcanreachaprecisionof
94.7%.
behaviors of candidates’ followers. A rumor is defined as 3. Wegiveathoroughanalysisonrumortweetsdetectedon
a controversial and fact-checkable statement (DiFonzo and 8milliontweetscollectedfromcandidates’followers.We
Bordia2007).WeusethecheckedrumorsfromSnope.com produce insights into followers’ rumor spreading behav-
astheobjectiverumorsourceinthispaper.Beforeanyfur- iorsfromvariousaspects.
theranalysis,weneedamethodtodetectrumorseffectively
andpreciselyonTwitter.Existingmachinelearningmethods RelatedWork
for rumor detection (Castillo, Mendoza, and Poblete 2011; Onlinesocialmediahavegainedhugepopularityaroundthe
Wu,Yang,andZhu2015;Jinetal.2016a)failtomeetthere- world and become a vital platform for politics. With thou-
quirementofthistaskforseveralreasons:1)Thesemethods sands of millions of tweets available, many existing stud-
need extensive labeled training data, which is expensive to iesfocusonsentimentanalysis(Wangetal.2012)andelec-
labelfortherumordetectionproblem;2)Theircomputation tionprediction(Tumasjanetal.2010)onTwitter.However,
efficiencyislowespeciallywhendealingwiththousandsof the openness and convenience of social media also fosters
millions of tweets; 3) It is difficult to tell what types of ru- a large amount of fake news and rumors which can spread
mors are posted as their binary results are not easily inter- wildly(Friggerietal.2014).Comparedwithexistingrumor
pretable. detectionworksthatarefocusedongeneralsocialeventsor
Considering these limitations, we propose to detect ru- emergencyevents(Jinetal.2014),thispaperpresentsafirst
morsasatextmatchingtaskinthispaper(Figure3).Inthis analysisofrumorsinapoliticalelection.
scheme,asetofverifiedrumorarticlesarecollectedasgold Most existing rumor detection algorithms follow the tra-
standardsamplesforreference.Eachtweetiscomparedwith ditionalsupervisedmachinelearningscheme.Featuresfrom
these verified rumors to see if they match closely. As text text content (Castillo, Mendoza, and Poblete 2011; Kwon
matchingisawell-studiedproblemininformationretrieval, et al. 2013), users (Morris et al. 2012), propagation pat-
many unsupervised algorithms, such as tf-idf word vector terns(Wu,Yang,andZhu2015)andmultimediacontent(Jin
model and word embedding, can be efficiently employed. et al. 2015; Jin et al. 2016b) are extracted to train a clas-
Moreover, the matching result can not only reveal whether sifier on labeled training data. Some recent works further
a tweet is rumor but also tell us which rumor the tweet is improvetheclassificationresultwithgraph-basedoptimiza-
relatedto.Thisapproachisbetterthantheexistingmachine tion methods (Gupta, Zhao, and Han 2012; Jin et al. 2014;
learning approaches in the aspects of efficiency and inter- Jin et al. 2016a). Although machine learning approaches
pretability. areveryeffectiveundersomecircumstances,theyalsohave
Weuseallthe1,723checkedrumorsfromSnopes.comas drawbacks.Thesupervisedlearningprocessrequiresalarge
theverifiedrumorarticles.Inordertofindthebestmatching amount of labeled training data which are expensive to ob-
algorithm,weconductacomparativestudyofseveralcom- tain for the rumor detection problem. They are often com-
peting algorithms. These algorithms are executed on a rea- putationally expensive, especially when dealing with thou-
sonably sized set of 5,000 manually labeled tweets to pro- sands of millions tweets. They derive features in a “black
videafairperformancecomparison. box”andtheclassificationresultsaredifficulttointerpret.
We then detect rumors with the selected most effective Toovercomethetimeefficiencyandinterpretabilityprob-
matchingalgorithmonover8millionoftweetsfrom14,000 lemsofsupervisedlearning,Zhao,Resnick,andMei(2015)
followersofthetwoprimarypresidentialcandidates:Hillary proposedalexicon-basedmethodfordetectingrumorsina
ClintonandDonaldTrump.Weinspecttherumordetection hugetweetstream.Theyextractedsomewordsandphrases,
resultsfromdifferentaspectstoanswerfollowingquestions: like “rumor”, “is it true”, “unconfirmed”, for matching ru-
which side posted the most rumors? who posted these ru- mor tweets. Their lexicon is relatively small, thus the de-
mors?whatrumorsdidtheyposted?whendidtheypostru- tection results tend to have high precision but low recall of
mors?Theseinsightshelpusunderstandtherumortweeting rumors.
behaviorsofdifferentgroupsoffollowersandcanbehelpful In this paper, we formulate the rumor detection as a text
for mining voters’ real intentions and accurately detecting matchingtask.Severalstate-of-the-artmatchingalgorithms
arecomparedforrumordetection.TF-IDF(Salton,Fox,and
Table1:Datasetusedforrumoranalysisduringelection.
Wu 1983) is the most commonly used method for comput-
#followers #tweets
ing documents similarity. BM25 algorithm (Robertson and
HillaryClinton 7,283 4,452,087
Zaragoza2009)canbeseenasanimprovedversionofTF-
DonaldTrump 7,339 4,279,050
IDF.Botharedocumentrelevancerankingmodelsbasedon
termmatching.Recentresearchindeeplearningfortextrep- All 14,622 8,731,137
resentationembedswordsordocumentsintoacommonvec-
torspace.Word2Vec(Mikolovetal.2013)andDoc2Vec(Le
andMikolov2014)aretwowidelyusedembeddingmodels as negative samples. In this set, not only is each tweet la-
atthewordandparagraphlevels,respectively.Inthispaper, beledasrumorornot,buttherumortweetsarealsolabeled
we give a comparative study of these methods to find the withtheircorrespondingverifiedrumorarticles.Therefore,
mostsuitablemodelforourrumordetectiontask. we can perform both general rumor classification and fine-
grainedrumoridentificationwiththisdataset.Thefollowing
Dataset isanexampleofaverifiedrumorarticleandthreeassociated
tweets.
Wecollectalarge-scaledatasetforanalyzingrumorsduring
Verifiedrumorarticle1:
the2016U.S.presidentialelectionfromTwitter.Forreliable
ShakyDiagnosis.Amontageofphotosandvideoclipsof
rumor detection, we obtain a set of verified rumor articles
DemocraticpresidentialcandidateHillaryClintonpurport-
fromSnopes.com.Wealsomanuallyconstructatestingset
edlydemonstratesshehassymptomsofParkinson’sdisease.
tofairlyevaluatetherumordetectionmethods.
Photosandvideoclipsnarratedbyamedicaldoctordemon-
TheamountoftweetsonTwitterisenormousandgrowing
stratethatDemocraticpresidentialcandidateHillaryClin-
every second. It is almost impossible and also unnecessary
ton likely has Parkinson’s disease. Hillary Clinton’s health
to process all the data. As we are interested in finding out
hasbeenthesubjectofintensespeculationduringthe2016
rumor spreading behaviors of different voter groups during
presidentialcampaign......
the 2016 U.S. presidential election, we design a data sam-
Associatedrumortweets:
plingschemetocollectthetweetsfromthefollowersofthe
twoprimarycandidates. 1. Hillary collapse at ground zero! game over, Clinton!
UsingtheTwitterAPI,wecollectalltheuserswhoarefol- Parkinson’sblackout!
lowing the Democratic presidential candidate Hillary Clin-
2. Wikileaks E-mails: Hillary looked into Parkinson’s drug
tonandtheRepublicpresidentialcandidateDonaldTrump.
aftersufferingfrom“decisionfatigue”.
Werandomlyselectabout10,000followersfromeachcan-
didate’sfollowerlist,whichcontainsmillionsoffollowers. 3. ExclusiveReport:Howtrueisthis??HillaryClintonhas
Parkinson’sdisease,doctorconfirms.
Weremoveuserswhofollowsbothcandidatestomakesure
eachuserfollowsonlyonespecificcandidateinourdataset.
RumorDetection
Userswhosetweetsarenotavailableduetotheirprivacyset-
tingarealsoexcluded.Wethencollectupto3,000mostre- WeformulaterumordetectiononTwitterasamatchingtask
centtweetsforeachuserusingtheTwitterAPI.Altogether, in this paper (Figure 3). Compared with detection methods
wehaveover8billiontweetsfrom14,000followersofthe based on supervised machine learning, our scheme is more
twocandidatesinourdataset(Table1). timeefficientandinterpretable,whichissuitableforanalyz-
We collect a set of verified rumor articles from ingrumorsfrommillionsoftweets.Withreliablerumorarti-
Snopes.com as gold standard samples for rumor matching. clescollectedfromSnopes.com,thekeypartofthisscheme
Snopes.comisaverypopularrumordebunkingwebsite.So- isthematchingalgorithm.Inthissection,wepresentaper-
cial media users can nominate any potential rumor to this formanceevaluationofseveralmatchingalgorithmstofind
site. The employed analysts then select some of these con- thebestmethodforrumordetectiontaskonTwitter.
troversialstatementstofact-checkthemasrumorsortruth. Compared with the traditional rumor classification algo-
Anarticleispresentedforeachcheckedrumorbythesepro- rithms,ourrumormatchingschemenotonlyoutputsatweet
fessionalanalysts,whichgivesconclusionoftherumorfol- as rumor or not but also identifies which rumor article it
lowedbyfulldescription,source,origin,suporting/opposing referstoifitisarumortweet.Weperformcomparativestud-
evidencesoftherumorstory.Wecollectthearticlesofallthe ies of different matching algorithms on both the classifica-
1,723 checked rumors on this website to form the verified tionandtheidentificationtaskofrumordetection.
rumorarticleset.
To quantitatively evaluate the performance of rumor de- RumorDetectionAlgorithms
tectionmethods,webuildamanuallylabeledtweetset.We
We compare the performance of five matching algorithms
randomly select 100 rumors from the verified rumor set.
with respect to the rumor classification task. The first set
For each verified rumor article, we search our large tweet
of methods includes two widely used term-based match-
set with keywords extracted from the article. Each tweet
ing methods: TF-IDF and BM25. The second set includes
in the search result is manually examined to check if it
two recent semantic embedding algorithms: Word2Vec and
matches the rumor article. After these procedures, we ob-
tainasetof2,500rumortweetsfrom86rumorarticles.We 1Thefullarticleisavailableat:http://www.snopes.com/hillary-
thenrandomlysamplesthesamenumberofunrelatedtweets clinton-has-parkinsons-disease/
Doc2Vec.Thethirdsetisalexicon-basedalgorithmforru-
mordetectiononTwitterstream.
TF-IDF (Salton, Fox, and Wu 1983) is a widely used
model in text matching. In this model, both the tweets
and the verified rumor articles are represented as a v-
dimensional vector, where v is the size of the dictionary
of the corpus. Each element in the vector stands for the
TF-IDF score of the corresponding word in the text. TF is
the term frequency. IDF score is the inverse document fre-
quency,whichiscalculatedonthewholecorpus.
BM25(RobertsonandZaragoza2009)isalsoatextsim-
ilaritycomputingalgorithmbasedonthebag-of-wordslan-
guage model. It is an improvement of the basic TF-IDF
model by normalizing on term frequency and document
length. Both TF-IDF and BM25 have been widely used in
manyrelatedstudies.
Figure4:Thecomparativeperformanceoffourmatchingal-
Word2Vec (Mikolov et al. 2013) represents each word
gorithms.
in a corpus with a real-valued vector in a common seman-
tic vector space. Compared with traditional lexical-based
matching models, this algorithm evaluates the quality of
word representations based on their semantic analogies.
ThevectorrepresentationofWord2Vecissuccessfullyused
in many text mining applications. We use the pre-trained
Word2Vecmodelonacorpusof27billiontweets.Theword
dimension is 200. To aggregate a presentation for a whole
text,wetaketheaverageofwordvectorsinthetext.
Doc2Vec (Le and Mikolov 2014) is also an embedding
algorithm on the semantic space, which can directly learn
thedistributedrepresentationsofdocuments.Weuseallthe
tweets and rumor articles for the unsupervised training of
the model after standard pre-processing. We use the de-
faultparametersettingsasin(LeandMikolov2014).After Figure5:Theaccuracyofrumoridentificationtask.
training,tweetsandverifiedrumorsarerepresentedas400-
dimensionalvectors.
ForWord2VecandDoc2Vec,thematchingscorebetween Word2Vec and Doc2Vec, respectively. The red triangle is
atweetandarumorarticleiscomputedbasedonthecosine the fixed result of lexicon matching. These results show
distanceoftheirvectorrepresentions. that BM25 reaches the best performance among all the
Lexicon matching (Zhao, Resnick, and Mei 2015) is a five rumor classification methods under different metrics.
lexicon-basedrumordetectionalgorithmsforefficientlyde- The two term-based methods (TF-IDF and BM25) outper-
tecting in huge tweet streams. It mines a couple of signal formsthesemantic-embeddingandlexicon-basedmethods.
words or phrases for recognizing prominent rumor tweets. For semantic-embedding, Word2Vec is slightly better than
We use the same set of regular expression patterns as in Doc2Vec. Lexicon matching can reach a rumor classifica-
(Zhao,Resnick,andMei2015)tomatchrumortweets. tionprecisionof0.862,butitsrecall(0.008)istoolow.
EvaluationonRumorClassificationTask EvaluationonRumorIdentificationTask
TF-IDF,BM25,Word2VecandDoc2Vecrepresentstextsas One extra advantage of our proposed rumor matching
numericvectors.Thesimilaritybetweenatweetandaveri- scheme is its ability to identify what rumor article a rumor
fiedrumoriscomputedastheirmatchingscore.Bysettinga tweetrefersto,apartfromclassifyingitasarumortweet.To
thresholdhforeachmethod,weclassifytweetswithmatch- compare the rumor identification performance of the four
ing scores larger than h as rumor tweets. We can achieve algorithms, we compute the similarity score between each
differentprecisionandrecallofrumorclassificationbyvary- pairoftweetandverifiedrumorarticleforthe2,500labeled
ingthethreshold.Wetestallthefourmethodsonthe5,000 rumor tweets and 1,723 verified rumor articles. If the most
labeled tweet set. Figure 4 illustrates the precision-recall similar rumor article of a tweet is exactly the same labeled
curvesofthesefouralgorithms.Thelexiconmatchingalgo- rumorarticleforit,thenthisisanaccuraterumoridentifica-
rithm detects rumors by keywords matching, thus its result tion.
isactuallyfixed(asasingelpointinFigure4). Fromtheoverallrumoridentificationaccuracyofeachru-
The highlighted round points on each curve in Fig- mormatchingmethodsinFigure5,wecanobservethatthe
ure 4 are points where the F1-measures are maximized, BM25 algorithm achieves the best accuracy of 0.799. The
at 0.758, 0.82, 0.764 and 0.745 for TF-IDF, BM25, accuracyofBM25isonlyslightlybetterthanthatofTF-IDF,
Therumortweetratioalltimeandintheelectionperiod
for Clinton and Trump are illustrated in Figure 6. We can
observefromthisfigurethat:
• Rumor tweets only consistute a small part on Twitter in
general. Our rumor classification algorithm has a preci-
sionof94.7%andarecallof31.5%onthetestset.Less
than1.2%tweetsaredetectedasrumorsinour8million
tweetset.Basedonthesefacts,wecanassumethatonly
about4%tweetsarerumortweetsonTwitter.
• All time, Clinton’s followers are a slightly more active
in posting rumor tweets than Trump’s followers. 1.2%
Figure6:Rumortweetratioalltimeandinelectionperiod.
tweetsarerumortweetsfromClinton’sfollowers,which
isabout4%morethanthatofTrump’sfollowers.
• Peopletendtopostmorerumortweetsintheelectiontime
although it has major advantage in the rumor classification
thaninthewholetime,especiallyforTrump’sfollowers.
task. This is probably because BM25 can distinguish non-
Comparingtheirelectionperiodandalltimerumortweet-
rumortweetsmuchbetterthanTF-IDF.Anotherinteresting
ing,Trump’sfollowershavearumortweetratioof1.35%
finding is that Doc2Vec actually performs better on the ru-
during the election, which is 18% higher than that in all
mor identification task than Word2Vec, although the latter
time.
has slightly better performance on the rumor classification
task. • Duringtheelectiontime,Trump’sfollowersaremoreac-
Rumor classification and rumor identification are corre- tiveinrumortweetingthanHillary’sfollowers.Asthefig-
lated but also different tasks. With experiments on the la- ure suggests, although Trump’s followers are a slightly
beledset,wefindthattheBM25algorithmachievesthebest lessactiveinrumortweetinginthewholetime,theybe-
performanceonbothtasksamongothers.Forthefollowing comemuchmoreinvolvedinpostingrumorsattheelec-
analysisonthewhole8milliontweetset,weuseBM25to tiontime,comparedwithClinton’sfollowers.
detectrumortweetsandmatchingthemwiththecorrespond-
ingrumorarticles.Toconductareliableandaccurateanal- Whopostedtheserumors?
ysis,wepreferahighprecisionforourrumordetectionre-
WhoarebehindtherumorsspreadingonTwitter?Weinves-
sult.Wesetthesimilaritythresholdh=30.5sothatwecan
tigate this issue by analyzing rumor tweets posted by indi-
achieveaveryhighrumorclassificationprecisionof94.7%
vidualfollowersofthetwocandidates.
andtherecallof31.5%.
To obtain an overall impression of rumor spreaders, we
rankusersbythetotalnumberofrumortweetstheyposted
AnalyzingRumorTweetsduringtheElection
(Figure 7). From this analysis, we find that the number of
Thispaperanalyzesrumortweetsduringthe2016U.S.pres- rumortweetspostedbyindividualfollowersofeachcandi-
identialelection.Forrumoranalysisatalargescale,inthis date follows a perfect power law distribution. This means
section,weusetheBM25rumordetectionalgorithmtode- that the majority of rumors are posted by only a few users
tectrumortweetsfromover8milliontweetscollectedfrom onTwitter.Tobeexact,thetop10%userspostedabout50%
the followers of Hillary Clinton and Donald Trump. Based rumortweetsandthetop20%userspostedabout70%ofall
on the results, we obtain insights into the rumor tweeting rumor tweets. This is a common phenomenon revealed by
behaviorsfromvariousaspects. bothTrump’sandClinton’sfollowers.Wecanconcludethat
rumorspreadingonTwitterismostlydominatedbyasmall
Whichsidepostedthemostrumors? groupofactiveusers.
Moreover,Figure7alsodemonstratestherumortweeting
Twitter has become an online battle field during the elec-
patternofeachcandidate’sfollowergroupfromtheuseras-
tion.Thenumberofrumortweetsreflectstheinvolvementof
pect.Forexample,wefindthattheClinton’sfollowershave
candidates’followersintheelectioncampaign.Whichside
alargerrumortweetratiothanTrump’sfollowersmostlybe-
offollowerswereinvolvedmostinspreadingrumortweets?
cause of the difference in their the top-1000 followers (the
Toanswerthisquestion,weusetheBM25rumorclassifica-
magnifiedpartinthefigure).
tion method to detect rumors in the subset of tweets of the
Aretheserumor-prolificfollowersjustactiveintweeting
twocandidates,respectively.Givenourfocusonrumorsdur-
rumorsoractiveingeneraltweetingaswell?Tounderstand
ingtheelectionperiod,wealsoanalyzerumortweetsposted
fromApril20152 uptothepresent.Becausethetotalnum- this, we calculate the ratio of rumor tweets in all tweets
posted by a user. In Figure 8, we show the rumor tweet ra-
beroftweetsforeachcandidate’sfollowergroupisslightly
tioofthetop1000usersofClinton’sfollowers.Weomitthe
different,wecalculatethepercentageofrumortweetsfora
figureofTrump’sfollowerstosavespace,whichhaveavery
faircomparison.
similardistribution.
2Clinton announced her candidacy on April 12, 2015 and In this figure, the x-axis is user ranked in the same or-
TrumpannouncedhisonJune16,2015. derasinFigure7andthey-axisistherumortweetratioof
Figure9:KeywordsinthetweetspostedbyoneofTrump’s
Figure7:Thenumberofrumortweetspostedbyindividual follower.
followersofClintonandTrump.Theplotonthetoprightis
amagnificationofthepartinthedottedbox.
Figure10:RumorspostedbyfollowersofTrumpandClin-
Figure8:RumortweetratioofClinton’sfollowers. ton.
Whatrumorsdidtheypost?
eachuser.Wefindthattherumortweetratioalsofollowsa
power law distribution except for some outliers. Followers Rumors cover various topics of interests. During the elec-
who post more rumor tweets also tend to have a larger ru- tion,mostrumorsarefocusedonthecandidates(Figure1).
mortweetratio.Thismeanstherumor-prolificusersdidnot People post different rumors to convey certain sentiments.
randomlypostanytweets;theywereactuallymoreconcen- By analyzing what people from different groups tweeted
tratedonpostingrumortweetsthantheuserswhooccasion- about in rumors, we can understand their intentions in this
allypostafewrumortweets.Infact,therumortweetratioof election.
thetop400usersisabove0.012,whichistheaveragerumor We use BM25 to identify the content of each rumor
ratioforClinton’sfollowers. tweet by matching it with the verified rumor articles from
Snopes.com.AsshowninFigure5,thismethodhasarumor
Case Study After analyzing rumor spreaders at a large identificationaccuracyof80%.Givenourfocusonthetwo
scale,wecanalsoconductadetailedanalysisforaspecific primary presidential candidates, Hillary Clinton and Don-
user. ald Trump, we only analyze rumor tweets related to them.
Take one of Trump’s followers for example. This user Identifiedrumortweetsaregroupedintotwosubsetsbased
posted3,211tweetsinourdataset,307ofwhicharedetected onwhichcandidatetheircorrespondingrumorarticlesrefers
asrumors.Therumortweetratioisashighas9.6%,which to.Afternormalizingthenumberofcandidate-relatedrumor
meansthisuserisveryactiveinrumortweeting.Byexamin- tweets with the total number of rumor articles for this can-
ingthetopkeywordsinalltweetspostedbytheuser(Figure didate in our dataset, we plot the rumor content spread by
9), we find this person is very focused on posting tweets Trump’sandClinton’sfollowersinFigure10.Weoffersome
about the 2016 presidential election: “Clinton”, “Sanders”, analysisofthisfigurebasedonthenormalizedrumortweet
“Trump”and“election”arethemostmentionedwordsinthe number.
tweets.Afterrumordetection,wefindthattherumortweets First, both follower groups post rumors about their fa-
ofthisuseraremainlyaboutClintonandSandersratherthan voredcandidateaswellastheopponentcandidate.Rumors
Trump: 15% tweets about Clinton and 28% tweets about areoftenverycontroversialstories.Theyarenormallyneg-
Sanders are rumor tweets, while only 10% tweets about ative towards one candidate during the election (Figure 2).
Trump are rumors. Given these observations, this user is Supportersofonecandidatewouldspreadrumorsaboutthe
probablyasolidsupporterofTrumpandtheRepublicparty opponentasanegativecampaigntacticanddebunkrumors
duringthiselection. about their favored candidate. For example, we show two
Table2:Toprankedrumorarticles.
ToprankedrumorspostedbyTrump’sfollowers
T1 BillO’ReillyandMegynKellyareClintonoper-
atives.
T2 HillaryClintonrefusedtocongratulateAmerican
goldmedalistGinnyThrasher.
T3 DonaldTrump:“lazinessisatraitinblacks”
T4 MotherTeresaandHillaryClinton’stalkonabor-
tion.
T5 Donald Trump opposed the invasion of Iraq be-
foreittookplacein2003.
ToprankedrumorspostedbyClinton’sfollowers
(a) RumortweettimelineofClinton’sfollowers
C1 DonaldTrumpwasnominatedforaNobelPeace
Prize.
C2 BillO’ReillyandMegynKellyareClintonoper-
atives.
C3 HillaryClintonrefusedtocongratulateAmerican
goldmedalistGinnyThrasher.
C4 DonaldTrumpdeniedsayingthatnotpayingfed-
eraltaxesmadehimsmart.
C5 HillaryClintondidn’tpubliclysupportsame-sex
marriageuntil2013.
tweetsabouttherumor“HillaryClintonhasParkinson’sdis-
ease”fromourdataset:
Tweet 1: Medical experts watching debate said Hillary (b) RumortweettimelineofTrump’sfollowers
showed“TelltaleSigns”ofParkinson’sDisease.
Tweet 2: “I know her physician; I know some of her Figure11:Rumortweettimeline.
healthhistorywhichisreallynotsogood”Trump’sMDon
Hillary—herMDsharedherinfowithhim?
The first tweet comes from a follower of Trump. It is though some issues are more important in the election,
spreading the rumor by quoting medical experts. The sec- suchaseconomyandterrorism,fewrumorsarepostedon
ondtweetcomesfromafollowerofClinton.Itisobviously them.
questioningthetruthfulnessoftherumor.
• Basically,Trump’sfollowerstweetmoreonrumorsabout
Second, users would post more rumor tweets about the
Clintonandlikewise.Apartfromsomesharedrumors,the
opponent candidate than their favored candidate. Clinton’s
two follower groups have very different focuses on ru-
followerspost8%morerumortweetsaboutTrumpthanru-
mors.TakingtheabortionrumorT4asanexample,there
morsaboutClinton.Trump’sfollowerspost5%morerumor
are210tweetsaboutitfromTrump’sfollowersandonly
tweets about Clinton than rumors about Trump. Moreover,
60tweetsfromClintonfollowers.
Trump’s followers are more active in this rumor tweeting
behaviortowardsbothClintonandTrump.Thenumbersof
Whendidtheyposttheserumors?
rumor tweets about the two candidates posted by Trump’s
followersarebothlargerthanthoseofClinton’sfollowers. Timingiseverythinginpolitics.Analyzingthetimepatterns
of rumor tweeting can reveal insights of online campaign.
Case Study To seewhat exactlyis talkedabout inrumor
After counting the number of rumor tweets posted in each
tweetsfromthetwofollowergroups,werankrumorarticles
day, we plot the rumor tweeting of Clinton’s and Trumps
basedonthenumberofmatchedtweets.Thetoprankedru-
followersoversixmonths(April2016toSeptember2016).
mor articles about the two candidates are listed in Table 2.
AsFigure11shows:1)Rumorsintheearlierthreemonths
Wecandrawsomeconclusionsfromthistable:
were relative smooth. As the election day approached, ru-
• These top ranked rumors are mostly about hot voting is- mor tweeting became more and more intense; 2) The two
suesinthiselection:racialminorities(rumorT2,T3and followergroupshaddifferenttimepatternsofrumortweet-
C3),massmedia’sobjectivity(T1andC2),abortion(T4), ing:thefluctuationandpeaksweredifferent.
LGBT (C5), military policy (T5) and candidate’s quali- We annotate the key events for some rumor peaks in the
fication (C1 and C5). Some rumors are so popular that figuretounderstandtheinherentreasonbehindthem.From
they are ranked high in two follower groups (T1-C2 and SeptembertoApril,thefivemarkedeventsinClinton’stime-
T2-C3).Obviously,spreadingrumorsonthesehotissues line are: 1) the first presidential debate, 2) Clinton reap-
would influence people’s judgement on a candidate. Al- peared after diagnosed with pneumonia, 3) Clinton called
theTrump’ssupportersas“basketofdeplorables”,4)Clin- [Jinetal.2014] Jin, Z.; Cao, J.; Jiang, Y.-G.; and Zhang, Y.
ton was made the official party nominee, 5) the Orlando 2014. News credibility evaluation on microblog with a hi-
shooting. And the five marked peaks in Trump’s timeline erarchical propagation model. In 2014 IEEE International
correspondto:1)thefirstpresidentialdebate,2)Trumpde- ConferenceonDataMining(ICDM),230–239. IEEE.
clared he was opposing the Iraq war before it started, 3) [Jinetal.2015] Jin,Z.;Cao,J.;Zhang,Y.;andYongdong,Z.
MikePencewasannouncedasthevicepresidentcandidate, 2015. Mcg-ictatmediaeval2015:Verifyingmultimediause
4) Trump condemned the killing of five Dallas police offi- withatwo-levelclassificationmodel. InProceedingsofthe
cers,5)theOrlandoshooting. MediaEval2015MultimediaBenchmarkWorkshop.
We find that rumors peaked in three types of occasions:
[Jinetal.2016a] Jin, Z.; Cao, J.; Zhang, Y.; and Luo, J.
1) key point in the presidential campaign, such as “the
2016a. News verification by exploiting conflicting social
presidential debate” and “official nominee”; 2) controver-
viewpoints in microblogs. In Proceedings of the Thirtieth
sialemergencyevents,including“theOrlandoshooting”and
AAAIConferenceonArtificialIntelligence,February12-17,
“the Dallas shooting”; 3) events triggering rumors, such as
2016,Phoenix,Arizona,USA.
“reappearing after pneumonia” and “opposing Iraq war”.
[Jinetal.2016b] Jin, Z.; Cao, J.; Zhang, Y.; Zhou, J.; and
Thisinsightremindsustopaymoreattentiontorumorsdur-
Tian, Q. 2016b. Novel visual and statistical image fea-
ingthesetypesofeventsinthefuturepoliticalcampaigns.
tures for microblogs news verification. IEEE Transactions
onMultimediainpublish.
Conclusions
[Kwonetal.2013] Kwon, S.; Cha, M.; Jung, K.; Chen, W.;
This paper studies the rumors spreading phenomenon on
andWang,Y. 2013. Prominentfeaturesofrumorpropaga-
Twitter during the 2016 U.S presidential election. We pro-
tioninonlinesocialmedia. InDataMining(ICDM),2013
pose a reliable and interpretable approach to detecting ru-
IEEE13thInternationalConferenceon,1103–1108. IEEE.
mor tweets by matching them with verified rumor articles.
[LeandMikolov2014] Le, Q. V., and Mikolov, T. 2014.
After a comparative study, we find the BM25 matching al-
Distributedrepresentationsofsentencesanddocuments. In
gorithmoutperformsotherfourcompetingrumormatching
ICML,volume14,1188–1196.
methods in this task. With a rumor detection precision of
94.7%, we use this method to detect rumors in over eight [Mikolovetal.2013] Mikolov, T.; Sutskever, I.; Chen, K.;
million tweets collected from the followers of the two pri- Corrado, G. S.; and Dean, J. 2013. Distributed represen-
mary presidential candidates. We provide a thorough anal- tationsofwordsandphrasesandtheircompositionality. In
ysis on the detected rumor tweets from the aspects of peo- Advances in neural information processing systems, 3111–
ple,contentandtime.Manyinterestingrumortweetingpat- 3119.
terns are discovered, including: 1) more rumors are posted [Morrisetal.2012] Morris,M.R.;Counts,S.;Roseway,A.;
at election time than on the average; 2) rumor tweeting is Hoff,A.;andSchwarz,J. 2012. Tweetingisbelieving?:un-
dominatedbyasmallgroupofusers;3)userspostrumorre- derstanding microblog credibility perceptions. In Proceed-
latedtweetstodebunkrumorsabouttheircandidateorslan- ings of the ACM 2012 conference on Computer Supported
der the opponent; 4) rumor tweeting erupts mainly in three CooperativeWork,441–450. ACM.
typesofoccasions(keypointsinthepresidentialcampaign,
[RobertsonandZaragoza2009] Robertson, S., and
upon controversialemergency events,upon unique events), Zaragoza,H. 2009. Theprobabilisticrelevanceframework:
etc. These insights would help us better understand rumors BM25andbeyond. NowPublishersInc.
duringpoliticaleventsandinspireustobuildmoreeffective
[Salton,Fox,andWu1983] Salton,G.;Fox,E.A.;andWu,
rumordetectionalgorithmsinthefuture.
H. 1983. Extendedbooleaninformationretrieval. Commu-
nicationsoftheACM26(11):1022–1036.
References
[Tumasjanetal.2010] Tumasjan,A.;Sprenger,T.O.;Sand-
[Castillo,Mendoza,andPoblete2011] Castillo, C.; Men- ner, P. G.; and Welpe, I. M. 2010. Predicting elections
doza,M.;andPoblete,B. 2011. Informationcredibilityon withtwitter:What140charactersrevealaboutpoliticalsen-
twitter. InProceedingsofthe20thinternationalconference timent. ICWSM10:178–185.
onWorldWideWeb(WWW),675–684. ACM.
[Wangetal.2012] Wang,H.;Can,D.;Kazemzadeh,A.;Bar,
[DiFonzoandBordia2007] DiFonzo, N., and Bordia, P. F.;andNarayanan,S. 2012. Asystemforreal-timetwitter
2007. Rumor psychology: Social and organizational ap- sentimentanalysisof2012uspresidentialelectioncycle. In
proaches. AmericanPsychologicalAssociation. ProceedingsoftheACL2012SystemDemonstrations,115–
[Friggerietal.2014] Friggeri,A.;Adamic,L.A.;Eckles,D.; 120. AssociationforComputationalLinguistics.
and Cheng, J. 2014. Rumor cascades. In Proceedings of [Wu,Yang,andZhu2015] Wu,K.;Yang,S.;andZhu,K.Q.
the Eighth International AAAI Conference on Weblogs and 2015. Falserumorsdetectiononsinaweibobypropagation
SocialMedia. structures. InIEEEInternationalConferenceonDataEngi-
[Gupta,Zhao,andHan2012] Gupta,M.;Zhao,P.;andHan, neering,ICDE.
J. 2012. Evaluatingeventcredibilityontwitter. InProceed- [Zhao,Resnick,andMei2015] Zhao, Z.; Resnick, P.; and
ingsoftheSIAMInternationalConferenceonDataMining, Mei,Q. 2015. Enquiringminds:Earlydetectionofrumors
153. SocietyforIndustrialandAppliedMathematics. in social media from enquiry posts. In Proceedings of the
24th International Conference on World Wide Web, 1395–
1405. InternationalWorldWideWebConferencesSteering
Committee.