Optimal Real-Time Bidding for Display Advertising

189 Pages·2016·11.21 MB·English

by  Weinan Zhang

Optimal Real-Time Bidding for
Display Advertising
Weinan Zhang
Adissertationsubmittedinpartialfulfillment
oftherequirementsforthedegreeof
DoctorofPhilosophy
of
UniversityCollegeLondon.
DepartmentofComputerScience
UniversityCollegeLondon
June6,2016
2
3
I,WeinanZhang,confirmthattheworkpresentedinthisthesisismyown. Where
informationhasbeenderivedfromothersources,Iconfirmthatthishasbeenindicated
inthework.

Abstract
Real-Time Bidding (RTB) is revolutionising display advertising by facilitating a real-
time auction for each ad impression. As they are able to use impression-level data,
such as user cookies and context information, advertisers can adaptively bid for each
adimpression. Therefore,itisimportantthatanadvertiserdesignsaneffectivebidding
strategy which can be abstracted as a function - mapping from the information of a
specific ad impression to the bid price. Exactly how this bidding function should be
designed is a non-trivial problem. It is a problem which involves multiple factors,
such as the campaign-specific key performance indicator (KPI), the campaign lifetime
auctionvolumeandthebudget.
Thisthesisisfocusedonthedesignofautomaticsolutionstothisproblemofcreat-
ingoptimisedbiddingstrategiesforRTBauctions: strategieswhichareoptimal,thatis,
fromtheperspectiveofanadvertiseragent-tomaximisethecampaign’sKPIinrelation
totheconstraintsoftheauctionvolumeandthebudget. Theproblemismathematically
formulatedasafunctionaloptimisationframeworkwheretheoptimalbiddingfunction
can be derived without any functional form restriction. Beyond single-campaign bid
optimisation,theproposedframeworkcanbeextendedtomulti-campaigncases,where
a portfolio-optimisation solution of auction volume reallocation is performed to max-
imise the overall profit with a controlled risk. On the model learning side, an unbiased
learning scheme is proposed to address the data bias problem resulting from the ad
auction selection, where we derive a “bid-aware” gradient descent algorithm to train
unbiased models. Moreover, the robustness of achieving the expected KPIs in a dy-
namicRTBmarketissolvedwithafeedbackcontrolmechanismforbidadjustment.
To support the theoretic derivations, extensive experiments are carried out based
on large-scale real-world data. The proposed solutions have been deployed in three
commercial RTB systems in China and the United States. The online A/B tests have
6 Abstract
demonstratedsubstantialimprovementoftheproposedsolutionsoverstrongbaselines.
Acknowledgements
First,IwouldliketoexpressmysinceregratitudetomyfirstsupervisorProf. JunWang.
I got to know Jun in Mar. 2012 when I was reading one of his papers on personalised
recommendation techniques published in RecSys 2010. After a careful survey about
hisresearchoncomputationaladvertisingandrecommendersystems,Idecidedtoonly
applyhisresearchgroupinUCLforpursuingmyPh.D.Fortunately,Igottheofferand
thescholarshipfromJunandstartedmyPh.D.careerfromSep. 2012.
During these 3.5 years of Ph.D. research work with Jun, I have been always feel-
ing lucky that I have made the right choice. Jun is a nice and easy-going professor,
just like a senior friend of mine. I totally have no pressure but only enjoyment when
discussing research problems with him. Jun usually emphasises the novelty of a re-
search topic, including the research problem novelty and methodology contributions.
He dedicates quite a lot of time to discuss with Ph.D. students and helps revise their
papers very carefully. In addition, he encourages me to pursue research internships in
large companies like Google and Microsoft Research as well as technology startups
like MediaGamma. Without his invaluable and generous support, I do not believe I
couldcomplete4internshipsandpublish16researchpapersduring3.5years.
Luckily, I have got the offer and am going to join Shanghai Jiao Tong University
as an assistant professor in Aug. 2016, working on big data mining and artificial in-
telligence research. I believe Jun’s philosophy of being a good Ph.D. supervisor will
deeplyhelpmeinmyfutureacademiccareer.
Second,Iwanttoshowmygreatrespectandappreciationtomysecondsupervisor
Prof. Stephen Robertson. When he agreed to be my second supervisor, Steve was
abouttoretireandplannedtofullychangetheshapeofhislife. IsincerelythankSteve
for travelling to London to provide me both high-level research advices and specific
solutiontipsduringourmeetingsandmyfirst-yearandtransfervivas.
8 Acknowledgements
Third, I would like to thank my Ph.D. viva examiners Prof. Sebastian Riedel
and Prof. Maria Polukarov for taking time to review my thesis and offer constructive
commentsandsuggestions.
Fourth, I am grateful to the industrial collaborators Dr. Xuehua Shen and Hairen
LiaofromiPinYou,JieLiu,TianchiZhu,LeiGongfromBigTree,JianXuandDr. Quan
LufromYahoo!,YifeiRongfromYOYI,TianxiongZhoufromTukMob. Withouttheir
technique support, none of my research solutions could be deployed and tested on any
onlinecommercialplatform.
Fifth, I would like to thank my internship supervisors and colleagues Dr. Ulrich
Paquet,Dr. KatjaHofmann,Prof. ThoreGraepelfromMicrosoftResearchCambridge,
Jie Yang, Dr. Amr Ahmed, Dr. Vanja Josifovski, Prof. Alex Smola, Dr. Andrei
BroderfromGoogle,andtheCEORaelClinefromMediaGammafortheirpatientand
invaluablehelpinguidingmetowardsmoresuccessfulandusefulresearch.
Sixth, I would like to thank my UCL research colleagues Prof. Emine Yilmaz,
Prof. Shi Zhou, Prof. Ingemar Cox, Prof. Antonis Bikakis, Dr. Xiaoxue Zhao, Dr. Jie
Xiong, Dr. Shuai Yuan, Dr. Bowei Chen, Dr. Ye Pan, Dr. Marc Sloan, Dr. Jagadeesh
Gorla, Dr. Kleanthis Malialis, Dr. Thomas Stone, Dr. Tamas Jambor, Dr. Shangsong
Liang, Rui Yu, Manisha Verma, Rishabh Mehrotra, Savvas Karagiannidis, Jie Li, Bin
Zou,RuiLuoandYixinWufortheirkindhelpandmanyinsightfuldiscussions.
Moreover,Iwillnotforgetthepatienthelpfrommyhomeuniversityandindustrial
supervisors before my Ph.D. study. They are Prof. Yong Yu and Prof. Gui-Rong Xue,
Prof. XiaofanWangfromShanghaiJiaoTongUniversity,Dr. BinGao,Dr. Tie-YanLiu
and Dr. Feng Zhao from Microsoft Research Asia. I sincerely thank them for opening
thedooroftheresearchworldforme.
Lastbutnotleast,Iamverygratefultomyparentsandmygirlfriendfortheirlove
and unconditional support during these years. Without them, I cannot imagine how I
couldsurviveinmyPh.D.grind.
Contents
ListofFigures 17
ListofTables 20
1 Introduction 21
1.1 BackgroundofInternetAdvertisingMarket . . . . . . . . . . . . . . . 21
1.2 Real-TimeBiddingbasedDisplayAdvertising . . . . . . . . . . . . . . 24
1.3 ResearchProblemsandContributions . . . . . . . . . . . . . . . . . . 26
1.3.1 Single-CampaignBidding . . . . . . . . . . . . . . . . . . . . 27
1.3.2 Multi-CampaignBidding . . . . . . . . . . . . . . . . . . . . . 28
1.3.3 UnbiasedLearningandOptimisationonCensoredData . . . . . 29
1.3.4 FeedbackControlforHandlingDataDynamics . . . . . . . . . 30
1.3.5 SummarisedContributions . . . . . . . . . . . . . . . . . . . . 31
1.4 ThesisStructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.5 SupportingPublications . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2 RelatedWork 35
2.1 DisplayAdvertising . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.2 BidOptimisationforSponsoredSearch . . . . . . . . . . . . . . . . . 37
2.3 BidOptimisationforRTBDisplayAdvertising . . . . . . . . . . . . . 38
2.4 RiskManagementModelsandApplications . . . . . . . . . . . . . . . 40
2.4.1 ModernPortfolioTheory . . . . . . . . . . . . . . . . . . . . . 40
2.4.2 StatisticalArbitrage . . . . . . . . . . . . . . . . . . . . . . . 42
2.5 LearningonBiasedorCensoredData . . . . . . . . . . . . . . . . . . 43
2.6 FeedbackControlTheoryandApplications . . . . . . . . . . . . . . . 45
10 Contents
3 Single-CampaignOptimalReal-TimeBidding 47
3.1 BackgroundandMotivation . . . . . . . . . . . . . . . . . . . . . . . 47
3.2 ProblemDefinition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.3 OptimalSolutionsofBiddingFunctions . . . . . . . . . . . . . . . . . 55
3.3.1 Winning&BiddingFunction1 . . . . . . . . . . . . . . . . . 55
3.3.2 Winning&BiddingFunction2 . . . . . . . . . . . . . . . . . 56
3.3.3 DiscussionsonDerivedBiddingFunctions . . . . . . . . . . . 58
3.3.4 OptimalSolutionofλ . . . . . . . . . . . . . . . . . . . . . . 58
3.3.5 SpecialCaseDiscussion: LinearWinningFunction . . . . . . . 60
3.4 ExperimentalSetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.4.1 DatasetandAnalysis . . . . . . . . . . . . . . . . . . . . . . . 61
3.4.2 EvaluationMeasures . . . . . . . . . . . . . . . . . . . . . . . 63
3.4.3 KPIEstimatorTraining . . . . . . . . . . . . . . . . . . . . . . 64
3.4.4 TestEvaluationSetup . . . . . . . . . . . . . . . . . . . . . . . 65
3.4.5 ComparedBiddingStrategies . . . . . . . . . . . . . . . . . . 66
3.5 OfflineEmpiricalStudy . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.5.1 PerformanceComparison . . . . . . . . . . . . . . . . . . . . . 68
3.5.2 TheImpactofBudgetConstraints . . . . . . . . . . . . . . . . 69
3.5.3 Clicksvs. Impressions . . . . . . . . . . . . . . . . . . . . . . 70
3.5.4 ParameterTuning . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.5.5 ResultsforanAlternativeKPI . . . . . . . . . . . . . . . . . . 72
3.6 OnlineA/BTest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.8 ChapterAppendix: GameTheoreticAnalysis . . . . . . . . . . . . . . 76
3.8.1 ProblemSettings . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.8.2 EquilibriumBiddingFunctionintheFirstPriceAuctions . . . . 78
3.8.3 EquilibriumBiddingFunctionintheSecondPriceAuctions . . 80
3.8.4 Discussion: TragedyoftheCommonsinRTB . . . . . . . . . . 84
4 Multi-CampaignStatisticalArbitrageMining 87
4.1 BackgroundandMotivations . . . . . . . . . . . . . . . . . . . . . . . 87
4.2 ProblemDefinition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90