Targeting Uplift: An Introduction to Net Scores

373 Pages·2019·11.128 MB·English

by  René Michel, Igor Schnakenburg, Tobias von Martens

René Michel · Igor Schnakenburg 
Tobias von Martens
Targeting
Uplift
An Introduction to Net Scores
Targeting Uplift
René Michel (cid:129) Igor Schnakenburg (cid:129)
Tobias von Martens
Targeting Uplift
An Introduction to Net Scores
123
RenéMichel IgorSchnakenburg
DeutscheBankAG DeTeConInternationalGmbH
FrankfurtamMain,Germany Berlin,Germany
TobiasvonMartens
DeutscheBankAG
FrankfurtamMain,Germany
ISBN978-3-030-22624-4 ISBN978-3-030-22625-1 (eBook)
https://doi.org/10.1007/978-3-030-22625-1
MathematicsSubjectClassification(2010):62F03,62-02,62-07,62P20,90B60,91G70
©SpringerNatureSwitzerlandAG2019
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG.
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Uplift modeling is a data mining approach to a specific challenge that occurs in
variousareasofapplication:predictingtheimpactofatreatmentandselectingthose
items (such as customers, patients, or machines) on whom the predicted impact
is(most)beneficial.Consequently,upliftmodelingmaximizesthe effectivenessof
treatments.
The topic has received considerable attention by researchers and practitioners.
In 2015, a Gartner study (see [1]) mentioned it as one of the upcoming hot
topics of advanced analytics with much potential ahead. Currently, most of the
relevantarticlesandwhitepapersarefocusedonmethodsandapplicationsofuplift
modeling in direct marketing, churn prevention, and medicine. This book aims
at covering all relevant aspects of uplift modeling from a general point of view,
suchasdatapreparation,modelingmethods,assessmentofupliftmodels,software
implementations,andpotentialspecificareasofapplication.
The audience targeted by this book comprises data scientists, especially
researches and practitioners in predictive modeling and scoring, mainly (but not
exclusively)inthemarketingcontext.DataScienceisarelativelynewconcept,and
there is, along with the hypeword Big Data, no uniquedefinition. Data scientists
can be regarded as analysts from various areas, such as statistics, mathematics,
computerscience,physics,oreconomics,whoareconcernedwiththeextractionof
relevantinformationoutofdatabyanalyticalmethods,includingthedeploymentof
theresultstosolveacertain(business)problem.
Therefore, the way this book is presented is adjusted to suit that audience;
it is not a classical mathematical book with definitions, theorems, and proofs
throughoutits text. Definitions and introductionsof important notationsare made
inamathematicalfashion,though,andmostresultsarepresentedinmathematical
notation. However, not always a proof is given, and sometimes, mathematical
exactnessissacrificedinordertopresentresultsintuitivelyandeasiertoread.Some
importantresults,however,willbeprovidedwithproofs.Readersnotinterestedin
themathematicalbackgroundmayskiptheseproofswithoutlosingthestorylineof
thebook.Thereisaclearfocusontheapplicationoftheresultsandpracticalissues
v
vi Preface
ofupliftmodeling.Consequently,therespectivepartsofthebookaredescribedina
suitablefashionwithoutformulae,legiblefornon-mathematicians,too.
Netscoringisatopicwithmanyfacets,andthisbooktriestocoverallofthem.
Naturally, the net scoring methods themselves are a starting point. However, also
methods to judge the quality of models are presented as well as useful steps in
the data preparation. Many intricate questions, like required sizes of datasets, are
answered with the help of simulations. In order to supportthis, a suitable way of
generatingdataascloseaspossibletoreal-worlddataisshown.Currentlyavailable
software implementations are presented and compared to each other. In the end,
applicationsofnetscoringtomanyfieldsaresuggestedwithaspecialemphasison
applicationsindirectmarketing.
In order to follow the text throughout the book, the reader should be familiar
with the basics of data analytics. Especially the knowledge of scoring methods
is beneficial. The introduction to classical scoring methods presented in the book
mainly serves to clarify notation and to preparethe introductionof the new uplift
scoring methods. A sound knowledge of basic statistics like hypothesis testing
and the general statistical way of thinking will be useful. However, no specific
knowledge of certain application areas, like direct marketing or medicine, is
required,sinceallapplicationexamplesareexplainedaccordingly.
Wewanttothank
(cid:129) Springer Verlag for publishing this book, especially Veronika Rosteck for the
fruitfulcooperation
(cid:129) Michael Falk and the Lehrstuhlfür Statistik of the University of Würzburg for
providingthetechnicalplatformonwhichwedevelopedthebook
(cid:129) BurchardHillmann-KösterandRob SimpsonofPitneyBowesfortheelaborate
supportandguidanceregardingtheSpectrumMinersoftware
FrankfurtamMain,Germany RenéMichel
Berlin,Germany IgorSchnakenburg
FrankfurtamMain,Germany TobiasvonMartens
February2019
Reference
1. A. Linden. Hype cycle for advanced analytics and data science. 2015. Gartner
Study-ID:G00277762.
Contents
1 Introduction................................................................. 1
1.1 ProblemStatement.................................................... 1
1.2 State-of-the-Art ....................................................... 4
1.3 StructureoftheBook ................................................. 5
References.................................................................... 6
2 TheTraditionalApproach:GrossScoring............................... 7
2.1 ProblemStatement.................................................... 7
2.2 Methods ............................................................... 11
2.2.1 DecisionTrees................................................ 14
2.2.2 Regression.................................................... 17
2.2.3 NeuralNetworks ............................................. 20
2.2.4 NearestNeighbor............................................. 23
2.2.5 BayesianClassifiers.......................................... 24
2.3 Assessment............................................................ 25
2.3.1 CriteriaofModelQuality.................................... 25
2.3.2 MisclassificationandProfit/Loss............................ 28
2.3.3 ResponseandCaptured(Response)......................... 31
2.3.4 GainandLift ................................................. 35
2.3.5 ReceiverOperatingCharacteristic........................... 38
2.3.6 GiniCoefficient .............................................. 41
2.3.7 AverageSquaredError....................................... 41
2.3.8 ModelStability............................................... 42
References.................................................................... 43
3 BasicNetScoringMethods:TheUpliftApproach...................... 45
3.1 ProblemStatement.................................................... 45
3.2 Methods ............................................................... 48
3.2.1 TwoIndependentModels.................................... 49
3.2.2 TwoDependentModels...................................... 52
vii
viii Contents
3.2.3 EqualTargetandControlGroupSizes...................... 57
3.2.4 OneCommonModel......................................... 62
3.2.5 BayesianClassifiers.......................................... 85
3.3 Semi-continuousandContinuousReactionTargets................. 89
3.3.1 PossibleModificationsofModelAssumptions............. 89
3.3.2 AMethodfortheSemi-continuousCase ................... 90
3.3.3 MethodsforthePureContinuousCase ..................... 92
3.4 MoreThanBinaryTreatmentVariables.............................. 95
3.5 SummaryofNetScoringMethods................................... 97
References.................................................................... 98
4 Validation of Net Models: Measuring Stability
andDiscriminatoryPower................................................. 101
4.1 NotationforModelValidation........................................ 102
4.2 ModelStability........................................................ 103
4.2.1 AverageSquaredDeviation.................................. 104
4.2.2 ASignificance-BasedMeasure.............................. 105
4.2.3 ModelStabilityRankCorrelation........................... 105
4.3 DiscriminatoryPower................................................. 106
4.3.1 Qini ........................................................... 106
4.3.2 AUnROC ..................................................... 111
4.3.3 ASignificance-BasedMeasure.............................. 117
4.4 ModelValidationandAdjustmentOverTime....................... 117
References.................................................................... 119
5 SupplementaryMethodsforVariableTransformation
andSelection................................................................ 121
5.1 VariableTransformation.............................................. 121
5.1.1 BinningMethods............................................. 122
5.1.2 TransformationofCategoricaltoNumericalVariables .... 125
5.2 VariablePreselection.................................................. 126
5.2.1 SummaryofPreselectionMethods.......................... 126
5.2.2 EnhancementofPreselectionbyCrossValidation ......... 132
5.2.3 Comparison of the Preselection Methods
onaReal-WorldDataset..................................... 132
References.................................................................... 135
6 A SimulationFrameworkfor the Validationof Research
HypothesesonNetScoring ................................................ 137
6.1 Multi-StepApproachtotheGenerationofSimulatedData......... 137
6.2 GatheringofHistoricalReal-WorldData............................ 138
6.3 ProductionoftheHypotheticalData................................. 139
6.4 DatasetSelectionforSimulationStudies ............................ 145
References.................................................................... 146
Contents ix
7 SoftwareImplementations................................................. 147
7.1 SASBase/SASFoundationorSASEnterpriseGuide................ 148
7.1.1 GeneralComments........................................... 148
7.1.2 ModelPreparation............................................ 149
7.1.3 ModelBuildingandScoring................................. 150
7.1.4 EvaluationandModelComparison ......................... 150
7.1.5 DeploymentandConclusion................................. 150
7.2 SASEnterpriseMiner................................................. 151
7.2.1 GeneralComments........................................... 151
7.2.2 ModelPreparation............................................ 152
7.2.3 ModelBuildingandScoring................................. 154
7.2.4 EvaluationandModelComparison ......................... 154
7.2.5 Deployment................................................... 155
7.2.6 Conclusion.................................................... 155
7.3 SpectrumMiner(PitneyBowes)...................................... 156
7.3.1 GeneralComments........................................... 156
7.3.2 ModelPreparation............................................ 156
7.3.3 ModelBuildingandScoring................................. 157
7.3.4 EvaluationandModelComparison ......................... 158
7.3.5 Deployment................................................... 159
7.3.6 Conclusion.................................................... 159
7.4 R&RStudio .......................................................... 159
7.4.1 GeneralComments........................................... 159
7.4.2 ModelPreparation............................................ 161
7.4.3 ModelBuildingandScoring................................. 162
7.4.4 EvaluationandModelComparison ......................... 163
7.4.5 Deployment................................................... 163
7.4.6 Conclusion.................................................... 164
7.5 Dymatrix’sNodesinKNIME......................................... 164
7.5.1 GeneralComments........................................... 164
7.5.2 ModelPreparation............................................ 165
7.5.3 ModelBuildingandScoring................................. 165
7.5.4 EvaluationandModelComparison ......................... 167
7.5.5 Deployment................................................... 168
7.5.6 Conclusion.................................................... 168
7.6 FurtherSoftwareTargetingUplift.................................... 168
7.7 ModelComparisonBetweenDifferentSoftwareSolutions
ontheSameData...................................................... 169
7.7.1 Two Plots to Evaluate the Performanceof the
SoftwareImplementations................................... 171
7.7.2 PlotsoftheResults........................................... 174
7.7.3 SummaryonModelPerformance,Features,
andUsability ................................................. 186
References.................................................................... 189
x Contents
8 DataPrerequisites.......................................................... 191
8.1 DataRequirements.................................................... 191
8.2 DevelopmentofaSuitableDatabase................................. 195
8.2.1 GeneralApproachestotheProvisionofHistoricalData... 195
8.2.2 AggregationofHistoricalData.............................. 196
8.2.3 RetrievingDatafromHistoricalTreatments................ 197
8.2.4 SuitableTargetandControlGroupSizes ................... 203
8.2.5 AdjustmentofAvailableData ............................... 218
References.................................................................... 239
9 PracticalIssuesandBusinessCases...................................... 241
9.1 ApplicationinDirectMarketing ..................................... 241
9.1.1 IdentificationofCustomerTypes............................ 242
9.1.2 EconomicImpactofCustomerSelectionsBased
onEitherGrossorNetScoring.............................. 256
9.1.3 Combinationof Gross and UpliftApproaches
forCustomerSelection....................................... 263
9.1.4 EconomicImpactofHybridCustomerSelections.......... 267
9.1.5 SensitivityAnalysesofKeyPerformanceIndicators....... 279
9.1.6 ImplicationsforPracticalApplications..................... 294
9.1.7 GeneralizationofCustomerTypesandResulting
SelectionApproaches........................................ 296
9.1.8 EnhancementofCampaignReporting ...................... 299
9.2 ApplicationofNetScoringinOtherAreas.......................... 301
9.2.1 RiskManagement............................................ 302
9.2.2 MedicalTreatments.......................................... 304
9.2.3 Agriculture ................................................... 305
9.2.4 MaintenanceofEquipment.................................. 306
9.2.5 ProductDevelopment........................................ 307
9.2.6 PredictivePolicing........................................... 309
9.2.7 PreventionofFareDodging ................................. 310
9.2.8 Summary ..................................................... 311
9.3 ProcessModelfortheImplementationofUpliftModeling
inPractice ............................................................. 312
References.................................................................... 318
10 SummaryandOutlook..................................................... 321
AppendixA ....................................................................... 327
A.1 Proofoftheχ2 Distributionofχ2 (Theorem3.2) ................. 327
net
A.2 Sensitivity Analyses of Key Performance Indicators
(DetailedFindings).................................................... 330
A.2.1 VariationofRandomNoise.................................. 330
A.2.2 VariationoftheDifferenceofRandomNoise .............. 332