Waiting Time Estimation A Case of EVE Online and CCP Customer Support

126 Pages·2017·5.97 MB·English

Waiting Time Estimation
A Case of EVE Online and CCP
Customer Support
Gu(cid:240)mundur Helgason
FFaaccuullttyyooffPPhhyyssiiccaallSScciieenncceess
UUnniivveerrssiittyyooffIIcceellaanndd
22001177

WAITING TIME ESTIMATION:
A CASE OF EVE ONLINE AND CCP
CUSTOMER SUPPORT
Gu(cid:240)mundur Helgason
30 ECTS thesis submitted in partial ful(cid:28)llment of a
MAS degree in Applied Statistics
Advisors
Anna Helga J(cid:243)nsd(cid:243)ttir and
Matth(cid:237)as KormÆksson
Faculty Representative
Gunnar StefÆnsson
Master’s Examiner
Thor Aspelund
Faculty of Physical Sciences
School of Engineering and Natural Sciences
University of Iceland
Reykjavik, June 2017
Waiting Time Estimation: A Case of EVE Online and CCP Customer Support
Waiting Time Estimation In Customer Support
30 ECTS thesis submitted in partial ful(cid:28)llment of a MAS degree in Applied Statistics
Copyright ' 2017 Gu(cid:240)mundur Helgason
All rights reserved
Faculty of Physical Sciences
School of Engineering and Natural Sciences
University of Iceland
T(cid:230)knigar(cid:240)ur, Dunhaga 5
107, Reykjavik
Iceland
Telephone: [phone]
Bibliographic information:
Gu(cid:240)mundur Helgason, 2017, Waiting Time Estimation: A Case of EVE Online and CCP
Customer Support, MAS thesis, Faculty of Physical Sciences, University of Iceland.
Printing: HÆsk(cid:243)laprent ehf.
Reykjavik, Iceland, June 2017
Abstract
We test and compare several di(cid:27)erent prediction methods for the purpose of waiting
time estimation in the context of online (email) customer support. Using data
obtained from CCP Games, the makers of the computer game EVE Online. For
the most part using classi(cid:28)cation methods for predicting whether or not a reply is
received after a certain time. Continuous methods are however also utilized, both to
predictwaitingtimecontinuouslyandtopredictrepliesbinomiallyatdi(cid:27)erenttimes.
Besidesanalyticalmodelingsimplerempiricalapproachesforwaitingtimeestimation
are also presented. Research literature associated with customer support, quality
in service, the psychology of waiting and waiting time estimation is reviewed. The
bene(cid:28)ts and shortcomings of di(cid:27)erent methods are discussed along with possibilities
in practical application.
`grip
˝ (cid:254)essari ranns(cid:243)kn, me(cid:240) hjÆlp (cid:253)missa t(cid:246)lfr(cid:230)(cid:240)ia(cid:240)fer(cid:240)a, spÆum vi(cid:240) fyrir um bi(cid:240)t(cid:237)ma
eftirsvarivi(cid:240)t(cid:246)lvup(cid:243)stime(cid:240)g(cid:246)gnumfrÆ(cid:254)j(cid:243)nustuveriCCP,framlei(cid:240)andat(cid:246)lvuleiksins
EVE Online. A(cid:240) mestu leyti er notast vi(cid:240) tv(cid:237)kosta t(cid:246)lfr(cid:230)(cid:240)il(cid:237)k(cid:246)n (cid:254)ar sem spÆ(cid:240) er
fyrir um hvort a(cid:240) svar sØ ge(cid:28)(cid:240) fyrir Ækve(cid:240)inn t(cid:237)mapunkt e(cid:240)a ekki. Samfelldar a(cid:240)fer-
(cid:240)ir eru (cid:254)(cid:243) einnig nota(cid:240)ar, b(cid:230)(cid:240)i til a(cid:240) spÆ fyrir um bi(cid:240)t(cid:237)ma (cid:237) sjÆlfu sØr og hvort
svar sØ ge(cid:28)(cid:240) fyrir Ækve(cid:240)inn t(cid:237)ma e(cid:240)a ekki. Auk greiningarlegra a(cid:240)fer(cid:240)a til forspÆar
er einnig notast vi(cid:240) einfaldari emp(cid:237)r(cid:237)skar a(cid:240)fer(cid:240)ir til a(cid:240) meta drei(cid:28)ngu bi(cid:240)t(cid:237)ma og
l(cid:237)kindi Æ svari eftir Ækve(cid:240)inn t(cid:237)ma. Tilt(cid:230)kar ranns(cid:243)knir Æ svi(cid:240)i (cid:254)j(cid:243)nustuvera, g(cid:230)(cid:240)a
(cid:237) (cid:254)j(cid:243)nustu, Æhrifa (cid:254)ess a(cid:240) b(cid:237)(cid:240)a eftir (cid:254)j(cid:243)nustu og a(cid:240)fer(cid:240)a sem notast hefur veri(cid:240) vi(cid:240)
til a(cid:240) spÆ fyrir um bi(cid:240)t(cid:237)ma eru sko(cid:240)a(cid:240)ar. A(cid:240)fer(cid:240)irnar sem notast var vi(cid:240) til bi(cid:240)t(cid:237)ma
forspÆar eru bornar saman, kostir (cid:254)eirra og gallar r(cid:230)ddir, auk hugsanlegra hagn(cid:253)tra
eiginleika.
v

Contents
List of Figures ix
List of Tables xiii
Acknowledgments xv
1 Introduction 1
1.1 EVE Online . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 CCP Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 The Support Ticket . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 The Work Flow / Working Procedures . . . . . . . . . . . . . . . . . 5
1.5 Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.1 Past . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.2 Present . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Literature Review 15
2.1 Good Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.1 Bene(cid:28)ts of High Quality Customer Service . . . . . . . . . . . 15
2.1.2 What Is Quality in Customer Support? . . . . . . . . . . . . . 16
2.1.3 Waiting for Service . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 The Psychology of Waiting . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3 The Mathematics of Waiting . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.1 Queuing Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 Waiting Time Estimation . . . . . . . . . . . . . . . . . . . . 27
3 Data and Methods 31
3.1 Main Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.1.1 Final Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.1.2 Training and Testing Data . . . . . . . . . . . . . . . . . . . . 46
3.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.1 Kernel Density and Cumulative Probability Functions . . . . . 47
3.2.2 Classi(cid:28)cation Approach . . . . . . . . . . . . . . . . . . . . . . 48
3.2.3 Regression Approach . . . . . . . . . . . . . . . . . . . . . . . 52
3.2.4 Rank Violation Index . . . . . . . . . . . . . . . . . . . . . . . 54
vii
Contents
4 Results 55
4.1 Kernel Density and Cumulative Probability . . . . . . . . . . . . . . 55
4.2 Classi(cid:28)cation Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.2.1 Logistic Regression Models . . . . . . . . . . . . . . . . . . . . 58
4.2.2 Generalized Additive Models . . . . . . . . . . . . . . . . . . . 61
4.3 Regression Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.4 Model Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.5 Rank Violation Index . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5 Discussion 83
References 93
6 Appendices 103
6.1 Appendices A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6.2 Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.3 Appendix C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.4 Appendix D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
viii
List of Figures
1.1 The Support Ticket Process . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 The EVE Online Help Center Front Page . . . . . . . . . . . . . . . . 7
1.3 The Support Ticket Submission Page . . . . . . . . . . . . . . . . . . 7
1.4 Support Ticket Creation in 2016 . . . . . . . . . . . . . . . . . . . . . 9
1.5 Support Ticket Creation : June-November 2016 . . . . . . . . . . . . 9
3.1 Tickets Per Status Within The CS System In 2016 . . . . . . . . . . 35
3.2 Amount Of Tickets In 5 Main Queues At Creation and 48 Hours . . . 36
3.3 Creation and Reply Rates in 2016 . . . . . . . . . . . . . . . . . . . . 37
3.4 System Load in 2016 / Quantiles and Gradient . . . . . . . . . . . . . 39
3.5 First Reply Times / By Creation Date, Reply Date And Distribution 40
3.6 Tickets Submitted in 2016 / Replied To Within the 48 Hour SLA . . 41
3.8 Reply Bursts in Billing and Game Play Tickets in August 2016 . . . . 43
3.9 Reply Bursts in Billing by Categories / June 2016 . . . . . . . . . . . 44
3.10 Replies in Technical / September 3rd . . . . . . . . . . . . . . . . . . 45
3.11 Kernel Density Function and Cumulative Probability Function . . . . 48
3.12 Confusion Matrix Example (Wikipedia, 2017) . . . . . . . . . . . . . 52
4.1 KDEs and ECDFs by Load : Billing & Game Play Queues . . . . . . 56
ix
LIST OF FIGURES
4.2 KDEs and ECDFs by Load : Technical Queue . . . . . . . . . . . . . 57
4.3 Logistic Regression : Main E(cid:27)ects . . . . . . . . . . . . . . . . . . . . 59
4.4 Logistic Regression : Comparison of Models . . . . . . . . . . . . . . 60
4.5 Monotonically decreasing binomial GAM with 10 knot smoothing on
Load per Queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.6 Prediction Cut-O(cid:27) Point Comparisons - Billing and Game Play Queues 62
4.7 Prediction Cut-O(cid:27) Point Comparisons - Technical . . . . . . . . . . . 62
4.8 Binomial GAM by Queue with Timepoints as a predictor variable . . 64
4.9 GAM : Waiting Time by Load per Queue : No Transformations . . . 66
4.10 GAM : Waiting Time by Load per Queue : Log Transformations . . . 67
√
4.11 GAM : Waiting Time by Load per Queue : Transformations . . . 68
4
4.12 Residuals by Queue for continuous GAM : No transformation . . . . 69
4.13 Residuals by Queue for continuous GAM : Log transformation . . . . 70
√
4.14 Residuals by Queue for continuous GAM : transformation . . . . . 71
4
√
4.15 Fitted/Observed : Log Transformations (L) - Transformations (R) 73
4
√
4.16 Pred.Accuracy : Log Transformations (L) - Transformations (R) . 74
4
√
4.17 95% Prediction and Con(cid:28)dence Intervals, Model (Billing) . . . . . 75
4
√
4.18 95% Prediction and Con(cid:28)dence Intervals, Model (Billing FT) . . . 75
4
√
4.19 95% Prediction and Con(cid:28)dence Intervals, Model (Game Play) . . . 76
4
√
4.20 95% Prediction and Con(cid:28)dence Intervals, Model (Game Play FT) . 76
4
√
4.21 95% Prediction and Con(cid:28)dence Intervals, Model (Technical) . . . . 77
4
4.22 Rank Violation Index by Creation Date for each Queue . . . . . . . . 80
4.23 Rank Violation Index by Creation Date : Applied . . . . . . . . . . . 80
x