Modeling and Analyzing IVR Systems, as a Special Case of Self-services Research Thesis Nitzan

119 Pages·2015·1.54 MB·English

Modeling and Analyzing IVR Systems, as a
Special Case of Self-services
Research Thesis
SubmittedinPartialFulfillmentofthe
RequirementsfortheDegreeof
MasterofScienceinOperationsResearchandSystemAnalysis
Nitzan Carmeli
SubmittedtotheSenateofthe
Technion-IsraelInstituteofTechnology
Sivan,5775 Haifa June10,2015
ThisResearchThesisWasDoneUndertheSupervisionof
ProfessorHayaKaspiandProfessorAvishaiMandelbaumintheFacultyof
IndustrialEngineeringandManagement.
IwouldliketogratefullythankProfessorHayaKaspiandProfessor
AvishaiMandelbaumfortheirendlessguidanceandsupport,forgivingme
thewonderfulopportunitytoworkwiththemandlearnfromthem. Ithas
beenagreatprivilege,andapleasure.
IwouldalsoliketothankDr. GalitYom-Tovforhervaluableguidance,
ArikSenderovich,YuvalMichaelandAnatBernshteinfortheir
contributiontothiswork,andgreatlythanktheSEELabteam: Ella
Nadjharov,IgorGavakoandDr. ValeryTrofimovforalltheirsupportand
assistance.
TheGenerousFinancialHelpoftheTechnion
isGratefullyAcknowledged.
Contents
1 Introduction 2
2 LiteratureReview 5
2.1 MethodologiesforevaluatingIVRs . . . . . . . . . . . . . . . . . 5
2.2 DesigningandoptimizingIVRs . . . . . . . . . . . . . . . . . . 6
2.3 ModelingaCallCenterwithanIVR . . . . . . . . . . . . . . . . 7
2.4 StochasticSearchinaForest . . . . . . . . . . . . . . . . . . . . 8
2.5 PredictingSearchSuccessinSelf-ServiceSystems . . . . . . . . 9
3 ModelingCustomerFlowasaStochasticSearchonaTree 11
3.1 TheSearchModel . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1.1 BuildingBlocks . . . . . . . . . . . . . . . . . . . . . . 12
3.1.2 StateSpaceandSearchProtocol . . . . . . . . . . . . . . 14
3.2 PropertiesofCandidates . . . . . . . . . . . . . . . . . . . . . . 17
3.3 ModelingtheSearchProtocolasaRootedGraph . . . . . . . . . 24
4 AdmissibleTreeCreation 30
4.1 AlgorithmDescription . . . . . . . . . . . . . . . . . . . . . . . 31
4.2 AlgorithmFormulation . . . . . . . . . . . . . . . . . . . . . . . 34
4.3 The Equivalence between Admissible Tree and the Set of Admis-
sibleCandidates . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4 ProofsofLemmas5–7 . . . . . . . . . . . . . . . . . . . . . . . 41
4.5 ProofsofLemmas8–10 . . . . . . . . . . . . . . . . . . . . . . . 44
5 IndexCalculationsOvertheAdmissibleTree 50
5.1 IndexCalculations . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.2 FindingOptimalSearchPolicyAlgorithm . . . . . . . . . . . . . 52
5.2.1 OptimalPolicyAlgorithm . . . . . . . . . . . . . . . . . 54
6 ExploratoryDataAnalysis 56
6.1 DataDescription . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.2 DemandforIVRServices . . . . . . . . . . . . . . . . . . . . . . 59
ii
6.3 CustomerPaths . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
6.4 SuccessRateofIVRServices. . . . . . . . . . . . . . . . . . . . 65
6.5 FurtherSupporttoanAbandonmentHypothesis . . . . . . . . . . 72
6.5.1 CustomerExperienceEffectonTimeSpentinIVRServices 73
7 ModelImplications 79
7.1 ComparingDifferentIVRDesigns . . . . . . . . . . . . . . . . . 81
7.1.1 EstimatingtheModelParameters . . . . . . . . . . . . . 81
7.1.2 NumericalExample . . . . . . . . . . . . . . . . . . . . 86
7.2 FurtherImplications. . . . . . . . . . . . . . . . . . . . . . . . . 96
8 SummaryandDiscussion 98
8.1 FutureResearch . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
A IVRDataTable 101
B IVRLastServicesDistribution 103
C Measurements-ReadingTimeofMenuOptions 106
iii
List of Figures
1.1 CallcenterwithanIVR . . . . . . . . . . . . . . . . . . . . . . . 3
3.1 Anexampleofarootedtree,representinganIVRsystem . . . . . 15
3.2 An example of a proper candidate represented by a path in GF.
HereGistakenfromFigure3.1. . . . . . . . . . . . . . . . . . . 28
3.3 ApartialexampleofaProperGraphcreatedfromanIVRtreeG. 29
4.1 ExampleofavertexinF4 . . . . . . . . . . . . . . . . . . . . . 33
exc
4.2 ExampleofapartialProperGraphandexcludedpaths . . . . . . 35
4.3 ExampleofapartialAdmissibleTree . . . . . . . . . . . . . . . 37
4.4 ExamplesofProposition3,caseIV . . . . . . . . . . . . . . . . . 39
4.5 Exampleofpathsrepresentingunwantedsequences,resultingfrom
addingverticesinF2 . . . . . . . . . . . . . . . . . . . . . . . 43
exc
4.6 AnexampleofLemma8 . . . . . . . . . . . . . . . . . . . . . . 46
4.7 AnexampleofLemma8 . . . . . . . . . . . . . . . . . . . . . . 47
6.1 ILBankIVRmenu . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.2 ILBankHybridanimation(link) . . . . . . . . . . . . . . . . . . 62
6.3 ILBankNetworkanimation(link) . . . . . . . . . . . . . . . . . 63
6.4 RecentAccountActivityduration,N = 1,983,161 . . . . . . . . 66
6.5 AccountSummaryduration,N = 1,081,992 . . . . . . . . . . . 66
6.6 AccountActivityTodayduration,N = 353,328 . . . . . . . . . . 67
6.7 CreditCardVouchersduration,N = 608,529 . . . . . . . . . . . 67
6.8 IVRservicesduration . . . . . . . . . . . . . . . . . . . . . . . . 68
6.9 Fittingmixtureofdistributions,RecentAccountActivity . . . . . 70
6.10 Fittingmixtureofdistributions,AccountSummary . . . . . . . . 71
6.11 DistributionofthetimeintheIDphase,asafunctionofthenumber
ofpriorcalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.12 Zoomin-DistributionofthetimeintheIDphase,asafunctionof
thenumberofpriorcalls . . . . . . . . . . . . . . . . . . . . . . 74
6.13 Distributionofthetimein‘RecentAccountActivity’,asafunction
ofthenumberofpriorcalls . . . . . . . . . . . . . . . . . . . . . 75
6.14 Distributionofthetimein‘RecentAccountActivity’,asafunction
ofthenumberofpriorcalls,zoominfrom0to20seconds . . . . 76
iv
6.15 Distributionofthetimein‘RecentAccountActivity’,asafunction
ofthenumberofpriorcalls,zoominfrom45to75seconds . . . . 76
6.16 Distributionofthetimein‘RecentAccountActivity’,asafunction
ofthenumberofpriorvisitstotheservice,acrosscalls . . . . . . 77
6.17 Distributionofthetimein‘RecentAccountActivity’,asafunction
ofthenumberofpriorvisitstotheservice,withinonecall . . . . 78
7.1 Calculatingorganizationalprofit-Simpleexample. . . . . . . . . 80
7.2 OriginalIVRdesign. Boldrepresentsserviceswithpositivereward 89
7.3 ShallowIVRdesign. Boldrepresentsserviceswithpositivereward 90
7.4 DeepIVRdesign. Boldrepresentsserviceswithpositivereward . 91
A.1 IVRdatatable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
B.1 RecentAccountActivityduration . . . . . . . . . . . . . . . . . 103
B.3 AccountSummaryduration . . . . . . . . . . . . . . . . . . . . . 104
B.5 AccountActivityTodayduration . . . . . . . . . . . . . . . . . . 104
B.7 CreditCardVouchersduration . . . . . . . . . . . . . . . . . . . 105
C.1 TimemeasurementofILBankIVRmenuoptions . . . . . . . . . 108
v
List of Tables
6.1 TotalnumberofIVRcallsbytheiroutcome,May2008toJune2009 57
6.2 Total number of IVR calls, by customer type, May 2008 to June
2009 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.3 IVRservices-relativedemandfrequency,May2008toJune2009 60
6.4 Frequentpaths . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.5 Fittingmixtureofdistributions,RecentAccountActivity . . . . . 70
6.6 Fittingmixtureofdistributions,AccountSummary . . . . . . . . 71
6.7 Statistics of time in the ID phase, as a function of the number of
priorcalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
7.1 AveragesojourntimewithinILBankIVR,May2008toJune2009 85
7.2 Average patience, by customer type, based on Khudyakov et al.
[18] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7.3 Rewardsandcosts,byAksinetal. [2] . . . . . . . . . . . . . . . 87
7.4 Laplacetransformforsuccessfulandunsuccessfulservicedurations 88
7.5 Results,optimalpathsandexpectedutility,Highpriority . . . . . 92
7.6 Results,optimalpathsandexpectedutility,Mediumpriority . . . 93
7.7 Results,optimalpathsandexpectedutility,Lowpriority . . . . . 94
7.8 Numericalexample-averagenumberofrelevantcallsbycustomer
type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7.9 Organizationalrevenue . . . . . . . . . . . . . . . . . . . . . . . 95
vi
Glossary
Abbreviation FullForm
IVR InteractiveVoiceResponse
EDA ExploratoryDataAnalysis
Notation1 Explanation
G = (V,E) Arootedtree.
M Thesetoftreeleaves.
s TherootvertexofG = (V,E).
A(i) Thesetofallimmediatesuccessorsofvertexi.
pre(i) Theimmediatepredecessorofvertexi.
dep(i) Depthoftheuniquepathleadingfromtherootstovertexi.
Γ(i) Thesub-treespanningfromvertexi.
M Thesetofleavesinthesub-treespanningfromvertexi.
Γ(i)
c Costperunitoftime.
τ ∼ exp(θ) Customerpatience.
P Successprobabilityofvertexi.
i
r Rewardearnedfromasuccessfulvisitatvertexi.
i
t (i) Timespentinvertexigivenasuccessfulvisit.
ser
t (i) Timespentinvertexigivenanunsuccessfulvisit.
F
T Timespentinvertexi.
i
N(j) The number of times that option (vertex) j was previously ex-
plored.
N(j)
t Theexplorationtimeofedgee = (i,j) ∈ E, giventhatoptionj
i,j
inmenuiwaspreviouslyexploredN(j)times.
Place (j) Theplaceofoptionj inmenui.
i
l Theleftmostsonofvertexi.
i
N Avectorrepresentingthenumberofrepeatedexplorationsofeach
vertex.
R∗(v) Theexpectedutilityofthetreespanningfromvertexv.
R∗,v Theindexofedge(u,v).
u
L (s) TheLaplacetransformofX.
X
1Listedhereareonlynotationswhichappearinmorethanonechapter.Therestareexplainedin
theirrelevantchapter.
vii
Abstract
Callcentersplayaprominentroleintoday’seconomy. Theyserveasthemain
customer contact channel in various enterprises, which makes them highly labor-
intensive operations. Thus, call centers look for means to reduce the number of
agents handling calls, and trying to do so without degrading service level. Inter-
active Voice Response (IVR) systems are presently one of the main self-service
channels employed by call centers. They are used as means to reduce operating
expensesderivedfromagentemploymentcosts.
The goal of our research is to improve and enhance IVR systems, aiming to
create a body of knowledge that will generalize to other self-service systems. We
model customers flow within an IVR system as a stochastic search in a directed
tree. ThesearchgoalistofindtheoptimalpathontheIVRtree,whichwillresult
in maximal expected discounted utility for customers. We show that a calculable
index can be assigned to each feasible option, and the optimal policy is to choose
theoptionwiththehighestindexateachstage.
OurmodelbuildingblockswerecreatedthroughanExploratoryDataAnalysis
(EDA) of real IVR transactions, in a call center of a large Israeli bank. The EDA
revealed interesting phenomena regarding customer abandonments and learning
withintheIVR.
OurworkenablesthecomparisonbetweenalternativeIVRdesigns,bothfrom
the customer and the enterprise point of view. This complements related research
inotherfields,suchasHuman-Factor-EngineeringandTelecommunication.
The model for IVR systems that we developed can easily be implemented to
otherself-servicesystemssuchasInternetwebsiteswhichhavebecomeprevalent.
1
Chapter 1
Introduction
Call centers play a prominent role in today’s economy. Indeed, they serve as
the main customer contact channel in various enterprises [1], public or private,
product-based or service-based. Call centers are also highly labor-intensive; they
employ sometimes hundreds, or even thousands of Customer Service Representa-
tives(CSRsorAgents)tohandleincomingcalls. Typically,60%-70%oftheover-
all operating expenses of call centers are derived from agent employment costs
[10]. Reducing the number of agents handling calls, without degrading service
level, is thus of interest and importance, and enabling customers to self-serve is
one of the basic means for doing so. As customers self-serve, the agent workload
isbeingreduced,andlessagentsarerequiredinordertomaintainacertainservice
level. Interactive Voice Response (IVR) systems, also known as Voice Response
Units (VRU), are one of the main self-service channels [18], along with Internet
websitesanddesignatedsmart-phoneapplications.
IVR systems, if properly designed, can increase customer satisfaction and
loyalty, cut staffing costs and increase revenue by extending business hours and
market reach [3]. Poorly designed IVR systems, on the other hand, will cause the
oppositeeffectandleadtodissatisfiedcustomers,increasedcallvolumeandmight
even increase agent turnover, as agents would serve frustrated customers [7]. A
PurdueUniversitystudyshowedthatmorethan90%ofUSconsumersareforming
their image on a certain company based on their experience with its call center.
Furthermore, more than 60% stopped using the products of a company in which
they had a negative call center experience [7]. Since the IVR system is the front
gateofmostcallcenters,havinganeffective,efficient,andcustomer-friendlyIVR
systemisextremelyimportant.
The goal of our research is to improve and enhance IVR systems, aiming to
generalizetootherself-servicesystems. Todoso,wemodelandanalyzecustomer
flow within an IVR system. The model building blocks were established and in-
spired by an Exploratory Data Analysis (EDA) of real IVR transactions in a call
centerofalargeIsraelibank,basedonmorethanoneyearofdatawhichincludes
millions of calls. The theoretical basis for our model then relies on the work of
2