Table Of ContentT-Labs Series in Telecommunication Services
Marie-Neige Garcia
Parametric
Packet-Based
Audiovisual
Quality Model
for IPTV Services
T-Labs Series in Telecommunication Services
Series editors
Sebastian Möller, Berlin, Germany
Axel Küpper, Berlin, Germany
Alexander Raake, Berlin, Germany
For furthervolumes:
http://www.springer.com/series/10013
Marie-Neige Garcia
Parametric Packet-Based
Audiovisual Quality Model
for IPTV Services
123
Marie-Neige Garcia
Telekom InnovationLaboratories,
Assessmentof IP-based
Application (AIPA)
TU Berlin
Berlin
Germany
ISSN 2192-2810 ISSN 2192-2829 (electronic)
ISBN 978-3-319-04854-3 ISBN 978-3-319-04855-0 (eBook)
DOI 10.1007/978-3-319-04855-0
Springer ChamHeidelberg New YorkDordrecht London
LibraryofCongressControlNumber:2014940154
(cid:2)SpringerInternationalPublishingSwitzerland2014
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor
informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthe
work. Duplication of this publication or parts thereof is permitted only under the provisions of
theCopyright Law of the Publisher’s location, in its current version, and permission for use must
always be obtained from Springer. Permissions for use may be obtained through RightsLink at the
CopyrightClearanceCenter.ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Today, telecommunication services go beyond telephony and include video ser-
vices such as Internet Protocol Television. These video services are prone to
degradations including blurring, blockiness, or unexpected freezing during video
playback. These degradations may be perceived by the user and therefore may
spoil her/his experience of the service. As a consequence, the audiovisual quality
as perceived by the user needs to be measured and, if needed, improved.
This book presents a parametric comprehensive model to measure and predict
theaudiovisualqualityofInternetProtocolTelevisionservicesasitislikelytobe
perceivedbytheuser.Thecomprehensivemodelisdividedintothreesub-models
referred to as the audio model, the video model, and the audiovisual model. The
audio and video models take as input a parametric description of the audiovisual
processingpath,anddeliverdistinctestimatesforboththeaudioandvideoquality.
These distinct estimates are eventually used as input data for the audiovisual
model. This model provides an overall estimate of the perceived audiovisual
qualityintotal.Theparametricdescriptioncanbeusedasdiagnosticinformation.
The quality estimates and diagnostic information can be practically applied to
enhance network deployment and operations. Two applications come to mind in
particular: Network planning and network service quality monitoring. The audio
modelcanbeusedindifferentlyforbothapplications.However,twovariantsofthe
video model have been developed in order to address particular needs of the
applications mentioned above.
Thisbookiswrittenforthestudents,researchers,andengineersinvolvedinthe
assessment of the Quality of Experience (QoE) of telecommunication services. It
may in particular be useful for persons working on network deployment and
operations,andforthosedevelopinginstrumentalmethodstoassesstheperceived
quality of telecommunication services.
This book was created within the scope of my Ph.D. dissertation, which was
conducted under the supervision of Prof. Alexander Raake at the Assessment of
IP-based Application (AIPA) chair, Technische Universität Berlin. The work was
supported by the T-V-Model project coordinated by Dr. Bernhard Feiten at the
Telekom Innovation Laboratories (T-Labs). The scope of this work was also influ-
encedbytheITU-TP.NAMS(‘‘Parametricnon-intrusiveassessmentofaudiovisual
mediastreamingquality’’)competition.Themodelappliedtoqualitymonitoringwas
indeedsubmittedduringthiscompetition.ItresultedintheITU-TP.1201.2standard.
v
Acknowledgments
I owe my deepest gratitude to my colleague, supervisor, and friend Prof. Alex-
ander Raake. He was a guide in my work and scientific career, and pushed any
undertaken task to higher scientific levels. He never failed to support me and to
share his numerous ideas and his motivation.
I am also indebted to the whole T-V-Model team: Bernhard Feiten, Alexander
Raake,PeterList,SavvasArgyropoulos,UlfWüstenhagen,andJensKrollfortheir
expertise and friendly support. The model presented in this work would not have
won the ITU-T P.NAMS competition without them.
In the context of this competition, I would like to thank my colleagues of the
ITU-T StudyGroup12,inparticularJörgenGustafsson,David Lindegren,Martin
Pettersson, Simon Broom, Kazuhisa Yamagishi, Ning Liao, Simon Gao, Martin
Kastner, Akira Takahashi, Jens Berger, and all further ITU-T Study Group 12
colleaguesfortheirinvolvementintheworkgroupsQuestionsQ14/12,Q13/12and
in the Work Party WP2/12.
It was also a great pleasure to participate in Video Quality Expert Group
(VQEG) meetings, and I thank the VQEG community for sharing their expertise
and for being open-minded to newcomers. I am in particular grateful to Prof.
Patrick Le Callet for reviewing my Ph.D. thesis.
I am also thankful to Stephen Wilson, Markus P. Möller, and Rahul Swami-
nathan for their helpful advice on my Ph.D. dissertation writing and defense.
I would also like to thank my colleagues at the AIPA, Quality and Usability,
and Design Research Labs for a unique friendly and scientific atmosphere. It was
always a pleasure to come to work.
In particular, I would like to thank:
• Prof. Sebastian Möller, head of the Quality and Usability Lab, for his support,
expertise, and useful advice,
• Nicolas Côté, Marcel Wältermann, Dominik Strohmeier, Benjamin Belmudez,
SebastianEgger,Jan-NiklasAntons,andPierreLebretonforfruitfuldiscussions
on subjective quality assessment and modeling,
• Robert Schleicher, Ina Wechsung, Katrin Schoenenberg, and Tim Polzehl for
their expertise and help in statistical analysis and modeling,
• Prof. Sascha Spors and Prof. Marcus Barkowsky for their expertise and useful
advice during the set up of standardized usability/quality test rooms,
vii
viii Acknowledgments
• Irene Hube-Achter, Christine Kluge, and Ulrike Kehrberg for administrative
(and more!) support,
• Katharina Bredies, Benjamin Weiss, Klaus-Peter Engelbrecht, Rahul Swami-
nathan, Matthias Geier, Christine Kühnel, Matthias Rath, Anja Naumann,
Michael Nischt, Katrin Wolf, Blazej Lewcio, and Maija Poikela for their
friendly support, their communicative motivation in their work, and/or the
relaxing time spent playing table football.
Finally,andobviously,IdeeplythankOdile,mysisters,myparents,myfamily
in general, and my friends (to which many of the above-mentioned colleagues
belong) for their love and support.
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objectives and Scope. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Contribution by the Author. . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Structure of the Book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Quality Engineering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 The Concept of Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.1 The Jekosch School. . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.2 Comparison with Other Terminology
in the Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.3 Quality of Service and Quality of Experience. . . . . . . . . 19
2.2 From Quality Elements to Quality. . . . . . . . . . . . . . . . . . . . . . 23
2.2.1 From Quality Elements to Quality Features . . . . . . . . . . 24
2.2.2 From Quality Features to Quality Dimensions . . . . . . . . 45
2.2.3 From Quality Dimensions to Quality. . . . . . . . . . . . . . . 50
2.3 Subjective Quality Assessment . . . . . . . . . . . . . . . . . . . . . . . . 51
2.3.1 Standardized Test Methods . . . . . . . . . . . . . . . . . . . . . 52
2.3.2 Standardized Rating Scales . . . . . . . . . . . . . . . . . . . . . 56
2.4 Quality Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.4.1 Quality Model Types. . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.4.2 Model Development . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.4.3 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3 Model Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.1 Scope and Model Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.2 From Quality Elements to Quality. . . . . . . . . . . . . . . . . . . . . . 73
3.3 Subjective Quality Assessment . . . . . . . . . . . . . . . . . . . . . . . . 76
3.3.1 Database Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.3.2 TVM Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.3.3 P.NAMS Data Set. . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.3.4 Test Set-Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
ix
x Contents
3.4 Subjective Test Analysis Procedure. . . . . . . . . . . . . . . . . . . . . 99
3.4.1 Rejection Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.4.2 Normalization Procedure . . . . . . . . . . . . . . . . . . . . . . . 100
3.5 Model Development. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.5.1 Modeling Approach. . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.5.2 Model Training. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.5.3 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4 Audio Quality Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.1 Subjective Quality Assessment . . . . . . . . . . . . . . . . . . . . . . . . 110
4.1.1 Subjective Test Overview . . . . . . . . . . . . . . . . . . . . . . 110
4.1.2 Subjective Test Results . . . . . . . . . . . . . . . . . . . . . . . . 110
4.2 Parametric Audio Quality Model. . . . . . . . . . . . . . . . . . . . . . . 122
4.2.1 State of the Art. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.2.2 Audio Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.2.3 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5 Video Quality Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.1 Subjective Quality Assessment . . . . . . . . . . . . . . . . . . . . . . . . 130
5.1.1 Subjective Test Overview . . . . . . . . . . . . . . . . . . . . . . 130
5.1.2 Subjective Test Results . . . . . . . . . . . . . . . . . . . . . . . . 131
5.2 Video Quality Model for Network Planning . . . . . . . . . . . . . . . 143
5.2.1 State of the Art. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.2.2 Packet-Based Video Model . . . . . . . . . . . . . . . . . . . . . 144
5.2.3 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
5.3 Perceptually Motivated Frame-Based Parameters. . . . . . . . . . . . 150
5.3.1 Parametric Description of Content. . . . . . . . . . . . . . . . . 150
5.3.2 Parametric Description of Slicing Degradation. . . . . . . . 155
5.3.3 Parametric Description of Freezing Degradation. . . . . . . 163
5.4 Video Quality Model for Quality Monitoring . . . . . . . . . . . . . . 164
5.4.1 State of the Art. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
5.4.2 Frame-Based Video Model. . . . . . . . . . . . . . . . . . . . . . 164
5.4.3 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6 Audiovisual Quality Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.1 Subjective Quality Assessment . . . . . . . . . . . . . . . . . . . . . . . . 172
6.1.1 Subjective Test Overview . . . . . . . . . . . . . . . . . . . . . . 172
6.1.2 Subjective Test Results . . . . . . . . . . . . . . . . . . . . . . . . 173
6.2 Modeling Approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.2.1 State of the Art. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.2.2 Modeling Approach. . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Contents xi
6.2.3 Quality-Based Modeling . . . . . . . . . . . . . . . . . . . . . . . 178
6.2.4 Impairment-Factor-Based Modeling. . . . . . . . . . . . . . . . 180
6.2.5 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.3 Parametric Audiovisual Quality Model. . . . . . . . . . . . . . . . . . . 190
6.3.1 Audiovisual Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
6.3.2 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
7 Conclusion and Outlook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.1 Conclusion and Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.2 Towards Quality Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . 200
7.3 Towards QoE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.4 Model Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
Appendix A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
Appendix B. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
