approximated transform and quantisation for complexity-reduced high efficiency video coding

244 Pages·2017·11.54 MB·English

APPROXIMATED TRANSFORM AND
QUANTISATION FOR COMPLEXITY-REDUCED
HIGH EFFICIENCY VIDEO CODING

A thesis submitted for the degree of Doctor of Philosophy

by
Mohd Mohd Sazali

College of Engineering, Design, and Physical Sciences
Brunel University
March 2017
i

Acknowledgement
My sincere acknowledgement goes to my principal supervisor, Prof. Dr
Abdul H. Sadka, for his productive guidance, constructive criticisms, invaluable
knowledge sharing, brilliant thoughts, and continuous support throughout my
research at Brunel University London. My gratitude is extended to my second
supervisor Dr Nikolaos V. Boulgouris for his valuable inputs and wonderful help. I
am also thankful to my current and past colleagues, Salim Al Amri, Dr Sagir Lawan,
Taha Alfaqheri, Dr Mohamed Rafiq Swash, Dr Hamdullah Mohib, Dr Obaid Fatah,
Dr Abdulkadir Audu, Dr Abdulkareem Ibrahim, and many others for their technical
advice and friendly company during the course of my research. Special thanks also
to Tony Morris, John Morse, and their respective teams for their technical assistance.
My genuine appreciation is dedicated to my great parents, Dr Mohd Sazali
Khalid and Shamsinar Jaafar, along with my helpful siblings for their continuous
encouragement, financial support, and endless prayers. Most of all, I am deeply
indebted to my wonderful wife, Siti Nazmin, for her long sacrifice and kind
understandings to allow me completing my study far away from the family. I owe
our beloved sons so much, Iman Shahdan and Irshad Shahidin, for having to spend a
big part of their childhood time without me being around. To our newly born son,
Imdad Sharfan, hopefully one day you can appreciate the hardships that we had gone
through and be as strong as your brothers.
Last but not least, I wish to express my appreciation to my generous
sponsors, Universiti Sains Malaysia (USM) and Malaysian Ministry of Higher
Education (MoHE) for funding my study. Even though far from being perfect, it is
hoped that this thesis will be beneficial in one way or another to the reader and the
body of knowledge.
ii

Abstract
The transform-quantisation stage is one of the most complex operations in the state-
of-the-art High Efficiency Video Coding (HEVC) standard, accounting for 11–41%
share of the encoding complexity. This study aims to reduce its complexity, making
it suitable for dedicated hardware accelerated architectures. Adopted methods
include multiplier-free approach, Multiple-Constant Multiplication architectural
designs, and exploiting useful properties of the well-known Discrete Cosine
Transform. Besides, an approximation scheme was introduced to represent the
original HEVC transform and quantisation matrix elements with more hardware-
friendly integers. Out of several derived approximation alternatives, an approximated
transform matrix (T16) and its downscaled version (ST16) were further evaluated.
An approximated quantisation multipliers matrix (Q) and its combination with one
transform matrix (ST16 + Q) were also assessed in HEVC reference software, HM-
13.0, using test video sequences of High Definition (HD) quality or higher. Their
hardware architectures were designed in IEEE-VHDL language targeting a Xilinx
Virtex-6 Field Programmable Gate Array technology to estimate resource savings
over original HEVC transform and quantisation. T16, ST16, Q, and ST16 + Q
approximated transform or/and quantisation matrices provided average Bjøntegaard-
Delta bitrate differences of 1.7%, 1.7%, 0.0%, and 1.7%, respectively, in
entertainment scenario and 0.7%, 0.7%, -0.1%, and 0.7%, respectively, in interactive
scenario against HEVC. Conversely, around 16.9%, 20.8%, 21.2%, and 25.9%
hardware savings, respectively, were attained in the number of Virtex-6 slices
compared with HEVC transform or/and quantisation. The developed architecture
designs achieved a 200 MHz operating frequency, enabling them to support the
encoding of Quad Full HD (3840 × 2160) videos at 60 frames per second.
Comparing T16 and ST16 with similar designs in the literature yields better
hardware efficiency measures (0.0687 and 0.0721, respectively, in mega
sample/second/slice). The presented approximated transform and quantisation
matrices may be applicable in a complexity-reduced HEVC encoding on hardware
platforms with non-detrimental coding performance degradations.
Keywords: Hardware complexity, HEVC, FPGA, quantisation, transform
iii

TABLE OF CONTENTS
CHAPTER TITLE PAGE

ACKNOWLEDGEMENT i
ABSTRACT ii
TABLE OF CONTENTS iii
LIST OF TABLES viii
LIST OF FIGURES xiii
LIST OF ABBREVIATIONS xix
LIST OF APPENDICES xxiv

1 INTRODUCTION 1
1.1 Video Coding 1
1.2 Motivation and Problem Description 4
1.3 Aim and Objectives 7
1.4 Scope of Work 7
1.5 Thesis Contributions 8
1.6 Thesis Outline 9

2 BACKGROUND 11
2.1 Digital Video Capture and Representation 11
2.1.1 Digital Video Capture 11
2.1.2 Digital Video Representation 12
2.2 Video Quality 15
2.3 Bjøntegaard Delta PSNR (BD-PSNR) and bitrate 20
(BD-rate)
2.4 Brief History of Video Coding 21
2.5 Prediction Structure/Configuration 23
2.5.1 All Intra (AI) 23
2.5.2 Random Access (RA) 24
2.5.3 Low Delay with P pictures (LP) 25
iv

2.5.4 Low Delay with B pictures (LB) 25
2.6 Overview of the High Efficiency Video Coding 26
(HEVC) standard
2.6.1 Video coding layer of HEVC 27
2.6.2 Profiles, Levels, and Tiers in HEVC 29
2.7 Summary 31

3 HEVC FORWARD TRANSFORM, INTERMEDIATE 32
SCALING, AND QUANTISATION
3.1 Introduction 32
3.2 HEVC Transforms 34
3.3 Basis Vectors of HEVC Core and Alternative 36
Transforms
3.4 Complexity Analysis 41
3.4.1 Even–Odd Decomposition 41
3.4.2 Multiplier-free Implementation 46
3.4.3 Multiple-Constant Multiplication 49
(MCM)
3.5 Intermediate Scaling 57
3.6 Quantisation 62
3.7 Related work on Transform and Quantisation 64
3.7.1 Related work on Transform 64
3.7.2 Related work on Quantisation 68
3.8 Summary 69

4 APPROXIMATED FORWARD CORE TRANSFORM, 70
INTERMEDIATE SCALING, AND QUANTISATION
FOR HEVC
4.1 Introduction 70
4.2 Approximated Forward Core Transform 72
4.2.1 Algorithmic Modelling 72
4.2.2 Degrees of Approximation 77
4.2.3 Arithmetic Complexity Analysis 78
v

4.2.4 Transform and Intermediate Scaling 82
4.3 Approximated Forward Quantisation 87
4.4 Summary 90

5 SOFTWARE-BASED PERFORMANCE EVALUATION 91
OF APPROXIMATED FORWARD TRANSFORM AND
QUANTISATION
5.1 Pilot Study 91
5.1.1 Peak Signal to Noise Ratio (PSNR) 94
5.1.2 Structural Similarity (SSIM) Index 98
5.1.3 Bjøntegaard-Delta Bitrate (BD-rate) 101
5.1.4 Visual Observations 101
5.1.5 Encoder-Decoder Compatibility 108
5.1.6 Conclusions 109
5.2 Approximated Transforms 110
5.2.1 Experimental Settings 110
5.2.2 Peak Signal to Noise Ratio (PSNR) 111
5.2.3 Bjøntegaard-Delta Bitrate (BD-rate) 113
5.2.4 Visual Observations 114
5.2.5 Conclusions 124
5.3 Approximated Quantisation 125
5.3.1 Experimental Settings 125
5.3.2 Peak Signal to Noise Ratio (PSNR) 125
5.3.3 Bjøntegaard-Delta Bitrate (BD-rate) 128
5.3.4 Visual Observations 129
5.3.5 Conclusions 137
5.4 Approximated Transform and Quantisation 138
5.4.1 Experimental Settings 138
5.4.2 Peak Signal to Noise Ratio (PSNR) 138
5.4.3 Bjøntegaard-Delta Bitrate (BD-rate) 141
5.4.4 Visual Observations 142
5.4.5 Conclusions 150
5.5 Summary 151
vi

6 DEDICATED HARDWARE ARCHITECTURE 153
DESIGNS FOR APPROXIMATED TRANSFORM,
INTERMEDIATE SCALING, AND APPROXIMATED
QUANTISATION
6.1 Hardware-Software Co-design Methodology 153
6.2 Hardware Architecture Designs for Approximated 157
Transform and Intermediate Scaling
6.2.1 Top-level Transform Module (TM) 157
6.2.2 Data path Module (DM) 159
6.2.3 Control Module (CM) 164
6.2.4 Functional Verification 166
6.2.5 Results and Discussions 166
6.2.6 Conclusions 171
6.3 Hardware Architecture Designs for Approximated 172
Quantisation
6.3.1 Top-level Quantisation Module (QM) 172
6.3.2 Functional Verification 174
6.3.3 Results and Discussions 174
6.3.4 Conclusions 175
6.4 Hardware Architecture Designs for Approximated 176
and Scaled Transform and Quantisation
6.4.1 Top-level Transform and Quantisation 176
Module (TQM)
6.4.2 Results and Discussions 177
6.4.3 Conclusions 178
6.5 Summary 179

7 CONCLUSIONS AND FUTURE WORK 181
7.1 Conclusions 181
7.2 Future Work 183

REFERENCES 185
vii

APPENDICES 193
Appendix A 193
Appendix B 202
Appendix C 211
viii

LIST OF TABLES
TABLE NO. TITLE PAGE
1.1 Size of a 10-minute raw video in several resolution 2
formats
1.2 Average shares of the most complex HEVC encoding 6
stages
1.3 Average shares of the most complex HEVC decoding 6
stages
2.1 A 10-second 1080p video in different YUV sampling 14
patterns
2.2 Five quality levels of video quality 16
2.3 Supported levels in Main profile of HEVC 31
3.1 Several properties of DCT 35
3.2 Computational complexity in 1-D N-point HEVC core 45
transforms
3.3 Computational complexity in 1-D N × N HEVC core 45
transforms
3.4 Computational complexity in 2-D N × N HEVC core 45
transforms
3.5 Equivalent shift-add operations for HEVC core transform 47
elements
3.6 Complexity in multiplier-free N-point/N × N 1-D HEVC 48
core transform using even–odd decomposition
3.7 Complexity in multiplier-free N-point/N × N 1-D HEVC 55
core transform using even–odd decomposition and
Multiple-Constant Multiplication (MCM)
3.8 Computational savings in multiplier-free N-point/N × N 56
1-D HEVC core transform using even–odd
decomposition and Multiple-Constant Multiplication
(MCM)
3.9 Intermediate scaling factors in 2-D HEVC forward 61
ix

transform
3.10 Intermediate scaling factors in 2-D HEVC inverse 62
transform
4.1 Constants in 32 × 32 HEVC core transform matrix 70
4.2 Equivalent shift-add operations of LUT integers 73
4
4.3 Decision criteria of approximation alternatives 75
4.4 Matrix elements in the first column of different 32 × 32 76
core transform alternatives
4.5 Complexity in multiplier-free N-point/N × N 1-D 80
approximated core transform using even–odd
decomposition and Multiple-Constant Multiplication
(MCM)
4.6 Computational savings in multiplier-free N-point/N × N 82
1-D approximated core transform using even–odd
decomposition and Multiple-Constant Multiplication
(MCM)
4.7 Complexity in multiplier-free N-point/N × N 1-D 84
approximated and scaled core transform using even–odd
decomposition and Multiple-Constant Multiplication
(MCM)
4.8 Computational savings in multiplier-free N-point/N × N 86
1-D approximated and scaled core transform using even–
odd decomposition and Multiple-Constant Multiplication
(MCM)
4.9 Intermediate scaling factors in 2-D forward 87
approximated and scaled transform
4.10 Several alternative quantiser multipliers 89
5.1 Comparison of approximated transform matrix in pilot 92
study, V with T16, HEVC, D and D
sf sr
5.2 Test video sequences used in pilot study on 93
approximated transform, V
5.3 Experimental settings in pilot study on approximated 94
transform, V