Table Of ContentDOCUMENT ROOM., 2-827
RESEARCH LABORATORY OF ELECTRONIC
ASACJHUSET''S INSTITUTE OF TECHNOL/Y
THRESHOLD DECODING
I.
1% JAMES L. MASSEY
(cid:3)O (cid:3) hS coe'P
TECHNICAL REPORT 410
APRIL 5, 1963
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
RESEARCH LABORATORY OF ELECTRONICS
CAMBRIDGE, MASSACHUSETTS
The Research Laboratory of Electronics is an interdepartmental
laboratory in which faculty members and graduate students from
numerous academic departments conduct research.
The research reported in this document was made possible in part
by support extended the Massachusetts Institute of Technology, Re-
search Laboratory of Electronics, jointly by the U.S. Army, the
U.S. Navy (Office of Naval Research), and the U.S. Air Force
(Office of Scientific Research) under Contract DA36-039-sc-78108,
Department of the Army Task 3-99-25-001-08; and in part by Con-
tract DA-SIG-36-039-61-G14.
Reproduction in whole or in part is permitted for any purpose of
the United States Government.
A
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
RESEARCH LABORATORY OF ELECTRONICS
Technical Report 410 April 5, 1963
THRESHOLD DECODING
James L. Massey
Submitted to the Department of Electrical Engineering,
M. I. T., August 20, 1962, in partial fulfillment of the
requirements for the degree of Doctor of Philosophy.
(Manuscript received August 31, 1962)
Abstract
Two procedures for decoding linear systematic codes, majority decoding and a pos-
teriori probability decoding, are formulated. The essential feature of both methods is
a linear transformation of the parity-check equations of the code into "orthogonal parity
checks." The decoding decisions are then made on the basis of the values assumed by
these orthogonal parity checks. For binary codes, the principal component required
in the circuitry for instrumenting these decoding rules is an ordinary threshold logical
element. For this reason, we refer to these. decoding rules as forms of "threshold
decoding. n
It is shown that threshold decoding can be applied effectively to convolutional codes
up to approximately 100 transmitted bits in length over an interesting range of rates.
Very simple decoding circuits are presented for such codes. However, it is also shown
that the probability of error at the receiver cannot be made as small as desired by
increasing the length of the code that is used with threshold decoding, rather this prob-
ability approaches a nonzero limit as the code length is increased indefinitely. A large
- number of specific convolutional codes, suitable for threshold decoding, are tabulated.
Some of these codes are obtained by hand construction and others by analytical tech-
niques.
It is shown that threshold decoding is applicable to certain low-rate block codes,
and that a generalization of the method is applicable to several other classes of block
codes. It is shown that simple decoding circuits can be used for such codes. The theo-
retical limits of threshold decoding with block codes are still not clear, but the results
presented here are promising.
JK
TABLE OF CONTENTS
Glossary vi
I. THE CONCEPT OF THRESHOLD DECODING 1
1.1 Threshold Decoding of Linear Codes 3
a. Linear Codes and the Decoding Problem 3
b. Orthogonal Parity Checks 5
c. Majority Decoding 6
d. A Posteriori Probability Decoding 7
e. Threshold Decoding 8
1.2 Summary 10
II. CONVOLUTIONAL ENCODING AND PARITY CHECKING 11
2.1 Convolutional Encoding 11
2.2 Algebraic Structure 14
2.3 The Gilbert Bound 15
2.4 An Upper Bound on Minimum Distance 17
2.5 Random-Coding Bound 19
2.6 Encoding Circuits 22
a. RnA-Stage Encoder 22
b. (l-R)nA-Stage Encoder 23
c. Serializing of Symbols 24
d. Remarks on Encoding Circuits 24
2.7 Parity Checking 24
III. CONVOLUTIONAL CODES FOR THRESHOLD DECODING 28
3.1 Introduction 28
3.2 Decoding Problem for Convolutional Codes 28
3.3 Parity-Check Structure 30
3.4 Bounds on Codes 33
3.5 Self-Orthogonal Codes 38
3.6 Trial-and-Error Codes 40
3.7 Uniform Convolutional Codes 46
3.8 "Reed-Muller-Like" Codes 49
3.9 Summary 51
iii
CONTENTS
IV. THRESHOLD-DECODING CIRCUITS FOR CONVOLUTIONAL CODES 53
4.1 Decoding for the Binary Symmetric Channel 54
a. Type I Decoder 54
b. Type II Decoder 56
c. Summary 59
4.2 Decoding for Time-Variant Channels 59
a. Weighting Factors 60
b. Analog Circuit for Computing Weighting Factors 61
4.3 Summary 63
V. PERFORMANCE DATA FOR THRESHOLD DECODING OF
CONVOLUTIONAL CODES 64
5.1 The Binary Symmetric Channel 64
a. A Bound on Error Probability 64
b. Data for the Trial-and-Error Codes 67
c. Tolerances for the Weighting Factors and Threshold 75
d. Modifications of Threshold Decoding 78
5.2 The Binary Erasure Channel 80
5.3 The Gaussian Channel 82
5.4 Summary 85
VI. THRESHOLD DECODING OF BLOCK CODES 86
6.1 Introduction 86
6.2 Maximal-Length Codes 88
a. Number of Orthogonal Parity Checks 89
b. Complete Orthogonalization 90
6.3 Threshold-Decoding Circuits for Cyclic Codes 91
6.4 Bose-Chaudhuri (15,7) Code 95
6.5 A Sufficient Condition for Complete Orthogonalization 95
6.6 Summary 96
VII. GENERALIZED THRESHOLD DECODING FOR BLOCK CODES 97
7.1 L-Step Orthogonalization 98
7.2 First-Order Reed-Muller Codes 98
7.3 Hamming Codes 100
7.4 Bose-Chaudhuri Codes 101
7.5 Nonapplicability to Convolutional Codes 103
7.6 Summary 104
iv
CONTENTS
VIII. CONCLUSIONS AND RECOMMENDATIONS FOR FURTHER
RESEARCH 106
8.1 Summary of Research 106
8.2 Conclusions 106
8.3 Recommendations for Further Research 108
APPENDIX A Basic Definitions and Properties of Modern Algebra 110
APPENDIX B Proof of Theorem 8 (Random-Coding Bound) 111
APPENDIX C Proof of Theorems 10 and 11 114
APPENDIX D Proof of Theorem 21 118
Acknowledgement 121
References 122
v
____
GLOSSARY
Symbol Definition
A An orthogonal parity check on some noise digit, say em
i
B Sum of the received digits, r excluded, checked by A
i m i
D Delay operator
G(j)(D) Code-generating polynomial in a convolutional code
J Number of orthogonal parity checks
k Number of information symbols in a code word
k Number of information symbols per time unit in a convolutional code
o
L Number of steps required to orthogonalize a block code
m Degree of code-generating polynomials
n Number of symbols in a code word
n Number of symbols per time unit in a convolutional code
o
nA Actual constraint length of a convolutional code
nE Effective constraint length of a convolutional code
n Number of symbols checked by B
i i
p Pr[em=l], where em is the bit to be determined by decoding
0
pO Transition probability of a binary symmetric channel
Pi Probability of an odd number of errors in the bits checked by B
i
p Erasure probability of a binary erasure channel
Pl(e) Error probability in determining set of first information symbols of a
convolutional code
vi
GLOSSARY
Symbol Definition
q Number of elements in a finite field or GF(q)
A
T Threshold
w. Weighting factor
({) Binomial coefficient
LIJ Greatest integer equal to or less than I
Least integer equal to or greater than I
[I]
{Xi} Set of all elements Xi., where the index i runs over all elements in the
set
{f(D)} Polynomial obtained from f(D) by dropping all terms with power of D
greater than m.
vii
(cid:3)
__
