Table Of ContentNotes on Coding Theory
J.I.Hall
Department of Mathematics
Michigan State University
East Lansing, MI 48824 USA
9 September 2010
ii
Copyright (cid:13)c 2001-2010 Jonathan I. Hall
Preface
These notes were written over a period of years as part of an advanced under-
graduate/beginning graduate course on Algebraic Coding Theory at Michigan
State University. They were originally intended for publication as a book, but
that seems less likely now. The material here remains interesting, important,
and useful; but, given the dramatic developments in coding theory during the
last ten years, significant extension would be needed.
TheoldestsectionsareintheAppendixandareovertenyearsold,whilethe
newest are in the last two chapters and have been written within the last year.
The long time frame means that terminology and notation may vary somewhat
fromoneplacetoanotherinthenotes. (Forinstance, Z , Z , andF alldenote
p p p
a field with p elements, for p a prime.)
There is also some material that would need to be added to any published
version. This includes the graphs toward the end of Chapter 2, an index, and
in-line references. You will find on the next page a list of the reference books
thatIhavefoundmostusefulandhelpfulaswellasalistofintroductorybooks
(of varying emphasis, difficulty, and quality).
These notesare not intended forbroad distribution. If you want to usethem in
any way, please contact me.
Please feel free to contact me with any remarks, suggestions, or corrections:
[email protected]
For the near future, I will try to keep an up-to-date version on my web page:
www.math.msu.edu\~jhall
Jonathan I. Hall
3 August 2001
Thenoteswerepartiallyrevisedin2002. Anewchapteronweightenumeration
was added, and parts of the algebra appendix were changed. Some typos were
fixed,andothersmallcorrectionsweremadeintherestofthetext. Iparticularly
thank Susan Loepp and her Williams College students who went through the
iii
iv PREFACE
notes carefully and made many helpful suggestions.
I have been pleased and surprised at the interest in the notes from people who
have found them on the web. In view of this, I may at some point reconsider
publication. For now I am keeping to the above remarks that the notes are not
intended for broad distribution.
Please still contact me if you wish to use the notes. And again feel free to
contact me with remarks, suggestions, and corrections.
Jonathan I. Hall
3 January 2003
Further revision of the notes began in the spring of 2010. Over the years I
have received a great deal of positive feedback from readers around the world.
I thank everyone who has sent me corrections, remarks, and questions.
Initially this revision consists of small changes in the older notes. I plan to
add some new chapters. Also a print version of the notes is now actively under
discussion.
Please still contact me if you wish to use the notes. And again feel free to
send me remarks, suggestions, and corrections.
Jonathan I. Hall
9 September 2010
Contents
Preface iii
1 Introduction 1
1.1 Basics of communication . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 General communication systems. . . . . . . . . . . . . . . . . . . 5
1.2.1 Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.2 Encoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.3 Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.4 Received word . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2.5 Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3 Some examples of codes . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.1 Repetition codes . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.2 Parity check and sum-0 codes . . . . . . . . . . . . . . . . 11
1.3.3 The [7,4] binary Hamming code . . . . . . . . . . . . . . 11
1.3.4 An extended binary Hamming code . . . . . . . . . . . . 12
1.3.5 The [4,2] ternary Hamming code . . . . . . . . . . . . . . 13
1.3.6 A generalized Reed-Solomon code . . . . . . . . . . . . . 13
2 Sphere Packing and Shannon’s Theorem 15
2.1 Basics of block coding on the mSC . . . . . . . . . . . . . . . . . 15
2.2 Sphere packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3 Shannon’s theorem and the code region . . . . . . . . . . . . . . 22
3 Linear Codes 31
3.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 Encoding and information . . . . . . . . . . . . . . . . . . . . . . 39
3.3 Decoding linear codes . . . . . . . . . . . . . . . . . . . . . . . . 42
4 Hamming Codes 49
4.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2 Hamming codes and data compression . . . . . . . . . . . . . . . 55
4.3 First order Reed-Muller codes . . . . . . . . . . . . . . . . . . . . 56
v
vi CONTENTS
5 Generalized Reed-Solomon Codes 63
5.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.2 Decoding GRS codes . . . . . . . . . . . . . . . . . . . . . . . . . 67
6 Modifying Codes 77
6.1 Six basic techniques . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.1.1 Augmenting and expurgating . . . . . . . . . . . . . . . . 77
6.1.2 Extending and puncturing . . . . . . . . . . . . . . . . . . 78
6.1.3 Lengthening and shortening . . . . . . . . . . . . . . . . . 80
6.2 Puncturing and erasures . . . . . . . . . . . . . . . . . . . . . . . 82
6.3 Extended generalized Reed-Solomon codes . . . . . . . . . . . . . 84
7 Codes over Subfields 89
7.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
7.2 Expanded codes. . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.3 Golay codes and perfect codes . . . . . . . . . . . . . . . . . . . . 92
7.3.1 Ternary Golay codes . . . . . . . . . . . . . . . . . . . . . 92
7.3.2 Binary Golay codes . . . . . . . . . . . . . . . . . . . . . 94
7.3.3 Perfect codes . . . . . . . . . . . . . . . . . . . . . . . . . 95
7.4 Subfield subcodes . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7.5 Alternant codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
8 Cyclic Codes 101
8.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
8.2 Cyclic GRS codes and Reed-Solomon codes . . . . . . . . . . . . 109
8.3 Cylic alternant codes and BCH codes . . . . . . . . . . . . . . . 111
8.4 Cyclic Hamming codes and their relatives . . . . . . . . . . . . . 117
8.4.1 Even subcodes and error detection . . . . . . . . . . . . . 118
8.4.2 Simplex codes and pseudo-noise sequences . . . . . . . . . 120
9 Weight and Distance Enumeration 125
9.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
9.2 MacWilliams’ Theorem and performance . . . . . . . . . . . . . . 126
9.3 Delsarte’s Theorem and bounds . . . . . . . . . . . . . . . . . . . 131
9.4 Lloyd’s theorem and perfect codes . . . . . . . . . . . . . . . . . 138
9.5 Generalizations of MacWilliams’ Theorem . . . . . . . . . . . . . 149
A Some Algebra A-155
A.1 Basic Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-156
A.1.1 Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-156
A.1.2 Vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . A-160
A.1.3 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-163
A.2 Polynomial Algebra over Fields . . . . . . . . . . . . . . . . . . . A-168
A.2.1 Polynomial rings over fields . . . . . . . . . . . . . . . . . A-168
A.2.2 The division algorithm and roots . . . . . . . . . . . . . . A-171
A.2.3 Modular polynomial arithmetic . . . . . . . . . . . . . . . A-174
CONTENTS vii
A.2.4 Greatest common divisors and unique factorization . . . . A-177
A.3 Special Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-182
A.3.1 The Euclidean algorithm . . . . . . . . . . . . . . . . . . A-182
A.3.2 Finite Fields . . . . . . . . . . . . . . . . . . . . . . . . . A-188
A.3.3 Minimal Polynomials . . . . . . . . . . . . . . . . . . . . . A-194
viii CONTENTS
Chapter 1
Introduction
Claude Shannon’s 1948 paper “A Mathematical Theory of Communication”
gave birth to the twin disciplines of information theory and coding theory. The
basicgoalisefficientandreliablecommunicationinanuncooperative(andpos-
sibly hostile) environment. To be efficient, the transfer of information must not
require a prohibitive amount of time and effort. To be reliable, the received
datastreammustresemblethetransmittedstreamtowithinnarrowtolerances.
These two desires will always be at odds, and our fundamental problem is to
reconcile them as best we can.
At an early stage the mathematical study of such questions broke into the
two broad areas. Information theory is the study of achievable bounds for com-
munication and is largely probabilistic and analytic in nature. Coding theory
then attempts to realize the promise of these bounds by models which are con-
structed through mainly algebraic means. Shannon was primarily interested in
theinformationtheory. Shannon’scolleagueRichardHamminghadbeenlabor-
ing on error-correction for early computers even before Shannon’s 1948 paper,
and he made some of the first breakthroughs of coding theory.
Although we shall discuss these areas as mathematical subjects, it must
always be remembered that the primary motivation for such work comes from
its practical engineering applications. Mathematical beauty can not be our sole
gauge of worth. Here we shall concentrate on the algebra of coding theory,
but we keep in mind the fundamental bounds of information theory and the
practical desires of engineering.
1.1 Basics of communication
Information passes from a source to a sink via a conduit or channel. In our
view of communication we are allowed to choose exactly the way information is
structuredatthesourceandthewayitishandledatthesink,butthebehaviour
of the channel is not in general under our control. The unreliable channel may
take many forms. We may communicate through space, such as talking across
1
2 CHAPTER 1. INTRODUCTION
a noisy room, or through time, such as writing a book to be read many years
later. Theuncertaintiesofthechannel,whateveritis,allowthepossibilitythat
the information will be damaged or distorted in passage. My conversation may
be drowned out or my manuscript might weather.
Ofcourseinmanysituationsyoucanaskmetorepeatanyinformationthat
you have not understood. This is possible if we are having a conversation (al-
though not if you are reading my manuscript), but in any case this is not a
particularly efficient use of time. (“What did you say?” “What?”) Instead to
guarantee that the original information can be recovered from a version that is
nottoobadlycorrupted,weaddredundancytoourmessageatthesource. Lan-
guages are sufficiently repetitive that we can recover from imperfect reception.
When I lecture there may be noise in the hallway, or you might be unfamiliar
with a word I use, or my accent could confuse you. Nevertheless you have a
goodchanceoffiguringoutwhatImeanfromthecontext. Indeedthelanguage
has so much natural redundancy that a large portion of a message can be lost
withoutrenderingtheresultunintelligible. Whensittinginthesubway, youare
likely to see overhead and comprehend that “IF U CN RD THS U CN GT A
JB.”
Communication across space has taken various sophisticated forms in which
codinghasbeenusedsuccessfully. IndeedShannon,Hamming,andmanyofthe
other originators of mathematical communication theory worked for Bell Tele-
phoneLaboratories. Theywerespecificallyinterestedindealingwitherrorsthat
occur as messages pass across long telephone lines and are corrupted by such
things as lightening and crosstalk. The transmission and reception capabilities
of many modems are increased by error handling capability embedded in their
hardware. Deep space communication is subject to many outside problems like
atmosphericconditionsandsunspotactivity. Foryearsdatafromspacemissions
hasbeencodedfortransmission,sincetheretransmissionofdatareceivedfault-
ily would be very inefficient use of valuable time. A recent interesting case of
deep space coding occurred with the Galileo mission. The main antenna failed
to work, so the possible data transmission rate dropped to only a fraction of
what was planned. The scientists at JPL reprogrammed the onboard computer
to domore code processingof the databefore transmission, and sowereable to
recover some of the overall efficiency lost because of the hardware malfunction.
It is also important to protect communication across time from inaccura-
cies. Data stored in computer banks or on tapes is subject to the intrusion
of gamma rays and magnetic interference. Personal computers are exposed to
much battering, so often their hard disks are equipped with “cyclic redundancy
checking” CRC to combat error. Computer companies like IBM have devoted
much energy and money to the study and implementation of error correcting
techniques for data storage on various mediums. Electronics firms too need
correction techniques. When Phillips introduced compact disc technology, they
wanted the information stored on the disc face to be immune to many types of
damage. If you scratch a disc, it should still play without any audible change.
(But you probably should not try this with your favorite disc; a really bad
scratchcancauseproblems.) Recentlythesoundtracksofmovies,pronetofilm
