On the Capacity of Finite State Channels and the Analysis of Convolutional Accumulate-m Codes

203 Pages·2003·0.99 MB·English

UNIVERSITY OFCALIFORNIA, SAN DIEGO
Onthe Capacity ofFiniteStateChannels and
the AnalysisofConvolutional Accumulate-m Codes
A dissertationsubmittedinpartial satisfactionofthe
requirementsforthedegree
DoctorofPhilosophy
in
Electrical Engineering
(CommunicationsTheory andSystems)
by
Henry D.Pfister
Committeeincharge:
ProfessorPaulH.Siegel,Chair
ProfessorEdwardBender
ProfessorAlonOrlitsky
ProfessorRuthWilliams
ProfessorJackWolf
2003
Copyright
Henry D. Pfister, 2003
Allrightsreserved.
ThedissertationofHenry D. Pfisteris approved,and
itisacceptableinqualityandformforpublicationon
microfilm:
Chair
UniversityofCalifornia,San Diego
2003
iii
CONTENTS
SignaturePage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
ListofFigures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
ListofTables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
VitaandPublications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
AbstractoftheDissertation . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Chapter1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Layman’sSummary . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 OutlineofDissertation . . . . . . . . . . . . . . . . . . . . . . . . 3
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Chapter2 TheSerialConcatenation ofRate-1CodesThroughUniformInterleavers . . 6
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 WeightEnumeratorsandSerialConcatenation . . . . . . . . . . . . 8
2.2.1 TheUnionBound . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 SerialConcatenation through aUniformInterleaver . . . 8
2.2.3 ASimpleExample-TheAccumulateInnerCode . . . . . 9
2.3 MultipleRate-1SerialConcatenations . . . . . . . . . . . . . . . . 10
2.3.1 TheInputOutputWeightEnumerator . . . . . . . . . . . 10
2.3.2 ALargeNumberofConcatenations . . . . . . . . . . . . 14
2.4 Properties ofRate-1Codes . . . . . . . . . . . . . . . . . . . . . . 17
2.4.1 Conditions forPrimitivity . . . . . . . . . . . . . . . . . 17
2.4.2 Recursivevs. Non-RecursiveRate-1/1CCs . . . . . . . . 19
2.5 BoundsontheMinimumDistance . . . . . . . . . . . . . . . . . . 20
2.5.1 TheMinimumDistanceDistribution . . . . . . . . . . . 20
2.5.2 Convolutional Accumulate-m (CAm)Codes . . . . . . . 23
2.5.3 Expurgated Ensembles . . . . . . . . . . . . . . . . . . . 25
2.6 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.6.1 TheErrorExponent . . . . . . . . . . . . . . . . . . . . 26
2.6.2 MaximumLikelihood DecodingPerformance . . . . . . . 28
2.6.3 IterativeDecodingPerformance . . . . . . . . . . . . . . 29
2.7 Conclusions andFutureWork . . . . . . . . . . . . . . . . . . . . . 30
Appendix 2A ProofofTheorem2.4.1 . . . . . . . . . . . . . . . . . . 37
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
iv
Chapter3 CodingTheoremsforConvolutional Accumulate-m Codes . . . . . . . . . 42
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2.1 WeightEnumerators andtheUnionBound . . . . . . . . 43
3.2.2 SerialConcatenation through aUniformInterleaver . . . 44
3.2.3 CodeEnsemblesandSpectralShape . . . . . . . . . . . 45
3.2.4 AsymptoticOrderofFunctions . . . . . . . . . . . . . . 47
3.2.5 TheIGEConjecture . . . . . . . . . . . . . . . . . . . . 48
3.2.6 NoiseThresholds . . . . . . . . . . . . . . . . . . . . . . 48
3.2.7 TypicalSetDecoding Bound . . . . . . . . . . . . . . . . 49
3.3 TerminatedConvolutional Codes . . . . . . . . . . . . . . . . . . . 53
3.3.1 Analytical Bounds . . . . . . . . . . . . . . . . . . . . . 53
3.3.2 Analytical BoundExamples . . . . . . . . . . . . . . . . 55
3.3.3 TheExactSpectralShape . . . . . . . . . . . . . . . . . 57
3.4 TheAccumulateCode . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.4.1 ASimpleBoundontheIOWTP . . . . . . . . . . . . . . 59
3.4.2 AnExponentially TightBoundontheIOWTP . . . . . . 60
3.4.3 ASimpleBoundontheCumulativeIOWTP . . . . . . . 61
3.5 SingleAccumulateCodes . . . . . . . . . . . . . . . . . . . . . . . 62
3.5.1 RepeatAccumulateCodes . . . . . . . . . . . . . . . . . 62
3.5.2 Convolutional AccumulateCodes . . . . . . . . . . . . . 63
3.5.3 Properties oftheBounds . . . . . . . . . . . . . . . . . . 64
3.6 Convolutional Accumulate-m Codes . . . . . . . . . . . . . . . . . 66
3.6.1 Description . . . . . . . . . . . . . . . . . . . . . . . . 66
3.6.2 TheIGEConjectureforCAm Codes . . . . . . . . . . . 66
3.6.3 TheWorstCaseMinimumDistance . . . . . . . . . . . . 68
3.6.4 WeightEnumeratorBound . . . . . . . . . . . . . . . . . 68
3.6.5 TheMainTheorem . . . . . . . . . . . . . . . . . . . . . 69
3.6.6 TheExactSpectralShape . . . . . . . . . . . . . . . . . 73
3.6.7 TheTypicalMinimumDistance . . . . . . . . . . . . . . 74
3.7 IterativeDecoding ofCAm Codes . . . . . . . . . . . . . . . . . . 77
3.7.1 DecodingGraphs . . . . . . . . . . . . . . . . . . . . . . 77
3.7.2 MessagePassingRules . . . . . . . . . . . . . . . . . . . 78
3.7.3 MessagePassingSchedule . . . . . . . . . . . . . . . . 80
3.7.4 DensityEvolution . . . . . . . . . . . . . . . . . . . . . 81
3.8 NumericalMethodsfortheSpectralShape . . . . . . . . . . . . . . 84
3.8.1 TheQuantized SpectralShape . . . . . . . . . . . . . . . 84
3.8.2 NoiseThresholds . . . . . . . . . . . . . . . . . . . . . . 85
3.8.3 MinimumDistanceRatio . . . . . . . . . . . . . . . . . 87
3.9 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 87
Appendix 3A BinomialCoefficientBounds . . . . . . . . . . . . . . . 89
Appendix 3B Convolutional CodeBounds . . . . . . . . . . . . . . . . 91
Appendix 3C “Accumulate”CodeBounds . . . . . . . . . . . . . . . . 98
v
Appendix 3D ProofofCAm CodeBounds . . . . . . . . . . . . . . . . 103
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Chapter4 TheCapacityofFiniteStateChannels . . . . . . . . . . . . . . . . . . . . . 110
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.2 ChannelModels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.2.1 Discrete-TimeLinearFilterChannelswithAWGN . . . . 113
4.2.2 TheDicodeErasureChannel . . . . . . . . . . . . . . . . 113
4.2.3 TheFiniteStateZ-Channel . . . . . . . . . . . . . . . . 114
4.3 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.3.1 TheIndecomposable FiniteStateChannel . . . . . . . . . 115
4.3.2 TheMarkovInputProcess . . . . . . . . . . . . . . . . . 116
4.3.3 CombiningtheInputProcessandtheFiniteStateChannel 116
4.3.4 TheFiniteStateProcess . . . . . . . . . . . . . . . . . . 117
4.4 TheEntropyRateofaFiniteStateProcess . . . . . . . . . . . . . . 118
4.4.1 ASimpleMonteCarloMethod . . . . . . . . . . . . . . 119
4.4.2 TheStatistical MomentsofEntropy . . . . . . . . . . . . 121
4.4.3 AMatrixPerspective . . . . . . . . . . . . . . . . . . . . 123
4.4.4 TheAnalytical Approach . . . . . . . . . . . . . . . . . 124
4.4.5 EntropyRateBounds . . . . . . . . . . . . . . . . . . . 129
4.4.6 Connections withLyapunovExponents . . . . . . . . . . 130
4.5 CapacityBounds . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.5.1 LowerBounds . . . . . . . . . . . . . . . . . . . . . . . 133
4.5.2 TheVontobel-Arnold UpperBound . . . . . . . . . . . . 134
4.5.3 AConjectured UpperBound . . . . . . . . . . . . . . . . 136
4.6 MonteCarloResults . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.6.1 PartialResponseChannels . . . . . . . . . . . . . . . . . 138
4.6.2 TheFiniteStateZ-Channel . . . . . . . . . . . . . . . . 140
4.7 Analytical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.7.1 TheSymmetricInformation RateoftheDEC . . . . . . . 141
4.7.2 TheMarkov-1RateoftheDEC . . . . . . . . . . . . . . 144
4.7.3 DensityEvolutionforFiniteStateChannels . . . . . . . . 148
4.8 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 151
Appendix 4A FormalChannelDefinitions . . . . . . . . . . . . . . . . 151
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Chapter5 JointIterativeDecodingofLDPCCodesandChannelswithMemory . . . . 156
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
5.2 SystemModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
5.2.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . 158
5.2.2 TheGeneralized ErasureChannel . . . . . . . . . . . . . 158
5.2.3 TheDicodeErasureChannel . . . . . . . . . . . . . . . . 160
5.2.4 Irregular LDPCCodes . . . . . . . . . . . . . . . . . . . 160
vi
5.3 AnalysisofJointIterativeDecoding . . . . . . . . . . . . . . . . . 162
5.3.1 SingleParameterRecursion . . . . . . . . . . . . . . . . 162
5.3.2 Conditions forConvergence . . . . . . . . . . . . . . . . 163
5.3.3 AchievingtheSymmetricInformation Rate . . . . . . . . 166
5.3.4 DegreeSequences withRegularCheckDistributions . . . 170
5.4 ResultsforArbitraryGECs . . . . . . . . . . . . . . . . . . . . . . 173
5.4.1 TheExistence ofArbitraryGECs . . . . . . . . . . . . . 173
5.4.2 NumericalOptimizationviaLinearProgramming . . . . . 175
5.4.3 AStabilityConditionforGeneralChannels . . . . . . . . 178
5.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 178
Appendix 5A ExactAnalysisoftheBCJRDecoderfortheDEC . . . . 179
Appendix 5B JointIterativeDecodingDEforGeneralChannels . . . . 183
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
vii
LISTOFFIGURES
2.1.1 Oursystemconsistsofanyrater < 1codefollowedbymrate-1codes. . . . . 7
2.5.1 EncoderforaCAm Codewiththeblocksizeindicated ateachstage. . . . . . . 20
2.5.2 Probabilistic boundontheminimumdistanceofvariousCAm codes. . . . . . . 24
2.7.1 Analytical and simulation results for a rate 1/2 RAm code with k = 1024 and
m =1;2;3. Simulationsarecompletedusing50decodingiterationsandthetop
plot shows theword error rate (WER)whilethe bottom plot showsthe bit error
rate (BER).Thelabel XVVsignifies the Viterbi-Viterbi (VV)Bound applied to
theexpurgated ensembles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.7.2 Analytical and simulation results for a rate 1/4 RAm code with k = 1024 and
m =1;2;3. Simulationsarecompletedusing50decodingiterationsandthetop
plot shows theword error rate (WER)whilethe bottom plot showsthe bit error
rate (BER).Thelabel XVVsignifies the Viterbi-Viterbi (VV)Bound applied to
theexpurgated ensembles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.7.3 Analytical and simulation results for a rate 8/9 PAm with k = 1024 and m =
1;2;3. Simulations are completed using 50 decoding iterations andthe top plot
shows the word error rate (WER)while the bottom plot shows the biterror rate
(BER). The label XVV signifies the Viterbi-Viterbi (VV) Bound applied to the
expurgated ensembles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.7.4 Simulation results forrate 1/2 RAm codes for 30 decoding iterations withm =
1;2 and k = 1024;2048;4096;8192;16384. Thetopplot showstheword error
rate(WER)whilethebottom plotshowsthebiterrorrate(BER). . . . . . . . . 35
2
2.7.5 Simulationresultsforrate8/9PA codeswithk = 1024;2048;4096;8192;16384
and20decodingiterations. Thetopplotshowstheworderrorrate(WER)while
thebottom plotshowsthebiterrorrate(BER). . . . . . . . . . . . . . . . . . . 36
2A.1 Theweightmappinggraph,G4,withtheG3 subgraph drawninsolidlines. . . 39
3.3.1 ThetrueWEandupperbounds fortheHamming(7,3)code. . . . . . . . . . . 56
3.3.2 ThetrueWEandupperbounds forthesingleparitycheck(9,8)CC. . . . . . . 57
3.3.3 ThetrueWEandupperbounds fortheG(D) = [1;1+D]CC. . . . . . . . . . 58
3.5.1 Anupperboundonthespectral shapeofRAcodes. . . . . . . . . . . . . . . . 63
3.7.1 AGTWgraphfortherate-1“accumulate” code. . . . . . . . . . . . . . . . . . 78
2
3.7.2 ATannergraphforanarbitraryCA code. . . . . . . . . . . . . . . . . . . . . 80
3.8.1 TypicalsetdecodingEb=N0 thresholds forRAm andPAm codesinAWGN. . . 84
3.8.2 The spectral shape of a (2,1) single parity code and the associated PAm codes
withm = 1;2;3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.8.3 Thespectral shapeofa[1;1+D]CCandtheassociated CAm codeswithm =
1;2;3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.2.1 Thestatetransition diagram ofthedicodechannel. . . . . . . . . . . . . . . . 114
viii
4.2.2 The state transition diagram of the finite state Z-channel. The symbol B(p)
refers toabinary random variable whichequals 1withprobability pand 0with
probability 1−p. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.3.1 The state diagram of atwostate input process which sends a1 withprobability
pfromthe0stateandwithprobability q fromthe1state. . . . . . . . . . . . . 116
4.3.2 The combined state diagram for a general two state Markov input process and
thedicodechannel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
7
4.6.1 TheSIRforvarious partialresponsechannels, estimatedwithn = 10 . . . . . 139
4.6.2 MonteCarlolowerboundsontheachievableinformationrateofthedicodechan-
nelusingoptimizedMarkovinputdistributions. . . . . . . . . . . . . . . . . . 140
4.6.3 MonteCarlolowerboundsontheachievableinformationrateoftheEPR4chan-
nelusingoptimizedMarkovinputdistributions. . . . . . . . . . . . . . . . . . 141
4.6.4 Monte Carlo upper and lower bounds on the achievable information rate of the
dicode channelusingoptimizedMarkovinputdistributions. . . . . . . . . . . . 142
4.6.5 TheSIRandtheMarkov-1rateofthefinitestateZ-channel. . . . . . . . . . . 143
4.7.1 TheSIRandMarkov-1rateoftheDECcomparedwiththecapacityofthebinary
erasure channel(BEC)andtheternaryerasurechannel (TEC). . . . . . . . . . 148
5.1.1 Blockdiagram ofthesystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
5.2.1 Gallager-Tanner-Wiberg graphofthejointiterativedecoder. . . . . . . . . . . 158
5.2.2 Thestatediagram ofthedicodechannelwithandwithoutprecoding. . . . . . . 161
5.3.1 Thisshowsthebitdegreedistributions(cid:21)(k)(x)resultingfromconstructingcheck
regular codes for the precoded DEC with (cid:15) = 1=2. The vertical axis of each
subplot isscaleddifferently tohighlight theirsimilarity. . . . . . . . . . . . . . 171
5.4.1 Theresultsofapproximating thenon-precoded DECwith(cid:15) = 0:5andn = 10. 174
ix
LISTOFTABLES
2.1 Input-output sequences andweightmappingsforn = 3“accumulate” code. . . 10
3.1 Numerical results for various CAm codes. (C = outer code, R = code rate, γ(cid:3)=
(cid:3)
Shannon limit, γ = typical set decoding threshold with m accumulates, (cid:14) =
m GV
(cid:3)
Gilbert-Varshamov bound, (cid:14) =normalized distance threshold with maccumu-
m
lates, and(cid:11) =density evolution threshold withmaccumulates) . . . . . . . . 88
m
4.1 Thetransferfunction andnormalizedresponse ofafewpartialresponse targets. 112
5.1 Codeconstruction resultsfortheprecoded DECwith(cid:15) = 1=2. . . . . . . . . . 172
5.2 Codeconstruction resultsfortheDECwith(cid:15) = 0:85andnoprecoding. . . . . 173
x