Anytime Channel Coding with Feedback by Hidayet Tunc Simsek BS

94 Pages·2004·2.18 MB·English

Anytime Channel Coding with Feedback
by
Hidayet Tunc Simsek
B.S. (Middle East Technical University, Ankara, Turkey) 1996
M.S. (University of California, Berkeley) 2000
A dissertation submitted in partial satisfaction of the
requirements for the degree of
Doctor of Philosophy
in
Engineering – Electrical Engineering
and Computer Sciences
in the
GRADUATE DIVISION
of the
UNIVERSITY OF CALIFORNIA, BERKELEY
Committee in charge:
Professor Pravin P. Varaiya, Chair
Professor Anant Sahai
Professor John K. Hedrick
Fall 2004
The dissertation of Hidayet Tunc Simsek is approved:
Chair Date
Date
Date
University of California, Berkeley
Fall 2004
Anytime Channel Coding with Feedback
Copyright 2004
by
Hidayet Tunc Simsek
1
Abstract
Anytime Channel Coding with Feedback
by
Hidayet Tunc Simsek
Doctor of Philosophy in Engineering – Electrical Engineering
and Computer Sciences
University of California, Berkeley
Professor Pravin P. Varaiya, Chair
We build on the Anytime information theory of Anant Sahai. Within that broad range we focus on the
Anytime channel coding problem for discrete-time memoryless channels with noiseless feedback. Our
main result is an upper bound on the exponent of anytime coding. For the special case of a Binary
ErasureChannelwithfeedbacktheresultistightor, inotherwords, theupperboundisachievable. The
upper bound turns out to have some interesting properties. For any discrete-time memoryless channel
the bound is a concave function, it is strictly positive for all rates up to capacity and identically zero
at capacity. Furthermore, for symmetric channels, the bound is expressed parametrically in terms of
Gallager’s exponent which is generally denoted as E . The discussion occupies Chapters 1-3.
0
In Chapter 4, we specialize to the Binary Symmetric Channel and design a family of ‘time-
sharing’ codes that achieve an exponent close to the upper bound. In particular, the achievable region
beats the sphere-packing bound which was thought to be a hard bound on performance with fixed
delays [21]. This shows that the sphere-packing bound is misleading when it comes to the opportunities
provided by feedback.
The two additional chapters may be read independently of the former material. In Chapter 5
we consider the problem of anytime relaying. We are interested in this problem because it represents
the simplest instance of a more general problem of real-time data processing. The relaying node must
be able to decode, interpret and re-encode the incoming data in a streaming fashion. We are unable to
treattherelayingproblemingeneralandthereforerestrictourattentiontothe specialcase ofaone-hop
relay in which the link from the source node to the relaying node is an Erasure Channel with feedback.
In this case we are able to give a precise characterization of the anytime capacity of the end-to-end
relay.
In the final chapter we consider an infinite state Markovian jump system. The material pre-
sentedinthischapterunfoldedduringoneofourmanyattemptstomodelouranytimecodes. Wepresent
stability criteria for jump systems. Our results may be viewed as a simpler and finite alternative to the
traditional criteria which requires the solution of an infinite set of balance equations.
Professor Pravin P. Varaiya
Dissertation Committee Chair
2
i
To my mother, father and brother.
ii
Contents
List of Figures iv
1 Introduction 1
1.1 Summary of main results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 List of notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.1 Conventional symbols used in equations and derivations . . . . . . . . . . . . . . 6
1.2.2 Symbols used to describe channels . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.3 Notation relating to fixed-length and variable-length block-coding . . . . . . . . 7
1.2.4 Notations specific to Anytime coding . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.5 List of reliability functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 The role of feedback in traditional communication 9
2.1 The reliability function for block coding . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 The reliability function for variable-length block coding . . . . . . . . . . . . . . . . . . 11
2.3 An asymptotically optimal variable-length block coding technique for the Binary Sym-
metric Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.1 The more general case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4.1 Schalkwijk’s one-up J-down method . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4.2 Limited feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3 Anytime channel coding with feedback 23
3.1 Problem set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 The random coding bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3 A time-sharing upper bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3.1 A parametric representation for symmetric channels . . . . . . . . . . . . . . . . 28
3.3.2 The bound is tight for the Binary Erasure Channel with feedback. . . . . . . . . 30
3.4 Anytime coding and estimation of unstable systems. . . . . . . . . . . . . . . . . . . . . 31
3.5 Discussion: Tree codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4 A time-sharing anytime code 38
4.1 The estimation problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2 The best estimator for the perfect channel . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.3 A decent estimator for a non-perfect channel . . . . . . . . . . . . . . . . . . . . . . . . 40
4.3.1 The decoder. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.3.2 The encoder. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
iii
4.3.3 Mathematical model of encoder-decoder . . . . . . . . . . . . . . . . . . . . . . . 42
4.3.4 The formulas are tight for the Binary Erasure Channel . . . . . . . . . . . . . . . 48
4.3.5 The formulas are decent for a good Binary Symmetric Channel . . . . . . . . . . 49
4.4 Improving the estimator for a non-perfect channel . . . . . . . . . . . . . . . . . . . . . 50
4.4.1 The inner encoder-decoder: creating the virtual channel . . . . . . . . . . . . . . 50
4.4.2 The outer decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.4.3 The outer encoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.4.4 Mathematical model of the modified system . . . . . . . . . . . . . . . . . . . . . 54
4.4.5 Performance for a noisy Binary Symmetric Channel . . . . . . . . . . . . . . . . 57
4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5 Regenerative relaying 60
5.1 Producing an estimate at the relay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.2 Relaying the estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.3 Analysis of the queuing system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6 Recurrence and stability in infinite Markovian jump systems 67
6.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.2 Main results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7 Future work: Noisy feedback 78
Bibliography 81
iv
List of Figures
1.1 The dynamic programming framework for tracking a random walk. . . . . . . . . . . . . 4
1.2 The anytime framework for tracking a random walk. . . . . . . . . . . . . . . . . . . . . 4
1.3 Plot of the Anytime upper bound E+(R) as a function of the transmission rate R for a
a
Binary Erasure Channel with erasure probability ε = 1/4. The curve is described as the
greatest lower bound for a family of curves parameterized by 0 ≤ λ ≤ 1. . . . . . . . . . 5
1.4 The Anytime exponent for various N-up J-down codes for a Binary Symmetric Channel
with crossover probability (cid:15) = 1/4. The curve E+(R) is the Anytime upper bound and
a
E (R) is the sphere-packing bound. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
sp
2.1 The reliability function of a Binary Symmetric Channel ((cid:15) = .1) with feedback. . . . . . 13
2.2 Illustration of one block of transmission for the asymptotically optimal encoder-decoder. 14
2.3 The achievable reliability region for a Binary Erasure Channel with erasure probability
0 ≤ e ≤ 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4 Illustration of 1-up J-down method for J = 2. . . . . . . . . . . . . . . . . . . . . . . . . 20
3.1 A block diagram illustrating the set-up of anytime coding with feedback. . . . . . . . . . 24
3.2 Sketch of E (ρ).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
0
3.3 Plot of E+(R) vs. R for a Binary Erasure Channel with erasure probability ε = 1/4. . . 32
a
3.4 A block diagram illustrating the construction of an anytime code from a state estimator. 35
3.5 A rate R = 1/3 Tree-code with two branches, each labeled with q = 3 channel symbols.
The source bits s , n = 0,1,2,··· are indicated below branches and the channel symbols
n
are indicated above the branches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.1 Illustration of three possible cases in the transmission of the control and information parts. 43
(N)
4.2 Anytime exponent E , N = 2,3,··· ,11 of time-sharing code for a Binary Symmetric
a
Channel with cross-over probability (cid:15) = 0.01 and no forward error protection. . . . . . . 51
4.3 The improved anytime exponent for a noisy Binary Symmetric Channel with cross-over
probability (cid:15) = 0.25. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.1 An abstract relaying system. The L-bit erasure channel is a discrete-time memoryless
channel described by P(b = a | a = a) = 1−ε and P(b = ∅ | a = a) = ε for all
n n n n
a ∈ {1,2,··· ,2L}. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.2 Illustration of the series queuing system for the source and relay. . . . . . . . . . . . . . 66
v
6.1 A simple infinite Markov jump linear system. The state θ(k) of the chain takes values
in S = {(0,0),(1,0),(2,0),(2,1),(3,0),(3,1),(3,2),···}. The transition probabilities are
q = Cλ−i, i ≥ 0, λ > 16/3 and C = 1/P∞ λ−i. The coefficient process is A = 1/2
i i=0 (0,0)
and for all 1 ≤ j +1 ≤ i, A = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
(i,j)
7.1 ThenoisyfeedbackexponentE andtheSphere-packingboundE . Theforwardchannel
f sp
is a Binary Symmetric Channel (BSC) with cross-over probability .1 and the reverse
channel is a BSC with cross-over probability 10−6. . . . . . . . . . . . . . . . . . . . . . 80