Binary Causal-Adversary Channels

0.17 MB·

by  Michael Langberg

Binary Causal-Adversary Channels
M. Langberg S. Jaggi B. K. Dey
Computer Science Division Department of Information Engineering Department of Electrical Engineering
Open University of Israel Chinese University of Hong Kong Indian Institute of Technology Bombay
Raanana 43107, Israel Shatin, N.T., Hong Kong Mumbai, India, 400 076
[email protected] [email protected] [email protected]
Abstract—In this work we consider the communication of
1
information in the presence of a causal adversarial jammer.
In the setting under study, a sender wishes to communicate 0.9 1−4p
a message to a receiver by transmitting a codeword x =
9 (x1,...,xn) bit-by-bit over a communication channel. The 0.8
0 adversarial jammer can view the transmitted bits xi one at 0.7
0 a time, and can change up to a p-fraction of them. However, McEliece bound
2 the decisions of the jammer must be made in an online or 0.6
n causal manner. Namely, for each bit xi the jammer’s decision C0.5
onwhethertocorruptitornot(andonhow tochangeit)must
Ja dependonlyonxj forj ≤i.Thisisincontrasttothe“classical” 0.4
adversarialjammerwhichmaybaseitsdecisionsonitscomplete
8 knowledge of x. We present a non-trivial upper bound on the 0.3 1−H(p)
2 amount of information that can be communicated. We show 0.2
that the achievable rate can be asymptotically no greater than
] min{1−H(p),(1−4p)+}. Here H(.) is the binary entropy 0.1
T function, and (1−4p)+ equals 1−4p for p ≤ 0.25, and 0 0Gilbert−Varshamov bound
I otherwise. 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
. p
s
c I. INTRODUCTION
[ Fig. 1. Bounds on capacity of the adversarial channel. The bold line in
Consider the following adversarial communication sce- purpleisourupperboundofmin{1−H(p),(1−4p)+}.
2 nario. A sender Alice wishes to transmit a message u to a
v
receiver Bob. To do so, Alice encodes u into a codeword eachbitx ,weassumethatCalvindecideswhethertochange
3 i
5 x and transmits it over a binary channel. The codeword it or not (and if so, how to change it) based on the bits xj,
8 x = x1,...,xn is a binary vector of length n. However, for j ≤i alone,i.e., the bits thathe hasalready observed.In
1 Calvin, a malicious adversary,can observe x and corruptup this case we refer to Calvin as a causal adversary.
1. to a p-fraction of the n transmitted bits, i.e., pn bits.
Causal adversaries arise naturally in practical settings,
0 In the classical adversarial channel model, e.g., [4], it is
where adversaries typically have no a priori knowledge of
9 usuallyassumed thatCalvin has fullknowledgeof the entire
0 Alice’s message u. In such cases they must simultaneously
codewordx, and basedon this knowledge(togetherwith the
: learn u based on Alice’s transmissions, and jam the corre-
v knowledgeof the code shared byAlice and Bob) Calvin can
spondingcodewordxaccordingly.Thiscausalityassumption
i maliciously plan what error to impose on x. We refer to
X is reasonable for many communicationchannels, both wired
such an adversary as an omniscient adversary. For binary
r and wireless, where Calvin is not co-located with Alice. For
a channels, the optimal rate of communicationin the presence
example consider the scenario in which the transmission of
of an omniscient adversary has been an open problem in
x = x ,...,x is done during n channel uses over time,
classical coding theory for several decades. The best known 1 n
where at time i the bit x is transmitted over the channel.
lower bound is given by the Gilbert-Varshamov bound [10], i
Calvin can only corrupt a bit when it is transmitted (and
[18],whichimpliesthatAlicecantransmitatrate1−H(2p)
thus its error is based on its view so far). To decode the
to Bob. Conversely, the tightest upper bound was given by
transmittedmessage,Bobwaitsuntilallthebitshavearrived.
McEliece et al. [12], and has a positive gap from the lower
Asintheomniscientmodel,Calvinisrestrictedinthenumber
bound for all p∈(0,1/4) (see Fig. 1).
of bits pn he can corrupt. This might be because of limited
In this work we initiate the analysis of coding schemes
processing power or limited transmit energy.
thatallowcommunicationagainstcertainadversariesthatare
weaker than the omniscient adversary. We consider adver- Recently, the problem of codes against causal adversaries
saries that behave in a causal or online manner. Namely, for was considered and solved by the authors [6] for large-
q channels, i.e., channels where Alice’s codeword x =
0TheworkofB.K.Deywassupported byBharti Centre forCommuni- x ,...,x is considered to be a vector of length n over a
1 n
cation in IIT Bombay, that of M. Langberg was supported in part by ISF
field of “large”size q. Each symbolx mayrepresenta large
grant 480/08, and that of S. Jaggi was partially supported by MS-CU-JL i
grants. packet of bits in practice. Calvin is allowed to arbitrarily
corrupt a p-fraction of the symbols, rather than bits. A tight codewordrandomvariableX(u) with codeworddistribution
characterization of the rate-region for various scenarios is p over X(u). For any value U = u of the message,
X(u)
givenin[6],andcomputationallyefficientcodesthatachieve Alice’s encoder choses a codeword from X(u) randomly
these rate-regions are presented. However, the techniques fromthedistributionp . Alice’smessagedistributionp ,
X(u) U
used in characterizing the rate-region of causal adversaries codebookX,andallthecodebookdistributionsp areall
X(u)
over large-q channels do not work over binary channels. knowntobothBobandCalvin,butthevaluesoftherandom
Thisisbecauseeachsymbolinalarge-qchannelcancontain variables U and X(.) are unknown to them. If X(u) =
withinita“small”hashthatcanbeusedtoverifythesymbol. {x(u,r):r∈Λ },thenthetransmittedcodewordX(U)has
u
This is the crux of the technique used to achieve the lower the probability distribution given by Pr[X(U) =x(u,r)] =
boundsin [6]. We currently do not know how to extend this p (u)p (x(u,r)). Let p be the overall distribution of
U X(u)
method to binary channels. Conversely, for upper bounds, codewords x = x(u,r) of Alice. It holds that p(x(u,r)) =
the geometryof the space of length-ncodewordsoverlarge- p (u)p (x) and p(x)= p (u)p (x).
U X(u) U U X(u)
q alphabets is significantly different than that corresponding Calvin/Channel:CalvinpossPessesnjammingfunctionsg (.)
i
to binary alphabets. For instance, for large-q channels the andn arbitraryjammingrandomvariablesJ thatsatisfy the
i
volume of an n-sphere of radius αn (0≤α≤1) over Fq is following constraints.
∼qnα, This leads to simpler bounds for large-q channels.
Causality constraint:For each i∈[n], the jammingfunction
In this work we initiate the study of binary causal- g (.) maps xi = (x ,...,x ) and Ji = (J ,...,J ) to an
i 1 i 1 i
adversary channels, and present two upper bounds on their
element of {0,1}.
capacity: 1 − H(p), and (1 − 4p)+. The upper bound of
Power constraint: The number of indices i ∈ [n] for which
1−H(p) is very “natural”. Namely, it is not hard to verify
the value of g (.) equals 1 is at most pn. That is, for all
i
that if Calvin attacks Alice’s transmission by simulating the
xn,Jn, g (xi,Ji)≤pn.
well-studied Binary Symmetric Channel [4], he can force a i i
TheoutpPutofthechannelisthesetofbitsy =x ⊕g (xi,Ji)
communication rate of no more than 1−H(p). The upper i i i
for i=1,...,n.
bound of (1−4p)+ presented in this work is non-trivial for
Bob: Bob’s decoder is a (potentially) probabilistic function
both its implications and its proof techniques. The bound
h(.) of the received vector y. It maps the vectors y =
demonstratesthatatleastforsomevaluesofp,theachievable
(y ,...y ) in {0,1}n to the messages in U.
rate is bounded away from 1 − H(p). For p ∈ (p ,0.5), 1 n
0
1−4p is strictly less than 1−H(p) (here p is the value Code parameters: Bob is said to make a decoding error
0
of p satisfying H(p) = 4p, and can be computed to be if the message u′ he decodes differs from the message
approximately 0.15642...). In fact for p ∈ (0.25,0.5) our u encoded by Alice. The probability of error for a given
bound implies that no communication at positive rate is message u is defined as the probability, over Alice, Calvin
possible, which is much stronger than the result obtained and Bob’s random variables, that Bob makes a decoding
by the upper bound of 1 − H(p) (see Fig. 1). Our proof error. The probability of error of the code C is defined as
techniques include a combination of tools from the fields the average over all u ∈ U of the probability of error for
of Extremal Combinatorics (e.g. Tura´n’s theorem [17]), and message u.
classical Coding Theory (e.g. the Plotkin bound [14], [2]). Wedefinetwotypesofratesandcorrespondingcapacities.
The rate R is said to be weakly achievable if for every
II. MODEL
ε > 0, δ > 0 and every sufficiently large n there exists an
Foranyintegerilet[i]denotetheset{1,...,i}.LetR≥0 (n,(R−δ)n)-code that allows communication with proba-
be Alice’s rate. An (n,Rn)-code C is defined by Alice’s bilityoferroratmostε.Thesupremumovernoftheweakly
encoder and Bob’s corresponding decoder, as below. achievable rates is called the weak capacity and is denoted
Alice:Alice’smessageuisassumedtobearandomvariable by Cw.
UwithentropyRn,overalphabetU. We considertwotypes The rate R is said to be strongly achievable1 if for every
of encoding schemes for Alice.
δ > 0, ∃α > 0 so that for sufficiently large n there
For deterministic codes, Alice’s message U is assumed to
existsan(n,(R−δ)n)-codethatallowscommunicationwith
be uniformly distributed over U = [2Rn]. Her deterministic probability of error at most e−αn. The supremum over n of
encoder is a deterministic function fD(.) that maps every thestronglyachievableratesiscalledthestrongcapacityand
u in [2Rn] to a vector x(u) = (x1,...,xn) in {0,1}n. is denoted by Cs.
Alice’s codebook X is the collection {x(u)} of all possible
Remark: Since a rate that is strongly achievable is always
transmitted codewords.
weakly achievable but the converse is not true in general,
More generally, Alice and Bob may use probabilistic Cw ≥Cs.
codes.Forsuchcodes,therandomvariableUcorresponding
to Alice’s message p may have an arbitrary distribution
U
p (with entropy Rn) over an arbitrary alphabet U. Alice’s 1Thisdefinitionismotivatedbytheextensiveliteratureonerrorexponents
U
ininformationtheory–forlargeclassesofinformation-theoretic problems,
codebook X is an arbitrary collection {X(u)} of subsets of
e.g. [9], [5], the probability of error of the coding scheme is required to
{0,1}n.ForeachsubsetX(u)⊂X,thereisacorresponding decayexponentially inblocklength.
III. RELATEDWORK AND OURRESULTS is 1−H(p−ε). Taking the limit when ε → 0 implies our
bound.
To the best of our knowledge, communication in the
presence of a causal adversary has not been explicitly ad- The upper bound of (1−4p)+ is more involved. For the
dressedintheliterature(otherthanourpriorworkforcausal case where Alice’s encoder is deterministic, the proof of
Theorem 1 has the following overall structure. Assume for
adversariesoverlarge-qchannels).Nevertheless,wenotethat
the model of causal channels, being a natural one, has been sake of contradiction that Alice attempts to communicate at
“on the table” for several decades and the analysis of the rate greater than R=(1−4p)+. To prove our upper bound
we design the following wait-and-push attack for Calvin.
online/causal channel model appears as an open question in
thebookofCsisza´randKorner[5](inthesectionaddressing Calvin starts by waiting for Alice to transmit approxi-
Arbitrary Varying Channels [1]). Various variants of causal mately Rn bits. As Alice is assumed to communicateat rate
adversarieshave beenaddressed in the past, for instance [1], greater than R, the set of Alice’s codewords X′ consistent
[11],[15],[16],[13]–howeverthemodelsconsideredtherein with the bits Calvin has seen so far is “large” with “high
differ significantly from ours. probability”. Calvin constructs X′ and chooses a codeword
At a high level, we show that for causal adversaries, for a x′ uniformly at random from X′. He then actively “pushes”
large range of p (for all p>0.25), the maximum achievable x in the direction of x′ by flipping, with probability 1/2,
rate equals that of the classical “omniscient” adversarial each future xi that differs from x′i. If Calvin succeeds in
pushing x to a word y roughly midway between x and
model (i.e., 0). This may at first come as a surprise, as
the online adversary is weaker than the omniscient one, x′, a careful analysis demonstrates that regardless of Bob’s
decodingstrategy, Bob is unableto determine whetherAlice
and hence one may suspect that it allows a higher rate of
communication. transmitted x or x′ — causing a decoding error of 1/2
We have two main results. Theorem 1 gives an upper in this case. So, to prove our bound, we must show that
bound on the weak capacity Cw if Alice’s encoder is de- with constant probability (independent of the block length
terministic. Theorem 2 gives an upper bound on the strong n) Calvin will indeed succeed in pushing x to y. Namely,
capacity Cs in the more general case where Alice’s encoder thatAlice’s codewordx andthecodewordchosenatrandom
is probabilistic. Due to certain limitations of our proof byCalvin x′ areof distanceatmost2pn.Roughlyspeaking,
techniques, we do not present any bounds on the weak we prove the above by a detailed analysis of the distance
capacity in the latter setting. The upper bound in both cases structure of the set of codewords in any code using tools
equals min{1−H(p),(1−4p)+}. from extremal combinatorics and coding theory.
Theorem 1 (Deterministic encoder): For deterministic The case where Alice’s encoder may be randomized is
codes, Cs ≤Cw ≤min{1−H(p),(1−4p)+}. more technically challenging, and is considered in Theo-
Theorem 2 (Probabilistic encoder): For probabilistic rem 2. At a high level, the strategy of Calvin for a prob-
codes, Cs ≤min{1−H(p),(1−4p)+}. abilistic encoder follows that outlined for the deterministic
Wenotethatunderaveryweaknotionofcapacityinwhich case. However, there are two main difficulties in its ex-
one only requires the success probability to be bounded tended analysis. Firstly, the symmetry between x and x′
away from zero (instead of approaching 1), the capacity of no longer exists. Namely, the fact that Bob may not be
the omniscientchannel,andthusthe binarycausal-adversary able to distinguish which of the two were transmitted by
channel, approaches 1−H(p). This follows by the fact that Alicedoesnotnecessarilycauseasignificantdecodingerror,
for n sufficiently large and ℓ≥4 there exists (n,Rn) codes since the probability of x′ being transmitted by Alice may
which are (ℓ,pn) list decodable with R = 1−H(p)(1 + well be significantly smaller than the probability that x
1/ℓ) [7]. Communicating using an (ℓ,pn) list decodable was transmitted. Secondly, the fact that both x and x′ may
code allows Bob to decode a list of size ℓ of messages correspond to the same message u places the entire scheme
which includes the message transmitted by Alice. Choosing in jeopardy. As it now no longer matters if Bob decodes to
a message uniformly at random from his list, Bob decodes x or x′, in both cases the decoded message will be that sent
correctly with probability at least 1/ℓ. by Alice.
Toovercomethesedifficulties,wedescribeamoreintricate
A. Outline of proof techniques
analysisof Calvin’s attack. Roughlyspeaking,we provethat
Theupperboundof1−H(p)followsdirectlybydescribing a “large” subset X′′ of X′ behaves “well”. Any x′ chosen
an attack for Calvin wherein he approximately simulates uniformlyatrandomfromX′, with “significant”probability,
a BSC(p) (Binary Symmetric Channel [4] with crossover is in X′′, and has three properties corresponding to those
probability p). More precisely, for each i ∈ [n] and any when Alice uses a deterministic encoder. That is, x′ is
sufficiently small ε > 0, Calvin flips x with probability sufficiently close to x as desired, it has approximately the
i
p − ε until he runs out of his budget of pn bit-flips. By sameprobabilityoftransmissionthatxdoes(thuspreserving
the Chernoff bound [3], with very high probability he does the needed symmetry), and it also correspondsto a message
not run out of his budget, and is therefore indistinguishable that differs from that corresponding to x. All in all, we
from a BSC(p − ε). But it is well-known [4] that in this show that the above three properties hold with probability
case the optimal rate of communication from Alice to Bob 1/poly(n), which suffices to bound the strong capacity of
the channel at hand (but not the weak capacity). X |x1,...,xℓ =xℓ}.Hethenchoosesanelementx′ ∈X|xℓ
In case of a randomizedencoderof Alice, we assume that uniformly at random. In the second stage, Calvin follows a
themessagesmayhavenonuniformdistribution,andalsoany randombit-flipstrategy.Thatis,foreachremainingbitx′ of
i
messageisencodedintooneofasetofpossiblecodewordsas x′ thatdiffersfromthecorrespondingbitx ofxtransmitted,
i
per some probability distribution in that set. One may think he flips the transmitted bit with probability1/2, until he has
ofvariousotherwaysofencoding,forexamplethefollowing, either flipped pn bits, or until i=n.
to confuse Calvin. But as we discuss in the next paragraph, We analyze Calvin’s attack by a series of claims. We first
such schemes are also covered in our setup. showthatwithhighprobability(w.h.p.)thesetX|xℓ islarge.
Multiple codebooks: In this scheme, Alice maintains a Claim 4.1: With probability at least 1−2−εn/4, the set
set of codes C1,C2,...,CL. For transmitting a message u, X|xℓ is of size at least 2εn/4.
she randomly selects the code Ci with probability qi. If the Proof: The number of messages u for which X|xℓ(u)
set of messages is U = {1,2,...,M} with a probability is of size less than 2εn/4 is at most the number of distinct
distribution given by p =△ Pr{u = i}, and the code C prefixes xℓ times 2εn/4, which in turn is at most 2ℓ+εn/4 =
i r
contains the codewords {x(u,r) | u = 1,2,...,M}, then 2(R−ε/4)n.
in our setup, the correspondingcodebook for the message u Nowassumethatthemessageuissuchthatitscorrespond-
will be X(u) = {x(u,r) | r = 1,2,...,L}. This codebook ingsetX|xℓ(u)isofsizeatleast2εn/4.Wenowshowthatthis
impliesthatthetransmittedcodewordxandthecodewordx′
may have less than L codewordsdue to commoncodewords
chosenbyCalvinaredistinctandofsmallHammingdistance
in the originalcodes. The induced probabilitydistribution in
this codebook of u is given by Pr{x|u}= q . apart with a positive probability (independentof n).
r:x(u,r)=x r Claim 4.2: Conditioned on Claim 4.1, with probability at
If Alice picks a code and uses it toPencode several
least ε , x6=x′ and d (x,x′)<2pn−εn/8.
messages, even then she does not gain anything. First, if 64p H
Proof: Consider the undirected graph G = (V,E) in
she uses the same code to encode too many messages (and
calvin knows the encoding scheme, as assumed), then both which the vertex set V consists of the set X|xℓ and two
nodes are connected by an edge if their Hamming distance
Bob and Calvin will know the code used after receiving or
is less than d = 2pn−εn/8. An independent set I in G
‘reading’some codewords.On the otherhand,if a randomly
corresponds to a subset of codewords in {0,1}n that are all
chosencodeis used onlyto encodea blockof few messages
(pairwise) at distance greater than d.
this is equivalent to using a longer (‘superblock’) code in
oursetup. The onlydifferenceis thatthe probabilityof error Since the codewords in X|xℓ all have the same prefix xℓ,
one may consider only the suffix (of length n−ℓ = 4pn−
analysed in ourset up is the probabilityof error in decoding
the ‘superblocks’ rather than the smaller blocks/codewords. εn/2) of the codewordsin X|xℓ. Here we assume p≤0.25,
minormodificationsintheproofareneededforlargerp.The
Theproofsoftheupperboundscorrespondingto1−H(p)
set of vectors defined by the suffixes in an independent set
have already been sketched in Section III-A. Hence we only
I of G now corresponds to a binary error-correcting code
provide proofs of the upper bounds corresponding to (1−
4p)+ in Theorems 1 and 2. of length 4pn − εn/2, with |I| codewords and minimum
distance d.
IV. PROOF OF THEOREM1 By Plotkin’sbound[2] there donotexistbinaryerrorcor-
recting codes with more than 2d +1 codewords.
Let R=(1−4p)++ε for some ε>0. Let log(.) denote 2d−(4pn−εn/2)
Thus I, any maximal independent set in G, must satisfy
thebinarylogarithm,hereandthroughout.Byassumptionfor
deterministic codes,Alice’s message space U is of size 2Rn. 2(2pn−εn/8) 16p
|I|≤ +1= (1)
Here we assume for that2Rn in an integer.Thisimplies that 2(2pn−εn/8)−4pn+εn/2 ε
the set X of Alice’s transmitted codewordsis of size 2Rn. 2 By Tura´n’s theorem [17], any undirected graph G of size
We now present Calvin’s attack. We show that for any |V| and average degree ∆ has an independent set of size
fixed ε > 0, regardless of Bob’s decoding strategy, there at least |V|/(∆+1). This, along with (1) implies that the
is a decoding error with constant probability (namely, the average degree of our graph G satisfies
error probability is independent of n). Calvin’s attack is in
|V| 16p
two stages. First Calvin passively waits until Alice transmits ≤|I|≤
∆+1 ε
ℓ = (R−ε/2)n bits over the channel. Let xℓ ∈ {0,1}ℓ be
This in turn implies that
thevalueofthecodewordobservedsofar.Hethenconsiders
thesetofcodewordsthatareconsistentwiththeobservedxℓ. ε|V| ε|V|
∆≥ −1≥
Namely,CalvinconstructsthesetX|xℓ ={x=x1,...,xn ∈ 16p 32p
The second inequality is for large enough n, since |V| is of
2In fact, X may be smaller, however we note that for codes of optimal
rate, |X|is ofsize exactly 2Rn.If|X|<2Rn,then forsometransmitted size at least 2Rn. To summarize the above discussion, we
codeword x at least two messages u and u′ must both be encoded to x. have shown that our graph G has large average degree of
TOhnerreefcoerievicnhganxg,inBgobth’secporodbebaboioliktysooafsetroroernicsodmeauxi′masalso–miteixs′a∈t/lXeascta1n/no2t. size ∆ ≥ ε3|2Vp|. We now use this fact to analyze Calvin’s
increase theprobability ofdecodingerror. attack.
By the definition of deterministic codes, any codeword in p(x′|y) = p(y|x′)p(x′)/p(y). Taking the ratio and noting
X is transmitted with equal probability. Also, by definition that for deterministic codes p(x)=p(x′), we have
both x (the transmitted codeword) and x′ (the codeword
p(x|y)/p(x′|y)=p(y|x)/p(y|x′). (4)
chosenbyCalvin)areinV =X|xℓ.Hencebothxandx′ are
uniform in X|xℓ. This implies that with probability |E|/|V|2 Since Calvin’s random bit-flip strategy involves him flip-
the nodes corresponding to codewords x and x′ are distinct ping bits of x (which are different from the corresponding
andconnectedbyanedgeinG.Thisinturnimpliesthatwith bits of x′) with probability 1/2, for all y satisfying (2), the
probability |E|/|V|2, x 6= x′ and d (x,x′) < 2pn−εn/8, probabilities p(y|x) and p(y|x′) are equal. This observation
H
as required. Now and (4) together imply p(x|y)=p(x′|y). Thus, Bob cannot
distinguishwhetherxorx′ weretransmitted.Namely,onthe
|E| ∆|V| ε pairofeventsinwhichAlicetransmitsxandCalvinchooses
= ≥
|V|2 2|V|2 64p x′ andinwhichAlice transmitsx′ andCalvinchoosesx, no
matter which decoding process Bob uses, he will have an
averagedecodingerrorof atleast1/2.Thissufficestoprove
Conditioned on Claim 4.2, Calvin’s codeword x′ is
our assertion.
very close to Alice’s transmitted codeword x. Specifically,
Thus a decoding error happens if the con-
d (x,x′) ∈ (0,2pn−εn/8). We now show that if Calvin
H ditions of Claims 4.1, 4.2, 4.3 and 4.4 are
follows the random bit-flip strategy, from Bob’s perspective
all satisfied. This happens with probability at
(w.h.p.), both x or x′ were equally likely to have been
least 1−2−εn/4 ε 1−2−Ω(ε2n) 1 ≥
transmitted by Alice. 64p 2
(cid:0) (cid:1)(cid:16) (cid:17)(cid:16) (cid:17)(cid:0) (cid:1)
WefirstshowthatduringCalvin’srandombit-flipprocess, 1 ε 1 1 ≥ ε for large enough n.
2 64p 2 2 512p
w.h.p.,Calvindoesnot“runout”ofhisbudgetofpnbitflips. (cid:0) (cid:1)(cid:16) (cid:17)(cid:0) (cid:1)(cid:0) (cid:1) (cid:4)
Claim 4.3: Conditioned on Claim 4.2, with probability at
least 1−2−Ω(ε2n) V. PROOF OF THEOREM2
WestartbyprovingthefollowingtechnicalLemmathatwe
d εn d εn useinourproof.Letq beanarbitraryprobabilitydistribution
d (x,y)∈ − , + . (2)
H
(cid:18)2 16 2 16(cid:19) over an index set I = {1,...,k}. Let A1,...,Ak be
arbitrary discrete random variables with probabilitydistribu-
Proof: The expected number of locations flipped by tions q1,...,qk over alphabets A1,...,Ak respectively. Let
Calvinisd/2≤pn−εn/16.Assumethatd/2=pn−εn/16 ki =|Ai|.LetAbearandomvariablethatequalstherandom
(forsmallervaluesofdtheboundisonlytighter).BySanov’s variableAi withprobabilityq(i).ThenthefollowingLemma
theorem[4,Theorem12.4.1],theprobabilitythatthenumber describing an elementary property of the entropy function
of bits flipped by Calvin deviates from the expectation d/2 H(.) is useful in the proof of Theorem 2.
by more than εn/16 is at most e−Ω(ε2n2/d) ≤ e−Ω(ε2n) for Lemma 5.1: TheentropiesofA,A1,...,Akandqsatisfy
large enough n. H(A)≤ ki=1q(i)H(Ai)+H(q), withequalityif andonly
It should be noted that d/2 + εn/16 ≤ pn, and so if for eachPi,i′ for which both q(i) and q(i′) are positive it
dH(x,y) ≤ d/2+ εn/16 implies that the number of bits holds that Prqi,qi′[Ai =Ai′]=0.
Proof: For any a ∈ A, the probability Pr{A = a} =
flippedby Calvin doesnotexceedpn. Since Calvin possibly
p(a) of occurrence of a, equals q(i)q (a). Hence
flips only the bits of x which differ from the corresponding i:a∈Ai i
bits in x′, (2) also implies P
H(A) = − p(a)log(p(a))
d (x′,y)∈ d − εn,d + εn . (3) a∈XSiAi
H (cid:18)2 16 2 16(cid:19) k ki
≤ − q(i)q (j)log(q(i)q (j)) (5)
i i
We conclude by provingthat if the number of bits flipped Xi=1Xj=1
byCalvinliesintherange(d/2−εn/16,d/2+εn/16),then k ki
indeed Bob cannot distinguish between the case in which x = q(i)(q (j)log(q (j)))
i i
or x′ were transmitted. Xi=1Xj=1
Claim 4.4: Conditioned on Claim 4.3 Bob makes a de- k ki
coding error with probability at least 1/2. + q (j)(q(i)log(q(i)))
i
Proof: By Bayes’ Theorem [8], if Bob receives y, Xi=1Xj=1
the a posteri probability that Alice transmitted x, denoted k
p(x|y),equalsp(y|x)p(x)/p(y).Herep(x)istheprobability = q(i)H(Ai)+H(q).
(over her encoding strategy) that Alice transmits x, p(y|x) Xi=1
istheprobability(overCalvin’srandombit-flippingstrategy) Here (5) follows from Jensen’s inequality, e.g. [4], with
that Bob receives y given that Alice transmits x, and p(y) equality if and only if for each positive Pr{A=a}, there is
is the resulting probability that Bob receives y. Similarly, a unique i such that q(i)q (j)>0 (here a (j)=a).
i i
We now turn to proveTheorem 2. Recall our notation:let to “push”x(u,r) to a codewordy atapproximatelythe
U be the random variable correspondingto Alice’s message same distance from x(u,r) and x′(u′,r′).
and pU its distribution (with entropy Rn). Throughout we • Given y, Bob is unable to distinguish whether x(u,r)
assume the message set U (the support of U) is at most or x′(u′,r′) was transmitted.
of size 2n. Let X be Alice’s codebook. X is a collection Tothisend,wepartitionthesetX|xℓ inton2 disjointsubsets
{X(u)} of subsets of {0,1}n. For each subset X(u) ⊂ X, X for i,j ∈ {1,2,...,n}. Let p(X ) be the probability
ij ij
there is a corresponding codeword random variable X(u) mass of X . Let p and p be the probability
ij U|ij X(u)|ij
with codeword distribution pX(u) over X(u). For any value distributions pU and pX(u) respectively conditioned on the
U = u of the message, Alice’s encoder choses a codeword event that Alice transmitted x(u,r) in X . The partition
ij
from X(u) randomly from the distribution pX(u). Alice’s Xij is obtained in two steps – first we partition X|xℓ into n
message distribution pU, codebook X, and all the codebook subsets Xi, then we partition each Xi into n sets Xij. We
distributions pX(u) are all known to both Bob and Calvin, also use the probability distribution p(Xi), pU|i and pX(u)|i
but the values of the random variables U and X(.) are defined accordingly. All in all, we prove the existence of a
unknown to them. If X(u) = {x(u,r) : r ∈ Λu}, then the subset Xij with the following properties
tgriavnesnmbitytePdrc[oXd(eUwo)r=d Xx((Uu,)r)h]as=thpeU(puro)pbXab(iul)it(yx(duis,trri)b)u.tLioent •• pH(X(piUj|)iji)silsar“glearwgeit”h. respect to p(xℓ).
p the the overall distribution of codewords x = x(u,r) • For any x(u,r) ∈ Xij it holds that p(x(u,r)) has
of Alice. It holds that p(x(u,r)) = pU(u)pX(u)(x) and approximately the same value.
p(xF)or=aPnyUεpU>(u)0p,X(lue)t(xR). = (1 − 4p)+ + ε. We start Ro•ugphUly|ijspiseaakpipnrgo,xpimroavtienlgytuhneisfeorpmroopnerittisessuopnpoXrt. reduces
ij
by specifying Calvin’s attack. Calvin uses a very similar
us to the case of a deterministic encoder (addressed in
attack to the one described in the proof of Theorem 1.
Theorem 1) and allows us to complete our proof.
That is, Calvin first passively waits until Alice transmits
We now present our proof for the existence of X as
ij
ℓ = (R − ε/2)n bits over the channel. Let xℓ ∈ {0,1}ℓ
specified above. We first show that with positive probability
be the value of the codeword observed so far. He then
the set X|xℓ has high entropy.
considers the set of codewords x(u,r) consistent with the Claim 5.1: With probability at least ε/4, H(p ) ≥
observed xℓ. Here and throughout this section, we denote U|xℓ
εn/4.
codewords by their corresponding message u and index r Proof: Letq betheprobabilitydistributionover{0,1}ℓ
in X(u). As it may be that x(u,r) is exactly the same for which q(xℓ) = p(xℓ) for all possible xℓ ∈ {0,1}ℓ.
caonddeiwnotrhdisassecxt(iuon′,ra′r)e,itnheasseetnsseinmtuhletisdeetfis.niNtiaomnseltyo, Cfoalllvoiwn Let qxℓ be the probability distribution pU|xℓ. Now using
Lemma 5.1 we obtain
constructs the set X|xℓ = {x(u,r) = x1,...,xn ∈ X |
x1,...,xℓ = xℓ}. Let p(xℓ) = p(X|xℓ) be the probability, H(pU)≤ q(xℓ)H(pU|xℓ)+H(q). (6)
under the probability distribution p, corresponding to the Xxℓ
eventthatCalvinobservesxℓ inthefirstℓtransmissions.Let By our definitions H(p ) = Rn. Moreover, H(q) ≤ ℓ =
U
p and p be the probability distributions p and (R−ε/2)n (since q is defined over an alphabet of size 2ℓ).
U|xℓ X(u)|xℓ U
p alsorespectivelyconditionedonthesameevent.Calvin Thus (6) becomes
X(u)
then chooses an element x′(u′,r′)∈X|xℓ with probability3 q(xℓ)H(p )≥Rn−(R−ε/2)n=εn/2.
pU|xℓ(u′)pX(u′)|xℓ(x′(u′,r′)). In the second stage he then Xxℓ U|xℓ
follows exactly the same random bit-flip strategy as in the
proof of Theorem 1. AstheaverageofH(pU|xℓ)isatleastεn/2,thenH(pU|xℓ)≥
εn/4withprobabilityatleastε/4(byaMarkovtypeinequal-
Recall that in the proof of Theorem 1, our goal was
ity, here we use the fact that H(p )≤n).
to prove that with some constant probability, the distance U|xℓ
betweenx(u,r)andx′(u′,r′)isapproximately2pn.Loosely We now define the sets Xi. For i = 1,...,n−1, let Xi
speaking, this allows the success of Calvin’s attack (i.e., be the set of codewordsin X|xℓ for which p(x(u,r))/p(xℓ)
is in the range (2−3i,2−3i+3]. The set X is defined to be
imply a decoding error). Following the same outline of n
proof, we now show that with probability 1/poly(n) the the set of codewords in X|xℓ for which p(x(u,r))/p(xℓ)
codewordx′(u′,r′)chosenbyCalvinhasthefollowingthree is in the range [0,2−3n+3]. Let p(Xi) be the probability
mass of X . Namely p(X ) ≃ 2−3i|X |p(xℓ). Let q be the
properties: i i i
distribution over {1,2,...,n} taking i w.p. p(X )/p(xℓ).
i
• It’scorrespondingmessagediffersfromthatcorrespond- Notice that H(q)≤ log(n)= o(n) (as its support is of size
ing to x(u,r) (i.e., u6=u′).
n). Conditioning on Claim 5.1 and using Lemma 5.1 it can
• x′(u′,r′)isclosetox(u,r)andthusCalvinwillbeable be verified that
Claim 5.2:
3This is one significant difference from the attack in the proof of
Theorem 1 – there Calvin chooses each x′ uniformly at random from the q(i)H(p )≥H(p )−H(q)≥εn/8 (7)
U|i U|xℓ
corresponding consistent set. Xi
ConsidersetsX with(relative)massq(i)≥1/n2.Itholds included many messages, each with the same probability,
i
that details follow.
q(i)H(p )≥εn/16 Consider the graph G = (V,E) in which the vertex set V
U|i
i≤n−1X;q(i)≥1/n2 consists of the set Xij and two nodes are connected by an
edgeif theirHammingdistanceislessthand=2pn−εn/8.
The above follows from the fact that
Now, it is can be verified (using analysis almost identical
q(i)H(p ) + q(n)H(p ) ≤
i≤n−1;q(i)≤1/n2 U|i U|i to that given in the proof of Theorem 1) that
PnP).i≤Hne−re1;qw(ie)≤u1s/en2thne/fna2ct+th2a−tnq+(3nn)≤≤2|X(fo|2r−su3nffi+c3i.entlylarge 1) With probability at least 1−2−Ω(εn) the codewords
i x(u,r) and x′(u′,r′) satisfy u 6= u′. Here one needs
We conclude the existence of a set X such that q(i) ≥
i
to take into consideration the slight difference in the
1/n2 and H(p ) ≥ εn/16. We now further partition X .
U|i i group sizes and the probabilities for each codeword.
Forj =1,...,n−1,letX bethesetofcodewordsx(u,r)
ij
in X for which p (u) is in the range (2−3j,2−3j+3]. X 2) WithprobabilityΩ ε theverticesinG corresponding
is deifined to be Uth|ie set of codewords x(u,r) in X foinr to x(u,r) and x′(u(cid:16)′,pr(cid:17)′) are connected by an edge.
i
which p (u) is in the range[0,2−3n+3]. Let p(X ) be the 3) During Calvin’s random bit-flip process, with high
probabilUit|yi mass of X . Namely p(X )≃2−3i|Xij |p(xℓ). probabilityof 1−2−Ω(ε2n), Calvin does not “run out”
ij ij ij
Let q′ be the distribution over {1,2,...,n} taking j w.p. of his budget of pn bit flips.
p(X )/p(X ). Notice that H(q′) ≤ log(n) = o(n) (as its 4) Conditioning on the above, Bob cannot distinguish
ij i
support is of size n). As before, conditioning on Claim 5.2 between the case in which x(u,r) or x′(u′,r′) were
and using Lemma 5.1 it can be verified that (for the index i transmitted.
specified above), 5) Finally, on the pair of events in which Alice trans-
Claim 5.3: mits x(u,r) and Calvin chooses x′(u′,r′), and Alice
transmits x′(u′,r′) and Calvin chooses x(u,r), no
q′(j)H(p )≥H(p )−H(q′)≥εn/32 (8)
U|ij U|i matter which decoding process Bob uses, he has an
Xj averagedecodingerrorthatisboundedawayfromzero.
Again,considersets X with mass q′(i)≥1/n2. Itholds
ij Here again we take into account the slight differences
that
between p(x(u,r)) and p(x′(u′,r′)).
q′(j)H(p )≥εn/64
U|ij To summarize, Calvin causes a decoding error with prob-
j≤n−1;Xq′(j)≥1/n2
abilityΩ(poly(ε)/poly(n))=Ω(1/poly(n)) as desired.This
We conclude the existence of a set Xij such that concludes our proof. (cid:4)
• H(pU|ij)≥εn/64.
• p(Xij)≥p(xℓ)/n4. VI. CONCLUSIONS
• For any x(u,r) ∈ Xij it holds that p(x(u,r)) is We analyze the capacity of the causal-adversarial channel
approximately 2−3ip(xℓ). and show (for both deterministic and probabilistic encoders)
• For any x(u,r)∈Xij it holds that pU|ij(u) is approx- thatthecapacityisboundedbyabovebymin{1−H(p),(1−
imately equal. 4p)+}. For a large range of p (for all p > 0.25), the
The set X is exactly what we are looking for. Roughly maximumachievablerateequalsthatofthestrongerclassical
ij
speaking, by Claim 5.1, with probability at least ε/4 Calvin “omniscient” adversarial model (i.e., 0).
viewsa prefix xℓ for whichH(p )≥εn/4.Conditioning Several questions remain open. In this work we do not
U|xℓ
on this event, both Alice and Calvin choose codewords address achievability results (i.e., the construction of codes).
x(u,r), x′(u′,r′) in X with probability at least 1/n8. It would be very interesting to obtain codes for the causal-
ij
We now sketch to remainder of the proof which closely adversary channel which obtain rate greater than that know
follows that of Theorem 1. We partition X into groups for the “omniscient” adversarial model (i.e., the Gilbert-
ij
of messages X (u) consisting of all codewords in X Varshamov bound) for p < 0.25). As we do not believe
ij ij
correspondingtou.Recallthateachcodewordx(u,r)∈X that the upper bound of (1−4p)+ presented in this work
ij
has approximately the same probability p(x(u,r)), and for is actually tight, such codes, if they exist, may give a hintto
each x(u,r) ∈ X it holds that p (u) is approximately the correct capacity.
ij U|ij
the same value. This implies that each group X (u)⊆X As done in our work on large alphabets [6], one may
ij ij
has approximately the same size. Moreover, as H(p ) ≥ also consider the more general channel model in which for
U|ij
εn/64 it holds that there are at least 2εn/64 non-empty a delay parameter d ∈ (0,1), the jammer’s decision on
subsets X (u) in X . the corruption of x must depend solely on x for j ≤
ij ij i j
So, all in all, X has a very symmetric structure: it i−dn. This might correspond to the scenario in which the
ij
includes many groups, each consisting of elements with the error transmission of the adversarial jammer is delayed due
same transmission probability, and each of approximately to certain computational tasks that the adversary needs to
the same size and mass (w.r.t. p). This reduces us to the perform.The capacityof the causalchannelwith delayis an
case considered in Theorem 1 in which our subset X|xℓ intriguing problem left open in this work.
REFERENCES
[1] D. Blackwell, L. Breiman, and A. J. Thomasian. The capacities
of certain channel classes under random coding. The Annals of
Mathematical Statistics, 31(3):558–567, 1960.
[2] A. E. Brouwer. Bounds on the size of linear codes. In V. S. Pless
andW.C.Huffman, editors, Handbook ofCoding Theory, volume1,
chapter 4, pages 295–461. Elsevier Science, New York, NY, USA,
1998.
[3] H.Chernoff.Measureofasymptoticefficiencyfortestsofahypothesis
basedonthesumofobservations. AnnalsofMathematical Statistics,
23:493–507, 1952.
[4] T.M. Cover and J.A. Thomas. Elements ofinformation theory, 2nd
edition. Wiley-Interscience, NewYork,NY,USA,2006.
[5] I.CsiszarandJ.G.Korner. InformationTheory:codingtheoremsfor
discretememorylesssystems.AcademicPress,Inc,Orlando,FL,USA,
1982.
[6] B. K. Dey, S. Jaggi, and M. Langberg. Codes against online adver-
saries. Manuscript. Available athttp://arxiv.org/abs/0811.2850.
[7] P.Elias. Error-correcting codes forlist decoding. IEEETransactions
onInformationTheory. 37(1):512, 1991.
[8] W.Feller. AnIntroductiontoProbabilityTheoryandItsApplications,
VolumeII(2nded.). JohnWiley&Sons,NewYork,1972.
[9] R. G.Gallager. Information Theory and Reliable Communication. J.
WileyandSons,NewYork,1968.
[10] E. N. Gilbert. A comparison of signalling alphabets. Bell Systems
Technical Journal, 31:504–522, 1952.
[11] S.Jaggi,M.Langberg,T.Ho,andM.Effros.CorrectionofAdversarial
ErrorsinNetworks. InproceedingsofIEEEInternational Symposium
onInformationTheory(ISIT),pages 1455–1459,2005.
[12] R. McEliece, E. Rodemich, H. Rumsey, and L. Welch. New upper
boundsontherateofacodeviathedelsarte-macwilliamsinequalities.
IEEETrans.Inform.Theory,23(2):157–166, March1977.
[13] L. Nutman and M. Langberg. Adversarial Models and Resilient
Schemes for Network Coding. In proceedings of IEEE International
Symposium onInformationTheory,pages 171–175,2008.
[14] M.Plotkin.Binarycodeswithspecifiedminimumdistance.IRETrans.
Inform.Theory,6:445–450, 1960.
[15] A.Sahai andS.Mitter. Thenecessity andsufficiency ofanytime ca-
pacityforstabilization ofalinearsystemoveranoisycommunication
link,PartI:scalarsystems.IEEETransactionsonInformationTheory,
52(8):3369–3395, 2006.
[16] A. Sarwate. Robust and adaptive communication under uncertain
interference. PhDthesis, Berkeley, 2008.
[17] P.Tura´n. OntheTheoryofGraphs.,Colloq.Math.3(1954),19-30.
Colloq. Math.3:19-30,1954.
[18] R.R.Varshamov.Estimateofthenumberofsignalsinerrorcorrecting
codes. Dokl.Acad.Nauk,117:739–741, 1957.