Communicating Correlated Sources Over an Interference Channel

0.43 MB·

by  Arun Padakandla

1
Communicating Correlated Sources over an
Interference Channel
Arun Padakandla
Abstract—A new coding technique, based on fixed block- S1 Tx1 X1 YY11 RRxx11 S
length codes, is proposed for the problem of communicating a ~ 1
pItasirpeorffocromrraenlacteedissoaunracleyszeodvetroade2r−ivuesear ninetwerfseertenocfescuhffiancineenlt. WS1S2 WY1Y2|X1X2
cthoendciutirornesn.tTkhneowlanttberesits, wprhoicvhenistodubeetsotrLiciutlyanledssCbhienndi[n1g].tOhaunr S~2 Tx2 X2 Y2 Rx2 S2
7 findings are inspired by Dueck’s example [2].
1
Fig.1. Transmissionofcorrelatedsourcesover2-IC.
0
2 I. INTRODUCTION

b Network information theory has provided us with elegant
restrictedtosub-blocksoffixedlengthl,thentheencoderscan
e techniques to exploit correlation amongst distributed infor-
enjoy the benefits of separation and co-ordination on a good
F
mation sources. Such a correlation is handled at two levels.
fraction (at least ∼ (1−lξ)) of these l−length sub-blocks.
7 Probabilistic (soft) correlation is exploited via binning or
Indeed, we prove in Section III-A that the latter technique
transferring them via test channels [3]. When the sources
outperformsLiuandChen’scodingtechnique(LCtechnique).
]
T possess common bits - Ga´cs-Ko¨rner-Witsenhausen (GKW) In Section IV, we build on this idea to propose a general
commonpart-,conditionalcodingprovidesenhancedbenefits.
I coding technique for an arbitrary problem instance.
s. In this article, we propose a new coding technique to exploit Joint source-channel coding over multi-user channels has
c thepresenceofnearGKWparts,amongstdistributedsources.
received considerable attention with regard to characterizing
[
Our primary focus is the scenario depicted in Fig. 1. A fundamental limits [4]–[6] and designing feasible strategies
4 pair S ,S of correlated sources have to be communicated
1 2 [7].FundamentalperformancelimitsforcommunicatingGaus-
v over a 2−user interference channel (IC). Receiver (Rx) j
sian sources over Gaussian channels have been studied in [8],
7
wishes to reconstruct S losslessly. We undertake a Shannon-
7 j [9] [10] and the latter considers communication over on IC.
theoreticstudyandrestrictattentiontocharacterizingsufficient
9 Ourfindingshighlightthesub-optimalityof(currentknown)
6 conditions under which Sj can be reconstructed at Rx j. S-Ljointsource-channelcodingtechniques(Rem.2).Notwith-
0 The current known best set of sufficient conditions (LC
standing this, we derive a S-L characterization (Rem. 4) of
. conditions) for this problem is due to Liu and Chen [1, Thm.
1 a new inner bound that strictly enlarges the current known
0 1]andareproventobeoptimalforaclassofdeterministicICs best (LC bound). Indeed, the fixed B-L coding technique
7 [1,Thm.2].Inthisarticle,weproposeanewcodingtechnique
is an l−letter technique. An important second contribution
1 based on fixed block-length (B-L) codes and derive a new set
is therefore, a framework - codes and tools (interleaving) -
:
v ofsufficientconditions.Throughanexample(Ex.1),weprove
for stitching together S-L techniques in a way that permits
i (Lem. 2) the latter conditions are strictly less binding.
X performance analysis of the resulting l−letter technique and
Presence of GKW part enables encoders co-ordinate their
r characterization via S-L expressions. Stepping beyond perfor-
a inputs, and thereby eliminate interference for the correspond- mancecharacterization,ourthirdcontributionisanewcoding
ing component of the channel input. Moreover, GKW part
technique for communicating correlated sources over an IC.
enablesco-ordinationevenwhileenjoyingseparation.Inother
This is part of an evolving work [11]–[13] on joint source-
words, one can design a channel code corresponding to an
channel coding, and is inspired by Dueck’s novel example [2]
optimizing input pmf, unconstrained by the source pmf. If
and his very specific, yet ingenious, fixed B-L coding. Here,
S ,S do not possess a GKW part, a single-letter (S-L)
1 2 we restrict attention to separation based schemes1 and focus
techniqueisconstrainedbytheS-LlongMarkovchain(LMC)
on providing a clear step-by-step description of the ideas.
X −S −S −X . The S-L LMC can, in general, severely
1 1 2 2 Unifying fixed B-L coding and inducing source correlation
constrain the set of achievable input pmfs (Ex. 1, Rem. 1). If
onto channel inputs [14] involves additional challenges, and
S ,S possess a near GKW part, i.e., K = f (S ) : j ∈ [2]
1 2 j j j is dealt in a concurrent submission [12, Sec. V].
such that ξ = P(K (cid:54)= K ) is ‘quite’ small, (relatively)
1 2
large sub-blocks of length l could agree with high probability.
II. PRELIMINARIES:NOTATION,PROBLEMSTATEMENT
Indeed, ξ[l] = P(Kl (cid:54)= Kl) = 1 − (1 − ξ)l ≤ lξ can be
1 2
We let an underline denote an appropriate aggregation of
heldsmallbyappropriatelychoosingl.Iftheencodersemploy
related objects. For example, S will be used to represent
conditional coding, i.e., identical source to channel mappings,
a pair S ,S of RVs. S will be used to denote either the
1 2
ThisworkwassupportedbytheCenterforScienceofInformation(CSoI),
anNSFScienceandTechnologyCenter,undergrantagreementCCF-0939370. 1Aswasdonein[11].
2
cpthlaeeiarrcSof1mr,opSml2emcooernntttehxient.dCeWaxr,theeis.neia.,nj{pj∈r,ojd{}u1c,=t2}S,{11t×,h2e}nS.2j,Foawrnidlmlwdi∈elnloNbtee, 0kS1 k−k1 S20k kaa6k6(k1a−k−11)= α 0W⋯S0011,kS2 k00−kk1 0αβ⋯01 0α (aα0−1)k
[m] : = {1,··· ,m}. For a pmf pU on U, b∗ ∈ U will ka6k(ak−1)= β
denote a symbol with the least positive probability wrt p .2
U
Boldfaces letters such as A denote matrices. For a m × l 0 0 β 0
matrix A, (i) A(t,i) denotes the entry in row t, column i, (a−1)k 0 0 0 β
(a−1)k (a−1)k
(ii) A(1 : m,i) denotes the ith column, A(t,1 : l) denotes
tth row.“withhighprobability”,“single-letter”,“longMarkov
Fig. 2. On the left, the source pmf is depicted through a bipartite graph.
chain”, “block-length” are abbreviated whp, S-L, LMC, B-L Larger probabilities are depicted through edges with thicker lines. On the
respectively. right,wedepicttheprobabilitymatrix.
For a point-to-point channel (PTP) (U,Y,W ), let
Y|U
Ec1o0rn.(2sR]t.a,nSptpUce,ocWimfiYpcao|Ulsliy)t,iodEnenc(ooRtde,epstho,efWtryapnedo)pmUisacdnoeddfiinnreagdteeaRxspo[1n5e,ntThfomr S~1 Tx1 UX11 WY1|X1 Y1 RRxx11 S1
VmYi|nU(cid:8)D(VY|U||WYr|U|pUU)+|YI(|UpU;VY|U)−R|+(cid:9). SW~S1S2 Tx U2 WWY0|U1U2 Y0 Rx2 S
2 2 X Y2|X2 Y 2
For RVs A ,A , we let ξ[l](A):= P(Al (cid:54)=Al), and ξ(A): 2 2
1 2 1 2
=ξ[1](A).ThroughoutSec.III,ξ[l] =ξ[l](S)andξ =ξ(S).If
A is IID, we note3 ξ[l] =1−(1−ξ)l ≤lξ. We let τ (K)= Fig.3. SourcechannelsetupofExample1.
l,δ
2|K|exp{−2δ2p2 (a∗)l} denote an upper bound on P(Kl ∈/
K
Tl(K)) where Tl(K) denotes our typical set.
δ δ TheICisdepictedinFig.3anddescribedbelow.Theinput
Consider a 2−user IC with input alphabets X ,X , output
1 2 alphabetsareU×X andU×X .TheoutputalphabetsareY ×
alphabetsY ,Y ,andtransitionprobabilitiesW .Let 1 2 0
S : = (S ,1S ),2taking values over S : = S ×Y1SY2|Xw1iXth2 pmf Y1 andY0×Y2.U =Y0 ={0,1,··· ,a−1}.(Uj,Xj)∈U×
1 2 1 2 X denotes encoder j’s input and (Y ,Y )∈Y ×Y denotes
W , denote a pair of information sources. For j ∈ [2], j 0 j 0 j
encSo1Sd2er j observes Sj, and decoder j aims to reconstruct Sj sbyomthbdoelscordeecresivaegdrebeywdiethcopdreorbajb.iTlithye1s.yWmbols Y0 received =at
with arbitrarily small probability of error (Fig. 1). If this is W W W , where Y0Y1Y2|X1U1X2U2
possible, we say S is transmissible over IC W . In this Y1|X1 Y2|X2 Y0|U1U2
Y|X
(cid:40)
article, our objective is to characterize sufficient conditions 1 if y =u =u
under which (S,WS) is transmissible over IC WY|X. WY0|U1U2(y0|u1,u2)= 1 if u0 (cid:54)=u1,y =2 0, and
1 2 0
0otherwise.ThecapacitiesofPTPchannelsW :j =1,2
III. FIXEDB-LCODINGOVERISOLATEDCHANNELS are C := h (2)+ 2 loga and C+h ( 2 ) reYsjp|Xejctively.
b k k b kaηk
We consider a simple generalization (Ex. 1) of Dueck’s We identify key aspects of Ex. 1. Let a,k be chosen
example [2] and propose a coding technique that enables sufficiently/quiet large. While S does not possess a GKW
transmissibility of the sources over the corresponding IC. part,S andS agreeonmost,butnotall,realizations.Indeed,
1 2
We also prove all current known joint source-channel coding ξ(S)= 1 isverysmall.WealsohaveH(S ),H(S ),H(S)
kaηk 1 2
techniques,andinparticularLC,isincapableofthesame.On are5 ∼ loga and naturally H(S |S ) : j ∈ [2] is very small.
j j
the one hand, this proves strict sub-optimality of the latter4, Each decoder benefits a lot by decoding either source, or
andontheotherhand,highlightstheneedforfixedBLcoding. any function thereof. Secondly, W is ‘very far’ from the
S
uniform pmf, and hence any S-L function g (S ) will remain
j j
Example 1: Source alphabets S = S = {0,1,··· ,a − ‘considerably’ non-uniform.
1 2
1}k. Let η ≥8 be a positive even integer. The source PMF is The IC supports a sum capacity of at most loga+2C +
h ( 2 ). Since sup I(X;Y) = 2C + h ( 2 ), the
W (ck,dk)=k−ka1ηk−1 iiff cckk ==ddkk,=ck0(cid:54)=k 0k, Wk)b.Y0kT|aUhηk−e clhatatnenreclhmanunsteplXca1crXar2yrribeuslkveorfythleittilnefoirnmfoartmiboantki(aofηnokrwlahregne
S1S2 kaηk(ak−1)
 1 if ck =0k,dk (cid:54)=0k, and U1 (cid:54)= U2, and moreover, it is necessary that U1 = U2 and
kaηk(ak−1)
U =U beclosetouniform,inordertocommunicate∼loga
1 2
0otherwise.Notethatintheaboveeqn.ck,dk ∈S abbreviate bits over W .
1 Y0|U
the k ‘digits’ c c ···c and d d ···d respectively. Fig. 2 We first prove Ex. 1 does not satisfy LC conditions. The
1 2 k 1 2 k
depicts the source pmf with η =6. proof is based on the following argument. Suppose LC tech-
nique enables decoder j reconstruct S for j ∈[2], then both
j
2TheunderlyingpmfpU willbeclearfromcontext. the decoders can reconstruct S1 and S2 if each of them is
3(1−x)l≥1−xlforx∈[0,1]. provided Y and Y (and Y ). We then prove that this is not
4Strictsub-optimalityofLCtechniquecanbeinferredfrom[2].Toverify 1 2 0
permissible,byfollowinganargumentsimilarto[2,Sec.III.c].
this, modify the MAC therein, to an IC with identical outputs, and use the
arguments presented in proof of Lemma 1. Surprisingly, this has not been
documentedin[1]. 5Infact,(1−aη1k)loga−lokg2 ≤H(S1H),(HS)(S2),≤loga+hb(k1)+lokg2
3
Lemma 1: Consider Ex. 1 with any η ∈N. There exists an Case 1a : For some u ∈ U, P(U = u) ≥ 1 and P(U =
1 2 2
a ∈ N,k ∈ N, such that for any a ≥ a and any k ≥ k , u) ≥ 1. Then P(Y = u) ≥ 1 (independence of U ,U ) and
∗ ∗ ∗ ∗ 2 0 4 1 2
the sources and the IC described in Ex. 1 do not satisfy LC hence H(Y )≤log2+ 3loga.
0 4
conditions that are stated in [1, Thm. 1]. Case 1b : For some u ∈ U, P(U = u) ≥ 1 and P(U =
1 2 2
The proof is detailed in [13]. u) ≤ 1. Then P(U (cid:54)= u) ≥ 1 and hence P(Y = 0) ≥ 1
2 2 2 0 4
Proof: Since the sources do not have a GKW and hence H(Y )≤log2+ 3loga.
0 4
part, it suffices to prove that Ex. 1 does not satisfy Case 2a : For every u∈U, P(U =u)≤ 1. Then for any
conditions stated in [1, Corollary 1]. Let SQWXUY u ∈ U, P(U (cid:54)= U ) = (cid:80) (cid:80) 1P(U = u2)P(U = z) ≥
2 1 u z(cid:54)=u 2 1
be any collection of RVs whose pmf factorizes as 1(cid:80) P(U = u) = 1, implying P(Y = 0) ≥ 1 and hence
WSpQpW1|QpW2|QpX1U1|S1W1QpX2U2|S2W2QWY|XU. We H2 (Yu0)≤lo2g2+ 43log2a. 0 2
prove In all cases, we have H(Y )≤log2+3loga. Substituting
0 4
through (3) and above, we conclude
H(S)>I(S X U ;Y Y |QW)+I(W S X U ;Y Y |Q)
1 1 1 1 0 1 2 2 2 2 0
−I(S ;S ) (1) 2 3 log|Y|
1 2 I(XU;Y|Q)≤2log2+2C+h ( )+ loga+
b kaηk 4 k
andtherebycontradicting[1,Eqn.(44),Corollary2].Towards <loga
that end, we first note
for sufficiently large k,a.
1 1 1 kak
H(S) ≥ H(S )=h ( )+ log(ak−1)≥ log Remark 1: Why is the LC technique incapable of commu-
2 b k k k 2
nicating S? Any valid pmf p induced by a S-L coding
1 k U1U2
≥ loga+ log( )≥loga, schemeisconstrainedtotheLMCU −S −S −U .Forj ∈
1 1 2 2
k 2
[2],p canequivalentlybeviewedasU =g (S ,W ),for
whenever ak ≥ 2. Secondly, the RHS of (1) can be bounded someUfju|Snjction g and RV W , that satisfyjW ⊥⊥j Wj . Ojwing
j j 1 2
above by to the latter, W and/or W being non-trivial RVs, reduces
1 2
P(U =U ).Ifwelet,W ,W bedeterministic,theonlyway
I(S X U ;Y Y |QW)+I(W S X U ;Y Y |Q)−I(S ;S ) 1 2 1 2
1 1 1 1 0 1 2 2 2 2 0 1 2 to make U uniform is to pool less likely symbols. However,
j
≤I(S X U ;Y Y |QW)+I(WS X U ;Y|Q)−I(S ;S )
1 1 1 1 0 2 2 2 1 2 the source is ‘highly’ non-uniform, and even by pooling all
≤ I(WSXU;Y|Q)+I(S X U ;Y|QW) thelesslikelysymbols,wecangatheraprobability,ofatmost,
1 1 1
−I(S1X1U1;Y|QWS2X2U2)−I(S1;S2) k1.Consequently,anypU1U2 inducedviaaS-Lcodingscheme
is sufficiently far from any pmf that satisfies U = U whp
= I(XU;Y|Q)+I(S X U ;Y|QW) 1 2
1 1 1 and U =U close to uniform.
1 2
−I(S X U ;YX U |QWS )−I(S ;S ) (2)
1 1 1 2 2 2 1 2 Remark 2: An l−letter (multi-letter with l > 1) coding
= I(XU;Y|Q)+I(S1X1U1;Y|QW) schemeisconstrainedbyanl−letterLMCU1l−S1l−S2l−U2l.6
−I(S X U ;YX U S |QW) Suppose we choose l reasonably large such that 1) ξ[l](S) is
1 1 1 2 2 2
not high, and 2) Sl is reasonably uniform on its typical set
≤ I(XU;Y|Q) j
Tl(S ), and define U : j ∈ [2] through identical functions
δ j j
where (2) follows from Markov chains QSW − XU − Y Ul = g(Sl) : j ∈ [2], then one can easily visualize the
j j
and S X U − S QW − X U and (3) follows from the existence of g such that p satisfies the twin objectives of
1 1 1 2 2 2 UlUl
independence of S and WQ. Ul = Ul whp and Ul = U1l 2is close to uniform. Our coding
1 2 1 2
We now argue the RHS above is strictly lesser than loga. scheme, will in fact, identify such g maps. This portrays the
Our argument pretty much follows Dueck [2, Sec. III.c] sub-optimalityofS-Lschemesforjointsource-channelcoding.
verbatim and is provided here for the sake of completeness.
Let R=1 .
{(S1,S2)=(0k,0k)} A. Fixed Block-length Coding over a Noiseless Channel
I(XU;Y|Q)≤I(XUR;Y|Q)≤log2+I(XU;Y|Q,R)
In order to input codewords on the W −channel, that
log|Y ×Y ×Y | 1 Y0|U
≤log2+ 0 1 2 +(1− )I(XU;Y|Q,R=1) agree, we employ the same source code, same channel code
k k and same mapping, each of fixed B-L7 l, at both encoders. l
We focus on the third term in the above sum. Conditioned on ischosenlargeenoughsuchthatthesourcecanbereasonably
R=1,thesourcesareequalto(0k,0k).Itcanbeverifiedthat efficientlycompressed,andyetsmallenough,toensureξ[l](S)
X1U1−S1Q−S2Q−X2U2. Given Q=q,R=1, (X1,U1) isreasonablysmall.Werefertothesel−lengthblocksassub-
is independent of (X2,U2) and hence blocks. Since l is fixed, there is a non-vanishing probability
that these source sub-blocks will be decoded erroneously. An
I(XU;Y|Q,R=1)≤ max I(XU;Y|Q,R=1)
pX1U1pX2U2 outercode,operatingonanarbitrarilylargenumbermofthese
2 sub-blocks,willcarryinformationtocorrectforthese‘errors’.
≤2C+hb(kaηk)+pU1pmU2aWxY0|UH(Y0) (3) TheoutercodewilloperateoversatellitechannelWYj|Xj.We
begin with a description of the fixed B-L codes.
We now evaluate an upper bound on the maximum value of
H(Y0) subject to U1,U2 being independent. We evaluate the 6ConstraintwillnotbeviaaS-LLMC.
following three possible cases. 7irrespectiveofthedesiredprob.oferror
4
1
theWseouermceplocoydae,siamndplelefitxCed B=-LU(linbneert)hecocdhea.nLneetlTcδol(dSe1.)Lbeet LSl (2k3a−η2k,|Sj|)≤ 4k loga (8)
U
lA = (cid:98)logal(cid:99) bits, of the (cid:100)log|Tδl(S1)|(cid:101) bits output by the for sufficiently large a,k. Recall lB = (cid:100)log|Tδl(S1)|(cid:101)−lA.
source code, be mapped to CU. Both encoders use the same Substituting δ = 1, verify13
source code8, channel code and mapping. k
Suppose we communicate an arbitrarily large number m B ≤(2/l)+(1/k)loga+(1+(1/k))h (1/k). (9)
b
of these sub-blocks on W as above. Moreover, suppose
Y0|U
encoderj communicatestherestofthelB =(cid:100)log|Tl(S )|(cid:101)− Since h (2)−(1+ 1)h (1) ≥ 1 log k for large enough
δ 1 b k k b k 2k 256
lA bits output by its source code to decoder j on its satellite k, RHS of (8), (9) sum to at most 2 +h (2)+ 5 loga for
l b k 4k
channel W .9 How much more information needs to be large enough a,k. Furthermore, H(S |S )≤h ( 1 )+
Yj|Xj 2 1 b (k−1)aηk
communicated to decoder j, to enable it reconstruct Slm? We 2 loga ≤ h ( 2 )+ 1 loga for sufficiently large a,k. It
j aηk b kaηk 4k
do a simple analysis that suggests a natural coding technique. can now be easily verified that the satellite channels support
View the m sub-blocks of Sj as the rows of the matrix these rates for large enough a,k.
Sj(1 : m,1 : l) ∈ Sjm×l. Let Kˆj(1 : m,1 : l) ∈ S1m×l A few details with regard to the above coding technique is
denote decoder j’s reconstruction of S1(1 : m,1 : l)10. The worth mentioning. pKˆlKˆl|SlSl can in principle be computed,
m sub-blocks once the fixed block-1len2gt1h 2codes, encoding and decoding
(cid:110)(cid:16)S (t,1:l),Kˆ (t,1:l):j =1,2(cid:17):t∈[m](cid:111) (4) maps are chosen. Sjlm will be binned at rate H(Sjl|Kˆjl) and
j j the decoder can employ a joint-typicality based decoder using
aSrienciei,d1in1 pwriitnhciapnle,l−wleencgatnhodpiesrtaritbeubtiyontreWatiSn1lgS2lthpeKˆs1leKˆl2l−|Sl1leSn2lg1t2h. theThcoeomrpeumte1d:pTShjl|eKˆjlL.CWceocnodnitciloundsestthaetefdolilnow[1in,gT.hm. 1] are
sub-blocks as a super-symbol, and employ standard binning not necessary. Refer to Ex. 1. There exists a∗ ∈ N and k∗ ∈
technique over these m super-symbols, decoder j needs only N such that for any a ≥ a∗ and any k ≥ k∗, S1,S2 and
H(Sjl|Kˆjl) bits per source sub-block. We have no characteri- the IC WY|XU do not satisfy LC conditions, and yet, S is
zation of pKˆlKˆl|SlSl, and hence we derive an upper bound. transmissible over IC WY|XU.
1 2 1 2 Remark 3: Theaboveschemecruciallyreliesonthechoice
H(Sjl|Kˆjl)≤H(Sjl,1{Kˆl(cid:54)=Sl}|Kˆjl)≤hb(P(Kˆjl (cid:54)=S1l))+ ofl -neithertoobig,nortoosmall.Thisiselegantlycaptured
j 1 as follows. As l increases, ξ[l](S)→1, τ (and g )14 →0.
+P(Kˆl (cid:54)=Sl)log|Sl|+P(Kˆl =Sl)H(Sl|Sl). (5) l,δ ρ,l
j 1 j j 1 j 1 As l decreases, ξ[l](S) → ξ(S), and τ (and g ) → 1. If
l,δ ρ,l
≤lLSl (P(Kˆjl (cid:54)=S1l),|Sj|)+lH(Sj|S1), where φ→0.5, LSl (φ,|Sj|)→0.5log|Sj|= k2 loga.
1
LS(φ,|K|):= h (φ)+φlog|K|, (6)
l l b
IV. FIXEDBLCODESOVERANARBITRARYIC
represents the additional source coding rate needed to com-
pensate for the errors in the fixed B-L decoding. It suffices to Our analysis (Sec. III-A) focused on proving
prove LS(P(Kˆl (cid:54)=Sl),|S |)+B+H(S |S ) < capacity of
W .lSinceLjS(φ,1|K|)jisnon-decreasinjgi1nφifφ< 1,we LSl (φ,|Sj|)+B+H(Sj|S1)≤I(Xj;Yj) (10)
Yj|Xj l 2
tbhoautnedndP,(nKoˆtjle(cid:54)=thaSt1l{)Sb1ly(cid:54)=a qKˆujla}nt⊆ity{tSh1lat(cid:54)=isSl2el}ss∪t{hSan1l ∈21/.TTδlo(wSa1r)d}s. wouhrerseufφfici<ent12cwonadsitaionnsupwpeilrl btaokuendthoisn fPor(mKˆ.jlT(cid:54)=heSl1la)c.kAolfl
Indeed, Sl = Sl ∈ Tl(S ) implies both encoders input same isolationbetweenchannelscarryingfixedB-LandinfiniteB-L
1 2 δ 1
C −codeword and agree on the lB bits communicated to codes will throw primarily two challenges.15 We present our
U
their respective decoders. Therefore P(Sl (cid:54)=Kˆl)≤φ, where generalization in three pedagogical steps.
1 j
φ=τl,δ(S1)+ξ(S), In general, P(S1l (cid:54)=Kˆjl)≤τl,δ+ξ[l]+gρ,l, where the first
two terms are as in (7), and g is the probability that any
δ2l l ρ,l
τ (S )≤ 2akexp{− } and ξ[l](S)≤ . (7) of the decoders incorrectly decodes the C −codeword, con-
l,δ 1 2k2a2k kaηk U
ditionedonboth encoderschoosingthe same C codeword.16
U
Choose l = k4aη2k,δ = k1, substitute in (7). Since η ≥ 6, Our fixed B-L code CU will be a constant composition code,
verify φ≤2k3a−η2k < 1 for sufficiently large a,k. Verify and in the statements of all theorems, gρ,l is defined as
2
2
(cid:88)
noi89sTEehnlercsoosud.geIhrn2tohuaerlseodnedes,mcwrpieplotpiyorsnos,voewutrehcaetacstshouedmeraeTteδlc(owSme1m)d,eu(mnainacndadtinooonftWTovδle(rS2W))Y.ijs|Xlejsseisr gρ,l := j=1exp{−l(Er(A+ρ,pU,pYj|U)−ρ)}.
Yj|Xj In all our theorems, LS(φ,|S |) is defined as in (6), g as
thanitscapacity,justifyingthisassumption. l j ρ,l
101)EncoderjcouldinputanyarbitraryCU−codewordwhenitssub-block above, φ = τl,δ(K1)+ξ[l](K)+gρ,l will serve as an upper
Sitjlob∈/seTrvδle(sS1Y)0,la=nd0ld.e2c)odKˆer2j(1c:oumld,1de:cll)ariesaanlsoarvbiietrwareydraescorencsotrnuscttriuocntiownheonf bound on P(S1l (cid:54)=Kˆjl) that is less than 12.
S1(1:m,1:l).
11The l−length mappings from the source sub-blocks to the l−length 13UseH(S1)≤loga+hb(k1)and|Tδ(S1)|≤exp{l(1+δ)H(S1)}.
channel codewords input on the WY0|U are identical across the sub-blocks 14gρ,l isdefinedinthesequel.
andthesourcesub-blocksaremutuallyindependent. 15Andanadditionallossinthechannelrate,denotedLC(·,·).
12pKˆlKˆl|SlSl doesnotnecessarilyfactor,owingtothel−lengthmaps. 16ForEx.1,gρ,l=0,andweignoredit.ForgeneralIC,gρ,l isnon-zero.
1 2 1 2
5
A. Designing independent streams ignoring self-interference The above fact can be therefore be stated as Aπ(1 : m,i) ∼
(cid:81)m {1(cid:80)l p }.
The main challenges in generalizing pertain to 1) multi- t=1 l i=1 Ai
One can now easily prove that, if C is a constant compo-
plexing a fixed B-L code with an infinite B-L code through a U
sition code of type p , and m codewords are independently
singlechannelinput,and2)theeffectoferroneousconditional U
chosen18 from C and placed as rows of U , then for any
codingontheoutercode.WeadapttoolsdevelopedbyShirani U j
i ∈ [l], the interleaved vector Uπ(1 : m,i) ∼ (cid:81)m p . We
and Pradhan [16], [17] in the context of distributed source t=1 U
now build l−codebooks (independently drawn), one for each
coding. The following very simple generalization is chosen to
of these interleaved vectors.
illustrateourideas.Inparticular,welivewithself-interference
Following is our channel code structure. C is constant
between the two streams. U
Theorem 2: (S,W ) is transmissible over IC composition code of type pU and B-L l. Encoder j picks l
(X,Y,WY|X) if thSere exists (i) a finite set K, maps independent codes CVji : i ∈ [l], each iid ∼ (cid:81)mt=1pVj(·),
fj :Sj →K, with Kj =fj(Sj) for j ∈[2], (ii) l∈N,δ >0, Ueacπh(1B:-mL,im)..OCuVtejir ciosdemmuletispslaegxeedisaslpolnitginwtoithl esquuba-lvepcatrotsr
(iii) finite set U,V ,V and pmf p p p p p j
defined on U ×V×1X,2where pU isUa tVy1peVo2fXse1q|UuVe1ncXes21|U7Vin2 (Mj1,··· ,Mjl). Vj ∈ Vm×l is defined as Vπj(1 : m,i) =
Ul, (iv) A,B ≥0, ρ∈(0,A) such that φ∈[0,0.5), CVji(Mji). For each (t,i)∈[m]×[l], Xj(t,i) is chosen IID
wrtp (·|U (t,i),V (t,i)).SymbolsinX ∈Xm×l are
A+B ≥(1+δ)H(K ), and for j ∈[2], Xj|UVj j j j j
1 input on the channel. It can be verified that (1) the codewords
B+H(Sj|K1)+LSl (φ,|Sj|)<I(Vj;Yj)−LCj (φ,|V|), of CU and CVji pass through an IID memoryless channels
whosetransitionprobabilitiesare‘characterized’inthesequel.
where, φ:= g +ξ[l](K)+τ (K ), LC(φ,|U|)=h (φ)+
ρ,l l,δ 1 j b Since each codebook C and each codeword is IID,
φlog|U|+|Y||U|φlog 1. Vji
Remark 4: The charaφcterization provided in Thm. 2 (and Uenjsu(tr,e1s :Vl)π→(1Y: jm(t,,i1):→l) iYs IπI(D1p:Yjm|U,−i)chisanInIDel.. IBnutetr,leuanvliensgs
j j
those in Thm. 3, 4)) is via S-L PMFs and S-L expressions.
U = U , we are not guaranteed the latter channel is
1 2
Remark 5: In the above, the fixed B-L code operates over (cid:81)p .19 In fact, we only know certain marginals of
Kl instead of Sl. LC(φ,|U|) quantifies the loss in rate of the Yj|Vj
j j j
outer code due to erroneous conditional coding. Note that, in Uπ(1:m,i),Vπ(1:m,i),Xπ(1:m,i),Yπ(1:m,i) : j∈[2]
j j j j
Ex.1,satellitechannelremainedunaffectedwhentheencoders Let (Vπ(1 : m,i),Yπ(1 : m,i)) ∼ (cid:81)p , where
placed different CU codewords. The latter events imply, the p = jp . We wish toj bound the differenceVˆIj(YˆVj ;Y ) −
dVacijftfu−earPleYrncjochoecafhn:iannneWnlt.heeeNl eiomsltaeunbtotuohtaraalptteYiLnjf|CjoVonj(rφm. ,Ltah|tUCjeio(|n)φn→,eb|weUt0w|)eaeliesesmnφaep→nbYtojs|.u0Vn.jTdahnoednrttehhseet tIphVV(ˆˆeVjjˆYˆjjp;|mYUˆˆj1f)UˆV2oj(ffvro,tmyh|eu,aiubno)tevre=l.eavUpeVsdjinYjgv|eUct(thvoe,ry|rUuel)π1a,t(i1wonhs:erepmUˆ(cid:81),ji)pUj=Uˆ1π2Uˆ(j21pUis:,
(cid:80)
follows from standard arguments [13]. The source-coding m,i), u1(cid:54)=u2pUˆ1Uˆ2 ≤ φ, we can prove |pVjYj(v,y) −
module, and the mappings to the channel-coding module are pVˆjYˆj(v,y)| ≤ φ. These steps are analogous to those in [12,
identical to Section III-A. We describe the structure of C Appendix B]. Using [15, Proof of Lemma 2.7], we conclude
U
and how it is multiplexed with the outer code built on Vj. I(Vj;Yj)−I(Vˆj;Yˆj)≤LC(φ,|U|).Werefertoreaderto[13]
If we build a single code C of B-L lm and multiplex it for the rest of the proof which is quite standard.
Vj
with m blocks of C , then C does not experience an IID Lemma 2: The conditions stated in Thm. 2 are strictly
U Vj
memoryless channel. weaker than those stated in [1, Thm. 1].
Let U (t,1 : l) denote encoder j’s chosen codeword from Proof: Ex. 1, with a,k chosen sufficiently large, satisfies
j
CU corresponding to the tth sub-block of Kj(t,1 : l). We conditions stated in Thm 2. In particular, choose δ = k1,ρ =
seek to identify sub-vectors of Uj that are IID, and whose 1,A = (1− k13)loga,B = H(S1)−A,l = k4aη2k,K = Sj,
pmf we know. We can then multiplex the outer code along fj = identity, Vj = Xj,pU uniform,pVj capacity achieving.
these sub-vectors. Interleaving [17] enables us do this. The result now follows from Lemma 1.
Suppose, for t ∈ [m], A(t,1 : l) ∼ p and the m vectors
Al
A(1,1:l),··· ,A(m,1:l)areiidp .Letπ :[l]→[l]:t∈
Al t B. Additional information via Message-Splitting
[m]bemsurjectivemaps,thatareindependentanduniformly
We now employ Han-Kobayashi technique to communicate
chosen among the collection of surjective maps (permuters).
therestoftheinformation(LHSof(10)).Towardsthatend,let
Then, for each i∈[l], the m−length vector
HK (p p ) be the Han-Kobayashi inner bound
A(1,π (i)),A(2,π (i)),··· ,A(m,π (i))∼(cid:89)m {1(cid:88)l p }. defined iVn1W[11,XP1roVp2oWs2itXio2n 3].
1 2 m l Ai Theorem 3: (S,W ) is transmissible over IC W if
t=1 i=1 S Y|X
there exists (i) a finite set K, maps f : S → K, with K =
j j j
[12, Appendix A] contains a proof. The following notation
f (S ) for j ∈[2], (ii) l∈N,δ >0, (iii) finite set U,V ,W :
will ease exposition. For A ∈ Am×l, and a collection π : j j j j
t j ∈[2] and pmf p p p p p defined
[l] → [l] : t ∈ [m] of surjective maps, we let Aπ ∈ Am×l be U V1W1 V2W2 X1|UV1W1 X2|UV2W2
such that Aπ(t,i) : = A(t,π (i)) for each (t,i) ∈ [m]×[l].
t 18Notnecessarilyuniformly.Infacttheindexoutputfromthesourcecode,
owingtothefixedblock-lengthlisnotnecessarilyuniform
17Pleasereferto[15,Defn2.1] 19WeareunawareofthetransitionprobabilitiesofthisIIDPTP.
6
on U ×V ×W ×X, where pU is a type of sequences in Ul, [9] C.Tian,S.Diggavi,andS.Shamai,“TheAchievableDistortionRegion
(iv) A,B ≥0, ρ∈(0,A) such that φ∈[0,0.5), of Sending a Bivariate Gaussian Source on the Gaussian Broadcast
channel,”Trans.onInfo.Th.,vol.57,no.10,pp.6419–6427,Oct2011.
A+B ≥(1+δ)H(K ), and for j ∈[2],
1 [10] I. E. Aguerri and D. Gu¨ndu¨z, “Correlated Gaussian sources over
(cid:16) (cid:17)
B+LC(φ,|UVW|) :j ∈[2] ∈HK (p p ), Gaussian weak interference channels,” in 2015 IEEE ITW, Oct 2015,
H(Sj|K1)+LSl(φ,|Sj|) V1W1X1 V2W2X2 pp.84–88.
1 [11] A.Padakandla,“CommunicatingcorrelatedsourcesoveraMACinthe
LCj (φ,|U|)=hb(φ)+5φlog|UVW|+||UVW|3φlogφ. absence of a Ga´cs-Ko¨rner common part,” in 2016 IEEE International
SymposiumonInformationTheory(ISIT),July2016,pp.2014–2018.
[12] ——, “Communicating Correlated Sources Over a MAC in
where, φ,LS(φ,|S |) are as defined in Thm 2.
l j the absence of a Ga´cs-Ko¨rner Common Part,” available at
Remark 6: For simplicity and compact description, we de- http://arxiv.org/abs/1601.07091.
rive a uniform upper bound on all the mutual-information [13] ——,“CommunicatingCorrelatedSourcesOveranInterferenceChan-
nel,”availableathttps://arxiv.org/abs/1701.06977.
quantities involved in the description of the Han-Kobayashi
[14] T. Cover, A. Gamal, and M. Salehi, “Multiple access channels with
region. This explains the large constant multiplying φlog 1. arbitrarily correlated sources,” Information Theory, IEEE Transactions
φ
Our third step is to use the decoded fixed B-L channel code- on,vol.26,no.6,pp.648–657,nov1980.
[15] I. Csisza´r and J. K¨orner, Information Theory: Coding Theorems for
words towards conditional decoding of the outer code. The
DiscreteMemorylessSystems,2nded. CambridgeUniv.Press,2011.
outer code is built on X and is superimposed on (interleaved [16] F. Shirani and S. Pradhan, “Finite block-length gains in distributed
j
vectors of) C . The challenge is that a fraction φ of the sourcecoding,”inProc.IEEE2014ISIT,June2014,pp.1702–1706.
U [17] F.Chaharsooghi,A.Sahebi,andS.Pradhan,“Distributedsourcecoding
decoded codewords are erroneous. The approach is to treat
inabsenceofcommoncomponents,”inProc.2013ISIT,pp.1362–66.
theinterleavedcolumnsofthedecodedUˆ asanoisystate/side
information. Interleaving ensures that these sub-vectors have
a S-L IID pmf. Proof is similar to [12, Proof of Thm. 1].
Theorem 4: (S,W ) is transmissible over an IC W if
S Y|X
there exists (i) a finite set K, maps f : S → K, with K =
j j j
f (S ) for j ∈ [2], (ii) l ∈ N,δ > 0, (iii) finite set U and
j j
pmf p p p defined on U ×X, where p is a type
U X1|U X2|U U
of sequences in Ul, (iv) A,B ≥ 0, ρ ∈ (0,A) such that φ ∈
[0,0.5), where
A+B ≥(1+δ)H(K ), and for j ∈[2],
1
B+H(S |K )+LS(φ,|K|)<I(X ;Y |U)−LC(φ,|U|)
j 1 j j j
1
LC(φ,|U|)=h (φ)+φlog|U|+|X ||Y||U|(1+|X |)φlog
j b j j φ
and φ, LS(φ,|K|) are as defined in Thm 2.
The final step in our generalization will combine the tech-
niques of Thm. 3, 4. In particular, we employ Han-Kobayashi
technique in the superposition layer. The message to be
communicated through the outer code is split into private and
public parts and coded using separate codebooks. Decoder j
uses the decoded fixed B-L channel codeword and employs a
conditional Han-Kobayashi decoder. We omit a characteriza-
tion in the interest of brevity.
REFERENCES
[1] W.LiuandB.Chen,“InterferenceChannelswithArbitrarilyCorrelated
Sources,”Trans.onInfo.Th.,vol.57,no.12,pp.8027–8037,Dec2011.
[2] G.Dueck,“Anoteonthemultipleaccesschannelwithcorrelatedsources
(corresp.),”Trans.onInfo.Th.,vol.27,no.2,pp.232–235,mar1981.
[3] T. Cover, A. Gamal, and M. Salehi, “Multiple access channels with
arbitrarilycorrelatedsources,”Trans.onInfo.Th.,pp.648–657,Nov
1980.
[4] K. Khezeli and J. Chen, “A Source-channel Separation theorem with
application to the source broadcast problem,” IEEE Trans. Info. Th.,
vol.62,no.4,pp.1764–1781,2016.
[5] Y.Murin,R.Dabora,andD.Gunduz,“OnJointSource-ChannelCoding
forCorrelatedsourcesoverMultiple-AccessRelayChannels,”Trans.on
Info.Th.,vol.60,no.10,pp.6231–6253,Oct2014.
[6] A.LapidothandM.Wigger,“Anecessaryconditionforthetransmissi-
bilityofcorrelatedsourcesoveraMAC,”in2016ISIT,July2016.
[7] P. Minero, S. H. Lim, and Y.-H. Kim, “A unified approach to Hybrid
Coding,”Trans.onInfo.Th.,vol.61,no.4,pp.1509–1523,April2015.
[8] A. Lapidoth and S. Tinguely, “Sending a Bivariate Gaussian over a
GaussianMAC,”Trans.onInfo.Th.,vol.56,pp.2714–2752,2010.