Secret Key Agreement under Discussion Rate Constraints

0.22 MB·

by  Chung Chan

Secret Key Agreement under
Discussion Rate Constraints
Chung Chan, Manuj Mukherjee, Navin Kashyap and Qiaoqiao Zhou
Z Z Z
lemA,bnsterwacts—inFgloer-letthteermlouwlteitrerbmoiunnadlsseacrreetobkteayinaedgreoenmtehnetppurbolbic- 11 Xa 22 Xb 33
discussion rate required to achieve any given secret key rate X
c
below the secrecy capacity. The results apply to general source
7 model without helpers or wiretapper’s side information but can Fig. 1: The graphical representation of the PIN (2.1). Each
1 bestrengthenedforhypergraphicalsources.Inparticular,forthe edgecorrespondstoanindependentrandomvariableobserved
0 pairwiseindependentnetwork,theresultsgiverisetoacomplete
by the incident nodes.
2 characterization of the maximum secret key rate achievable
n under a constraint on the total discussion rate.
a
J I. INTRODUCTION lower bound was furthersingle-letterized and simplified to an
9 We consider the multiterminal secret key agreement by easily computable bound in [10], where the condition for the
1 optimality of omniscience was also generalized from PINs to
public discussion in [1] under the source model without
hypergraphical sources [11], using the idea of decremental
helpersor wiretapper’sside information.While the maximum
]
secret key agreement in [12] for the upper bound [13].
T achievable secret key rate with unlimited public discussion,
Unfortunately, the lower bound can be loose even for simple
I called the secrecy capacity, was characterized in [1] using an
. PINs. It was also conjectured that the lower bound failed to
s achieving scheme through the omniscience of the source, it
c givetheconditionfortheoptimalityofomniscienceforgeneral
[ waspointedout[1]thattheproposedschememaynotachieve
theminimumpublicdiscussionrate,referredtoasthecommu- sources.
2 By resolving the conjecture in [10], we discovered new
nication complexity.While a multi-letter characterization was
v techniquesthatcanimprovethelowerboundfurther.Although
derived in [2] for the 2-user case, a computable single-letter
8
the techniques are also based on the idea of MMI, they work
0 characterization is a challenging open problem.
quite differently compared to the idea of Wyner common
0 Simpler versions of the problem have been considered,
5 such as the introduction of the vocality constraints in [3– information [8]. We apply these techniques to obtain an
0 outer bound on the region of achievable secret key rate and
5]. Using the result of [3] with silent users and viewing
. discussion rate tuples. In particular, for PIN models on trees
1 the secrecy capacity as the multivariate mutual information
0 our outer bound turns out to be an exact characterization.
measure (MMI) [6], these simpler problems can be resolved
7 In contrast with the rate region characterized in [14] for
completely [7]. Combining the idea of Wyner common infor-
1 two terminals using the idea of two-way interactive source
: mation and the MMI, a multi-letter lower bound on the com-
v coding [15], the result is the first instance of an exact and
munication complexity was derived in [8]. For the pairwise
i easily computable characterization for the case with at least
X independent network (PIN) [9], the bound leads to a precise
threeterminalswithunlimitednumberofroundsofinteractive
r single-letter condition in [8] under which the omniscience
a strategy in [1] achieves the communication complexity. The discussion. We also use the outer bound to characterize the
communicationcomplexity,andmoregenerally,themaximum
secretkeyrateachievableunderanygiventotaldiscussionrate,
C.Chan(email:[email protected]),andQ.ZhouarewiththeInstitute
of Network Coding at the Chinese University of Hong Kong, the Shenzhen referred to as the rate-constrained secrecy capacity.
KeyLaboratoryofNetworkCodingKeyTechnologyandApplication,China,
andtheShenzhenResearchInstituteoftheChineseUniversityofHongKong. II. MOTIVATION
N. Kashyap and M. Mukherjee are with the Department of Electrical
Communication Engineering attheIndianInstitute ofScience, Bangalore. We first motivate the idea of secret key agreement and the
TheworkdescribedinthispaperwassupportedbyagrantfromUniversity main results informally using a simple example. Let X , X
Grants Committee of the Hong Kong Special Administrative Region, China a b
(ProjectNo.AoE/E-02/08),andsupportedpartiallybyagrantfromShenzhen andXc be uniformlyrandomandindependentbits, anddefine
ScienceandTechnologyInnovationCommittee(JSGG20160301170514984),
theChinese University ofHongKong(Shenzhen), China. Z1 :=Xa
The work of C. Chan was supported in part by The Vice-Chancellor’s Z :=(X ,X ,X ) (2.1)
One-offDiscretionaryFundofTheChineseUniversityofHongKong(Project 2 a b c
Nos.VCF2014030andVCF2015007),andagrantfromtheUniversityGrants Z :=( X ,X ).
3 b c
CommitteeoftheHongKongSpecialAdministrative Region,China(Project
No.14200714). Consider 3 users 1, 2 and 3 observing Z , Z and Z
1 2 3
The work of N. Kashyap and M. Mukherjee is supported in part by a respectivelyinprivate.Theprivatesource(Z ,Z ,Z )iscalled
Swarnajayanti Fellowship awarded to N. Kashyap by the Department of 1 2 3
Science &Technology (DST),GovernmentofIndia. aPIN[9,16]inthesensethatitsstatisticaldependencycanbe
describedbya(multi-)graphasshowninFig.1withthenodes source denoted by the random vector
representing the users, X represented by an edge incident
on nodes 1 and 2, and Xa and X represented by two edges ZV :=(Zi|i∈V)∼PZV taking values from
b c
incident on nodes 2 and 3. ZV := Zi, assumed to be finite.
i∈V
If user 2 reveals F := X ⊕X in public so that everyone
a b Y
canobserveit, thenuser3 canrecoverX asF⊕X .K:=X N.b., capital letters in sans serif font are used for random
a b a
variables and the corresponding capital letters in the usual
is called a secret key bit generated by the public discussion
F because K is not only recoverable by all users but also math italic font denote the alphabet sets. PZV denotes the
joint distribution of Z ’s. The protocol can be divided into
uniformly random and independent of the public discussion i
F. A general asymptotic secret key agreement protocol by the following phases:
interactive public discussion was formulated in [1], where
Private observation: Each user i∈V observes an n-sequence
the maximum achievablekey rate, called the secrecy capacity
anddenotedbyCS, wascharacterizedby a single-letterlinear Zni :=(Zit|t∈[n])=(Zi1,Zi2,...,Zin)
program.Forthecurrentexample,itiseasytoseethatC =1,
sinceuser1observesatmost1bitinprivateand1bitofSsecret i.i.d. generated from the source Zi for some block length n.
key is achievable by the above discussion scheme. Private randomization: Each user i ∈ V generates a random
variable U independent of the private source, i.e.,
A quantity of interest but not characterized in [1] is the i
smallestpublicdiscussionrate requiredtoachievethesecrecy
H(U |Z )= H(U ). (3.1)
V V i
capacity, called the communication complexity and denoted
i∈V
by R . For the current example, R ≤ 1 because the above X
S S
capacity-achievingdiscussion F is 1 bit. However, the precise For convenience, we denote the entire private observation of
user i∈V as
characterizationof R hasbeenunknownevenforthe current
simple example. S Z˜i :=(Ui,Zni). (3.2)
In this work, we introduce new techniques that not only Public discussion: Using a public authenticated noiseless
impliesRS =1 for the currentexamplebutalso characterizes channel, each user i ∈ V broadcasts a message in round t
themaximumkeyrateunderatotalpublicdiscussionrateR≥
0, called the rate-constrainedsecrecy capacityand denotedby F :=f (Z˜ ,F˜ ) where (3.3a)
it it i it
C (R). For the current example, it will follow that
S F˜ :=(F ,Ft−1), (3.3b)
it [i−1]t V
CS(R)=min{R,1}. (2.2) t ∈ [ℓ] for some positive integer ℓ number of rounds, F[i−1]t
consistsofthepreviousmessagesbroadcastinthesameround,
Although it is easy to see that C (0) ≥ 0 and C (R) = 1, while Ft−1 denotes the messages broadcast in the previous
S S V
for R ≥ 1, and that C (R) ≥ min{R,1} by time sharing, rounds. Without loss of generality, we assume this interactive
S
proving the reverse inequality is non-trivial and calls for new discussionisconductedintheascendingorderofuserindices.
techniquesnotcoveredby[8,10]. Indeed,ourtechniqueswill We also write
also imply that only user 2 needs to discuss in public, and so
F :=F =(F |t∈[ℓ]) (3.3c)
a secret key rate of r ∈ [0,1] is achievable by a discussion i i[ℓ] it
rate tuple (r1,r2,r3)Kiff they belong to the region F:=FV =(Fi|i∈V) (3.3d)
to denote the aggregate message from user i ∈ V and
R ={(r ,(r ,r ,r ))|r ∈[0,1],
K 1 2 3 K (2.3) the aggregation of the messages from all users respectively.
r1 ≥0,r2 ≥rK,r3 ≥0}. Key generation: A random variable K, called the secret key,
is required to satisfy the recoverability constraint that
This matches our intuition, since users 1 and 3 have indepen-
dent private observations, i.e., Z1 is independent of Z3, and lim Pr(∃i∈V,K6=θi(Z˜i,F))=0, (3.4)
so only user 2 can help them share a non-trivial secret key. n→∞
It turns out that the techniques apply to more general source for some function θi, and the secrecy constraint that
model with private randomization and interactive discussion 1
lim [log|K|−H(K|F)]=0, (3.5)
allowed as in [1]. It also completely characterizes CS(R) for n→∞n
the PIN model.
whereK denotesthefinitealphabetsetofpossiblekeyvalues.
III. PROBLEMFORMULATION Definition 3.1 Given the private source Z , a secret key rate
V
r is achievable by the public discussion rate tuple r :=
We consider the multiterminal secret key agreement [1] K V
(r |i∈V) iff
without helpers or wiretapper’s side information. It involves i
a finite set V := [m] := {1,2,...,m} of m ≥ 2 users. The 1 1
r ≤liminf log|K| and r ≥limsup log|F |, (3.6)
users have access to a private (discrete memoryless multiple) K n→∞ n i n→∞ n i
in addition to (3.4) and (3.5). The set of achievable (r ,r ) Proposition 3.2 ([9, 16]) For a PIN with weight c, there is a
K V
is denoted by R. The rate-constrained secrecy capacity is secretkeyagreementscheme,calledthetree-packingprotocol,
defined for R≥0 as which achieves (r ,r )∈R with
K V
CS(R):=max{rK |(rK,rV)∈R,r(V)≤R}, (3.7) rK := ηj and ri := (dTj(i)−1)ηj for i∈V,(3.13a)
where, for convenience, r(B):= i∈Bri for B ⊆V. ✷ jX∈[k] jX∈[k]
where k is a non-negativeinteger; η ∈R is a non-negative
Proposition 3.1 CS(R) is continPuous, non-decreasing and j +
real number; T := (V,E ) is a spanning tree with edge set
concave for R≥0. ✷ j j
E ⊆V2\{(i,i)|i∈V} satisfying
j
PROOF Continuity is because the liminf and limsup in (3.6)
always exist, since CS(R) is boundedwithin [0,H(ZV)]. The ηj ≤c(B) ∀B ∈2V \{∅}, (3.13b)
monotonicityisobvious,andconcavityfollowsfromtheusual
j∈[kX]:B∈Ej
time sharing argument. (cid:4)
which is the constraint for fractional tree-packing [17]; and
The unconstrained secrecy capacity defined and character- d (i) is the degree of node i in T . Furthermore, the un-
Tj j
ized in [1] is the special case constrained secrecy capacity C is the maximum r over the
S K
fractional tree packing {(ηj,Tj)|i∈[k]}. ✷
C := lim C (R) (3.8)
S S
R→∞ However, it was left as an open problem in [9] whether the
=C (R )=H(Z )−R
S CO V CO above scheme achieves R . We resolve this in the affirmative
S
where R is the smallest rate of communication for omni- by providing a matching converse.
CO
science, characterzied in [1] by the linear program
IV. MAIN RESULTS
R =min{r(V)|r(B)≥H(Z |Z ),∀B (V}. (3.9)
CO B V\B We will make use of the following alternative character-
It was also mentioned in [1] that the unconstrained capacity ization of the unconstrainted secrecy capacity in [11]: For
can be attained by a possibly smaller discussion rate, referred the no-helper case, CS = I(ZV) where I(ZV) is called the
to as the communication complexity multivariate mutual information (MMI) defined as
RS :=min{r(V)|(CS,rV)∈R} (3.10) I(ZV):= min IP(ZV), with (4.1a)
P∈Π′(V)
=min{R≥0|C (R)=C }≤R .
S S CO 1
I (Z ):= H(Z )−H(Z ) (4.1b)
Our goal is to characterize or bound CS(R) and R using P V |P|−1(cid:20) C∈P C V (cid:21)
only single-letter expressions. We will also specialize and X
=D(PZVkQC∈PPZC)
strengthen the results to the hypergraphicalsource model:
and Π′(V) being the se|t of partitio{nzs of V into} at least 2
Definition 3.2 (Definition 2.4 of [11]) Z is a hypergraphi-
V non-empty disjoint subsets of V. The conditional versions
cal source w.r.t. a hypergraph (V,E,ξ) with edge functions I(Z |W′) and I (Z |W′) are defined in the same way but
V P V
ξ : E → 2V \ {∅} iff, for some independent (hyper)edge with the entropy terms conditionedon W′ in addition. D(·k·)
variables X for e∈E with H(X )>0,
e e istheKullback–Leiblerdivergence,whichisnon-negative,and
soareI andI .Itwaspointedoutin[6]thatthesetofoptimal
Z :=(X |e∈E,i∈ξ(e)), for i∈V. (3.11) P
i e solutions form a lattice w.r.t. the partial order P′ (cid:23)P iff
The weight function c : 2V \{∅} → R of a hypergraphical
∀C ∈P,∃C′ ∈P′ :C ⊆C′.
source is defined as
c(B):=H(X |e∈E,ξ(e)=B) with support (3.12a) Hence, there exists a unique finest optimal partition, denoted
e
by P∗(Z ) and referred to as the fundamental partition.
supp(c):= B ∈2V \{∅}|c(B)>0 (3.12b) V
Furthermore, both the MMI and the optimal partitions can
The PIN mod(cid:8)el [9] such as (2.1) is an(cid:9)example, where the be computedin stronglypolynomialtime w.r.t. the numberof
corresponding hypergraph is the graph in Fig. 1 with weight evaluation of the entropies.
c({1,2}) = H(Xa) = 1, c({2,3}) = H(Xb,Xc) = 2 and 0 In the bivariate case when V ={1,2}, the MMI reducesto
otherwise. Shannon’s mutual information
Definition 3.3 ([9]) ZV is a PIN iff it is hypergraphicalw.r.t. I(Z )=I(Z ∧Z )=H(Z )+H(Z )−H(Z ,Z ),
{1,2} 1 2 1 2 1 2
a graph (V,E,ξ) with edge function ξ : E → V2 \{(i,i) |
i∈V} (i.e., no self loops). ✷ because {{1},{2}} is the unique partition in Π′({1,2}) (and
is therefore the fundamental partition P∗(Z )).
For this special source model, there is a protocol in [16, {1,2}
We begin with some general lower bounds on the public
ProofofTheorem3.3]thatachievestheunconstrainedsecrecy
discussion rates:
capacity [16, (15),(17)].
Theorem 4.1 For any (rK,rV)∈R, we have Z1
1
r(V \B)≥(|P|−1)[r −I (Z )] (4.2)
K P B
X X
for any B ⊆V with size |B|>1 and P ∈Π′(B). ✷ c a
PROOF See Appendix A. (cid:4) Z 3 Xb 2 Z
3 2
(4.2)is a lower boundon the totaldiscussion rate r(V \B)
of the subset V \B of users required to achieve a secret key Fig. 2: The triangle PIN defined in (4.6).
rate of r , for any choice of subset B of more than one user.
K
Choosing P to be the fundamentalpartition P∗(Z ) in (4.2),
B
I (Z ) = I(Z ), which gives the following lower bound in The lower bound is achievable by Proposition 3.2, hence
P B B
terms of the MMI. completing the proof of (4.5a). (cid:4)
Corollary 4.1 For any (r ,r )∈R, we have The current example has a weight function c with
K V
r(V \B)≥(|P∗(Z )|−1)[r −I(Z )] (4.3) supp(c)={{1,2},{2,3}},
B K B
which is a spanning tree with node degrees given by
for any B ⊆V with size |B|>1. ✷
NotethatI(Z )in (4.3)isthesecrecycapacitywhenusersin d(1)=d(3)=1 and d(2)=2,
B
V \B areremoved.Hence,toachieveasecretkeyratebeyond
which gives the lower bound (4.4) and hence the region
I(Z ), users in V \B must discuss. (4.3) states that the total
B in (2.3). The capacity is the minimum edge weight, i.e.,
discussionrateofusersinV\B isatleasttheadditionalsecret
keyrater −I(Z )amplifiedbyafactorof|P∗(Z )|−1≥1. C =min{c({1,2}),c({2,3})}=min{1,2}=1.
K B B S
Applying (4.2) to the example in Section II with B =
Unfortunately, the lower bound (4.2) can be loose for PIN
{1,3},P ={{1},{3}} (or simply (4.3)), we have
with cycles. E.g., consider a triangle PIN with V :=[3] and
r2 ≥(2−1)[rK−I(Z1∧Z3)]=rK (4.4) Z1 :=(Xa, Xc)
This is achievable as mentioned in Section II by time sharing Z2 :=(Xa,Xb ) (4.6)
between (rK,(r1,r2,r3)) = (0,(0,0,0)) and (1,(0,1,0)) ∈ Z3 :=( Xb,Xc)
R. Since C = I(Z ) ≤ I(Z ∧ Z ) = 1 and is
S {1,2,3} {1,2} 3 whereX ,X ,X areindependentuniformlyrandombits.This
achievable, we have (2.3) as the achievable rate region R. a b c
a PIN with correlation represented by a triangle in Fig. 2. It
More generally,
follows from (3.8), (3.9) and (3.10) that
Theorem 4.2 For PIN with weight c such that supp(c),
C =R =1.5≥R .
defined in (3.12), forms a spanning tree, we have S CO S
R ={(r ,r )|r ∈[0,C ], In particular, the secret key rate of 1 is achievable by the
K V K S (4.5a) scheme described in Section II.
r ≥(d(i)−1)r ,i∈V}, where
i K Applying (4.2) with B = {1,3} and P = {{1},{3}} as
CS =min{c({i,j})|{i,j}∈supp(c)}, (4.5b) before,
r ≥r −I(Z ∧Z )=r −1.
and d(i) is the degree of node i in the spanning tree. ✷ 2 K 1 3 K
PROOF Since the source model forms a Markov tree w.r.t. This is the best possible bound involving r2 over all possible
thespannnigtreegivenbysupp(c), theunconstrainedsecrecy choices of B and P, but it is trivial when rK ≤ 1. By
capacity (4.5b) follows from [1, (36)]. symmetry,the best boundsfor r1 and r3 are also trivial when
To prove (4.5a), consider any PIN with weight function c rK ≤1.
such that supp(c) forms a spanning tree. For any i ∈ V, Nevertheless,we discovereda differentboundingtechnique
choose B =V \{i} and let P be the connected components thatcangiveanon-trivialboundintheabovecase,byexploit-
of the spanning tree after node i and its incident edges are ing the hypergraphicaldependency structure of the source:
removed. It follows that P ∈Π′(B) with
Theorem 4.3 Forhypergraphicalsource,wehave(r ,r )∈
K V
|P|=d(i) and I (Z )=0 R only if
P B
due to the fact that supp(c) forms a spanning tree. By (4.1) α(P)r(V)≥[1−α(P)]rK ∀P ∈Π′(V), where (4.7a)
in Theorem 4.2, we have max |{C ∈P |C∩ξ(e)6=∅}|−1
α(P):= e∈E (4.7b)
r ≥(|P|−1)[r −I (Z )] |P|−1
i K P B
=(d(i)−1)r . and ξ is the edge function of the hypergraph in (3.11). ✷
K
N.b., it is easy to see that α(P) ∈ [0,1] because the maxi- unclear how one can generalize (4.7) to more generalsources
mization in the numerator of (4.7b) is the maximum number thatarepossiblynon-hypergraphical.Anotherinterestingopen
of blocks in P that an edge e ∈ E can intersect, which is problem is to characterize R for PINs with cycles, thereby
between 1 and |P|. If α(P) = 0 for some P ∈ Π′(V), then improving Theorem 4.2 to allow for cycles.
(4.7a) becomes r ≤ 0, i.e., C = 0. This happens when no Theboundin(4.7)canbelooseforhypergraphicalsources.
K S
edge crossesP, i.e., the source correspondsto a disconnected A trivial example is where V :=[3] and
hypergraph.
Z :=(X , X )
1 a c
PROOF See Appendix B. (cid:4) Z :=(X ,X ,X )
2 a b c
For the current example, choose P = {{1},{2},{3}}. For Z := (X ,X ).
3 b c
eachedgee, {C ∈P|C∩ξ(e)6=∅} simplifiestothenumber
The numerator of α(P) in (4.7b) is 0 for any P, as the
of incident n(cid:12)odes, which is always 2(cid:12)for graphs. Hence, mininum is achieved by the hyperedge c incident on all
(cid:12) (cid:12)
2(cid:12)−1 1 (cid:12) 1− 1 the nodes. Hence, α(P) = 0 and so (4.7) becomes trivial.
α(P)= 3−1 = 2 and so r(V)≥ 1 2rK =rK. However, with B = {1,3} and P = {{1},{3}}, (4.2) gives
2 r ≥ r −1, which is non-trivial for 1 < r ≤ 2 = C .
2 K K S
Since CS =RCO =1.5, the lower bound above is achievable We also conjecture that (4.2) and (4.7) are both loose for the
by time-sharing, which gives example where V :=[6] and
C (R)=min{R,1.5} and so R =1.5. Z :=(X , X )
S S 1 a d
Surprisingly, the argument can be extended to any PIN for a Z2 :=(Xa,Xb )
completecharacterizationofthecommunicationcomplexityas Z :=(X ,X , X )
3 a b d
well as the rate-constrained secrecy capacity. Z :=( X ,X ,X )
4 b c d
Theorem 4.4 For PIN, Z :=( X ,X )
5 b c
R Z6 := Xc.
C (R)=min ,C , (4.8)
S (cid:26)|V|−2 S(cid:27) We conjecture that (rK,rV)∈R only if
which gives RS =(|V|−2)CS. ✷ r(V)≥1.5r ,
K
PROOF The converse follows from (4.7a) with P ={{i}|i∈
which is achievable using the idea of secret key agreement
V}. More precisely, the minimization in the numerator of
by network coding [11]. It can be shown that the best lower
α(P) is always equal to 2 as it is the number of incident
nodes of an edge. Hence, bound from (4.2) and (4.7) is r(V) ≥ rK. Hence, we expect
thatresolvingtheconjectureintheaffirmativepotentiallyleads
1
α(P)= and so to new techniques for obtaining better lower bounds on the
|V|−1
public discussion rate required for secret key agreement.
r(V)≥(|V|−2)r by (4.7a).
K
APPENDIX A
The lower bound can be shown to be achievable by Proposi-
PROOF OF THEOREM4.1
tion 3.2. With (r ,r ) defined in (3.13a),
K V
To prove Theorem 4.1, we will first prove the mult-letter
k
version of the bound in terms of I :
r(V)= [d (i)−1]η P
Tj j
iX∈V Xj=1 Lemma A.1 For any B ⊆V with size |B|>1,
k
= ηj [dTj(i)−1]=(|V|−2)rK, H(FV\B)−H(F|Z˜B)≥(|P|−1) IP(Z˜B|F)−IP(Z˜B) (A.1)
j=1 i∈V
X X h i
where the last equality follows from the fact that for any P ∈Π′(B). ✷
i∈V dTj(i)=|Ej|=|V|−1 as Tj is a spanning tree. (cid:4) PROOF Consider any B ⊆ V such that |B| > 1, and P ∈
Π′(B) as stated in the lemma. Define
P V. EXTENSIONSAND CHALLENGES
While the lower bound (4.2) can be loose in the presence a :=I (Z˜ |Ft )−I (Z˜ |Ft−1) for t∈[ℓ], (A.2)
t P B V P B V
of cycles, it can be shown to be tight for hypergraphical
sources that correspond to hypergraphs that are minimally where F0 := 0 deterministically for notational convenience.
V
connectedinthesensethatremovinganyedgedisconnectsthe Then, we have the telescoping sum
hypergraphs.This generalizes the result of Theorem 4.2 from
ℓ
PINs to hypergraphicalsources. Both lower bounds (4.2) and a =I (Z˜ |F)−I (Z˜ ),
t P B P B
(4.7) can also be extended to include helpers. However, it is
t=1
X
and so it suffices to show that Since ℓ b =H(F )and ℓ c =H(F |Z˜ )bythe
t=1 t V\B t=1 t V B
chain rule, the above inequality and (A.4) gives
ℓ P P
(|P|−1) at ≤r.h.s. of (A.1). (A.3) ℓ
Xt=1 (|P|−1) at ≤H(FV\B)−H(FV|Z˜B),
By the definition (4.1b) of IP, Xt=1
H(Z˜ |Ft )−H(Z˜ |Ft ) which establishes (A.3) as desired. (cid:4)
at = C∈P C V B V We now single-letterize (A.1) to give the desired lower
|P|−1
P bound (4.2) in Theorem 4.1:
H(Z˜ |Ft−1)−H(Z˜ |Ft−1)
− C∈P C V B V
|P|−1 PROOF (THEOREM4.1) Consider any B ⊆ V with size
(|P|−1)a =I(Z˜P ∧F |Ft−1) |B|>1andP ∈Π′(B)asstatedinthetheorem.l.h.s.of(A.1)
t B Vt V in Lemma A.1 can be bounded by the total discussion rate as
,2
,1 (A.4) follows:
− I(Z˜ ∧F |Ft−1),
| {z } C Vt V H(F )−H(F |Z˜ )≤H(F )≤ log|F |
Cz∈P }| { V\B V B V\B i
X
wherewehavegroupedtheentropytermsindifferentbrackets i∈XV\B
intothemutualinformationtermsinthelastexpressionbythe ≤n r(V \B)+δ(1) (A.5)
n
definition of conditional mutual information. Using standard h i
techniques (cf. [18, Lemma B.1]), for some δ(1) →0 as n→∅ by (3.6). Next, we simplify first
n
term on the r.h.s. of (A.1) as follows:
,2 (=a) I(Z˜C ∧Fit|F˜it) H(Z˜ |F)−H(Z˜ |F)
CX∈PiX∈V IP(Z˜B|F)(=a) C∈P C B
(b) |P|−1
≥ H(F |F˜ ) P
it it (b)
CX∈PXi∈C ≥H(K|F)+IP(Z˜B|F,K)−nδn(2)
(=c) H(Fit|F˜it) (≥c)n(r −δ(2)−δ(3)) (A.6)
K n n
i∈BC∈P:i∈C
X X
(=d) H(F |F˜ ) • where (a) is by the definition 4.1b of IP;
it it
• (b) is obtained by applying the inequalities
i∈B
X
(≥e) H(Fit|FtV−1,F[i−1]∩Bt,FV\Bt) H(Z˜C|F)+nδn(2)|P|P|−|1 ≥H(K,Z˜C|F)
i∈B =H(K|F)+H(Z˜ |F,K)
X C
=(f)H(F |Ft−1,F )
Bt V V\Bt for some δ(2) →0, by (3.4) and Fano’s inequality, and
n
• where (a) follows from the chain rule and the defini-
tion (3.3b) of F˜ ; H(Z˜B|F)≤H(K,Z˜B|F)
it
• (b) is because =H(K|F)+H(Z˜B|F,K),
I(Z˜C ∧Fit|F˜it) =H(Fit|F˜it) if i∈C by (3.3a), Iand(Z˜the|nFg,Kro)u;pingtheentropytermsinvolvingZ˜C toform
(≥0 otherwise; P B
• (c) is because IP(Z˜B|F,K) ≥ 0 by the positivity of
• (c) is obtained by interchanging sums; divergence in (4.1b), and H(K|F) ≥ n[rK − δn(3)] for
• (d) is because the summand on r.h.s. of (c) is constant some δn(3) →0 by (3.5).
w.r.t.C,andsotheinnersummationgivesamultiplicative Finally, the last term on the r.h.s. of (A.1) can be single-
factor of 1. letterized as follows:
• (oen)FisVo\Bbtta,inwehdicbhyd(o3e.3sbn)oatnidncarneaaseddthiteioennatlrocpoyn.ditioning IP(Z˜B)(=d) C∈PH(Z˜C)−H(Z˜B)
• (f) follows from the chain rule. P |P|−1
Hence, (=e) C∈P i∈CH(Ui)− i∈BH(Ui)
|P|−1
,1 −,2 ≤ H(FVt|FtV−1)−H(FVt|FtV−1,Z˜B) +P C∈PPnH(ZC)−nHP(ZB)
−h H(F |Ft−1)−H(F |Ft−1)i |P|−1
Vt V V\Bt V P
= H(cid:2)(FV\Bt|FtV−1) −H(FVt|FtV−1,Z(cid:3)˜B) =(f)nIP(ZB) (A.7)
≤bt:=H(FV\Bt|FtV−\1B) ct • where (d) is by the definition (4.1b) of IP;
| {z } | {z }
• (e) is obtained by the expansion It follows that, if f is modular, the summation in (B.2a) is
constant for all feasible µ satisfying (B.2b).1 ✷
H(Z˜ )=H(U ,Zn)= H(U )+nH(Z )
C C C i C
Xi∈C The algorithm is illustrated in Fig. 3a, which is a plot of ws
H(Z˜ )=H(U ,Zn)= H(U )+nH(Z ) against s ∈ S. In particular, the horizontal axis enumerates
B B B i B
i∈B the elements S in a descending order of their weights w as
X
desired by the greedy algorithm in Step 1. The set of first j
by the definition (3.2) of Z˜ , the independence assump-
V elementsformthesetS ,andtheµ∗(S )isthedropinheight
tion(3.1)andthefactthatZn isi.i.d.generatedfromthe j j
V fromthe j-th barto the (j+1)-thbar,with the exceptionthat
source Z ;
V µ∗(S ) (or equivalently µ∗(S)) is the height of the last bar.
• (f) is because the expression in the first pair of brackets k
evaluates to 0 by exchanging the first two summation, The proof is by a lamination procedure that can turn any
and the expression in the second pair brackets evaluate µ to µ∗ gradually without increasing the sum in (B.2a) or
to nI (Z ). violating (B.2b):
P B
Applying (A.5), (A.6) and (A.7) to (A.1) and dividing both
sidesby n, we havethe desiredlowerbound(4.2)inthe limit Lamination:ForeveryB1,B2 ∈supp(µ)suchthatB1crosses
as n→∞. (cid:4) B2 in the sense that
APPENDIX B {B1,B2}6={B1∩B2,B1∪B2},
PROOF OF THEOREM4.3 reduce µ(B ) and µ(B ) by δ and increase µ(B ∩B ) and
1 2 1 2
To prove Theorem 4.3, we will make use of Edmonds’ µ(B ∪B ) by δ, where
1 2
greedy algorithm in combinatorial optimization [17]. A set
δ :=min{µ(B ),µ(B )}≥0,
function f : 2S → R with a finite ground set S is said to be 1 2
submodular iff for all B ,B ⊆S, where the non-negativity is by the assumption that µ is non-
1 2
negative. Doing so reduces µ(S)f(S) by
B⊆S
f(B )+f(B )≥f(B ∩B )+f(B ∪B ). (B.1)
1 2 1 2 1 2
δ[f(B1)+f(B2)−f(BP1∩B2)−f(B1∪B2)]≥0,
f is said to be supermodular if −f is submodular. If f is
where the non-negativity is by the submodularity (B.1) of f.
both submodular and supermodular, it is said to be modular.
f is said to be normalized if f(∅)=0. The entropy function
B 7→ H(ZB) [19], for instance, is a well-known normalized The procedureturns the supportof µ to that of µ∗, namely
submodular function [20]. Edmonds’ greedy algorithm states {S | 1 ≤ j ≤ k}, which forms a laminar family (or more
j
that: specifically, a chain).
Proposition B.1 ([17, Theorem 44.3]) For any normalized
submodular function f : 2S → R with a finite ground set PROOF (THEOREM4.3) For any P ∈ Π′(V), by (3.4) and
Fano’s inequality,
S, and any non-negativeweight vector w :=(w |s∈S)∈
S s
RS, consider the linear program
+
min µ(B)f(B) (B.2a)
µ nδn ≥ H(K|Z˜C,F)
B⊆S
X C∈P
X
such that µ : 2S → R+ is a non-negative set function = H(Z˜C,F,K)− H(Z˜C)− H(F|Z˜C) (B.4)
satisfying
C∈P C∈P C∈P
X X X
µ(B)=w , ∀s∈S. (B.2b)
s ,1 ,2 ,3
B⊆S:s∈S
X | {z } | {z } | {z }
Then, the optimal solution µ∗ to the above problem is given
as follows: for some δ →0 as n→∞, where the last equality is by the
n
1) Enumerate S as {s1,...,sk} (with k:=|S|) such that chain rule expansion.We will bound,1,,2 and,3 to obtained
the desired lower bound (4.7).
w ≥···≥w .
s1 sk ,3 can be bounded by the usual technique (cf. [18,
2) With S :={s |1≤j′ ≤j} for 1≤j ≤k, set
j j′
µ∗(S ):=w −w for 1≤j <k (B.3a)
j sj sj+1
µ∗(Sk):=µ∗(S)=wsk (B.3b) 1This is because −f is submodular and so the same µ∗ defined in (B.3)
both minimizes and maximizes the sum in (B.2a), the value of which must
and µ∗(B)=0 otherwise, i.e., if B 6=Sj for 1≤j ≤k. therefore beaconstant.
ws ws
S1={0}
ws1 w0=|P|
µ∗(S1)=|P|−we1
we1
wsj Sj :={sj′ |j′≤j} wej Sj+1={0}∪{ej′ |j′≤j}
wsj+1 µ∗(Sj):=wsj −wsj+1 wej+1 µ∗(Sj+1)=wej −wej+1
we|E| S|E|+1={0}∪E
wsk Sk=S 1 S|V|+|E|+1=S µ∗(S|E|+1)=we|E|−1
µ∗(S):=wsk V µ∗(S)=1
s∈S s∈S
s1 sj sj+1 sk s1=0s2=e1 =sj+ej1 =sj+ej2+1 =s|Ee||E+|1s|E|+2 s|E|+|V|+1
(a) µ∗ in general (B.3). (b) µ∗ applied to the proof of (B.10).
Fig. 3: Illustration of Edmonds’ greedy algorithm in Lemma B.1.
Lemma B.1]): with
ℓ Y =(F,K) (B.6a)
0
,3 (=a) H(Fit|F˜it,Z˜C) Y =U for i∈V (B.6b)
i i
CX∈PXt=1iX∈V Y =Xn for e∈E. (B.6c)
ℓ e e
(b)
≤ H(F |F˜ )
it it Note that Z˜ = (U ,Zn ) = (U ,Xn ), where the first
CX∈PXt=1i∈XV\C equality is bCy (3.2),CandECthe seconCd eqEuCality is by (3.11).
ℓ
(=c) H(Fit|F˜it) Hence, we can rewrite ,1 as the sum B⊆Sµ(B)f(B) in
(B.2a) with
Xt=1iX∈V C∈PX:i6∈C P
(=d)(|P|−1) ℓ H(F |F˜ ) f(B):=H(YB) for B ⊆S.
it it
t=1i∈V 1, B ={0}∪C∪EC,C ∈P
XX µ(B):=
(=e)(|P|−1)H(F) (B.5) (0, otherwise.
• where (a) follows from the chain rule expansion on Then, f is normalized and submodular as it is an entropy
F (3.3); function of Y [20], and (B.2b) holds with the non-negative
S
• (b) is because weights defined as
=0 if i∈C by (3.3a),
H(Fit|F˜it,Z˜C)(≤H(Fit|F˜it) otherwise; w0 := µ(B)
B⊆S:0∈S
X
• (c) is obtained by interchanging sums; = µ({0}∪C∪E )=|P|, (B.7a)
C
• (d) is because the summand on r.h.s. of (c) is constant C∈P
X
w.r.t.C,andsotheinnersummationgivesamultiplicative
w := µ(B) for i∈V
factor of |P|−1. i
B⊆S:i∈S
• (e) follows again from the chain rule expansion on X
F (3.3). = µ({0}∪C∪EC)=1 (B.7b)
Next, we will bound ,1 and ,2 using Edmonds’ greedy C∈XP:i∈C
algorithm in PropositionB.1. For notationalsimplicity, define w := µ(B) for e∈E
e
B⊆S:e∈S
E :={e|i∈ξ(e)} for i∈V X
i
= µ({0}∪C∪E )
C
E := E for C ⊆V,
C i
C∈PX:e∈EC
i∈C
[ =|{C ∈P |C∩ξ(e)6=∅}|. (B.7c)
which denote the collection of edges incident on node i∈V
andnodesinC ⊆V respectively.LetS ={0}∪V∪E,where As an example, for the triangle PIN Z defined in
{1,2,3}
we assume 0 6∈ V ∪E without loss of generality. Define Y (4.6) and illustrated in Fig. 2, and the partition P :=
S
{{1},{2},{3}} into singletons, whichisidenticalto(B.10)exceptthat(F,K)isremovedfrom
every entropy term. We also have equality here because f is
w =|P|=3
0 modularoverV ∪E due to the fact that Y for s∈V ∪E de-
s
w =w =w =1
1 2 3 fined in (B.6b) and (B.6c) are mutually independent because
w =w =w =2, of(3.1)andtheindependenceoftheedgevariables.Itfollows
a b c
that
as w in (B.7c) reduces to the number of incident nodes of
e
edge e for singleton partition. ,1 −,2 ≥(|P|−we1)H(F,K)
It follows that =(f)(|P|−1)[1−α(P)]H(F,K)
w0 =|P|≥we ≥1=wi ∀e∈E,i∈V. (=g)(|P|−1)[1−α(P)][H(F)+H(K)−nδn′]
Enumerate E as {e1,...,e|E|} such that for some δn′ →0 as n→∞, where
• (f) is because by (B.8) and (B.7c),
w ≥w ≥···≥w . (B.8)
e1 e2 e|E| we1 :=me∈aExwe =me∈aEx|{C ∈P |C∩ξ(e)6=∅}|
Then, the desired ordering in Step 1 of the greedy algorithm =(|P|−1)α(P)+1 by (4.7b).
in Proposition B.1 satisfies
|P|−w =(|P|−1)[1−α(P)]
e1
s1 ={0} (B.9a) • (g) is by the secrecy constraint (3.5).
{s ,...s }={e ,...,e } (B.9b) Applying the above inequality and (B.5) to (B.4) and simpli-
2 |E|+1 1 |E|
fying, we have
{s ,...s }=V (B.9c)
|E|+2 |E|+|V|+1
H(F) H(K) δ
and so µ∗ defined in (B.3) can be evaluated as shown in α(P) ≥[1−α(P)] −δ′ − n ,
n n n |P|−1
Fig. 3b, with possibly non-zero values at (cid:20) (cid:21)
which implies (4.7a) by (3.6) in the limit as n→∞. (cid:4)
S ={s }={0}
1 1
S ={0}∪{e |1≤j′ ≤j} for 1≤j ≤|E| ACKNOWLEDGMENT
j+1 j′
S =S ={0}∪E∪V. The authors would like to thank Dr. S. W. Ho for inspiring
k
discussion and helpful comments.
By Proposition B.1, we can lower bound ,1 with
µ∗(B)f(B) , which simplifies to REFERENCES
B⊆S
[1] I. Csisza´r and P. Narayan, “Secrecy capacities for multiple terminals,”
P µ∗(S1) YS1 IEEETrans.Inf.Theory,vol.50,no.12,pp.3047–3061, Dec.2004.
,1 ≥(|P|−we1)H(F,K) [2] aHc.tiToynasgio,n“CInofmormmoantioinnfoTrmheaotiroyn, avnodl.s5e9c,rentok.ey9,capppa.ci5ty6,2”7I–E5E64E0T,rSaneps-t
z |E|−}|1 {µ∗(Sjz+}1|){ YSj+1 2013.
[3] A.GohariandV.Anantharam,“Information-theoretic keyagreementof
+ Xj=1 (cid:0)zwej −}|wej+1(cid:1){H(zF,K,Xn{e}j|′|1≤j′≤j}{) (B.10) [4] Mvmou.l.lMti5pu6lk,ehnteeorr.jme8ei,,npNapl.s.—K3a9Ps7ah3ryta–Ip3,”,9aI9nn6df,oYrAm.uSagat.ino2kn0a1rTa0hs.ueobrray,mIaEnEiaEmT,r“aAncshaicetvioinngsSonK,
µ∗(S|E|+1) YS|E|+1 YS capacity in the source model: When must all terminals talk?” in 2014
+(w −1)H(F,K,Xn)+H(F,K,Xn,U ). IEEEInternationalSymposiumonInformationTheoryPreceedings,June
e|E| E E V 2014,pp.1156–1160.
[5] H. Zhang, Y. Liang, and L. Lai, “Secret key capacity: Talk or keep
Using thze tri}a|ngle{PINzan}d| si{ngletonzpartit}i|on ag{ain as an
silent?” in Proc.IEEEInt. Symp. on Inf. Theory, June 2015, pp. 291–
example, we have 295.
[6] C. Chan, A. Al-Bashabsheh, J. Ebrahimi, T. Kaced, and T. Liu,
µ∗({0})=µ∗({0,a,b,c})=µ∗({0,a,b,c,1,2,3})=1 “Multivariate mutual information inspired by secret-key agreement,”
Proceedings oftheIEEE,vol.103,no.10,pp.1883–1913, Oct2015.
the above inequality evaluates to [7] C.Chan,A.Al-Bashabsheh,Q.Zhou,N.Ding,T.Liu,andA.Sprintson,
“Successive omniscience,” IEEETrans.Inf. Theory, vol. 62,no. 6,pp.
H(Z˜ ,F,K)+H(Z˜ ,F,K)+H(Z˜ ,F,K) 3270–3289, June2016.
1 2 3 [8] M.Mukherjee,N.Kashyap,andY.Sankarasubramaniam,“Onthepublic
≥H(F,K)+H(X ,F,K)+H(U ,X ,F,K). communicationneededtoachieveskcapacityinthemultiterminalsource
{a,b,c} {1,2,3} {a,b,c}
model,” IEEE Transactions on Information Theory, vol. 62, no. 7, pp.
Wecanfollowasimilarargumenttobound,2.Notethatthe 3811–3830, July2016.
[9] S. Nitinawarat and P. Narayan, “Perfect omniscience, perfect secrecy,
entropy in,2 is the same as that in,1 except it does not have andsteinertreepacking,” IEEETrans.Inf.Theory,vol.56,no.12,pp.
(F,K), and so we can eliminate Y0 from the above argument 6490–6500, Dec.2010.
to obtain [10] C. Chan, M. Mukherjee, N. Kashyap, and Q. Zhou, “When is omni-
science a rate-optimal strategy for achieving secret key capacity?” in
|E|−1 2016 IEEE Information Theory Workshop (ITW), Sept 2016, pp. 354–
,2 = wej −wej+1 H(Xn{ej′|1≤j′≤j}) [11] 3C5.8C.hanandL.Zheng,“Mutualdependenceforsecretkeyagreement,”
Xj=1 (cid:0) (cid:1) inProceedingsof44thAnnualConferenceonInformationSciencesand
+(we|E| −1)H(XnE)+H(XnE,UV), Systems,2010.
[12] C.Chan,A.Al-Bashabsheh,andQ.Zhou,“Incrementalanddecremental
secret key agreement,” in Proc. IEEE Int. Symp. on Inf. Theory, July
2016,pp.2514–2518.
[13] M. Mukherjee, C. Chan, N. Kashyap, and Q. Zhou, “Bounds on the
communicationrateneededtoachieveSKcapacityinthehypergraphical
sourcemodel,”inProc.IEEEInt.Symp.onInf.Theory,July2016,pp.
2504–2508.
[14] J. Liu, P. W. Cuff, and S. Verdu´, “Common randomness and key
generation withlimited interaction,” CoRR,vol.abs/1601.00899, 2016.
[15] A.Kaspi,“Two-waysourcecodingwithafidelitycriterion,”IEEETrans.
Inf.Theory,vol.31,pp.735–740,Nov.1985.
[16] S.Nitinawarat, C.Ye,A.Barg,P.Narayan,andA.Reznik,“Secretkey
generationforapairwiseindependentnetworkmodel,”IEEETrans.Inf.
Theory,vol.56,no.12,pp.6482–6489,Dec2010.
[17] A. Schrijver, Combinatorial Optimization: Polyhedra and Efficiency.
Springer, 2002.
[18] I.Csisza´randP.Narayan,“Secrecycapacitiesformultiterminalchannel
models,”IEEETransactions onInformation Theory,vol.54,no.6,pp.
2437–2452, June2008.
[19] R.W.Yeung,InformationTheoryandNetworkCoding. Springer,2008.
[20] S. Fujishige, “Polymatroidal dependence structure of a set of random
variables,” Information andControl, vol.39,no.1,pp.55–72,1978.