On MBR codes with replication

0.71 MB·

by  M. Nikhil Krishnan

On MBR codes with replication
M. Nikhil Krishnan and P. Vijay Kumar, Fellow, IEEE
Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore.
Email: [email protected], [email protected]
Abstract
An early paper by Rashmi et. al. presented the construction of an (n,k,d = n−1) MBR regenerating code featuring the
inherent double replication of all code symbols and repair-by-transfer (RBT), both of which are important in practice. We first
showthatnoMBRcodecancontainevenasinglecodesymbolthatisreplicatedmorethantwice.Wethengoontopresenttwo
newfamiliesofMBRcodeswhichfeaturedoublereplicationofallsystematicmessagesymbols.Thecodesalsopossessasetof
dnodeswhosecontentsincludethemessagesymbolsandwhichcanberepairedthroughhelp-by-transfer(HBT).Asacorollary,
6 weobtainsystematicRBTcodesforthecased=(n−1)thatpossessinherentdoublereplicationofallcodesymbolsandhaving
1 a field size of O(n) in comparison with the general, O(n2) field size requirement of the earlier construction by Rashmi et. al.
0 For the cases (k = d = n−2) or (k+1 = d = n−2), the field size can be reduced to q = 2, and hence the codes can be
2 binary. We also give a necessary and sufficient condition for the existence of MBR codes having double replication of all code
symbolsandalsosuggesttechniqueswhichwillenableanarbitraryMBRcodetobeconvertedtoonewithdoublereplicationof
n
all code symbols.
a
J
Index Terms— Distributed storage, regenerating codes, exact repair, MBR codes.
9
2
I. INTRODUCTION
]
T Dimakis et. al. [1] introduced a class of distributed storage codes called regenerating codes. In the regenerating code
I framework,afileofB symbolswillbeencodedtoavector-codehavingnnodesstoringαsymbolseach.Duringasinglenode
.
s failure, the failed node can be regenerated by downloading β ≤ α symbols each, from any d surviving nodes (node-repair
c
property).Also,byaccessingsymbolsinanyk nodes,thefilecanberetrieved(data-collectionproperty).[1]showedexistence
[
of a trade-off between α (storage) and dβ (bandwidth) for given n, k, d, β and file-size B given by

1
k−1
v (cid:88)
B ≤ min{α,(d−i)β}. (1)
0
9 i=0
1
The Minimum Storage Regeneration (MSR) and Minimum Bandwidth Regeneration (MBR) points are the two extremal points
8
in the trade-off, where α and dβ are minimized first respectively.
0
. There are two models of repair for regenerating codes; functional and exact repair. The bound (1) in [1] was based on the
1
functional repair model. In this model, the contents of a node are permitted to change following repair, while retaining the
0
node-repair and data-collection properties. In the exact repair case, node contents remain the same after repair.
6
1 At the MBR point, α, β are given respectively by: α=dβ, β = B . Our focus here is on MBR codes and exact repair,
kd−(k)
v: with parameter β =1. 2
i During node-repair, if repair is carried out simply by reading precisely 1 symbol from each of the d nodes, the repair is
X
termed as help-by-transfer (HBT). If in addition to help-by-transfer, no computation is needed at the replacement node either,
r we will speak of repair-by-transfer (RBT)1. Minimizing reads during repair is advantageous, as it translates to savings in the
a
utilization of computational resources, storage-disk durability etc.
Replication of data is a second important consideration in practical distributed storage platforms such as Hadoop [2]. In
additiontoimprovingresiliencyagainsterrorsanderasures,italsohelpswithdataavailabilitywhentherearetransientfailures
in the system. It also increases chances of local data computation in nodes (data locality), thereby reducing job execution
delays (jobs are compute operations to be performed on the data) and bandwidth [3].
In [4], fractional repetition (FR) codes are introduced which generalize the RBT-MBR construction in [5], focusing more
on the replication aspect and relaxing the requirement of any d for repair to a table-based repair-model involving d nodes. In
contrasttoFRcodes,ourconstructionsprovideageneralizationwhilestayingwithintheMBRregime,andretainingproperties
such as reduced reads, systematicity of codes and the best-possible MBR replication level of 2.
P. Vijay Kumar is also an Adjunct Research Professor at the University of Southern California. This work is supported in part by the National Science
FoundationunderGrantNo.1421848andinpartbythejointUGC-ISFresearchprogram.
1someauthorsuseRBTnotationforboththescenarios
A. Other Related Work
The Product-Matrix MBR (PM-MBR) codes [6] are a family of MBR codes that exists for all possible parameter sets
(n,k ≤d,d≤n−1,β =1). (n,k ≤d,d=n−1)-MBR codes having the RBT property for all node-repairs are provided in
[5] and [7]. These codes are formed via concatenation of an outer MDS code of length (cid:0)n(cid:1) and an inner replication-2 code. In
2
[8], the authors present repair-by-transfer MBR codes for d=(n−1) based on congruent transformations on skew-symmetric
matrices. This approach requires only O(n) field-size and lesser computational complexity over the RBT construction in [7].
[9] gives new encoding matrices for MSR and MBR codes that have the least update complexity in PM framework. [9]
also proposes new decoding schemes that have improved error correction capability. The paper [10] proves non-existence of
d<(n−1) MBR codes with HBT for all nodes. This paper also gives PM based constructions for two relaxations, namely,
HBT for only a specific set of d nodes and HBT recovery from d specific nodes (for all nodes).
B. Our Contributions
Wepresentheretwonewfamiliesof(n,k ≤d,d≤n−2,β =1)MBRconstructions.BothconstructionshaveanRBT-MBR
code [5] contained in (d+1) nodes, as a component of the construction. These (d+1) nodes contain systematic data with
double replication and a subset d of these nodes can be repaired via HBT, irrespective of the choice of d helpers.
In Section-II, we show that it is not possible to have a replication level > 2 even for a symbol in an MBR code, when
k ≥ 2. Section-IV describes the two MBR constructions. The first family of codes are motivated by the RBT construction
appearingin[5]and[7].For(n,k =n−2,d=n−2)and(n,k =n−3,d=n−2),thesecodescanbeimplementedoverF .
2
The second family of codes are based on internal node transformations of PM-MBR, combining ideas from [7] and [10]. This
construction also has the replication, HBT properties. Systematicity of the code is ensured by an additional precoder, which is
related to the underlying MDS code appearing in the PM-MBR generator matrix. For d=k, both codes have smaller update
complexity than any code in the conventional PM framework. In Section-V, we discuss, to what extent double replication can
be brought in an MBR code with given parameters (n,k,d,β =1) and modified constructions to achieve that.
II. NON-EXISTENCEOFMBRCODESWITHREPLICATION>2
For i∈[n](cid:44){1,2,...,n}, let W , Sj denote random variables corresponding to node-i content and repair-data supplied
i D i
by node-i (i ⊆ D, |D| = d) to repair node-j, respectively. Sj is a function of W . Let k ≥ 2. From [7], we list a few
D i i
equations on entropies of these random variables as the following lemma.
Lemma II.1. 1) H(W )=α=dβ
i
2) H(W |W )=H(W | Si)=(d−1)β
i j i D j
3) H( Si, Si)=2β
D j D k
Theorem II.2. It is not possible to replicate a symbol more than twice in an exact-MBR code with k ≥2.
Proof:
WOLOG assume a symbol x is present in nodes-1, 2 and 3. Note that Lemma-II.1.1 implies:
H(x)=1
From Lemma-II.1.2,
(d−1)β = H(W |W )
1 2
= H(W |W ,x)
1 2
≤ H(W | S1,x)
1 D 2
≤ H(W | S1)
1 D 2
= (d−1)β.
∴H(W | S1)−H(W | S1,x)=0.
1 D 2 1 D 2
Hence,
H(x| S1) = H(x| S1,W )+I(x;W | S1)
D 2 D 2 1 1 D 2
= H(x| S1,W )+H(W | S1)−H(W | S1,x)
D 2 1 1 D 2 1 D 2
= H(x| S1,W )
D 2 1
(a)
= 0
where (a) follows as node-1 contains x. Similarly, H(x| S1) = 0.
D 3
Therefore, we have
H( S1, S1) = H( S1)+H( S1| S1)
D 2 D 3 D 2 D 3 D 2
= H( S1)+H( S1| S1,x)+I( S1;x| S1)
D 2 D 3 D 2 D 3 D 2
= H( S1)+H( S1| S1,x)+H(x| S1)−H(x| S1, S1)
D 2 D 3 D 2 D 2 D 2 D 3
= H( S1)+H( S1| S1,x)
D 2 D 3 D 2
≤ H( S1)+H( S1|x)
D 2 D 3
= H( S1)+H( S1)−(cid:0)H(x)−H(x| S1)(cid:1)
D 2 D 3 D 3
= β+β−(1−0)
= 2β−1.
This contradicts Lemma-II.1.3 and proves that it is not possible to replicate a symbol more than twice.
Remark 1. Fork =1,∃MBRcodeswithreplication>2.Foreg.,considera(vector)replicationcodewithddistinctsymbols
in a node, and every node 1,2,...,n identical. This is clearly an (n,k = 1,d,β = 1) MBR code with each symbol having
replication n.
III. PRODUCT-MATRIXMBRANDREPAIR-BY-TRANSFERMBRCODES
In this section, we give a brief summary of PM-MBR codes [6] and RBT-MBR codes appearing in [5], [7].
A. Product-Matrix (PM) MBR [6]
Consider a symmetric (d×d) message matrix M.
(cid:20) (cid:21)
S T
M = (2)
Tt 0
where S is a (k×k) symmetric matrix holding (cid:0)k+1(cid:1) message symbols, T is a (k×(d−k)) matrix holding k(d−k) message
2
symbols. There is a (d×n) encoding matrix ψ=[φ ∆ ]t, where φ and ∆ are (n×k) and (n×(d−k))
PM PM PM PM
matrices respectively. Also, any k rows of φ and any d columns of ψ are linearly independent. Let ψ denote ith column
PM i
of ψ. Node-i content is given by:
n =Mψ (3)
i i
For the systematic version of Product-Matrix MBR,
(cid:20) (cid:21)t
I 0
ψ = k (4)
φ˜ ∆˜
where I is the (k×k) identity matrix, 0 is a (k×(d−k)) matrix and [φ˜ ∆˜] is an ((n−k)×d) Cauchy matrix. The
k
repair-data transmitted from node-i to j (or vice-versa) during repair is given by:
Sj =ψtMψ (5)
i j i
B. Repair-by-transfer MBR codes for d=(n−1) [7]
The construction can be visualized in terms of a complete graph on n nodes. B message symbols will be encoded using an
[(cid:0)n(cid:1),B]-MDS code. A code symbol will be assigned to each edge of the graph and a node will be storing symbols assigned
2
to the edges incident on it. During single-node failures, repair can be performed by merely transferring one symbol each, from
the remaining (n−1) nodes.
IV. MBRCONSTRUCTIONSWITHDOUBLEREPLICATIONFORSYSTEMATICMESSAGESYMBOLS
A. Construction-A for k =d≤(n−2)
Consider a field F with characteristic 2. For d=k, B =kd−(cid:0)k(cid:1)=(cid:0)d+1(cid:1). The starting point of this construction will be
q 2 2
the code described in III-B. The code in III-B can be viewed in terms of a (d×d) symmetric matrix M and it’s diagonal.
(cid:0)d+1(cid:1) message symbols {m }B will be used to populate M. Column entries of the augmented matrix [M|diag(M)] gives
2 i i=1
an equivalent description of the code in III-B, for parameters (n =d+1,k =d,d =d). These (d+1) columns
RBT RBT RBT
(having systematic message symbols with replication-2) form the first (d+1) node contents n , n , ..., n .
1 2 (d+1)
Consider a (d×(n−(d+1))) Cauchy matrix φ over F (char. 2). φ = [φ |φ |...|φ ] (where φ ∈ Fd×1). Let
q (d+2) (d+3) n i q
[φ |φ |...|φ ]=I . Node-i ∈[n]\{(d+1)} will store n =Mφ =[n |n |...|n ]φ .
1 2 d d i i 1 2 d i
[n |n |...|n ]=[M|diag(M)|Mφ]
1 2 n
Remark 2. As d=k, it is clear from (4) that [n |n |...|n |n |...|n ] gives a PM-MBR code on (n−1) nodes, with
1 2 d (d+2) n
ψ =φ 1≤j ≤n, j (cid:54)=(d+1). Also, any d-subset of {φ } is independent.
j j j
Lemma IV.1. (φ (cid:12)φ )tn =φtn , 1≤i≤n, i(cid:54)=(d+1) ((cid:12) indicates element-wise multiplication).
i i (d+1) i i
Proof: RHS = φtMφ = (cid:80)d (cid:80) [φ ] M [φ ] +(cid:80)d [φ ] M [φ ] (=b)(cid:80)d [φ ] M [φ ] = LHS, where (b)
i i a=1 b(cid:54)=a i a a,b i b a=1 i a a,a i a a=1 i a a,a i a
follows as M is symmetric and F has char. 2.
q
Proposition 1. The vector-code construction described above is MBR with parameters (n,k,d=k,β =1,α=d)
Proof: Node-repair: From Remark-2, it is clear that only repairs involving n need to be checked.
(d+1)
Assume node-(d+1) failed. Let an arbitrary set D = {i ,i ,...,i } of d nodes, (i (cid:54)= (d+1)), help in the repair. Each
1 2 d j
node-i ∈ D provides φtn . Applying Lemma-IV.1, replacement node has access to φt n , where φ = φ (cid:12)φ ,
i i rep (d+1) rep D D
φ =[φ φ ...φ ].Invertibilityofφ followsfromtheinvertibilityofφ ,asfieldcharacteristicis2.Thus,replacement
D i1 i2 id rep D
node can retrieve n from φt n .
(d+1) rep (d+1)
Now,considerthefailureofnode-i(i(cid:54)=(d+1)).LetthehelpernodescomefromasetD(cid:48) ={i ,i ,...,i }∪{(d+1)}
1 2 (d−1)
of d nodes, (i (cid:54)= i). Helper node-(d+1) provides (φ (cid:12)φ )tn . Any other helper node-i gives φtn = φtMφ =
j i i (d+1) j i ij i ij
φt Mφ = φt n . Applying Lemma-IV.1, replacement node has access to φt n , where φ = [φ φ φ ...φ ].
ij i ij i D(cid:48) i D(cid:48) i i1 i2 i(d−1)
Clearly φ is invertible and hence, replacement node can retrieve n from φt n .
D(cid:48) i D(cid:48) i
Data-collection: As d=k, this follows from node-repair property.
Remark 3. (Help-by-transfer) To repair a node-i in [d], a helper node-j will be providing φtn or (φ (cid:12)φ )tn . As φ =e ,
i j i i j i i
ith column of I , this is equivalent to providing a stored symbol with out computation.
d
Remark 4. When d = k = (n−2), φ = φ can be chosen to be 1 (all-one vector) and hence, implementation over F is
n 2
possible.
B. Linearized Polynomials [11]
A polynomial of the form f(x)=(cid:80)t a xqi, a (cid:54)=0 is called a linearized polynomial of q-degree t. The coefficients {a }
i=0 i t i
arecomingfromF .Itisknownthatf(b x +b x )=b f(x )+b f(x ),whereb ∈F ,x ∈F .Alinearizedpolynomial
qm 1 1 2 2 1 1 2 2 i q i qm
with q-degree t can be uniquely determined from (t+1) evaluations at points {θ }t+1 ⊆ F , which are independent over
i i=1 qm
F . [12] gives maximum rank distance Gabidulin codes based on linearized polynomial evaluations.
q
C. Construction-A for k <d≤(n−2)
The file-size, B =kd−(cid:0)k(cid:1) symbols in F (with char. 2). Here, qm is chosen in such a way that ∃ n =d2−(cid:0)d(cid:1)=(cid:0)d+1(cid:1)
2 qm c 2 2
symbols {θi}ni=c1 ⊆ Fqm, which are independent over Fq. A systematic [nc, B, nc−B+1]-Gabidulin code will be used to
encode B message symbols to n code symbols {c }nc . A vector-code C will be constructed treating these code symbols as
c i i=1
message symbols {m } in IV-A (with φ over F ).
i q
Proposition 2. C is an MBR code with parameters (n,k <d,d≤(n−2),β =1,α=d).
Proof:
Node-repair: Node-repair property is inherited from the underlying construction in IV-A.
Data-collection (DC): If all the k nodes are in [d+1], DC follows from the embedded RBT code. Consider the case when
k(cid:48) <k nodes are read from [d+1] and (k−k(cid:48)) nodes are read from K(cid:48)(cid:48)⊆ [n]\[d+1] (|K(cid:48)(cid:48)|=k−k(cid:48)). Using the notation in
IV-A,eachnode-j inK(cid:48)(cid:48) hasaccesstosymbols{φtn } ∪{(φ (cid:12)φ )tn },whichare(d+1)equationsonn variables
j i i∈[d] j j (d+1) c
{c }nc . Set of variables V ⊆ {c }nc arising in each equation-i can be determined using a complete graph G on (d+1)
i i=1 i i i=1
vertices. G will be identical to the ‘RBT graph’ (described in III-B) contained in the first (d+1) nodes of C (Fig. 1 gives an
example). After removing the (known) interference from k(cid:48) nodes, let V(cid:48) denote the variable set corresponding to a (modified)
i
equation-i coming from a node in K(cid:48)(cid:48). {V(cid:48)} are determined by the induced (complete) graph G(cid:48) of G on the remaining vertices
i
in [d+1] (illustrated in Fig. 1). Contents of V(cid:48) are exactly the edges incident on a vertex-i in G(cid:48). Let i ,i ,...,i be
i 1 2 (d−k(cid:48)+1)
the vertices in G(cid:48). Any one among the (d−k(cid:48)+1) equations given by a node in K(cid:48)(cid:48) is dependent on the others (application of
Lemma-IV.1). Hence, one can remove a vertex (i.e., a modified equation) i from G(cid:48). All the remaining equations are
(d−k(cid:48)+1)
independent as there is an unshared edge (unshared c ) for each vertex. Each vertex contains (d−k(cid:48)) c ’s, i.e., |V(cid:48)|=d−k(cid:48).
j i i
Now, we will obtain a lower bound on the number of independent equations arising from all the (k−k(cid:48)) nodes ∈ K(cid:48)(cid:48). Let
equations corresponding to {i ,...,i } be already added to a set S. Considering equations corresponding to i (1 ≤ l ≤
1 l−1 l
(d−k(cid:48))) from all nodes ∈ K(cid:48)(cid:48), there will be at least min{(d−k(cid:48) −(l−1)),(k−k(cid:48))} independent equations that can be
added to S, as |V(cid:48) \∪l−1V(cid:48) |=d−k(cid:48)−(l−1) (follows from G(cid:48) structure) and {φ } comes from columns of a Cauchy
il j=1 ij i i∈K(cid:48)(cid:48)
Node-6 Node-7 Node-8
p c + p c + p c +p c q c + q c + q c +q c r c + r c + r c +r c
1 1 2 2 3 3 4 4 1 1 2 2 3 3 4 4 1 1 2 2 3 3 4 4
p c + p c + p c +p c q c + q c + q c +q c r c + r c + r c +r c
1 2 2 5 3 6 4 7 1 2 2 5 3 6 4 7 1 2 2 5 3 6 4 7
p c + p c + p c +p c q c + q c + q c +q c r c + r c + r c +r c
1 3 2 6 3 8 4 9 1 3 2 6 3 8 4 9 1 3 2 6 3 8 4 9
p c + p c + p c +p c q c + q c + q c +q c r c + r c + r c +r c
1 4 2 7 3 9 4 10 1 4 2 7 3 9 4 10 1 4 2 7 3 9 4 10

p 2c + p 2c + p 2c +p 2c q 2c + q 2c + q 2c +q 2c r 2c + r 2c + r 2c +r 2c
1 1 2 5 3 8 4 10 1 1 2 5 3 8 4 10 1 1 2 5 3 8 4 10

Node-2 *c +*c +*c
5 6 7
c c
DC 2 5
c c
6 7 *c +*c +*c
c c 6 8 9
2 6
c c c c
1 2 c 3 6
c c 3 c c
3 4 c c 8 9
5 7
Node-1 c c Node-3
4 8
c c
*c +*c +*c 1 9 *c +*c +*c
5 8 10 7 9 10
c c c c
1 5 4 7
c c c c c
8 10 10 9 10
Node-5 Node-4
Fig. 1: An example construction for n=8,k =3,d=4 Here, file-size is 9. Data-collector is connected to nodes-1, 6 and 7.
k(cid:48) =1.
matrix. It can be easily verified that nodes ∈ K(cid:48)(cid:48) will be providing at least (d−k+1)(k−k(cid:48))+(cid:0)k−k(cid:48)(cid:1) independent equations,
2
in total. Note that each independent equation (over F ) corresponds to an independent evaluation of the underlying linearized
q
polynomial. Thus, data-collector will be having access to at least k(cid:48)d−(cid:0)k(cid:48)(cid:1)+(d−k+1)(k−k(cid:48))+(cid:0)k−k(cid:48)(cid:1)=kd−(cid:0)k(cid:1)=B
2 2 2
polynomial evaluations, at points independent over F . Hence, data-collection is possible.
q
D. Construction-A for k =(n−3), d=(n−2) over F
2
Here, file-size, B = (cid:0)(d+1)(cid:1)−1. A parity symbol p will be obtained by taking the XOR of {m }B . Define c = m ,
2 i i=1 i i
1 ≤ i ≤ B and c = p. These (cid:0)(d+1)(cid:1) symbols {c }B+1 will be used as message symbols {m } for the construction in
(B+1) 2 i i=1 i
IV-A, with φ as in Remark-4. The code thus formed will be MBR. Only DC property needs to be verified as node-repair
property is inherited from the underlying construction.
If all the k nodes are read from [n−1], DC follows from the MDS nature of the inner single parity check code. Consider
the remaining case where (k −1) = (n−4) nodes are read from [n−1] along with node-n. After removing the known
interference of symbols from (n−4) nodes in [n−1], there will be three sums on three unknowns (of which only two are
independent) given by node-n; (c +c ), (c +c ) & (c +c ). As (c +c +c ) is known from the single parity check nature
i j i k j k i j k
of the inner code, c , c and c can be solved. An example is illustrated in Fig. 2.
i j k
E. Construction-B for k ≤d≤(n−1)
Shah et. al. [10] gives an MBR construction whose d nodes (these nodes contain a fraction of systematic message symbols)
are repairable via HBT. Combining ideas from [10] and [7], we first modify the conventional PM-MBR code discussed in
III-A to the following:
n =χ(i)tMψ 1≤i≤(d+1) (6)
i i
where χ(i) is a (d×d) invertible matrix whose columns are ψ ’s, 1≤l≤(d+1),l(cid:54)=i, in any order.
l
n =χ(d+1)tMψ (d+2)≤i≤n (7)
i i
Node-5
m1+m2+m3
m2+ m4+m5
m3+m5+ p
m1+m2 m2+ m4
+m3 +m5
m+1p+ m 4 m+3 +p m 5
m m m m
1 2 2 4
m m m
3 2 5
Node-1 Node-2
m
3
m1 m4 m5
m1m4 p m3m5
p p
Node-4 Node-3
Fig. 2: An example MBR construction over F for (n=5,k =2,d=3). File-size is 5. All m ’s have replication-2 and
2 i
every solid edge coming to {1,2,3} carries HBT repair-data from the other end.
These are invertible linear transformations of a node in the conventional PM-MBR code and hence, the code is linear MBR,
with repair-data between two nodes same as that in (5). (6) implies that the first (d+1) nodes have a structure identical to the
RBT-MBR code in [7], with replication-2. (6) and (7) guarantee HBT property for repairs in [d]. We will be using an encoding
matrix of the form (4). ψ =e , 1≤i≤k where e denote ith column of the (d×d) identity matrix.
i i i
Togetsystematicityfortheconstruction,wefillMwithB precodedsymbols(structureasin(2)).Firstweconsideramatrix
M(cid:48) filled with raw message symbols {m }B as in (2). Let M , M (cid:48) denote ith columns of M and M(cid:48) respectively. Both
i i=1 i i
M and M(cid:48) are completely determined by their first k columns (or rows). The precoder does the following set of k operations
to obtain M from M(cid:48).
M =[Λ(i)t]−1M(cid:48) 1≤i≤k (8)
i i
where Λ(i) is a (d×d) invertible matrix whose jth column is given by:
(cid:26)
ψ j =i
[Λ(i)] = (d+1)
j ψ 1≤j ≤d,j (cid:54)=i
j
Theequationsin(8)areindeedconsistent,astheyinduceidentitymappingforallm ’soccurringmorethanonce.Thefollowing
i
example illustrates the precoder.
Remark 5. (Update complexity) Compared to PM-MBR, both the code families have lesser number of symbols modified for
a single message symbol change, when d=k.
Example 1. Let the MBR parameters be (n≥4,k =2,d=3,β =1). ∴ B =5 symbols. Let
 
m m m
1 2 3
M(cid:48) =m2 m4 m5, ψ1 =[1 0 0]t, ψ2 =[0 1 0]t
m m 0
3 5
∴ Λ(1) = [ψ ψ ψ ], Λ(2) = [ψ ψ ψ ], M = [Λ(1)t]−1M(cid:48), M = [Λ(2)t]−1M(cid:48), χ = [ψ ψ ψ ], χ =
4 2 3 1 4 3 1 1 2 2 1 2 3 4 2
[ψ ψ ψ ]. n = χtMψ =χtM = χt[Λ(1)t]−1M(cid:48)= [m m m ]t. Similarly, n =[m m m ]t.
1 3 4 1 1 1 1 1 1 1 2 3 1 2 2 5 4
V. CONSTRUCTIONOFMBRCODESWITHDOUBLEREPLICATIONFORALLSYMBOLS
Consider a linear (n,k,d,α,β) MBR code over F. Generator matrix, G = [G |G |...|G ], G∈ FB×nα, G ∈ FB×α.
1 2 n i
n =Gtf, where f is the file-vector. We define the following subspaces, as in [5]. The node subspace associated with node-i,
i i
W = col(G ), where col(.) denotes column space. To repair a failed node-i, a helper node-j will provide β symbols. This
i i
will be viewed as passing a subspace of W and termed a repair subspace of node-j. Node subspaces and repair subspaces
j
have dimensions α and β respectively [5].
Lemma V.1. The repair subspace from node-i to node-j is the intersection space of W and W .
i j
Proof: Lemma-3 in [5].
Node-1 Node-4
m +3m +2m
m m 1 2 3
1 2 m +2m +3m
1 2 3
m m
2 1 m +3m +2m m +2m +3m
1 2 3 1 2 3
m +3m +2m m +2m +3m
m m m m 1 2 3 1 2 3
2 3 m 1 3 m +m +m m +m +m m +m +m
3 1 2 3 1 2 3 1 2 3
Node-2 Node-3 Node-5 Node-6
Fig. 3: An example construction for n=6,k =2,d=2.
Lemma V.2. AlinearMBRcodecanbetransformedtoanotherlinearMBRcodebysubstitutinganode-icontentwithrepair-
data for that node. The repair-data can come from an arbitrary set D of d nodes and the substitution process is equivalent to
re-encoding node-i with a new basis set G (cid:48) for W , instead of G .
i i i
Proof: Lemma-3 & Theorem-4 in [5].
Theorem V.3. An (n,k,d,β =1) MBR code with all symbols replicated twice exists iff there is a simple d-regular graph on
n vertices. Therefore, MBR codes with inherent double replication exists iff nd is even.
Proof: If there is an MBR code with all symbols replicated twice, a corresponding d-regular graph can be constructed by
putting an edge between every node sharing a symbol. From Lemma-V.1, it is clear that the graph thus constructed will be
simple, as β =1. Conversely, suppose there exists a simple d-regular graph G on n vertices. Take an arbitrary (n,k,d,β =1)
MBR code and map it’s nodes to G. For each adjacent vertex-j of vertex-i, the repair-data between node-i and node-j will be
put in both node-i and node-j, replacing a symbol of the starting MBR code in each of these two nodes. Repeating this for
every node-i will yield an MBR code (follows from Lemma-V.2) with double replication. The second statement follows from
Erdos-Gallai Theorem [13].
Corollary V.4. Theconstructions inIV-A, IV-C,IV-Ehaving anRBTcode inthe first(d+1) nodes,can beeasily transformed
tocodeswithdoublereplicationforallsymbols,ifn≥2(d+1)andndiseven.Theseconstructionshaveacompactdescription
for the repair-data between nodes-i,j ∈ {(d+2),(d+3),...,n}, which is of the form ytMy . However, this transformation
i j
will be at the expense of HBT repairability property of first d nodes (from any other d nodes).
Corollary V.5. Inparticular,if(d+1)|n,onecanconstructanMBRcode,whichwillbeaconcatenationof n RBTcodes.
(d+1)
Example 2. (Concatenated RBT) Let the MBR parameters be (n=6,k =2,d=2). Hence, file-size = 3 symbols (over F ).
22
Consider transforming the construction in IV-A. See Fig. 3.
(cid:20) (cid:21) (cid:20) (cid:21)
m m 1 1 1
M = 1 2 φ=
m m 1 2 3
2 3
∴ φ =[1 1]t, φ =[1 2]t & φ =[1 3]t. n =[m m ]t, n =[m m ]t, n =[m m ]t, n =[φtMφ φtMφ ]t =
4 5 6 1 1 2 2 2 3 3 1 3 4 5 4 6 4
[m +3m +2m m +2m +3m ]t, n = [φtMφ φtMφ ]t = [m +3m +2m m +m +m ]t and n =
1 2 3 1 2 3 5 4 5 6 5 1 2 3 1 2 3 6
[φtMφ φtMφ ]t = [m +2m +3m m +m +m ]t. {n , n , n }, {n , n , n } form two RBT codes, where first
4 6 5 6 1 2 3 1 2 3 1 2 3 4 5 6
one has systematic data.
TheoremV.6. Ifndisodd,itispossibletoconstructan(n,k,d,β =1)MBRcode,whereallbutonesymbolshavereplication
2.
Proof: Let d(cid:48) =(d−1). Construct a d(cid:48)-regular graph, G on n vertices. Now, remove vertex-i and construct a 1-regular
1
graph G on the remaining (n−1) vertices. Form a graph union G = G ∪G and map the vertices to nodes of an arbitrary
2 1 2
(n,k,d,β =1) MBR code. Using the technique in proof of Theorem-V.3, all except one symbol in the transformed code will
have replication-2. For node-i with degree d(cid:48), repair-data coming from a non-adjacent node (w.r.t G) will be taken as the dth
symbol.
REFERENCES
[1] A. Dimakis, P. Godfrey, Y. Wu, M. Wainwright, and K. Ramchandran, “Network coding for distributed storage systems,” IEEE Trans. Inf. Theory,
vol.56,no.9,pp.4539–4551,Sep.2010.
[2] “Hadoop,”http://hadoop.apache.org.
[3] J.DeanandS.Ghemawat,“Mapreduce:simplifieddataprocessingonlargeclusters,”Commun.ACM,vol.51,no.1,pp.107–113,2008.
[4] S. El Rouayheb and K. Ramchandran, “Fractional repetition codes for repair in distributed storage systems,” in Proc. 48th Annu. Allerton Conf.
Communication,Control,andComputing,Urbana-Champaign,IL,Sep.2010,pp.1510–1517.
[5] K.Rashmi,N.Shah,P.Kumar,andK.Ramchandran,“Explicitconstructionofoptimalexactregeneratingcodesfordistributedstorage,”inProc.47th
Annu.AllertonConf.Communication,Control,andComputing,Urbana-Champaign,IL,Sep.2009,pp.1243–1249.
[6] K.V.Rashmi,N.B.Shah,andP.V.Kumar,“OptimalExact-RegeneratingCodesforDistributedStorageattheMSRandMBRPointsviaaProduct-Matrix
Construction,”IEEETrans.Inf.Theory,vol.57,no.8,pp.5227–5239,Aug.2011.
[7] N.Shah,K.Rashmi,P.Kumar,andK.Ramchandran,“DistributedStorageCodesWithRepair-by-TransferandNonachievabilityofInteriorPointson
theStorage-BandwidthTradeoff,”IEEETrans.Inf.Theory,vol.58,no.3,pp.1837–1852,Mar.2012.
[8] S. Lin and W. Chung, “Novel Repair-by-Transfer Codes and Systematic Exact-MBR Codes with Lower Complexities and Smaller Field Sizes,” IEEE
Trans.ParallelDistrib.Syst.,vol.25,no.12,pp.3232–3241,2014.
[9] Y.S.Han,H.Pai,R.Zheng,andP.K.Varshney,“Update-EfficientError-CorrectingProduct-MatrixCodes,”IEEETrans.Commun.,vol.63,no.6,pp.
1925–1938,Jun.2015.
[10] N.B.Shah,“OnMinimizingData-ReadandDownloadforStorage-NodeRecovery,”IEEECommun.Lett.,vol.17,no.5,pp.964–967,May2013.
[11] R.LidlandH.Niederreiter,IntroductiontoFiniteFieldsandTheirApplications. NewYork,NY,USA:CambridgeUniversityPress,1986.
[12] E.Gabidulin,“TheoryofCodeswithmaximumrankdistance,”ProblemsofInformationTransmission,vol.21,no.7,pp.1–12,1985.
[13] P.Erdo˝sandT.Gallai,“Graphswithprescribeddegreesofvertices,”Mat.Lapok,vol.11,pp.264–274,1960.