Optimal Distributed Channel Assignment in D2D Networks Using Learning in Noisy Potential Games

0.68 MB·

by  Mohd. Shabbir Ali

Optimal Distributed Channel Assignment in D2D
Networks Using Learning in Noisy Potential Games
Mohd. Shabbir Ali, Pierre Coucheney, and Marceau Coupechoux
Abstract—WepresentanovelsolutionforChannelAssignment A. Contributions
Problem(CAP)inDevice-to-Device(D2D)wirelessnetworksthat
takesintoaccountthethroughputestimationnoise.CAPisknown • Novel Approach: Our approach is to learn the optimal
tobeNP-hardintheliteratureandthereisnopracticaloptimal channel assignments in a D2D wireless network using a
learningalgorithmthattakesintoaccounttheestimationnoise.In noisy potential game that takes into account the estima-
thispaper,wefirstformulatetheCAPasastochasticoptimization tion noise. Distributed learning in a noisy environment
7 problemtomaximizetheexpectedsumdatarate.Tocapturethe for CAP is novel. We consider a Stochastic Optimiza-
1 estimation noise, CAP is modeled as a noisy potential game,
tion Problem (SOP) with the objective to maximize the
0 a novel notion we introduce in this paper. Then, we propose
2 a distributed Binary Log-linear Learning Algorithm (BLLA) expected sum data rate of an underlay D2D network.
that converges to the optimal channel assignments. Convergence We translate this problem into a noisy potential game.
n of BLLA is proved for bounded and unbounded noise. Proofs The notion of the noisy potential game is introduced to
a for fixed and decreasing temperature parameter of BLLA are
J accountforthefactthatonlynoisyestimatesoftheutility
provided.Asufficientnumberofestimationsamplesisgiventhat
7 guarantees the convergence to the optimal state. We assess the are available to the players.
1 performanceofBLLAbyextensivesimulations,whichshowthat • Learning algorithm: We propose a distributed Binary
the sum data rate increases with the number of channels and Log-linearLearningAlgorithm(BLLA)foraSOP.BLLA

] users. Contrary to the better response algorithm, the proposed solves CAP to achieve an optimal channel assignment,
I
N algorithmachievestheoptimalchannelassignmentsdistributively which corresponds to the optimal Nash equilibrium of
even in presence of estimation noise.
. the game. The convergence of BLLA is proved for
s
c I. INTRODUCTION fixed temperature and decreasing temperature parameter.
[ Everincreasingdemandforhigherdataratesofmobileusers We provide a sufficient number of estimation samples
1 andscarcityofwirelessfrequencyspectrumismakingefficient that guarantees the convergence for both the cases of
v utilization spectrum resources increasingly critical. Device- boundednoiseandunboundednoise.NotethatforSOPs,
7 to-Device (D2D) networks increase the utilization of the BLLA is distributed and more practical when compared
7 spectrum resources by providing spatial spectrum reuse [1]– toStochasticApproximation(SA)[14],FiniteDifference
5
4 [12]. In a D2D network, D2D users reuse the radio resources SA [15], and Simultaneous Perturbation SA [16] algo-
0 allocated to traditional cellular Users (UEs). The cellular UEs rithms, which are centralized and may not be desirable
. communicate with the Base Station (BS) while the D2D UEs in large networks. Note that compared to BLLA the
1
0 communicate among themselves without or with limited help algorithm in [17] considers only the fixed temperature.
7 fromBS.Thisispossibleprovidedthattheinterferencecaused • Simulations results: Extensive simulations show that
1 bytheD2DUEstocellularUEsislimited.Acrucialproblem BLLA achieves the maximum sum data rate of the
v: in underlay D2D networks is thus to assign channels to UEs network. It shows that BLLA tracks well the increase
i to increase its utilization while maintaining low interference. of sum data rate with the increase of UEs and with the
X
Channel Assignment Problem (CAP) in D2D networks is increase of the number of channels. We also show that
ar challenging due to the lack of Channel State Information contrary to better response algorithm, BLLA converges
(CSI)ofD2DlinksattheBSandbecausefeedbackoverheads to the optimum even in presence of estimation noise.
shouldbekeptatareasonablelevel.CSIestimationerrorsthat
are due to several factors such as randomly varying channel B. Related literature survey and comparison
gain, feedback errors, feedback delay errors, and quantization
In this subsection, we discuss and compare different ap-
errors [13] affect the performance of D2D network. It is,
proaches for CAP in the literature. CAP in wireless networks
therefore, essential to have low feedback distributed solutions
is a standard problem and it is known to be NP-hard [18].
achieving optimal channel assignment while taking into ac-
Extensive surveys of CAP can be found for underlay D2D
count CSI or throughput estimation errors.
networks [1] and in various contexts in [19], [20].
The CAP solution approaches adopted by the state-of-
Mohd. Shabbir Ali and M. Coupechoux are with LTCI, Telecom
ParisTech, Université Paris-Saclay. Emails: [email protected], the-art channel assignment algorithms are dynamic program-
[email protected]. P. Coucheney is with UVSQ, ming[10],graph-theoreticalandheuristicsolutions[11],[12],
David-Lab, France. Email: [email protected]. This work was sup-
game theory [3]–[9], linear programming (LP), non-linear
ported by NetLearn ANR project (ANR-13-INFR-004) and the Indo-French
CEFIPRAproject"D2DforLTE-Advanced" programming (NLP), and Markov Random Field [19]. Other
approaches for CAP are neural networks [21], simulated
annealing [22], tabu search, genetic algorithms [20].
In [10], the authors jointly optimize the mode selection and
channel assignment in a cellular network with underlay D2D
communications in order to maximize the weighted sum rate.
A dynamic programming (DP) algorithm is proposed but it is
exponentially complex. Therefore, a suboptimal greedy algo-
rithm is proposed. In contrast to our approach, this solution
relies on explicit closed form expressions of sum data rate
for different channel fading scenarios. Our method can be
appliedtoanyfadingscenarioasitisbasedonusersmeasured
throughput. In [11], a suboptimal graph-theoretical heuristic
solutionforCAPinD2Dnetworksisproposed.Theweighted
signal sum is maximized using maximum-weighted bipartite Fig. 1: D2D cellular network layout model.
matching and interference sum is minimized using minimum-
weighted partitioning. This approach is centralized since the
BS uses the partial CSI of all the UEs. On the contrary, communicate with other UEDs. The set of UEs is denoted
our approach is distributed, optimal, and maximizes the sum as D. We consider a set of orthogonal frequency channel
throughputinsteadofsignalsum.In[12],aheuristicalgorithm bands F. The UECs are assigned different channels by the
is proposed for joint mode selection, channel allocation and BS, whereas UEDs reuse these channels. A UE transmits on
power allocation in a D2D wireless network. Channel estima- a single channel. The UEs that transmit on the same channel
tion is assumed to be perfect. c ∈ F cause interference to each other, the amount of which
Different game-theoretic models such as non-cooperative depends on channel gains between transmitters and receivers.
games, coalition formation games, and auction games are A. Channel Model
used to study the radio resource allocation issues in D2D
We consider a channel model that captures the effect of
networks [3]–[9]. In [9], a game-theoretical reverse iterative
path-loss,shadowing,andsmall-scalefading.LetdenoteD(c)
combinatorial auction is proposed as the allocation mecha-
as the set of UEs on channel c ∈ F. Let P and P denote
nism. However, in this auction, the BS needs to have the i 0
thetransmitpowerofUEiandnoisepower,respectively.The
bid from all the D2D links that may create a huge feedback
signal-to-interference-plus-noise ratio (SINR) at the receiver
overhead.In[4],apricingmechanismisproposedtomaximize
of UE i on channel c is given as:
thenetworkthroughputunderQoSconstraint.However,theal-
P g
gorithm proposed is a heuristic algorithm whose performance γ (c)= i i , (1)
is only evaluated through simulations. In contrast, BLLA’s i (cid:80)j∈D(c)\iPjgj,i+P0
convergence is proven theoretically and confirmed through
where g is the channel power gain between UE i and its
i
simulations. In [5], the uplink resource allocation problem
receiver, g is the channel power gain between UEs i and j.
j,i
for multiple D2D and cellular users is modeled as a coalition
These g and g take into account the path-loss, shadowing,
i j,i
game.ConvergencetoaNashequilibriumisproved.However,
and small-scale fading. The theoretical data rate ν (c) of UE
i
the equilibrium may be sub-optimal and inefficient. In [6], [7]
i on the channel c of bandwidth W is given by the classical
c
also, the coalition formation algorithms proposed to jointly
Shannon capacity formula, ν (c)=W log (1+γ (c)).
i c 2 i
solve mode selection and spectrum sharing may not converge
Note that the channel power gains g , g are subject to
i i,j
to an optimal coalition structure. The contract-based game
random variations. These variations arise due to randomly
theoretic mechanism proposed in [8] is evaluated through
varying channel gain, feedback errors, feedback delay errors,
simulations only.
and quantization errors [13]. Therefore, all the quantities
This paper is organized as follows. The system model and definedareinfactrandomvariables.Wedenoteνˆandν asthe
problem formulation are described in Section II. A noisy po-
estimated data rate and the expected data rate, respectively.
tentialgameframeworkisdevelopedinSectionIII.BLLAand
B. Problem Formulation
itsconvergenceresultsaregiveninSectionIV.Simulationand
conclusions are presented in Sections V and VI, respectively. Our objective is to maximize the expected sum data rate of
Proofs are in Appendix of our arXiv paper [23]. the network by assigning channels to UEs. Let c¯= (c ,c )
i −i
denotes a channel assignment vector where UE i is assigned
II. D2DCELLULARNETWORKMODEL the channel c ∈F and UEs other than UE i are assigned the
i
In this section, we describe the D2D cellular network channelvectorc ∈F|D|−1.TheestimateddatarateofaUE
−i
model as shown in Fig. 1. This figure shows downlink (DL) depends on vector c¯ and is denoted as νˆ (c¯). The objective
j
and uplink (UL) models. We consider a single base station function is φˆ(c¯)=(cid:80) νˆ (c¯). Formally, CAP is stated as:
j∈D j
(BS) and two types of UEs: (i) cellular UEs (UECs) that
c¯∗ ∈ argmax φ(c¯),
communicate with the BS and (ii) D2D UEs (UEDs) that (2)
c¯∈F|D|
where φ(c¯) = E[φˆ(c¯)] is the expected value over all the where N is the number of estimation samples, and Uˆ is
i,k
randomness. We seek to maximize the average sum data given by:
rate by using only estimates of data rates. Hence, the above
problem is a SOP [24]. In the next sections, we develop a Uˆi,k(ai,a−i)= (cid:88) νˆj(ai,a−i)− (cid:88) νˆj(ai,a−i),
general solution framework for this kind of SOPs. j∈D(ai) j∈D(ai)\i
(5)
where νˆ is the measured data rate of player j and D(a ) =
III. NOISYPOTENTIALGAMEFRAMEWORK j i
{j ∈D :a =a }isthesetofUEsusingthesamechannelas
j i
In real scenarios, UEs don’t experience the theoretical i. Note that random utility Uˆ may have a large variance but
i,k
data rate and have access only to estimates of their average the variance of the utility UˆN can be reduced by increasing
i
throughput that is corrupted by noise. In order to develop thenumberofsamplesN.Wewillseeinthenextsectionthat
a distributed solution to the CAP, we model the channel the number of samples N must be designed carefully so as to
assignment problem (2) as a stochastic game. preserve the convergence properties of potential games.
Definition 1: [CAP game] A CAP game is defined by the We have the following result. The proof is straightforward.
tuple Gˆ:={D,{X } ,{Uˆ } }, where D is a set of UEs
i i∈D i i∈D
that are players of the game, {Xi}i∈D are action sets consist- Proposition 1: A CAP game GˆN :=
ing of orthogonal channels, Uˆ : X → R are random utility {D,{X } ,{UˆN} } with utilities defined in (4),
i i i∈D i i∈D
functionswithfiniteexpectation,andX :=X ×X ×...X . (5) is a noisy potential game with potential function φ(a).
1 2 |D|
Anactionprofilea:=(a ,a )wherea ∈X istheaction Intherestofthepaper,weconsidertheCAPnoisypotential
i −i i i
of player i and a ∈X is the action set of all the players game GˆN.
−i −i
exceptplayer i.Note thatthe actionvector a∈X isthe same
as the channel assignment vector c¯and X =F|D|. IV. LEARNINGINPRESENCEOFNOISE
Potential games are attractive class of games, using which
In this section, we first describe the proposed binary log-
distributedsolutionstooptimizationproblemscanbedesigned.
linearalgorithm(BLLA)forlearninginthepresenceofnoise.
Iftheobjectivefunctionoftheoptimizationproblemisaligned
Then, we give the results on convergence of BLLA.
with the potential function, then global maximizers of the
The details of BLLA are described in Algorithm 1 and
objective are also the optimal Nash Equilibria (NE) of the
shown in Fig. 2. Each time slot is divided into two phases of
game. The optimal NEs are the maximizers of the potential
size N samples each (Phase I and Phase II). At the beginning
function. Moreover, for potential games with deterministic
of each time slot t, BS randomly selects a player i and a trial
utilities, an NE always exists and there exist algorithms that
action aˆ ∈X with uniform probability. Also, BS informs all
areguaranteedtoconvergetoNEsortotheglobalmaximizers i i
the players j such that a (t−1)∈{a (t−1),aˆ } to estimate
of the potential function [25]. Let us thus recall the definition j i i
their data rates and feedback this information to the BS at
of potential games.
the end of the two phases. Player i plays action a (t − 1)
Definition 2: [Potential game] A game G := i
and aˆ during Phase I and Phase II, respectively. At the end
{D,{X } ,{U } } is a (deterministic) potential i
i i∈D i i∈D of Phase II, all players on a (t − 1) and aˆ feedback to
game if there is a potential function h : X → R such that i i
the BS their two estimates of their sampled mean data rates
∀i∈D, ∀a ,a(cid:48) ∈X and ∀a ∈X ,
i i i −i −i corresponding to Phases I and II. BS calculates the utility of
U (a ,a )−U (a(cid:48),a )=h(a ,a )−h(a(cid:48),a ). (3) player i according to (4) and selects an action from the set
i i −i i i −i i −i i −i
{a (t−1),aˆ } according to (6), where τ(t) is a temperature
i i
This framework cannot be directly used for our CAP game parameter that governs the convergence properties of BLLA.
because of the random utilities. We thus propose in this paper Then, BS informs player i with the selected action. This
a new class of games, namely noisy potential games. feedback requires only one bit. BLLA is distributed in nature
because only a few players have to feedback to the BS.
Definition3:[Noisypotentialgame]Lettheexpectedutility
of player i is denoted as U = E[Uˆ ]. The game Gˆ :=
i i
{D,{X } ,{Uˆ } }isanoisypotentialgameifthegame A. Convergence of BLLA
i i∈D i i∈D
G :={D,{Xi}i∈D,{Ui}i∈D} is a potential game. In this subsection, we present the results of convergence of
WenowdesigntheutilityfunctionoftheCAPgamesoasto BLLA for both the cases of bounded and unbounded noise.
obtain a noisy potential game and align the potential function BLLA generates an irreducible Markov chain over the action
with the objective function of the CAP optimization problem. space of the CAP game GˆN. However, as the parameter τ
Weconsiderthefollowingutilityfunctionwhichrepresentsthe goes to zero, the stationary distribution concentrates on a few
marginalcontributionofplayeritotheglobalutilityaveraged states. The states whose limit probability is strictly positive
over N samples: as τ goes to zero are called stochastically stable. It is known
that for exact potential games the stochastically stable states
N
UˆN(a ,a )= 1 (cid:88)Uˆ (a ,a ), (4) of BLLA are the maximizers of the potential function [26].
i i −i N i,k i −i We extend this result to noisy potential games.
k=1
TABLE I: Simulation parameters.
Parameter Variable Value
Numberoforthogonalchannels F 5
Channelbandwidth Wc 180KHz
Carrierfrequency fc 2.6GHz
NumberofUEs |D| 20
TotaltransmitpowerofBS PBS 46dBm
TransmitpowerofUE PUE 25dBm
MinimumSINR γmin -10dB
MaximumSINR γmax 23dB
Additivenoisepowerperchannel P0 −174+10log(Wi)dBm
Path-lossexponent η 3.5
Shadowingvariance σsh 6
tion of noise. Assume that M(θ) is finite. Let θ∗ =
argmax θ(1−ξ)τ−log(M(θ)).Thenumberofsam-
θ
ples used are
(cid:16) (cid:17)
Fig.2:Timeslots,phases,andstepsofBLLAareshowninthis log 4 + 2
figure.Stepsarerepresentedusingcircles.Step5isestimation N ≥ ξ τ . (8)
(cid:16) (cid:17)
duration of the UEs on two channels a (t−1) and aˆ . log eθ∗(1−ξ)τ
i i M(θ∗)
Algorithm 1 Binary Log-linear Learning Algorithm Proof: See Appendix A.
A small N is desired for practical implementations. We
1: Initialisation: Start with arbitrary action profile a.
choose the lowest N that satisfies Theorem 2. In Theorem 2,
2: While t≥1 do
we have a convergence in probability for fixed parameter τ.
3: Set parameter τ(t).
In Theorem 3, we consider the case of decreasing parameter
4: BS randomly selects a player i and a trial action aˆi ∈Xi
τ for which we obtain an almost sure convergence to optimal
with uniform probability. BS informs player i and all the
state as in simulated annealing with cooling schedule [27].
players with actions a (t − 1) and aˆ to estimate their
i i
Theorem 3: Consider BLLA with a decreasing parameter
sample mean data rates.
5: Player i plays action ai(t−1) and aˆi during Phase I and τ(t)=1/log(1+t),andthenumberofsamplesN(τ)isgiven
by Theorem 2. Then, BLLA converges with probability 1 to
Phase II, respectively.
6: At the end of Phase II, all players with actions ai(t−1) the global maximizer of the potential function.
and aˆ feedback to BS their two estimates of their sample
i Proof See Appendix B.
mean data rates corresponding to Phases I and II.
7: BS calculates UˆiN(a(t−1)), UˆiN(aˆi,a−i(t−1)), and V. SIMULATIONS
selects action aˆ with probability
i
A. Simulation Parameters
(cid:16) (cid:17)−1
1+e∆Ni /τ , (6)
In this Section, we present simulation results on the down-
where ∆N =UˆN(a(t−1))−UˆN(aˆ ,a (t−1)). link considering standard wireless system parameters shown
i i i i −i in Table I. We consider that a BS is located at the center
8: BS informs player i to play the selected action. All the
of a region of radius 200 m. Among 20 UEs there are
other players repeat their previous actions, i.e., a (t) =
−i
5 UECs. The UECs have dedicated channels and no two
a (t−1).
−i
UECs are on the same channel. These UECs serve as passive
playersofthegamebecausetheydonotchangetheirchannel.
The receivers of UED transmitters are located around them
Theorem 2: The stochastically stable states of BLLA are
uniformly random over a region of radius 20 m. The UEDs
the global maximizers of the potential function φ(a) if one of
learn their channel with the help of the BS and hence are the
the following holds.
active players of the game.
1) The estimation noise is bounded in an interval of size (cid:96)
The variations of Rayleigh fading over time are considered
and the number of estimation samples used are
as the noise component for all the simulations. Channel
(cid:18) (cid:18)4(cid:19) 2(cid:19) (cid:96)2 variations result in random UEs data rates. The data rates
N ≥ log + , (7) are bounded because the SINR is bounded between γ and
ξ τ 2(1−ξ)2τ2 min
γ .Weassumeonlyboundednoiseinthesimulationswhere
max
where 0<ξ <1. the noise interval is set to (cid:96) = 1 due to normalized utilities.
2) The estimation noise is unbounded with finite mean Besides, the additive white Gaussian noise with power P is
0
and variance. Let M(θ) be moment generating func- considered.
2.5×107 2.6×107
2.4 2.5
2.4
c2.3 m
Sum data rate in bits/se122...9122 BBLLLLAA,, ττ (=t) 0=. 10.1/log(1+t) Sum data rate of syste1222....91232 ττ == 00..51
τ = 0.05
1.8 1.8
1.7 1.7
0 100 200 300 400 500 0 100 200 300 400 500
Iterations Iterations
Fig. 3: Convergence of BLLA for fixed temperature and Fig.4:ConvergenceofBLLAfordifferentfixedtemperatures.
decreasing temperature.
2.6×107
B. Simulation Results
2.4
c
In Fig. 3, convergence to the maximum sum data rate of se
s/2.2
BLLAisshownwithfixedtemperatureτ =0.1anddecreasing bit
n
temperature τ(t)=0.1/log(1+t)1. The number of samples ate i 2
are calculated according to (7) for τ = 0.1 and ξ = 10−5. a r
at
BLLAreachesthemaximumsumdataratewithbothfixedand m d1.8
decreasing temperatures. However, it has more variations for Su
fixed temperature. For decreasing temperature, the probability 1.6 Samples = 1
Samples = 100
of staying at the maximum is higher. Samples wrt τ = 0.05
1.4
Tostudytheeffectoftemperatureweshowtheperformance 0 100 200 300 400 500
Iterations
of BLLA for different temperatures in Fig. 4. As before, the
samples are calculated corresponding to different tempera- Fig. 5: Effect of the number of samples on the convergence
tures. For higher temperature τ = 0.5, BLLA exhibits huge of BLLA.
variations. The probability of being at a local maximum de-
creaseswithincreasingtemperature.Astemperaturedecreases,
channel. As evident from the figure, BLLA correctly tracks
the variations also decrease. Also, the probability of being
this phenomenon.
at a local maximum increases with decreasing temperature.
We also study the performance of BLLA by varying the
Therefore, the temperature should be chosen carefully to
number of UEs in Fig. 7 for 10 orthogonal channels and 10
obtain the desired performance. BLLA with smaller τ =0.05
UECs.Asbefore,thesumdatarateisobtainedfromBLLAat
gives the desired performance.
the end of 500 iterations averaged over 1000 realizations. As
We study the effect of the number of samples on the
the number of UEs increases (up to approximately 60 in the
performanceofBLLAinFig.5.Ifplayerstakedecisionsafter
figure), the sum data rate first linearly increases because of
every single sample, i.e., if the estimation errors are ignored,
the increasing traffic. A linear growth is observed as long as
BLLA exhibits large variations due to noise. As the number
interferenceiscontrolled.Sixtyismuchlargerthanthenumber
of samples increases the performance of BLLA improves. If
of available channels, which means that BLLA manages to
the number of samples is taken according to Theorem 2 then
assign frequencies in such a way that UEs do not interfere
BLLA provides high and stable sum data rate.
too much. After 60 UEs, the increase is reduced because
WenowstudytheeffectofthenumberofchannelsinFig.6.
interference significantly affects the sum data rate. BLLA
This plot shows the average sum data rate obtained from
exactly tracks this behaviour as evident from Fig. 7.
BLLA at the end of 500 iterations averaged over 1000
We now compare in Fig. 8 the performance of BLLA and
realizations. We see that the sum data rate increases as
better response (BR) algorithm, which accepts the trial action
the number of channels increases. This is intuitive because
onlyifitsutilityisbetterthanthecurrentaction.Bestresponse
the optimal channel assignment has lower interference per
algorithm, which is same as better response with two actions,
wasappliedtoCAPwiththeobjectiveofminimizingthetotal
1Note that τ(t) = 0.1/log(1+t) works well even though it is smaller interference in Wireless Sensor Networks (WSN) in [28]. The
thanthatgivenbyTheorem3.Thereasonbeingthattheheightofthehighest
parameter of BLLA is τ(t) = 0.1/log(1+t). Each curve
local maximum is smaller than the global maximum [27]. We consider that
heighttobe10%oftheglobalmaximum,whichisreasonable. in the figure is obtained by averaging over 1000 realizations
5.5×107 2.5×107
2.4
m data rate in bits/sec4.55 BLLA, τ(t) = 0.1/log(1+t) m data rate in bits/sec222...1232
Su Su1.9
BLLA, τ(t) = 0.1/log(1+t)
BR, Samples = 1
1.8 BR, Samples = 200
BR, Samples = 2000
4 1.7
4 6 8 10 12 14 16 18 20 0 100 200 300 400 500
Number of channels Iterations
Fig. 6: Effect of the number of channels on the sum data rate. Fig. 8: Comparison of BLLA and better response algorithms.
×107
10 increaseswiththenumberofchannelsandwiththenumberof
9 UEs. BLLA performs better than better response algorithm.
c 8
se REFERENCES
s/ 7
bit
ate in 6 BLLA, τ(t) = 0.1/log(1+t) [1] cAo.mAmsuadnii,caQti.onWianngce,llaunldarVn.etMwaonrkcsu,s”oI,E“EAEsCurovmemyuonn.Sduervviceey-sto&-dTevuitcoe-
a r 5 rials,vol.16,no.4,pp.1801–1819,2014.
dat 4 [2] M. N. Tehrani, M. Uysal, and H. Yanikomeroglu, “Device-to-device
m communicationin5gcellularnetworks:challenges,solutions,andfuture
u
S 3 directions,”IEEECommun.Mag.,vol.52,no.5,pp.86–92,2014.
[3] L.Song,D.Niyato,Z.Han,andE.Hossain,“Game-theoreticresource
2
allocation methods for device-to-device communication,” IEEE Com-
1 mun.Lett.,vol.21,no.3,pp.136–144,2014.
0 20 40 60 80 100
Number of UEs [4] Q. Ye, M. Al-Shalash, C. Caramanis, and J. G. Andrews, “Distributed
resource allocation in device-to-device enhanced cellular networks,”
Fig. 7: Effect of the number of UEs on the sum data rate. IEEETrans.Commun.,vol.63,no.2,pp.441–454,2015.
[5] Y. Li, D. Jin, J. Yuan, and Z. Han, “Coalitional games for resource
allocationinthedevice-to-deviceuplinkunderlayingcellularnetworks,”
IEEETrans.WirelessCommun.,vol.13,no.7,pp.3965–3977,2014.
of the algorithms. BR performs the worst when noise is [6] H. Chen, D. Wu, and Y. Cai, “Coalition formation game for green
not taken into account, which corresponds to one estimation resource management in d2d communications,” IEEE Commun. Lett.,
vol.18,no.8,pp.1395–1398,2014.
sample case. When the number of samples is increased to
[7] Y.Cai,H.Chen,D.Wu,W.Yang,andL.Zhou,“Adistributedresource
200, BR improves. However, BLLA is better than BR. For managementschemeford2dcommunicationsbasedoncoalitionforma-
2000samples,BRperformanceisthesameasBLLA.Itshows tiongame,”inProc.ICC,pp.355–359,IEEE,2014.
[8] B.-Y.Huang,S.-T.Su,C.-Y.Wang,C.-W.Yeh,andH.-Y.Wei,“Resource
that the number of samples for BR has to be tuned carefully
allocation in d2d communication-a game theoretic approach,” in Proc.
to obtain the desired performance. On the contrary, BLLA ICC,pp.483–488,IEEE,2014.
performs better with the fixed number of samples without any [9] C.Xu,L.Song,Z.Han,Q.Zhao,X.Wang,X.Cheng,andB.Jiao,“Effi-
ciencyresourceallocationfordevice-to-deviceunderlaycommunication
tuning.NotethattheoreticallyBRneedsaninfinitenumberof
systems:areverseiterativecombinatorialauctionbasedapproach,”IEEE
samples. There is no theoretical guarantee for its convergence J.Sel.AreasCommun.,vol.31,no.9,pp.348–358,2013.
for a finite number of samples. On the contrary, as we have [10] R. Wang, J. Zhang, S. Song, and K. B. Letaief, “Optimal qos-aware
channel assignment in d2d communications with partial csi,” IEEE
proved,BLLAhasatheoreticalguaranteeofconvergencewith
Trans.WirelessCommun.,vol.15,no.11,pp.7594–7609,2016.
a finite number of samples. [11] S.MaghsudiandS.Stan´czak,“Hybridcentralized–distributedresource
allocationfordevice-to-devicecommunicationunderlayingcellularnet-
VI. CONCLUSIONS works,”IEEETrans.Veh.Technol.,vol.65,no.4,pp.2481–2495,2016.
[12] C.Gao,X.Sheng,J.Tang,W.Zhang,S.Zou,andM.Guizani,“Joint
A novel optimal solution for CAP in D2D wireless net- mode selection, channel allocation and power assignment for green
device-to-devicecommunications,”inProc.ICC,pp.178–183,2014.
works that takes into account throughput estimation noise is
[13] A. Goldsmith, Wireless communications. Cambridge university press,
presented.Tocapturethisnoise,anoisypotentialgameframe- 2005.
work is introduced. A distributed Binary Log-linear Learning [14] H. Robbins and S. Monro, “A stochastic approximation method,” The
annalsofmathematicalstatistics,pp.400–407,1951.
Algorithm that achieves the optimal channel assignments is
[15] J.Kiefer,J.Wolfowitz,etal.,“Stochasticestimationofthemaximumof
proposed. A sufficient number of estimation samples that a regression function,” The Annals of Mathematical Statistics, vol. 23,
guaranteestheconvergenceinbothcasesofboundednoiseand no.3,pp.462–466,1952.
[16] J.C.Spall,“Multivariatestochasticapproximationusingasimultaneous
unboundednoisearegiventhatarevalidatedusingsimulations.
perturbationgradientapproximation,”IEEETrans.onautomaticcontrol,
BLLA achieves the optimal sum data rate. The sum data rate vol.37,no.3,pp.332–341,1992.
(cid:8) (cid:9)
[17] gDa.mSe.sLweisthlienoainsdy Jr.ewRa.rdMs,”aridnenP,ro“Ec.quIEilEibEriuNmetwseolrekctiGonamiens,pCotoennttrioall tuhteiliCtieAsPUgam=eEG(cid:104)Uˆ:=N(cid:105).DI,t{iXsia}ni∈eDxa,c{tUpi}oit∈enDtia,lwgiatmhee.xBpeLcLteAd
andOptimization(NetGCooP),pp.1–4,2011. i i
[18] M.R.GareyandD.S.Johnson,Computersandintractability,vol.29. induces a regular Markov process over the action space X of
whfreemanNewYork,2002. G [17], [26], [29]. Let denote Pτ as the transition matrix of
[19] E.Ahmed,A.Gani,S.Abolfazli,L.J.Yao,andS.U.Khan,“Channel
the regular Markov process.
assignment algorithms in cognitive radio networks: Taxonomy, open
issues, and challenges,” IEEE Commun. Surveys & Tutorials, vol. 18, Definition 4 (Resistance of transition [26]): Let a =
no.1,pp.795–823,2016. (a ,a ) and b = (a(cid:48),a ) be action profiles such that only
[20] G.K.Audhya,K.Sinha,S.C.Ghosh,andB.P.Sinha,“Asurveyonthe i −i i −i
player i changes its action. Let Pτ be a strictly positive
channel assignment problem in wireless networks,” Wireless Commun. ab
andMobileComputing,vol.11,no.5,pp.583–609,2011. probability transition function with the parameter τ. A non
[21] P.T.Chan,M.Palaniswami,andD.Everitt,“Neuralnetwork-baseddy- negative number R is the resistance of transition a→b if
namicchannelassignmentforcellularmobilecommunicationsystems,” ab
IEEETrans.Veh.Technol.,vol.43,no.2,pp.279–288,1994. Pτ
[22] M. Duque-Antón, D. Kunz, and B. Rüber, “Channel assignment for 0< lim ab <∞. (9)
cellular radio using simulated annealing,” IEEE Trans. Veh. Technol., τ→0+ e−Rτab
vol.42,no.1,pp.14–21,1993. Todevelopeasyrulestocomputetheresistanceofafunction
[23] M. S. Ali, P. Coucheney, and M. Coupechoux, “Optimal distributed
wegiveageneraliseddefinitionofresistanceinthefollowing.
channel assignment in d2d networks using learning in noisy potential
games,”Availableonhttps://arxiv.org/,Jan.2017. Let“o”and“ω”denotelittle"o"orderandlittleomegaorder,
[24] J.C.Spall,Introductiontostochasticsearchandoptimization:estima- respectively. We call function g(τ) a sub-exponential function
tion,simulation,andcontrol,vol.65. JohnWiley&Sons,2005. that satisfies g ∈o(cid:0)ek/τ(cid:1) and g ∈ω(cid:0)e−k/τ(cid:1) for any k >0.
[25] D. Monderer and L. S. Shapley, “Potential Games,” Games and Eco-
nomicBehavior,vol.14,pp.124–143,May1996. Definition5(Resistanceofpositivefunction):Letf(τ)bea
[26] J.R.MardenandJ.S.Shamma,“Revisitinglog-linearlearning:Asyn- strictlypositivefunction.Ifthereisasub-exponentialfunction
chrony,completenessandapayoff-basedimplementation,”Gamesand
g(τ) and a number R such that (10) holds, then R is unique
EconomicBehaviour,vol.75,pp.788–808,July2012.
[27] B. Hajek, “Cooling schedules for optimal annealing,” Mathematics of (see Lemma 2) and is called the resistance of f, denoted by
operationsresearch,vol.13,no.2,pp.311–329,1988. Res(f).
[28] J. Chen, Q. Yu, P. Cheng, Y. Sun, Y. Fan, and X. Shen, “Game theo- f(τ)
retical approach for channel allocation in wireless sensor and actuator lim =1. (10)
networks,” IEEE Trans. Automatic Control, vol. 56, no. 10, pp. 2332– τ→0g(τ)e−τR
2344,2011.
[29] M. S. Ali, P. Coucheney, and M. Coupechoux, “Load balancing in Remark Remark that Definition 5 includes Definition 4, in
heterogeneous networks based on distributed learning in near-potential which g(τ)=κ,0<κ<∞.
games,”IEEETrans.WirelessCommun.,vol.15,no.7,pp.5046–5059,
2016. Remark Note that (10) is equivalent to
[30] H. P. Young, “The Evolution of Conventions,” Econometrica, vol. 61,
pp.57–84,Jan.1993. f(τ)=g(τ)e−Resτ(f) +h(τ), (11)
[31] S.AnilyandA.Federgruen,“Simulatedannealingmethodswithgeneral
1ac9c8e7p.tanceprobabilities,”JournalofAppliedProbability,pp.657–667, where h(τ)∈o(cid:16)g(τ)e−Resτ(f)(cid:17).
[32] P.Brémaud,Markovchains:Gibbsfields,MonteCarlosimulation,and
Lemma 1: Consider any two sub-exponential functions
queues,vol.31. SpringerScience&BusinessMedia,2013.
[33] V. Anantharam and P. Tsoucas, “A proof of the markov chain tree g (τ) and g (τ). Consider two real numbers R and R . If
1 2 1 2
theorem,” Statistics & Probability Letters, vol. 8, no. 2, pp. 189–192, R <R then
1989. 1 2
(cid:16) (cid:17)
[34] S. Anily and A. Federgruen, “Ergodicity in parametric nonstationary g (τ)e−R2/τ ∈o g (τ)e−R1/τ . (12)
markovchains:Anapplicationtosimulatedannealingmethods,”Oper- 2 1
ationsResearch,vol.35,no.6,pp.867–874,1987.
Proof: Let k be a real number. Then
A. Proof of ConvergenceAofPPBELNLDAIXwith Fixed τ lim g2(τ)e−R2/τ = lim g2(τ) (cid:20) g1(τ) (cid:21)−1.
τ→0g1(τ)e−R1/τ τ→0e−(R2−k)/τ e−(R1−k)/τ
If the utilities of the CAP game are deterministic and (13)
without noise then the CAP game becomes an exact potential The above limit goes to zero when we choose R1 <k <R2.
game. For an exact potential game, the stochastically stable
states of BLLA are the maximisers of potential [17], [26], Lemma 2: If Res(f) exists then it is unique.
[29].WeprovethesameevenforanoisypotentialCAPgame Proof: Assume that function f have two different resis-
GˆN if the number of samples is chosen carefully. Our proof tances R1 and R2. Then, from (11) we have
athpeprroeasicshtainscetoBsLhLoAwwthiatht feosrtimaaptaerdtiucutillaitriensuimsbsaemr eofassatmhaptleosf f(τ)=g1(τ)e−Rτ1 +h1(τ)=g2(τ)e−Rτ2 +h2(τ), (14)
(cid:16) (cid:17) (cid:16) (cid:17)
with the deterministic utilities2. This kind of proof idea based where h1(τ)∈o g1(τ)e−Rτ1 and h2(τ)∈o g2(τ)e−Rτ2 .
on resistance is similar to that of [17], [30]. (cid:16) (cid:17)
We first recall the computation of the resistance of BLLA LetR1 <R2.UsingLemma1,wehaveh2 ∈o g1(τ)e−Rτ1 .
in a deterministic potential game, as in [26]. Let consider Rearranging the term in above equation, we have
2Inalltheproofs,theconsideredutilitiesarenormalizedbythemaximum 1+ h1(τ) = g2(τ)e−Rτ2 + h2(τ) . (15)
potentialφmax. g1(τ)e−Rτ1 g1(τ)e−Rτ1 g1(τ)e−Rτ1
(cid:16) (cid:17)
Using Lemma 1, we arrive at contradiction that 1=0. whereh3(τ)∈o g1(τ)e−Res(τf1) .Accordingto(11),wehave
Res(f +f )=Res(f )=min{Res(f ),Res(f )}.
1 2 1 1 2
The following Lemma gives useful rules for computing The case of Res(f )=Res(f ) leads to the same result as
1 2
Res(f). shown below.
Lemma 3: Let f and f be strictly positive real valued
functions.Letκbea1positive2constant.IfRes(f )andRes(f ) f1(τ)+f2(τ)=e−Res(τf1) [g1(τ)+g2(τ)]+h1(τ)+h2(τ).
1 2
(21)
exist then
Note that sum of sub-exponential functions g (τ)+g (τ) is
1) f(τ) is sub-exponential if and only if Res(f) = 0. In 1 2
a sub-exponential function. Observe that h (τ) + h (τ) ∈
particular Res(κ)=0, (cid:16) (cid:17) 1 2
2) Res(e−κ/τ)=κ, o [g1(τ)+g2(τ)]e−Res(τf1) .Asinthepreviouscase,accord-
3) Res(f1+f2)=min{Res(f1),Res(f2)}, ing to (11) we have Res(f1+f2)=Res(f1)
4) Res(f −f )=Res(f ),if Res(f )<Res(f ), Proofofrule4:Also,itcanbeshownsimilarlytotheproof
1 2 1 1 2
5) Res(f1f2)=Res(f1)+Res(f2), of rule 3 that if Res(f1)<Res(f2) then
6) Res(1)=−Res(f),if Res(f)(cid:54)=0.
f Res(f1−f2)=Res(f1). (22)
7) If f (τ) ≤ f (τ), Res(f ) and Res(f ) exist then
1 2 1 2
Proof of rule 5:
Res(f )≤Res(f ).
2 1
8) RLeets(ff1)(τe)xi≤stsfa(nτd)R≤esf(2f()τ)=. IRfeRs(efs(f).1) = Res(f2) then lim f1(τ) lim f2(τ)
1 τ→0g1(τ)e−Res(τf1) τ→0g2(τ)e−Res(τf2)
RnoetmcaormkpIunteRRueles(f4, −if fRe)s(bfe1c)au=se Rthees(dfi2f)fertehnecnewoef scuabn-- = lim f1(τ)f2(τ) =1. (23)
exponential function1s may2not be a sub-exponential function. τ→0g1(τ)g2(τ)e−Res(f1)+τRes(f1)
Therefore, Res(f f )=Res(f )+Res(f ).
Remark In Rule 8, in general if f (τ) ≤ f(τ) ≤ f (τ) and 1 2 1 2
1 2 Proof of rule 6: Since Res(f) exists we have
Res(f )(cid:54)=Res(f ) then Res(f) may not exists.
1 2
f(τ)
Proof: Proof of rule 1: Let f(τ) be a sub-exponential lim =1. (24)
function. Choosing g(τ)=f(τ) from (10) we have τ→0g(τ)e−Resτ(f)
Inversing both sides of the above equation, we get
lime−Resτ(f) =1. (16) 1
τ→0
f(τ)
lim =1. (25)
ThAersesfuomree, wReesh(fav)e=Re0s.(fFr)o=m0(.11), we have f(τ) = g(τ)+ τ→0 g(1τ)e−−Reτs(f)
h(τ), which is a sub-exponential function. Therefore, we have Res(1)=−Res(f).
Letf(τ)=κandg(τ)=κtheng(τ)∈o(cid:0)eκτ(cid:1)andg(τ)∈ Proof of rule 7: Assufme that Res(f1) < Res(f2). Using
ω(cid:0)e−κτ(cid:1), κ>0. Substituting these in (10) we have Res(κ)= Lemma 1, we have g2(τ)e−Res(f2)/τ ∈ o(cid:0)g1(τ)e−Res(f1)/τ(cid:1)
0. and h2 ∈o(cid:0)g1(τ)e−Res(f1)/τ(cid:1).
Proof of rule 2: Substituting f(τ) = e−κ/τ and g(τ) = 1
f ≤f , (26)
in (10) we get Res(e−κ/τ)=κ. 1 2
Proof of rule 3: Consider that Res(f ) and Res(f ) be the g (τ)e−Res(f1)/τ +h (τ)≤g (τ)e−Res(f2)/τ +h (τ), (27)
1 2 1 1 2 2
resistances of functions f1 and f2, respectively. We have 1+ h1(τ) ≤ g2(τ)e−Res(f2)/τ +h2(τ). (28)
f1(τ)=g1(τ)e−Res(τf1) +h1(τ), (17) g1(τ)e−Res(f1)/τ g1(τ)e−Res(f1)/τ
f2(τ)=g2(τ)e−Res(τf2) +h2(τ), (18) AThseτreftoerned,sRteos(zfer)o≥wReeasr(rfive).at a contradiction that 1 ≤ 0.
1 2
where h1(τ) ∈ o(cid:16)g1(τ)e−Res(τf1)(cid:17), h2(τ) ∈ Pr(cid:16)oof o(cid:17)f rule 8: We have 1 ≤ ff1((ττ)) ≤ ff21((ττ)) and
o(cid:16)g2(τ)e−Res(τf2)(cid:17). Res ff12((ττ)) =Res(f1)−Res(f2)=0.ByRule1 ff12((ττ)) issub-
The sum of two functions can be written as exponential. This implies that f(τ) is also sub-exponential
f1(τ)
(cid:32) Therefore, there exists g (τ) such that
f1(τ)+f2(τ)=g1(τ)e−Res(τf1) 1+ g1(τh)1e(−τR)es(τf1) f(τ) 01
+ g2(τ)e−Res(τf2) + h2(τ) (cid:33), (19) 1=τli→m0gf011((ττ)), (29)
g1(τ)e−Res(τf1) g1(τ)e−Res(τf1) = lim f(τ) lim g1(τ)e−Res(τf1), (30)
Cwoenhsaidveerhth2e∈coas(cid:16)egw1(hτe)ne−RReess(τ(ff1)1(cid:17)).<ThReerse(ffo2r)e.,Using Lemma 1 τ→0g01(τ)g1f((ττ))e−Res(τf1) τ→0 f1(τ)
= lim , (31)
f1(τ)+f2(τ)=g1(τ)e−Res(τf1) +h3(τ), (20) τ→0g01(τ)g1(τ)e−Res(τf1)
where the product g (τ)g (τ) is also a sub-exponential Proofforboundednoisecase: LetdenotethenoiseZ =
01 1 (cid:16) (cid:17) i
function. Therefore, Res(f) exists and Res(f) = Res(f1) = Uˆi(a)−Ui(a)− Uˆi(b)−Ui(b) . Using Hoefding inequality
Res(f2). for bounded independent random variables, we have
As an illustration of application of the above rules we Pr(cid:0)A¯δ(cid:1)=Pr(cid:32) 1 (cid:88)N |Z |>δ(cid:33)≤2exp(cid:18)−2Nδ2(cid:19). (42)
calculate the resistance of BLLA with deterministic utilities N i (cid:96)2
i=1
using the above rules. Let m (t) denote the probability of
i
choosing player i to revise its action. In case of deterministic Substituting (42) in Lemma 4, we have
utilities, the traPnsτiti=onmpro(tb)abilityePτ1Uaτbi(bo)f BLL.A is (32) pi(cid:18)1− τδ(cid:19)−4e−2Nδl22 ≤pNi ≤pi(cid:18)1+ τδ(cid:19)+4e−2Nδ(cid:96)(224.3)
ab i eτ1Ui(a)+eτ1Ui(b) Substituting the number of samples N from (7) and δ =
Let ∆i =Ui(a)−Ui(b). (1−ξ)τ in above, we have
Using Lemma 3, we have Res(Pτ )
(cid:32) eτ1Ui(abb) (cid:33) ξ(cid:16)pi−e−τ2(cid:17)≤pNi ≤(2−ξ)pi+ξe−τ2. (44)
=Res(mi(t))+Res eτ1Ui(a)+eτ1Ui(b) , (33) As before, the transition probability Paτb of BLLA is
(cid:16) (cid:17) (cid:110) (cid:16) (cid:17) (cid:16) (cid:17)(cid:111) (cid:16) (cid:17)
=Res eτ1Ui(b) −min Res eτ1Ui(a) ,Res eτ1Ui(b) , ξmi(t) pi−e−τ2 ≤Paτb ≤(2−ξ)mi(t)pi+ξmi(t)e−τ2.
(34) (45)
In the following, we calculate the resistance of lower and
=∆+, (35)
i upper bound of the above Pτ using Lemma 3. Note that
ab
wherer ∆+i =max{0,∆i}. Res(pi)=∆+i , Res(e−τ2)=(cid:16)2, and ∆(cid:16)i ≤2. Th(cid:17)e(cid:17)resistance of
theInnothiseyfpoolltoewntiinagl,CwAePsghaomwethGˆaNt twheithreessistitmanacteedouftBiliLtiLeAsUˆfoNr lower bound of Paτb is Res ξmi(t) pi−e−τ2
i
(cid:16)(cid:16) (cid:17)(cid:17)
is Lsaemmemaas i4n:(3L5e)t. Fdeonrottheis,∆wNe n=eedUˆtNhe(af)ol−lowUˆiNng(bl)e,m∆ma.= =Res(ξmi(t))+Res pi−e−τ2 , (46)
i i i i (cid:110) (cid:16) (cid:17)(cid:111)
Ui(a)−Ui(b), =min Res(pi),Res e−τ2 , (47)
pNi =E(cid:20)(cid:16)1+e∆Ni /τ(cid:17)−1(cid:21), (36) =Res(pi). (48)
Similarly, the resistance of upper bound of Pτ is
pi =(cid:16)1+e∆i/τ(cid:17)−1, (37) Res(cid:16)(2−ξ)mi(t)pi+ξmi(t)e−τ2(cid:17) ab
and consider the event Aδ =(cid:8)(cid:12)(cid:12)∆Ni −∆i(cid:12)(cid:12)<δ(cid:9). Then =min(cid:110)Res((2−ξ)mi(t)pi),Res(cid:16)ξmi(t)e−τ2(cid:17)(cid:111), (49)
(cid:12)(cid:12)pNi −pi(cid:12)(cid:12)≤δτ−1pi+2Pr(cid:0)A¯δ(cid:1). (38) =min(cid:110)Res(pi),Res(cid:16)e−τ2(cid:17)(cid:111), (50)
Proof: Notice that the probability of transition of BLLA
from action a to b in noisy potential game GˆN is pN = =Res(pi). (51)
i
PrN(a→b)givenin(36)andindeterministicpotentialgame Sinceboththeboundshavethesameresistance,byRule8the
is pi = Pr(a→b) given in (37). Using the law of total resistanceofPaτb existsandisequaltoRes(pi).Therefore,the
probability, we can write resistance of transitions of BLLA with bounded noise is same
pN = PrN(cid:16)a→bAδ(cid:17)Pr(cid:0)Aδ(cid:1)+PrN(cid:16)a→bA¯δ(cid:17)Pr(cid:0)A¯δ(cid:1). as in the case of without noise (35).
i   Proof for unbounded noise case: In this case, we use
(39) ChernoffboundtocalculatePr(cid:0)A¯δ(cid:1)becauseoftheunbounded
and
noise as below. Let denote the noise Z with moment gener-
i
(cid:12)(cid:12)pNi −pi(cid:12)(cid:12)≤(cid:12)(cid:12)(cid:12) PrN(cid:16)a→bAδ(cid:17)−pi(cid:12)(cid:12)(cid:12)Pr(cid:0)Aδ(cid:1) ating function M(θ).
+(cid:12)(cid:12)(cid:12) PrN(cid:16)a→bA¯δ(cid:17)−pi(cid:12)(cid:12)(cid:12)Pr(cid:0)A¯δ(cid:1). (40) Pr(cid:0)A¯δ(cid:1)=Pr(cid:32)N1 (cid:88)N |Zi|>δ(cid:33)=2Pr(cid:32)N1 (cid:88)N Zi >δ(cid:33),
i=1 i=1
It can be shown that the absolute value of the derivative of p
i (52)
with respect to ∆ is τ−1p (1−p ) ≤ τ−1p . Therefore, we
i i i i (cid:18) (cid:18) eθ∗δ (cid:19)(cid:19)
have (cid:12) (cid:16)  (cid:17) (cid:12) ≤2exp −Nlog , (53)
(cid:12) PrN a→bAδ −p (cid:12)≤δτ−1p . (41) M(θ∗)
(cid:12)  i(cid:12) i
where, (52) is obtained by assuming symmetric probability
(cid:12) (cid:16)  (cid:17) (cid:12)
Also, we bound (cid:12)(cid:12) PrN a→bA¯δ −pi(cid:12)(cid:12) ≤ 2. Substituting, distributionofnoise.However,fornon-symmetricdistribution
this and (41) in (40) we have (38). a more complex expression can be obtained. Also, we used
theChernoffboundforindependentandidenticallydistributed We will show that the derivative p(cid:48)(τ) is positive in the
random variables to obtain the equation (53). ∂f(δ,τ)
vicinity of 0. Previous lemma shows that ≤ k/τ.
Substituting(53),δ =(1−ξ)τ,andthenumberofsamples ∂τ
Since the constant function k/τ is integrable w.r.t. to the
N from (8) in Lemma 4, we have
distribution of ∆ , then
N
ξpi−4ξe−τ2 ≤pNi ≤(2−ξ)pi+4ξe−τ2. (54) p(cid:48)(τ)=E(cid:20)∂f(∆N,τ)(cid:21).
Following the same steps as before, we get that the resis- ∂τ
tance of transitions of BLLA with unbounded noise is same By previous lemma, the point reaching the maximum of
as in the case of without noise (35). ∂f(δ,τ)
goes to zero when τ goes to zero, and the function
∂τ
is then decreasing. Hence, for any (cid:15), there is τ small enough
B. Proof of Convergence of BLLA with Decreasing τ(t) such that the minimum (resp. maximum) of the derivative on
the interval [δ −(cid:15),δ +(cid:15)] is attained at δ +(cid:15) (resp. δ −(cid:15)).
We give the proof in the case of bounded noise. The proof
Consider the event
for unbounded noise can be done similarly. The proof is
divided into several lemmas. For a given parameter τ, we fix A(cid:15) =(cid:8)(cid:12)(cid:12)∆Ni −∆i(cid:12)(cid:12)<(cid:15)(cid:9). (57)
N(τ) as in (7), and we consider p(τ) = pN(τ). Recall that
p(τ)=E[f(∆ ,τ)] with f(δ,τ)=(cid:0)1+exp(cid:0)δ(cid:1)(cid:1)−1. Following the proof techniques used to show the convergence
N τ of BLLA
∂f(δ,τ)
Lemma 5: Function is odd, has the sign of δ, (cid:20) (cid:21)
∂f(∆ ,τ)
∂τ p(cid:48)(τ)=E N , (58)
is bounded in absolute value by k/τ for some k > 0, and ∂τ
the maximum is attained (for positive value) at the point a∗τ, (cid:20) (cid:21) (cid:20) (cid:21)
∂f(∆ ,τ) ∂f(∆ ,τ)
where a∗ >0. =E N |A¯(cid:15) P[A¯(cid:15)]+E N |A(cid:15) P[A(cid:15)],
∂τ ∂τ
Proof: We have
(59)
∂f(δ,τ) δ 1 k ∂f(δ+(cid:15),τ)
= . (55) ≥− P[A¯(cid:15)]+ P[A(cid:15)], (60)
∂τ τ22+exp(δ/τ)+exp(−δ/τ) τ ∂τ
kξ 2 ∂f(δ+(cid:15),τ)
This is an odd function in δ that has the sign of δ. Hence, we ≥− exp(− )+0.5 . (61)
just consider the case δ >0. Then τ τ ∂τ
In the above (60) is obtained by using Lemma 5 and (61)
∂2f(δ,τ) 1 (cid:20) δ Y −Y−1 (cid:21)
= 1− , is obtained by choosing δ = (1−ξ)τ. Note that as in (55)
∂δ∂τ τ2(2+Y +Y−1)2 τ 2+Y +Y−1 δ+(cid:15) δ+(cid:15)
the above second term is equivalent to exp(− ),
τ2 τ
with Y = exp(δ/τ). This is first positive and then negative
kξ 2
when δ is positive. The maximum is reached when which is a dominant term compared to exp(− ) if τ is
τ τ
small enough. Hence the derivative is lower bounded by a
δ Y −Y−1
=1. (56) positive function and then is positive. More, by choosing (cid:15)=
τ 2+Y +Y−1
τ2, we see that the derivative is lower bounded by a function
We claim that the maximum in δ is attained for δ∗ = a∗τ, equivalent to τ12e−τδ, which has the resistance δ.
with a∗ > 0 a constant. Indeed, consider δ = aτ with a > 0 The upper bound is obtained with the following inequality:
in (56), which gives (cid:20) (cid:21) (cid:20) (cid:21)
∂f(∆ ,τ) ∂f(∆ ,τ)
p(cid:48)(τ)=E N |A¯(cid:15) P[A¯(cid:15)]+E N |A(cid:15) P[A(cid:15)],
2+exp(a)(1−a)+exp(−a)(1+a)=0. ∂τ ∂τ
k ∂f(δ−(cid:15),τ)
Considerthefunctiong(a)=2+exp(a)(1−a)+exp(−a)(1+ ≤ P[A¯(cid:15)]+ P[A(cid:15)],
τ ∂τ
a).Wehaveg(0)=4,andg tendsto−∞whenagoesto∞.
kξ 2 ∂f(δ−(cid:15),τ)
Furthermore, the derivative is −a(exp(a)+exp(−a)) which ≤ exp(− )+0.5 .
τ τ ∂τ
is strictly negative, hence there is a unique solution a∗ to the
equation (56). Replacing δ by δ∗ =a∗τ in (55) yields: And by choosing (cid:15) = τ2 we obtain the same equivalent
∂f(δ∗,τ) a∗ 1 function τ12e−τδ, which has the resistance δ,
= . Therefore,fromRule8wehavethattheresistanceof|p(cid:48)(τ)|
∂τ τ 2+exp(a∗)+exp(−a∗)
is δ.
a∗ Lemma 7: The non-homogeneous Markov chain generated
Hence the result follows with k = .
2+exp(a∗)+exp(−a∗) by the BLLA algorithm with decreasing parameter τ(t) =
1 is weakly ergodic.
log(1+t)
Lemma 6: If δ > 0 (resp. δ < 0), then p(τ) is increasing Proof:TheconditionsofvalidityofTheorem2in[31]are
(resp.decreasing)inthevicinityofτ =0.Furthermore,|p(cid:48)(τ)| checked by Lemma 6, Equation (54) and the classical choice
has resistance δ. ofdecreasingparameterτ.Moredetailsaboutweakergodicity
Proof: We consider δ >0. The case δ <0 is similar. can also be found in [32].