Optimal Medium Access Control in Cognitive Radios: A Sequential Design Approach

0.09 MB·English

by  Lifeng Lai

OPTIMALMEDIUMACCESS CONTROLIN COGNITIVERADIOS:ASEQUENTIAL
DESIGN APPROACH
Lifeng Lai1, HeshamEl Gamal2, HaiJiang3 andH. VincentPoor1
1Dept. ofElec. Eng.,Princeton Univ.,{llai,poor}@princeton.edu
2Dept. ofElec. and Comp. Eng.,OhioStateUniv.,[email protected]
3Dept. ofElec. and Comp. Eng.,Univ. ofAlberta, [email protected]
8
0
0 ABSTRACT opportunities. Thisdecisionprocesscanbeenhancedbytak-
2 ingintoaccountanyavailablestatisticalinformationaboutthe
The design of medium access controlprotocolsfor a cogni- primarytraffic. Forexample,withasinglecognitiveuserca-
n
tiveuserwishingtoopportunisticallyexploitfrequencybands
a pable of accessing (sensing) only one channelat a time, the
within parts of the radio spectrum having multiple bands is
J problembecomestrivialiftheprobabilitythateachchannelis
1 considered. In the scenario under consideration, the avail- freeisknownapriori. Inthiscase,theoptimalruleisforthe
abilityprobabilityofeachchannelisunknownaprioritothe
2 cognitiveusertoaccessthechannelwiththehighestprobabil-
cognitiveuser.Henceefficientmediumaccessstrategiesmust
ityofbeingfreeinalltimeslots. However,suchtime-varying
] strikeabalancebetweenexploringtheavailabilityofchannels traffic information is typically not available to the cognitive
T and exploiting the opportunitiesidentified thus far. Using a
users a priori. The need to learn this information on-line
I sequentialdesignapproach,anoptimalmediumaccessstrat-
. creates a fundamentaltradeoff between exploitation and ex-
s egyis derived. To avoidthe prohibitivecomputationalcom-
ploration. Exploitationreferstotheshort-termgainresulting
c plexityofthisoptimalstrategy,alowcomplexityasymptoti-
[ fromaccessingthechannelwiththeestimatedhighestproba-
callyoptimalstrategyisalsodeveloped. Theproposedstrat- bilityof beingfree(basedon theresultsof previoussensing
1 egydoesnotrequireanypriorstatisticalknowledgeaboutthe
decisions)whereasexplorationistheprocessbywhichacog-
v trafficpatternonthedifferentchannels.
nitiveuser learnsthe statistical behaviorofthe primarytraf-
9
IndexTerms— Cognitiveradio,banditproblem,medium fic (by choosing possibly different channels to probe across
8
2 accesscontrol. time slots). In the presence of multiple cognitive users, the
3 mediumaccessalgorithmmustalsoaccountforthecompeti-
. 1. INTRODUCTION tionbetweendifferentusersoverthesamechannel.
1
Inthispaper,wedevelopaunifiedframeworkforthede-
0
signandanalysisofcognitivemediumaccessprotocolsinthe
8 As a promising technique to increase spectral efficiency of
presence of a single cognitive user who can access a single
0 overcrowded parts of the radio spectrum, the opportunistic
: spectrum access problem has been the focus of significant channelineachtimeslot.Asarguedinthesequel,thisframe-
v work allows for the construction of strategies that strike an
research activities [1]. The underlying idea is to allow un-
i
X licensed users (i.e., cognitive users) to access the available optimal balance between exploration and exploitation. We
derive an optimal sensing rule that maximizes the expected
r spectrumwhenthelicensedusers(i.e.,primaryusers)arenot
a throughputobtainedbythecognitiveuser. Comparedwitha
active. The presence of high priority primary users and the
genie-aidedscheme,inwhichthecognitiveuserknowsapri-
requirementthatthecognitiveusersshouldnotinterferewith
oritheprimarynetworktrafficinformation,thereisathrough-
themintroducenewchallengesforprotocoldesign.Theover-
putlosssufferedbyanymediumaccessstrategy.Weobtaina
archinggoalofthecurrentworkistodevelopaunifiedframe-
lower boundon this loss andfurtherconstructa linear com-
workforthedesignofefficient,andlowcomplexity,cognitive
plexity single index protocol that achieves this lower bound
mediumaccessprotocols.
asymptotically (when the primary traffic behavior changes
Thespectralopportunitiesavailabletocognitiveusersare
slowly).Similarapproacheshavebeenconsideredin[3]and[4],
by their nature time-varying. To avoid interfering with the
butwithdifferentemphases.
primary network, cognitive users must first probe to deter-
Wehavealsoextendedourstudytonetworkswithmulti-
minewhetherthereareprimaryactivitiesbeforetransmission.
plecognitiveusersandnetworkswithmorecapablecognitive
Undertheassumptionthateachcognitiveusercannotaccess
users, and have developed optimal strategies for these sce-
alloftheavailablechannelssimultaneously,themaintaskof
narios. However,duetospacelimitations,wedonotdiscuss
themediumaccessprotocolistodistributivelychoosewhich
theseresultshere.Wealsoomittheproofsofresultspresented
channelseachcognitiveusershouldattempttouseindifferent
inthispaper.Interestedreaderscanreferto[5]fordetails.
timeslots,inordertofully(ormaximally)utilizethespectral
The rest of this paper is organized as follows. Our net-
workmodelisdetailedin Section2. Section3developsand
ThisresearchwassupportedbytheNationalScienceFoundationunder
GrantsANI-03-38807andCNS-06-25637. analyzesanoptimalstrategyforthesinglecognitiveusersce-
nario.Finally,Section4summarizesourconclusions. Intuitively,thecognitiveuserwouldliketoselectthechan-
nelwith thehighestprobabilityof beingfreein orderto ob-
2. NETWORKMODEL tainmoretransmissionopportunities. Ifθ isknownthenthis
problemistrivial: thecognitiveusershouldchoosethechan-
Figure 1 shows the channel model of interest. We consider neli∗ = argmaxθ to sense. The uncertaintyin θ imposes
i
a primary network consisting of N non-overlapping chan- i∈N
afundamentaltradeoffbetweenexploration,inordertolearn
nels, N = {1,··· ,N}, each with bandwidthB. The users
θ,andexploitation,byaccessingthechannelwiththehighest
in the primary network are operated in a synchronoustime-
estimatedfreeprobabilitybasedoncurrentavailableinforma-
slotted fashion. We assume that at each time slot, channeli
tion,asdetailedinthefollowingsection.
is free with probability θ . Let Z (j) be a random variable
i i
that equals 1 if channel i is free at time slot j and equals 0
3. OPTIMALMEDIUMACCESSPROTOCOLS
otherwise.Hence,givenθ ,Z (j)isaBernoullirandomvari-
i i
able with distribution h (z (j)) = θ δ(1)+ (1 − θ )δ(0),
whereδ(·)isadeltafunθcitioin. Furthermi ore,foragiveinθ = Waliezesdtaartssbuymdpetivoenlotphiantgft(hθe)oipstikmnaolwsnolautpiorniourindbeyrtthheeciodge--
(θ ,··· ,θ ),theZ (j)areindependentforeachiandj. We
1 N i nitive user. As we will see, this optimal medium access al-
consider a block varying model in which the value of θ is
gorithmsuffersfromaprohibitivecomputationalcomplexity
fixedfora blockofT timeslotsandthenrandomlychanges
that growsexponentiallywith the block length T. This mo-
atthebeginningofthenextblockaccordingtoajointproba-
tivates the design of low complexity asymptotically optimal
bilitydensityfunction(pdf)f(θ).
approaches,whichwealsoconsider.
Ourcognitivemediumaccessproblembelongstotheclass
t=1(cid:13) t=T(cid:13)
of banditproblems. In this setting, the decisionmaker must
Channel 1(cid:13)
Channel 2(cid:13) sequentiallychooseoneprocesstoobservefromN ≥2stochas-
tic processes. These processes usually have parameters that
areunknowntothedecisionmakerand,associatedwitheach
observation is a utility function. The objective of the deci-
Channel N(cid:13)
sionmakeristomaximizethesumordiscountedsumofthe
utilitiesviaastrategythatspecifieswhichprocesstoobserve
Occupied by the primary users(cid:13)
for every possible history of selections and observations. A
Spectrum opportunities(cid:13) comprehensivetreatment covering different variants of ban-
ditproblemscanbefoundin[2].
Fig.1. Channelmodel. We are now ready to rigorously formulate our problem.
ThecognitiveuseremploysamediumaccessstrategyΓ,which
In our model, the cognitive users attempt to exploit the willselectchannelS(j) ∈ N to senseattime slotj forany
availabilityoffreechannelsintheprimarynetworkbysens- possiblecausalinformationpatternobtainedthroughthepre-
ingtheactivityatthebeginningofeachtimeslot. Ourwork viousj−1observations:Ψ(j)={s(1),z (1),··· ,s(j−
s(1)
seeks to characterize efficient strategies for choosing which 1),z (j−1)},j ≥2,i.e.s(j)=Γ(f,Ψ(j)). Noticethat
s(j−1)
channels to sense (access). The challenge here stems from z (j)isthesensingoutcomeofthejth timeslot,inwhich
s(j)
thefactthatthecognitiveusersareassumedtobeunawareof
s(j)is thechannelbeingaccessed. If j = 1, thereis noac-
θ apriori. We considertwocases inwhicha cognitiveuser
cumulatedinformation,andthusΨ(1)=φands(1)=Γ(f).
eitherhasordoesnothavepriorinformationaboutthepdfof
Theutilitythatthecognitiveuserobtainsbymakingdecision
θ, i.e., f(θ). In thescenariopresentedin thispaper,attime
S(j)attimeslotjisthenumberofbitsitcantransmitattime
slotj,asinglecognitiveuserselectsonechannelS(j)∈N to
slotj,whichisBZ (j). Wedenotetheexpectedvalueof
access. IfthesensingresultshowsthatchannelS(j)isfree, S(j)
the payoffobtained by a cognitiveuser who uses strategy Γ
i.e., Z (j) = 1, the cognitive user can send B bits over
S(j) as
thischannel;otherwise,thecognitiveuserwillwaituntilthe
nexttimeslotandpickapossiblydifferentchanneltoaccess. T
Therefore,the total numberof bits that the cognitiveuser is W =E  BZ (j). (1)
Γ f S(j)
abletosendoveroneblock(ofT timeslots)is Xj=1 
 
T WefurtherdenoteV∗(f,T)=supW ,whichisthelargest
Γ
W = BZ (j).
S(j) Γ
X throughputthatthecognitiveusercouldobtainwhenthespec-
j=1
tralopportunitiesaregovernedbyf(θ)andtheexactvalueof
It is clear that W is a random variable that depends on eachrealizationofθisnotknownaprioribytheuser.
the traffic in the primary networkand, moreimportantlyfor Eachmediumaccessdecisionmadebythecognitiveuser
us, the medium access protocols employed by the cognitive has two effects. The first one is the short-termgain, i.e., an
user. Therefore,theoverarchinggoalofthispaperistocon- immediatetransmissionopportunityif thechosenchannelis
struct low complexity medium access protocols that maxi- found free. The second one is the long-term gain, i.e., the
mizeE{W}. updatedstatisticalinformationaboutf(θ). Thisinformation
willhelpthecognitiveuserinmakingbetterdecisionsinfu- argmaxV∗(f ,1). Similarly, after solving the T = 2
turestages. Thereisaninterestingtradeoffbetweentheshort i∈N zi∗(1)
problem,onecanproceedtosolvetheT =3case. Usingthis
andlong-termgains. Ifweonlywanttomaximizetheshort-
procedurerecursively,wecansolvetheproblemwithT −1
termgain,wecanchoosethechannelwiththehighestavail-
observations. Finally, our originalproblemwith T observa-
abilityprobabilitytosense,basedonthecurrentinformation.
tionsissolvedasfollows.
This myopic strategy maximally exploits the existing infor-
mation. On the other hand, by choosing other channels to V∗(f,T) = max [Bθ +θ V∗(f ,T −1)
sense, we gain statistical informationabout f(θ) which can i∈N Z i i zi=1
effectively guide future decisions. This process is typically +(1−θ )V∗(f ,T −1)]f(θ)dθ.
i zi=0
referredtoasexploration,asnotedpreviously.
Morespecifically,letfj(θ)betheupdatedpdfaftermak- Theoptimalsolutiondevelopedabovesuffersfromapro-
ing j −1 observations. We begin with f1(θ) = f(θ). Af- hibitive computationalcomplexity. In particular, the dimen-
terobservingz (j),weupdatethepdfusingthefollowing sionality of our search dimension grows exponentially with
s(j)
Bayesianformula. the blocklength T. Moreover,one can envisionmanyprac-
1. Ifzs(j)(j)=1,fj+1(θ)= Rθθss((jj))ffjj((θθ))dθ, ttiivcaelussceernatorioosbtianinwthhiechpritiowroinufldorbmeatdioifnficfu(lθt)f.orTthheiscmogontii--
vatesourpursuitoflowcomplexitynon-parametricprotocols
2. Ifzs(j)(j)=0,fj+1(θ)= R((11−−θθss((jj))))ffjj((θθ))dθ. wpehnidchonmfai(nθta)inexcpelrictaitilny.opHtiemncaeli,tyinptrhoepefortlileoswainngd,dwoenaoitmdteo-
The following result characterizes the optimal strategy that develop strategies that depend only on the information ob-
maximizestheaveragethroughputthecognitiveuserobtains tainedthroughobservationsΨ.
fromthenetwork. For a given strategy Γ, the expected number of bits the
Lemma1 Foranypriorpdff,thereexistsanoptimalstrat- cognitiveuser isabletotransmitthroughablockwithgiven
egyΓ∗ tothechannelselectionproblem(1),andV∗(f,T)is parametersθis
achievable.Moreover,V∗satisfiesthefollowingcondition:
T T N
V∗(f,T)=s(m1)a∈xNEf(cid:8)BZs(1)+V∗(cid:0)fZs(1),T −1(cid:1)(cid:9), (2) EXj=1BZS(j)(j)=Xj=1BXi=1θiPr{Γ(Ψ(j))=i}.
wherefZs(1) istheconditionaldistributionupdatedusingthe Recall that Γ(Ψ(j)) = i means that, following strategy
Bayesianruledescribedabove,asifthecognitiveuserchooses
Γ, the cognitiveusershouldchoose channeli in time slot j,
s(1)andobservesZ .Also,V∗ f ,T −1 isthevalue
s(1) Zs(1) basedontheavailableinformationΨ(j).HerePr{Γ(Ψ(j))=i}
ofabanditproblemwithpriorinfo(cid:0)rmationfZs(1(cid:1)) andT −1 istheprobabilitythatthecognitiveuserwillchoosechanneli
sequentialobservations.
attimeslotj,followingthestrategyΓ.
Inprinciple,Lemma1providesthesolutiontoproblem(1).
Compared with the idealistic case where the exact value
Effectively, it decouples the calculation at each stage, and
ofθisknown,inwhichtheoptimalstrategyforthecognitive
hence, allows the use of dynamicprogrammingto solve the
useristoalwayschoosethechannelwiththelargestavailabil-
problem. The ideais to solvethe channelselectionproblem
ityprobability,thelossincurredbyΓisgivenby
withasmallerdimensionfirstandthenusebackwarddeduc-
tiontoobtaintheoptimalsolutionforaproblemwithalarger T T N
dimension. Starting with T = 1, the secondterm inside the L(θ;Γ)= Bθi∗ − B θiPr{Γ(Ψ(j))=i},
expectationin(2)is0,sinceT−1=0.Hence,theoptimalso- Xj=1 Xj=1 Xi=1
lutionistochoosethechannelihavingthelargestE {BZ },
whichcanbecalculatedasE {BZ } = B θ f(θ)fdθ.Aind where θi∗ = max{θ1,··· ,θN}. We say that a strategy Γ
V∗(f,1) = mi∈aNxEf{BZi}. fWith tihe solutRioniforT = 1 at sisucchonthsiastteLn(tθi;f,Γf)osrcaanleysθas∈O([T0,β1).]NI,nththereeseexqiusetsl,βwe<us1e
hand, we can now solve the T = 2 case using (2). At first, the following notations 1) g (N) = ω(g (N)) means that
1 2
for every possible choice of s(1) and possible observation ∀c > 0,∃N , such that ∀N > N ,g (N) < cg (N); 2)
0 0 2 1
zs(1), we calculate the updated distribution fzs(1) using the g1(n)=O(g2(N))meansthat∃c1,c2 >0andN0,suchthat
Bayesianformula. Next,wecalculateV∗(f ,1)(whichis ∀N > N , c g (N) ≤ g (N) ≤ c g (N). For example,
zs(1) 0 1 2 1 2 2
equivalentto the T = 1 problemdescribed above). Finally, consider a loyal scheme in which the cognitive user selects
applying(2),wehavethefollowingequationforthechannel channeliatthebeginningofablockandstickstoit. Ifθ is
i
selectionproblemswithT =2: thelargestoneamongθ,L(θ;Γ) = 0. Ontheotherhand,if
θ isnotthelargestone,L(θ;Γ) ∼ O(T). Hence,thisloyal
i
V∗(f,2) = mi∈aNxZ [Bθi+θiV∗(fzi=1,1) schemeisnotconsistent. Thefollowinglemmacharacterizes
thefundamentallimitsofanyconsistentscheme.
+(1−θ )V∗(f ,1)]f(θ)dθ.
i zi=0 Lemma2 ForanyθandanyconsistentstrategyΓ,wehave
Hence,inthefirststep,thecognitiveusershouldchoosei∗(1)=
argmi∈aNxV∗(f,2)tosense. Afterobservingzi∗(1),thecogni- lim inf L(θ;Γ) ≥B θi∗ −θi , (3)
tiveuserhasΨ(1) = {zi∗(1)},anditshouldchoosei∗(2) = T→∞ lnT i∈NX\{i∗}D(θi||θi∗)
whereD(θ ||θ )denotestheKullback-Leiblerdivergencebe- The intuition behind this strategy is that as long as Y
i l i
tweenthetwoBernoullirandomvariableswithparametersθ grows as fast as O(lnT), Λ convergesto the true value of
i i
andθlrespectively:D(θi||θl)=θiln(cid:16)θθil(cid:17)+(1−θi)ln(cid:16)11−−θθil(cid:17). θnielinwpitrhobthaebillaitryg,easntdθitheevecnotgunailtliyv.eTuhseerlowssillocfhOo(olsneTth)eccohmaens-
fromthetimespentinsamplingtheinferiorchannelsinorder
Lemma2showsthatthelossofanyconsistentstrategyscales
to learn the valueof θ. Thisprice, however,is inevitableas
at least as ω(lnT). An intuitive explanation of this loss is
establishedinthelowerboundofLemma2.
thatweneedtospendatleastO(lnT)timeslotsonsampling
Finally, we observe that the difference between the my-
eachofthechannelswithsmallerθ ,inordertogetareason-
ablyaccurateestimateofθ,andheniceuseittodeterminethe opic rule and the order optimal single index rule is the ad-
channelhavingthelargestθi tosense. Wesaythatastrategy ditionalterm 2lnj/Yi(j)addedtothecurrentestimateθˆi.
ΓisorderoptimalifL(θ;Γ)∼O(lnT). Roughlyspeakping,thisadditionaltermguaranteesenoughsam-
Beforeproceedingtotheproposedlowcomplexityorder- plingtimeforeachchannel,sinceifwesamplechannelitoo
optimalstrategy,wefirstanalyzethelossorderofsomeheuris- sparsely, Yi(j) will be small, which will increasethe proba-
ticstrategieswhichmayappeartobereasonable. bilitythatΛi isthelargestindex. WhenYi(j)scalesaslnT,
Thefirst simple ruleis therandomstrategyΓr where, at θˆi will be the dominantterm in the index Λi, and hence the
eachtimeslot,thecognitiveuserrandomlychoosesachannel channel with the largest θi will be chosen much more fre-
fromtheavailableN channels. Thefractionoftimethecog- quently.
nitiveuserspendsoneachchannelistherefore1/N,leading
4. CONCLUSIONS
N
tothelossL(θ;Γr)= BiP=1(Nθi∗−θi)T ∼O(T). Thisworkhasdevelopeda unifiedframeworkforthedesign
ThesecondoneisthemyopicruleΓg inwhichthecogni- andanalysisofcognitivemediumaccessbasedontheclassi-
tiveuserkeepsupdatingfj(θ),andchoosesthechannelwith cal bandit problem. Our formulation highlights the tradeoff
the largest value of θˆ = θ fj(θ)dθ at each stage. Since betweenexplorationandexploitationincognitivechannelse-
i i
therearenoconvergencegRuaranteesforthemyopicrule,that lection. A linear complexitycognitivemedium access algo-
is θˆ may never convergeto θ due to the lack of sufficiently rithm,whichisasymptoticallyoptimalasthenumberoftime
slotsincreases,hasalsobeenproposed.
many samples for each channel [6], the loss of this myopic
strategyisO(T). 5. REFERENCES
Thethirdprotocolweconsiderisstayingwiththewinner
and switching from the loser rule ΓSW where the cognitive [1] J. Mitola, “Cognitive radio: Making software radios
userrandomlychoosesachannelinthefirsttimeslot. Inthe morepersonal,”IEEEPersonalCommunications,vol.6,
succeedingtime-slots1)iftheaccessedchannelwasfoundto pp.13–18,Aug.1999.
be free, it will choose the same channel to sense; 2) other-
[2] D.A.BerryandB.Fristedt,BanditProblems: Sequen-
wise,itwillchooseoneoftheremainingchannelsbasedona
tialAllocationofExperiments. London: Chapmanand
certainswitchingrule.
Hall,1985.
Lemma3 Nomatterwhattheswitchingruleis,L(θ;Γ )∼
SW [3] A. Motamedi and A. Bahai, “Dynamic channel selec-
O(T).
tion for spectrum sharing in unlicensed bands,” Euro-
There are several strategies that have loss of order O(lnT). pean Trans. on Telecommunicationsand Related Tech-
Weadoptthefollowinglinearcomplexitystrategyfrom[7]. nologies,2007. Submitted.
[4] Q. Zhao, L. Tong, A. Swami, and Y. Chen, “Decen-
Rule1 (Orderoptimalsingleindexstrategy)
tralized cognitive MAC for opportunistic spectrum ac-
ThecognitiveusermaintainstwovectorsXandY,where
cessinadhocnetworks: APOMDPframework,”IEEE
eachX recordsthenumberoftimeslotsinwhichthecogni-
i JournalonSelectedAreasInCommunications,vol.25,
tiveuserhassensedchannelitobefree,andeachY records
i pp.589–600,Apr.2007.
thenumberoftimeslotsinwhichthecognitiveuserhascho-
senchannelitosense. Thestrategyworksasfollows. [5] L.Lai,H.ElGamal,H.Jiang,andH.V.Poor,“Cogni-
tivemediumaccess: exploration,exploitationandcom-
1. Initialization:atthebeginningofeachblock,eachchan- petition,”IEEE/ACMTrans.onNetworking,2007.Sub-
nelissensedonce. mitted,availableatwww.princeton.edu/∼llai.
[6] P. R. Kumar, “A survey of some results in stochastic
2. After the initialization period, the cognitive user ob-
adaptive control,” SIAM Journal on Control and Opti-
tainsanestimateθˆatthebeginningoftimeslotj,given
mization,vol.23,pp.329–380,May1985.
byθˆ(j)=X (j)/Y (j),andassignsanindexΛ (j)=
i i i i
θˆ(j)+ 2lnj/Y (j)totheithchannel.Thecognitive [7] P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time
i i
analysis of the multiarmed bandit problem,” Machine
userchoposesthechannelwiththelargestvalueofΛ (j)
i
Learning, vol. 47, pp. 235–256, 2002. Kluwer Aca-
tosenseattimeslotj. Aftereachsensing,thecognitive
userupdatesXandY. demicPublishers.