Federated Learning Over Wireless Edge Networks

175 Pages·2022·5.069 MB·English

by  Wei Yang Bryan Lim, Jer Shyuan Ng, Zehui Xiong, Dusit Niyato, Chunyan Miao

Wireless Networks
SeriesEditor
XueminShermanShen,UniversityofWaterloo,Waterloo,ON,Canada
The purpose of Springer’s Wireless Networks book series is to establish the state
of the art and set the course for future research and development in wireless
communication networks. The scope of this series includes not only all aspects
of wireless networks (including cellular networks, WiFi, sensor networks, and
vehicular networks), but related areas such as cloud computing and big data.
The series serves as a central source of references for wireless networks research
and development. It aims to publish thorough and cohesive overviews on specific
topics in wireless networks, as well as works that are larger in scope than survey
articles and that contain more detailed background information. The series also
providescoverageofadvancedandtimelytopicsworthyofmonographs,contributed
volumes,textbooksandhandbooks.
**Indexing:WirelessNetworksisindexedinEBSCOdatabasesandDPLB**
Wei Yang Bryan Lim (cid:129) Jer Shyuan Ng (cid:129)
Zehui Xiong (cid:129) Dusit Niyato (cid:129) Chunyan Miao
Federated Learning Over
Wireless Edge Networks
WeiYangBryanLim JerShyuanNg
Alibaba-NTUJointResearchInstitute Alibaba-NTUJointResearchInstitute
Singapore,Singapore Singapore,Singapore
ZehuiXiong DusitNiyato
SingaporeUniversityofTechnologyand SchoolofComputerScienceand
Design Engineering
Singapore,Singapore NanyangTechnologicalUniversity
Singapore,Singapore
ChunyanMiao
SchoolofComputerScienceand
Engineering
NanyangTechnologicalUniversity
Singapore,Singapore
ISSN2366-1186 ISSN2366-1445 (electronic)
WirelessNetworks
ISBN978-3-031-07837-8 ISBN978-3-031-07838-5 (eBook)
https://doi.org/10.1007/978-3-031-07838-5
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNatureSwitzerland
AG2022
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse
ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Theconfluenceofedgecomputingandartificialintelligence(AI)hasdriventherise
ofedgeintelligence,whichleveragesthestorage,communication,andcomputation
capabilitiesofenddevicesandedgeserverstoempowerAIimplementationatscale
closertowheredataisgenerated.Anenablingtechnologyofedgeintelligenceisthe
privacy-preserving machine learning paradigm known as federated learning (FL).
Amidtheincreasinglystringentprivacyregulations,FLwillenablethedevelopment
of applications that have to be built using sensitive user data and will continue to
revolutionize service delivery in finance, Internet of Things (IoT), healthcare, and
transportindustries,amongothers.However,theimplementationofFLisenvisioned
toinvolvethousandsofheterogeneousdistributedenddevicesthatdifferintermsof
communication and computation resources, as well as the levels of willingness to
participateinthecollaborativemodeltrainingprocess.Thepotentialnodefailures,
devicedropouts,andstragglerseffectarekeybottlenecksthatimpedetheeffective,
sustainable,andscalableimplementationofFL.
In Chap.1, we will first present a tutorial and survey on FL and highlight
its role in enabling edge intelligence. This tutorial and survey provide readers
with a comprehensive introduction to the forefront challenges and state-of-the-art
approaches towards implementing FL at the edge. In consideration of resource
heterogeneity at the edge networks, we then provide multifaceted solutions for-
mulated via the interdisciplinary interplay of concepts derived from network
economics, optimization, game theory, and machine learning towards improving
theefficiencyofresourceallocationforimplementingFLatscaleamidinformation
asymmetry.InChap.2,wedeviseamulti-dimensionalcontract-matchingapproach
for optimized resource allocation for federated sensing and learning amid multi-
dimensional sources of heterogeneities. In Chap.3, we propose a joint-auction
coalition formation framework towards facilitating communication-efficient FL
networks aided by unmanned aerial vehicles (UAVs). In Chap.4, we propose a
two-level evolutionary game theoretic and auction approach to allocate and price
resourcestofacilitateefficientedgeintelligence.InChap.5,werecapthekeypoints
anddiscussthepromisingresearchdirectionsforfutureworks.
v
vi Preface
We sincerely thank our collaborators for their contributions to the presented
researchworks.SpecialthanksalsogotothestaffatSpringerNaturefortheirhelp
throughout the publication preparation process. Finally, we would like to take the
chancetodedicatethisbooktocelebratethebirthofLimChenHuanTheodore,son
ofDr.LimWeiYangBryanandFooFengLin.
Singapore,Singapore WeiYangBryanLim
Singapore,Singapore JerShyuanNg
Singapore,Singapore ZehuiXiong
Singapore,Singapore DusitNiyato
Singapore,Singapore ChunyanMiao
Contents
1 FederatedLearningatMobileEdgeNetworks:ATutorial ............. 1
1.1 Introduction ............................................................. 1
1.2 BackgroundandFundamentalsofFederatedLearning ............... 6
1.2.1 FederatedLearning ............................................. 6
1.2.2 StatisticalChallengesofFL ................................... 8
1.2.3 FLProtocolsandFrameworks ................................. 11
1.2.4 UniqueCharacteristicsandIssuesofFL ...................... 12
1.3 CommunicationCost ................................................... 12
1.3.1 EdgeandEndComputation ................................... 13
1.3.2 ModelCompression ............................................ 16
1.3.3 Importance-BasedUpdating ................................... 17
1.4 ResourceAllocation .................................................... 18
1.4.1 WorkerSelection ............................................... 21
1.4.2 JointRadioandComputationResourceManagement ....... 25
1.4.3 AdaptiveAggregation .......................................... 27
1.4.4 IncentiveMechanism .......................................... 28
1.5 PrivacyandSecurityIssues ............................................ 32
1.5.1 PrivacyIssues .................................................. 33
1.5.2 SecurityIssues ................................................. 37
1.6 Applications of Federated Learning for Mobile Edge
Computing .............................................................. 39
1.6.1 CyberattackDetection ......................................... 42
1.6.2 EdgeCachingandComputationOffloading .................. 44
1.6.3 BaseStationAssociation ...................................... 47
1.6.4 VehicularNetworks ............................................ 48
1.7 ConclusionandChapterDiscussion ................................... 50
2 Multi-dimensionalContractMatchingDesignforFederated
LearninginUAVNetworks ................................................. 53
2.1 Introduction ............................................................. 53
2.2 SystemModelandProblemFormulation ............................. 56
2.2.1 UAVSensingModel ........................................... 58
vii
viii Contents
2.2.2 UAVComputationModel ...................................... 59
2.2.3 UAVTransmissionModel ..................................... 60
2.2.4 UAVandModelOwnerUtilityModeling .................... 61
2.3 Multi-dimensionalContractDesign ................................... 61
2.3.1 ContractConditionAnalysis .................................. 62
2.3.2 ConversionintoaSingle-DimensionalContract ............. 63
2.3.3 ConditionsforContractFeasibility ........................... 64
2.3.4 ContractOptimality ............................................ 68
2.4 UAV-SubregionAssignment ........................................... 70
2.4.1 MatchingRules ................................................. 71
2.4.2 MatchingImplementationandAlgorithm .................... 72
2.5 PerformanceEvaluation ................................................ 73
2.5.1 ContractOptimality ............................................ 73
2.5.2 UAV-SubregionPreferenceAnalysis .......................... 76
2.5.3 Matching-BasedUAV-SubregionAssignment ................ 78
2.6 ConclusionandChapterDiscussion ................................... 80
3 Joint Auction–Coalition Formation Framework for
UAV-AssistedCommunication-EfficientFederatedLearning........... 83
3.1 Introduction ............................................................. 83
3.2 SystemModel .......................................................... 86
3.2.1 WorkerSelection ............................................... 88
3.2.2 UAVEnergyModel ............................................ 90
3.3 CoalitionsofUAVs ..................................................... 93
3.3.1 CoalitionGameFormulation .................................. 94
3.3.2 CoalitionFormationAlgorithm ............................... 97
3.4 AuctionDesign ......................................................... 98
3.4.1 Buyers’Bids .................................................... 99
3.4.2 Sellers’Problem ................................................ 100
3.4.3 AnalysisoftheAuction ........................................ 102
3.4.4 Complexity of the Joint Auction–Coalition
Algorithm ....................................................... 104
3.5 SimulationResultsandAnalysis ...................................... 105
3.5.1 CommunicationEfficiencyinFLNetwork ................... 106
3.5.2 PreferenceofCellsofWorkers ................................ 107
3.5.3 Profit-MaximizingBehaviorofUAVs ........................ 109
3.5.4 AllocationofUAVstoCellsofWorkers ...................... 111
3.5.5 ComparisonwithExistingSchemes .......................... 114
3.6 ConclusionandChapterDiscussion ................................... 115
4 EvolutionaryEdgeAssociationandAuctioninHierarchical
FederatedLearning.......................................................... 117
4.1 Introduction ............................................................. 117
4.2 SystemModelandProblemFormulation ............................. 120
4.2.1 SystemModel .................................................. 120
4.2.2 Lower-LevelEvolutionaryGame ............................. 121
4.2.3 Upper-LevelDeepLearningBasedAuction .................. 121
Contents ix
4.3 Lower-LevelEvolutionaryGame ...................................... 122
4.3.1 EvolutionaryGameFormulation .............................. 122
4.3.2 WorkerUtilityandReplicatorDynamics ..................... 123
4.3.3 Existence, Uniqueness, and Stability of the
EvolutionaryEquilibrium ...................................... 125
4.4 DeepLearningBasedAuctionforValuationofClusterHead ....... 128
4.4.1 AuctionFormulation ........................................... 128
4.4.2 Deep Learning Based Auction for Valuation of
ClusterHeads ................................................... 130
4.4.3 MonotoneTransformFunctions ............................... 132
4.4.4 AllocationRule ................................................. 133
4.4.5 ConditionalPaymentRule ..................................... 134
4.4.6 NeuralNetworkTraining ...................................... 134
4.5 PerformanceEvaluation ................................................ 135
4.5.1 Lower-LevelEvolutionaryGame ............................. 136
4.5.2 Upper-LevelDeepLearningBasedAuction .................. 140
4.6 ConclusionandChapterDiscussion ................................... 143
5 ConclusionandFutureWorks.............................................. 147
References......................................................................... 151
Index............................................................................... 165