Robust Latent Feature Learning for Incomplete Big Data

119 Pages·2023·4.049 MB·English

by  Di Wu

SpringerBriefs in Computer Science
Di Wu
Robust Latent Feature
Learning for Incomplete
Big Data
SpringerBriefs in Computer Science
SeriesEditors
StanZdonik,BrownUniversity,Providence,RI,USA
ShashiShekhar,UniversityofMinnesota,Minneapolis,MN,USA
XindongWu,UniversityofVermont,Burlington,VT,USA
LakhmiC.Jain,UniversityofSouthAustralia,Adelaide,SA,Australia
DavidPadua,UniversityofIllinoisUrbana-Champaign,Urbana,IL,USA
XueminShermanShen,UniversityofWaterloo,Waterloo,ON,Canada
BorkoFurht,FloridaAtlanticUniversity,BocaRaton,FL,USA
V.S.Subrahmanian,UniversityofMaryland,CollegePark,MD,USA
MartialHebert,CarnegieMellonUniversity,Pittsburgh,PA,USA
KatsushiIkeuchi,UniversityofTokyo,Tokyo,Japan
BrunoSiciliano,UniversitàdiNapoliFedericoII,Napoli,Italy
SushilJajodia,GeorgeMasonUniversity,Fairfax,VA,USA
NewtonLee,InstituteforEducation,ResearchandScholarships,LosAngeles,CA,
USA
SpringerBriefs present concise summaries of cutting-edge research and practical
applications across a wide spectrum of fields. Featuring compact volumes of 50 to
125pages,theseriescoversarangeofcontentfromprofessionaltoacademic.
Typicaltopicsmightinclude:
(cid:129) Atimelyreportofstate-of-theartanalyticaltechniques
(cid:129) A bridge between new research results, as published in journal articles, and a
contextualliteraturereview
(cid:129) Asnapshotofahotoremergingtopic
(cid:129) Anin-depthcasestudyorclinicalexample
(cid:129) A presentation of core concepts that students must understand in order to make
independentcontributions
Briefs allow authors to present their ideas and readers to absorb them with
minimal time investment. Briefs will be published as part of Springer’s eBook
collection, with millions of users worldwide. In addition, Briefs will be available
forindividualprintandelectronicpurchase.Briefsarecharacterizedbyfast,global
electronic dissemination, standard publishing contracts, easy-to-use manuscript
preparation and formatting guidelines, and expedited production schedules. We
aim for publication 8–12 weeks after acceptance. Both solicited and unsolicited
manuscriptsareconsideredforpublicationinthisseries.
**Indexing:ThisseriesisindexedinScopus,Ei-Compendex,andzbMATH**
Di Wu
Robust Latent Feature
Learning for Incomplete
Big Data
DiWu
CollegeofComputerandInformationScience
SouthwestUniversity
Chongqing,China
ISSN2191-5768 ISSN2191-5776 (electronic)
SpringerBriefsinComputerScience
ISBN978-981-19-8139-5 ISBN978-981-19-8140-1 (eBook)
https://doi.org/10.1007/978-981-19-8140-1
©TheAuthor(s),underexclusivelicensetoSpringerNatureSingaporePteLtd.2023
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseof
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by
similarordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors, and the editorsare safeto assume that the adviceand informationin this
bookarebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Preface
Intheeraof BigData, high dimensional andincomplete(HDI) data arefrequently
encountered in many industrial applications, such as recommender systems, the
Internet of Things, intelligent transportation, cloud computing, and so on. It is of
great significance to analyze them for mining rich and valuable knowledge and
patterns. Latent feature learning is one of the most popular representation learning
methods tailored for HDI data due to its high accuracy, computational efficiency,
and ease of scalability. The crux of analyzing HDI data lies in addressing the
uncertainty problem caused by their incomplete characteristics. However, existing
HDImethodsdonotfullyconsidersuchuncertainty.
Inthisbook,Iintroduceseveralrobustlatentfeaturelearningmethodstoaddress
suchuncertaintyforeffectivelyandefficientlyanalyzingHDIdata,includingrobust
latent feature learning based on smooth L -norm, improving robustness of latent
1
featurelearningusingL -norm,improvingrobustnessoflatentfeaturelearningusing
1
double-space, data-characteristic-aware latent feature learning, posterior-
neighborhood-regularizedlatentfeaturelearning,andgeneralizeddeeplatentfeature
learning.
Thisisthefirstbookthatcanhelpresearchersandengineersfullyunderstandhow
to employ robust latent feature learning to effectively and efficiently analyze HDI
data.Itisassumedthatreadershaveabasicknowledgeofmathematics,aswellasa
certainbackgroundindatamining.Readerscanobtainanoverviewofthechallenges
of analyzing HDI data. In addition, this book provides several algorithms and real
application cases, which can help readers quickly build their robust latent feature
learningmodelstoanalyzeHDIdata.
Chongqing,China DiWu
September2022
v
Acknowledgments
The work described in this book was supported by the National Natural Science
FoundationofChinaundergrant62176070.
vii
Contents
1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 SymbolsandNotations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 BookOrganization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 BasisofLatentFeatureLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Preliminaries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3 LatentFeatureLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.1 ABasicLFLModel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.2 ABiasedLFLModel. . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.3 AlgorithmsDesign. . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 13
2.4 PerformanceAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4.1 EvaluationProtocol. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4.2 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3 RobustLatentFeatureLearningbasedonSmoothL -norm. . . . . . . 19
1
3.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 RelatedWork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3 ASmoothL -NormBasedLatentFeatureModel. . . . . . . . . . . . . 21
1
3.3.1 ObjectiveFormulation. . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.2 ModelOptimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3.3 IncorporatingLinearBiasesintoSL-LF. . . . . . . . . . . . . . . 23
3.4 PerformanceAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4.1 GeneralSettings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4.2 PerformanceComparison. . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4.3 OutlierDataSensitivityTests. . . . . . . . . . . . . . . . . . . . . . 27
3.4.4 TheImpactofHyper-Parameter. . . . . . . . . . . . . . . . . . . . 28
ix
x Contents
3.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4 ImprovingRobustnessofLatentFeatureLearningUsing
L -Norm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
1
4.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2 RelatedWork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.3 AnL -and-L -Norm-OrientedLatentFeatureModel. . . . . . . . . . . 35
1 2
4.3.1 ObjectiveFormulation. . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.3.2 ModelOptimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.3.3 Self-AdaptiveAggregation. . . . . . . . . . . . . . . . . . . . . . . . 37
4.4 PerformanceAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4.1 GeneralSettings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4.2 L3F’sAggregationEffects. . . . . . . . . . . . . . . . . . . . . . . . 40
4.4.3 ComparisonBetweenL3FandBaselines. . . . . . . . . . . . . . 41
4.4.4 L3F’sRobustnesstoOutlierData. . . . . . . . . . . . . . . . . . . 41
4.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
5 ImproveRobustnessofLatentFeatureLearningUsing
Double-Space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.2 RelatedWork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.3 ADouble-SpaceandDouble-NormEnsembledLatentFeature
Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.3.1 PredictorBasedonInnerProductSpace(D2E-LF-1). . . . . 50
5.3.2 PredictoronEuclideanDistanceSpace(D2E-LF-2). . . . . . 53
5.3.3 EnsembleofD2E-LF-1andD2E-LF-2. . . . . . . . . . . . . . . . 56
5.3.4 AlgorithmDesignandAnalysis. . . . . . . . . . . . . . . . . . . . 57
5.4 PerformanceAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.4.1 GeneralSettings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.4.2 PerformanceComparison. . . . . . . . . . . . . . . . . . . . . . . . . 59
5.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6 Data-characteristic-awareLatentFeatureLearning. . . . . . . . . . . . . 67
6.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.2 RelatedWork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.2.1 RelatedLFL-BasedModels. . . . . . . . . . . . . . . . . . . . . . . 68
6.2.2 DPClustAlgorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.3 AData-Characteristic-AwareLatentFeatureModel.. . . . . . . . . .. 70
6.3.1 ModelStructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.3.2 Step1:LatentFeatureExtraction. . . . . . . . . . . . . . . . . . . 71
6.3.3 Step2:NeighborhoodandOutlierDetection. . . . . . . . . . . 71
6.3.4 Step3:Prediction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Contents xi
6.4 PerformanceAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6.4.1 PredictionRuleSelection. . . . . . . . . . . . . . . . . . . . . . . . . 76
6.4.2 PerformanceComparison. . . . . . . . . . . . . . . . . . . . . . . . . 77
6.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7 Posterior-neighborhood-regularizedLatentFeatureLearning. . . . . 85
7.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7.2 RelatedWork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7.3 APosterior-Neighborhood-RegularizedLatentFeatureModel. . . . 87
7.3.1 PrimalLatentFeatureExtraction. . . . . . . . . . . . . . . . . . . . 87
7.3.2 Posterior-NeighborhoodConstruction. . . . . . . . . . . . . . . . 88
7.3.3 Posterior-Neighborhood-RegularizedLFL. . . . . . . . . . . . . 89
7.4 PerformanceAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
7.4.1 GeneralSettings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
7.4.2 ComparisonsBetweenPLFandState-of-the-ArtModels. . . 91
7.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
8 GeneralizedDeepLatentFeatureLearning. . . . .. . . . . . .. . . . . . .. 97
8.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
8.2 RelatedWork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
8.3 ADeepLatentFeatureModel. . . . . . . . . . . . . . . . . . . . . . . . . . . 98
8.4 PerformanceAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
8.4.1 GeneralSettings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
8.4.2 EffectsofLayerCountinDLF. . . . . . . . . . . . . . . . . . . . . 101
8.4.3 ComparisonBetweenDLFandRelatedModels. . . . . . . . . 102
8.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
9 ConclusionandOutlook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
9.1 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
9.2 Outlook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111