Table Of ContentKah Phooi Seng · Li-minn Ang
Alan Wee-Chung Liew · Junbin Gao
Editors
Multimodal Analytics for
Next-Generation
Big Data Technologies
and Applications
Multimodal Analytics for Next-Generation
Big Data Technologies and Applications
(cid:129) (cid:129)
Kah Phooi Seng Li-minn Ang
(cid:129)
Alan Wee-Chung Liew Junbin Gao
Editors
Multimodal Analytics
for Next-Generation
Big Data Technologies
and Applications
Editors
KahPhooiSeng Li-minnAng
SchoolofEngineeringandInformation SchoolofInformationandCommunication
Technology Technology
UniversityofNewSouthWales GriffithUniversity
Canberra,ACT,Australia GoldCoast,QLD,Australia
AlanWee-ChungLiew JunbinGao
SchoolofInformationandCommunication TheUniversityofSydneyBusinessSchool
Technology UniversityofSydney
GriffithUniversity Sydney,NSW,Australia
GoldCoast,QLD,Australia
ISBN978-3-319-97597-9 ISBN978-3-319-97598-6 (eBook)
https://doi.org/10.1007/978-3-319-97598-6
©SpringerNatureSwitzerlandAG2019
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartofthe
materialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors, and the editorsare safeto assume that the adviceand informationin this
bookarebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG.
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
ToGraceAngYi-en,myblesseddaughter,for
all the joy you bring.
—Kah Phooi Seng
To my parents and loved ones, for your
unceasing support and love.
—Li-minn Ang
To Alana and Nicholas, my two lovely
children, who bring me joy and headache.
—Alan Liew Wee Chung
To Zhi Chen, my lovely wife, and Henry Yu
Gao, my amazing son, for your constant love
and support.
—Junbin Gao
Preface
Thedigitalagebringsmoderndataacquisitionmethodswhichallowthegatheringof
different types and modes of data in variety and volume (termed in this book as
multimodalBigData)tobeusedasmultipleinputsourcesintoincreasinglysophis-
ticated computing systems deploying intelligent techniques and machine-learning
abilities to find hidden patterns embedded in combined data. Some examples of
different types of data modalities include text, speech, image, video, and sensor-
baseddata.Thedatatypescouldoriginatefromvarioussourcesrangingfromsocial
medianetworkstowirelesssensorsystemsandcouldbeintheformofstructuredor
unstructured data. Big Data techniques are targeted toward large system-level
problems that cannot be solved by conventional methods and technologies. The
advantage of utilizing multimodal Big Data is that it facilitates a comprehensive
view of information and knowledge as it allows data to be integrated, analyzed,
modeled,andvisualizedfrommultiplefacetsandviewpoints.
While the use of multimodal data gives increased information-rich content for
informationandknowledgeprocessing,itleadstoanumberofadditionalchallenges
in terms of scalability, decision-making, data fusion, distributed architectures, and
predictive analytics. Addressing these challenges requires new approaches for data
collection,transmission,storage,andinformationprocessingfromthemultipledata
sources.TheaimofthisbookistoprovideacomprehensiveguidetomultimodalBig
Data technologies and analytics and introduce the reader to the current state of
multimodal Big Data information processing. We hope that the reader will share
ourenthusiasminpresentingthisvolumeandwillfindituseful.
Intended Audience
The target audience for this book is very broad. It includes academic researchers,
scientists, lecturers, and advanced students and postgraduate students in various
disciplineslikeengineering,informationtechnology,andcomputerscience.Itcould
vii
viii Preface
be an essential supportfor fellowship programs in BigDataresearch. This bookis
alsointendedforconsultants,practitioners,andprofessionalswhoareexpertsinIT,
engineering, and business intelligence to build decision support systems. The fol-
lowing groups will also benefit from the content of the book: scientists and
researchers, academic and corporate libraries, lecturers and tutors, postgraduates,
practitionersandprofessionals,undergraduates,etc.
ACT,Australia KahPhooiSeng
GoldCoast,Australia Li-minnAng
GoldCoast,Australia AlanWee-ChungLiew
Sydney,Australia JunbinGao
Acknowledgments
We express our deepest appreciation to all the people who have helped us in the
completionofthisbook.Wethankallthereviewersofthebookfortheirtremendous
service bycritically reviewing thechapters and offeringuseful suggestions, partic-
ularly Ch’ng Sue Inn, Chew Li Wern, and Md Anisur Rahman. We gratefully
acknowledgeourSpringereditorRonanNugentandthepublicationteam,fortheir
diligenteffortsandsupporttowardthepublicationofthisbook.
KahPhooiSeng
Li-minnAng
AlanWee-ChungLiew
JunbinGao
ix
Contents
PartI Introduction
1 MultimodalInformationProcessingandBigDataAnalytics
inaDigitalWorld. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
KahPhooiSeng,Li-minnAng,AlanWee-ChungLiew,
andJunbinGao
PartII Sentiment,AffectandEmotionAnalysisforBig
MultimodalData
2 Speaker-IndependentMultimodalSentimentAnalysis
forBigData. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
ErikCambria,SoujanyaPoria,andAmirHussain
3 MultimodalBigDataAffectiveAnalytics. . .. . . . . . . .. . . . . . . .. . 45
NusratJahanShoumy,Li-minnAng,andD.M.MotiurRahaman
4 HybridFeature-BasedSentimentStrengthDetection
forBigDataApplications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
YanghuiRao,HaoranXie,FuLeeWang,LeonardK.M.Poon,
andEndongZhu
PartIII UnsupervisedLearningStrategiesforBigMultimodalData
5 MultimodalCo-clusteringAnalysisofBigDataBased
onMatrixandTensorDecomposition. . . . . . . . . . . . . . . . . . . . . . . 95
HongyaZhao,ZhenghongWei,andHongYan
6 Bi-clusteringbyMulti-objectiveEvolutionaryAlgorithm
forMultimodalAnalyticsandBigData. . . . . . . . . . . . . . . . . . . . . . 125
MaryamGolchinandAlanWee-ChungLiew
7 UnsupervisedLearningonGrassmannManifoldsforBigData. . . . 151
BoyueWangandJunbinGao
xi
xii Contents
PartIV SupervisedLearningStrategiesforBigMultimodalData
8 Multi-productNewsvendorModelinMulti-taskDeepNeural
NetworkwithNormRegularizationforBigData. . . . . . . . . . . . . . 183
YanfeiZhang
9 RecurrentNeuralNetworksforMultimodalTimeSeries
BigDataAnalytics. . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . .. 207
MingyuanBaiandBoyanZhang
10 ScalableMultimodalFactorizationforLearningfromBigData. . . 245
QuanDoandWeiLiu
PartV MultimodalBigDataProcessingandApplications
11 BigMultimodalVisualDataRegistrationforDigitalMedia
Production. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
HansungKimandAdrianHilton
12 AHybridFuzzyFootballScenesClassificationSystemforBig
VideoData. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
SongWeiandHaniHagras
13 MultimodalBigDataFusionforTrafficCongestionPrediction. . . . 319
TaiwoAdetiloyeandAnjaliAwasthi
14 ParallelandDistributedComputingforProcessingBigImage
andVideoData. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
PraveenKumar,ApekshaBodade,HarshadaKumbhare,
RuchitaAshtankar,SwapnilArsh,andVatsalGosar
15 MultimodalApproachesinAnalysingandInterpretingBigSocial
MediaData. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
EugeneCh’ng,MengdiLi,ZiyangChen,JingboLang,andSimonSee
