Association Affinity Network Based Multi-Model Collaboration for Multimedia Big Data ...

204 Pages·2015·2.7 MB·English

University of Miami
Scholarly Repository
Open Access Dissertations Electronic Theses and Dissertations
2013-12-15
Association Affinity Network Based Multi-Model
Collaboration for Multimedia Big Data
Management and Retrieval
Tao Meng
University of Miami, [email protected]
Follow this and additional works at:https://scholarlyrepository.miami.edu/oa_dissertations
Recommended Citation
Meng, Tao, "Association Affinity Network Based Multi-Model Collaboration for Multimedia Big Data Management and Retrieval"
(2013).Open Access Dissertations. 1115.
https://scholarlyrepository.miami.edu/oa_dissertations/1115
This Open access is brought to you for free and open access by the Electronic Theses and Dissertations at Scholarly Repository. It has been accepted for
inclusion in Open Access Dissertations by an authorized administrator of Scholarly Repository. For more information, please contact
[email protected].
NOTE: This is the unnumbered page section.

UNIVERSITY OF MIAMI

ASSOCIATION AFFINITY NETWORK BASED
MULTI-MODEL COLLABORATION FOR MULTIMEDIA
BIG DATA MANAGEMENT AND RETRIEVAL

By

Tao Meng

A DISSERTATION

Submitted to the Faculty
of the University of Miami
in partial fulfillment of the requirements for
the degree of Doctor of Philosophy

Coral Gables, Florida

December 2013
NOTE: All Master’s and Ph.D. students must include this page in the front matter.
Substitute your name in place of “Student Name”.

©2013
Tao Meng
All Rights Reserved
UNIVERSITY OF MIAMI

A dissertation submitted in partial fulfillment of
the requirements for the degree of
Doctor of Philosophy

ASSOCIATION AFFINITY NETWORK BASED
MULTI-MODEL COLLABORATION FOR MULTIMEDIA
BIG DATA MANAGEMENT AND RETRIEVAL

Tao Meng

Approved:

__________________________ ___________________________
Mei-Ling Shyu, Ph.D. Xiaodong Cai, Ph.D.
Professor of Electrical and Associate Professor of Electrical

Computer Engineering and Computer Engineering

__________________________ ___________________________
Saman Aliari Zonouz, Ph.D. Nigel John, Ph.D.
Assistant Professor of Electrical Lecturer of Electrical and
and Computer Engineering Computer Engineering

__________________________ ___________________________
Shu-Ching Chen, Ph.D. M. Brian Blake, Ph.D.
Professor of School of Computing Dean of the Graduate School
and Information Sciences
Florida International University

NOTE:
(1) Do NOT add “Chairman,” “Committee Member,” or “Outside Member” to any signees’ name.
Signature lines should include only the signees’ name, highest degree earned, and title. Do not add
“Dr.” or “Professor” in front of a signee’s name.
(2) Check accuracy of each committee member’s name and title at www.miami.edu by clicking
“People” at the top of the landing page and typing in the last name of the committee member in the
field next to it.
page si
natures on this page will be accepted. Faxed or electronic s
MENG, TAO (Ph.D., Electrical and

Association Affinity Network Based Computer Engineering)

Multi-Model Collaboration for Multimedia (December 2013)

Big Data Management and Retrieval

Abstract of a dissertation at the University of Miami.

Dissertation supervised by Professor Mei-Ling Shyu.
No. of pages in text. (185)

With the rapid development of smart devices and ever-increasing popularity of social
media websites such as Flickr, YouTube, Twitter and Facebook, we have witnessed a huge
increase of multimedia data. Recently, social media technology and big data have
converged to provide rich content on what happens around the world via texts, images,
videos, audios, etc. Given the enormous volumes of multimedia data, efficient and effective
retrieval of relevant information according to users’ needs poses great challenges for
traditional text-oriented storage and retrieval systems. Since manually annotating and
managing the huge amount of information have become infeasible, data-driven approaches
have received more and more attention. As a result, mining interesting patterns and human
understandable semantic features automatically from raw multimedia data to facilitate
large-scale knowledge discovery and information retrieval has become an essential
research task in today’s multimedia big data analysis.
One of the central problems in multimedia big data analysis is automatic data
annotation. The automatic annotation for human readable patterns such as the semantic
concepts in video or image data sets provides the foundation for content-based search and
retrieval. The biggest challenge that the researchers face now is the semantic gap problem,
which is the gap between low-level features and high-level concepts. A lot of efforts have
been made in bridging this gap in the multimedia research field. This dissertation mainly
focuses on utilizing content information and inter-label correlations to improve annotation
accuracy. The multimedia data annotation problem is first converted to a multi-label or a
multi-class classification problem. Next, in order to model correlations among labels
mathematically, we design an association affinity network (AAN) to capture such
correlations.
In multimedia data sets, the label correlation is usually hidden information. In addition,
a large number of associations can be noisy. The scores from different models are also of
different scales. Facing these challenges, we propose several steps in utilizing the AAN to
address these issues. First, the output scores from different models are normalized using
the Bayesian posterior probability approach. Second, the nodes and links are modeled
properly under a given application scenario. Third, a link selection module is proposed to
filter the noisy links using association rule mining and correlation mining. Next, the
weights of different links are computed using different models, such as the collaboration
model and the regression model. Finally, the newly computed scores are used for
classification and/or data retrieval purposes. In addition, negative correlations, which have
rarely been explored, are studied and utilized in this dissertation.
The proposed framework is applied to two real-world applications: the multi-label high
level semantic concept detection and the multi-class biomedical image temporal stage
annotation. Experiments utilizing the benchmark data sets, such as the TRECVID semantic
indexing data sets and the IICBU biomedical image data set, have been conducted to
evaluate the effectiveness of the proposed framework. Generally speaking, the proposed
framework achieves promising results. The contributions of different components are
evaluated. The experimental results demonstrate that modeling and utilizing the inter-label
correlation properly could help improve multimedia data annotation accuracy and bridge
the semantic gap. As the extensions of the existing framework, several future research
directions are also proposed.
Tomydearparents
iii
Acknowledgments
I would like to thank my dissertation supervisor and chairman of the committee Dr. Mei-Ling
Shyuforhersupport,suggestionsandguidanceinmyPh.D.study. Alsomygreatappreciation
goes to Dr. Shu-Ching Chen of the School of Computing and Information Sciences at Florida
InternationalUniversity(FIU)forhisguidanceandsuggestions. Inaddition,mythanksgotoDr.
XiaodongCai,Dr. NigelJohnandDr. SamanAliariZonouzoftheDepartmentofElectricaland
ComputerEngineeringatUniversityofMiamiforacceptingtheappointmenttothedissertation
committeeandtheirconsistenthelpandsuggestions. Iwouldalsoliketothankthedepartment
for supporting my Ph.D. study. Moreover, my special thanks go to the National Oceanic and
Atmospheric Administration and TCL Research America for the financial support throughout
myPh.D.study. IwouldliketothankallmyfriendsandcolleaguesintheDatamining,Database
and Multimedia Research Group at UM and the Distributed Multimedia Information Systems
LaboratoryatFIU,fortheirhelp,considerationandsupport,especiallywhenImetdifficultyin
research. MythanksalsogotoMr. LeAn,Dr. XiaochenTang,Mr. LiangPeng,Dr. LifanGuo,
Ms. JiaHuang,Mr. JiaXu,Mr. YangLiu,andMs. MingMa.
Last but not the least, I would like to extend my utmost gratitude to my parents for their
loveandencouragement. Itwouldhavebeenimpossibleformetofinishthisworkwithouttheir
consistenthelpandsupport.
TAO MENG
UniversityofMiami
December2013
iv
Contents
ListofFigures ix
ListofTables xii
CHAPTER1 Introduction 1
1.1 MotivationsandChallenges . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2 ProposedSolutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.1 Multimedia AnnotationComponent . . . . . . . . . . . . . . . 12
1.2.2 AssociationAffinityNetwork(AAN)ModelingComponent . . 12
1.2.3 EvaluationComponent . . . . . . . . . . . . . . . . . . . . . . 17
1.3 Contributions andLimitations . . . . . . . . . . . . . . . . . . . . . . 20
1.4 OutlineoftheDissertation . . . . . . . . . . . . . . . . . . . . . . . . 24
CHAPTER2 LiteratureReview 26
2.1 Large-ScaleMultimedia DataMining . . . . . . . . . . . . . . . . . . 26
2.1.1 DataMiningforBigData . . . . . . . . . . . . . . . . . . . . 27
2.1.2 Mining MultimediaBigData . . . . . . . . . . . . . . . . . . . 29
2.1.3 BenchmarkDataSetsforMultimediaBigDataMining . . . . . 31
2.2 UtilizingInter-Concept Correlations inMultimedia ConceptDetection . 33
2.3 Multi-ClassClassification . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.3.1 NaturalExtensionofBinaryClassificationModel . . . . . . . . 41
v