Table Of ContentComputer Science I Internet-Scale
n
t
e
Internet-Scale Pattern Recognition r
n
e
t
New Techniques for Voluminous Data Sets and Data Clouds -
S Pattern Recognition
c
a
l
e
For machine intelligence applications to work successfully, P
a
machines must perform reliably under variations of data and must
t
t New Techniques for Voluminous
be able to keep up with data streams. Internet-Scale Pattern e
r
Recognition: New Techniques for Voluminous Data Sets n
and Data Clouds unveils computational models that address R
Data Sets and Data Clouds
e
performance and scalability to achieve higher levels of reliability. c
o
It explores different ways of implementing pattern recognition g
n
using machine intelligence. i
t
i
o
Based on the authors’ research from the past 10 years, the text n
draws on concepts from pattern recognition, parallel processing,
Anang Hudaya Muhamad Amin
K
distributed systems, and data networks. It describes fundamental ha M
n u
research on the scalability and performance of pattern recognition, , a ha
n m
addressing issues with existing pattern recognition schemes for d a Asad I. Khan
Na d A
Internet-scale data deployment. The authors review numerous s m
approaches and introduce possible solutions to the scalability utio in,
n Benny B. Nasution
problem.
By presenting the concise body of knowledge required for
reliable and scalable pattern recognition, this book shortens the
learning curve and gives you valuable insight to make further
innovations. It offers an extendable template for Internet-scale
pattern recognition applications as well as guidance on the
programming of large networks of devices.
K14810
K14810_Cover.indd 1 10/15/12 11:09 AM
Internet-Scale
Pattern Recognition
New Techniques for Voluminous
Data Sets and Data Clouds
Internet-Scale
Pattern Recognition
New Techniques for Voluminous
Data Sets and Data Clouds
Anang Hudaya Muhamad Amin
Asad I. Khan
Benny B. Nasution
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2013 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20121207
International Standard Book Number-13: 978-1-4665-1097-5 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
“Knowledge is the conformity of the object and the intellect.”— Averroes
Contents
Preface xi
Acknowledgments xv
About the Authors xvii
I Recognition: A New Perspective 1
1 Introduction 3
1.1 As We See, We Learn . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Recognition at a Large Scale . . . . . . . . . . . . . . . . . . 4
1.3 Computational Intelligence Approach for Pattern Recognition 8
1.4 Scalability in Pattern Recognition . . . . . . . . . . . . . . . 11
1.4.1 Common Barriers . . . . . . . . . . . . . . . . . . . . 11
1.4.2 Possible Solutions . . . . . . . . . . . . . . . . . . . . 12
1.4.3 Distributed Computing Solution for Scalability of PR
Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 Distributed Approach for Pattern Recognition 15
2.1 Scalability of Neural Network Approaches . . . . . . . . . . . 16
2.1.1 Pattern Storage Capacity . . . . . . . . . . . . . . . . 16
2.1.2 Inter-Neuron Communication Frequency . . . . . . . . 17
2.2 Key Components of DPR . . . . . . . . . . . . . . . . . . . . 20
2.2.1 Learning Mechanism . . . . . . . . . . . . . . . . . . . 20
2.2.2 Processing Approach . . . . . . . . . . . . . . . . . . . 21
2.2.3 Training Procedure . . . . . . . . . . . . . . . . . . . . 21
2.3 System Approaches . . . . . . . . . . . . . . . . . . . . . . . 22
2.4 Pattern Distribution Techniques . . . . . . . . . . . . . . . . 25
2.4.1 Subpattern Distribution . . . . . . . . . . . . . . . . . 25
2.4.2 Pattern Set Distribution . . . . . . . . . . . . . . . . . 26
2.5 Current DPR Schemes . . . . . . . . . . . . . . . . . . . . . 27
2.5.1 Graph Neuron . . . . . . . . . . . . . . . . . . . . . . 27
2.5.2 Hierarchical Graph Neuron . . . . . . . . . . . . . . . 29
2.5.3 Distributed HierarchicalGraph Neuron . . . . . . . . 30
vii
viii Contents
2.6 Resource Considerations for DPR Implementations . . . . . 30
2.6.1 Resource-Aware Approach. . . . . . . . . . . . . . . . 31
2.6.2 Message-PassingModel in DPR. . . . . . . . . . . . . 31
II Evolution of Internet-Scale Recognition 33
3 One-Shot Learning Considerations 35
3.1 One-Shot Learning Graph Neuron (GN) Scheme . . . . . . . 36
3.1.1 Pattern Representation . . . . . . . . . . . . . . . . . 37
3.1.2 Recognition Procedure . . . . . . . . . . . . . . . . . . 38
3.2 One-Shot Learning Model . . . . . . . . . . . . . . . . . . . . 41
3.2.1 Bias Array Design for Pattern Memorization . . . . . 42
3.2.2 Collaborative-ComparisonLearning Technique . . . . 42
3.3 GN Complexity Estimation . . . . . . . . . . . . . . . . . . . 44
3.4 Graph Neuron Limitations . . . . . . . . . . . . . . . . . . . 46
3.5 Significance of One-Shot Learning . . . . . . . . . . . . . . . 48
4 Hierarchical Model for Pattern Recognition 49
4.1 Evolution of One-Shot Learning: The Hierarchical Approach 49
4.1.1 Solution to Crosstalk Problem . . . . . . . . . . . . . 51
4.1.2 Computational Design for a Hierarchical One-Shot
Learning DPR Scheme . . . . . . . . . . . . . . . . . . 52
4.1.3 HGN Recognition Procedure . . . . . . . . . . . . . . 55
4.2 Complexity and Scalability of HierarchicalDPR Scheme . . 57
4.2.1 Complexity Estimation . . . . . . . . . . . . . . . . . 57
4.2.2 Scalability in HGN Approach . . . . . . . . . . . . . . 60
4.3 Reducing HierarchicalComplexity: A Distributed Approach 60
4.3.1 Distributed Neurons of HGN Network . . . . . . . . . 61
4.3.2 Distributed HGN Approach . . . . . . . . . . . . . . . 63
4.4 Design Evaluation for Distributed DPR Approach . . . . . . 65
4.4.1 Non-Uniform Distribution . . . . . . . . . . . . . . . . 65
4.4.2 Uniform Distribution. . . . . . . . . . . . . . . . . . . 69
5 Recognition via Divide-and-Distribute Approach 73
5.1 Divide-and-Distribute Approach for One-Shot Learning IS-PR
Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.1.1 Associative Memory (AM) Concept in Pattern Recog-
nition . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.1.2 DHGN Computational Design. . . . . . . . . . . . . . 75
5.1.3 Dual-Phase Recognition Procedure . . . . . . . . . . . 80
Contents ix
5.2 Dimensionality Reduction in Pattern Pre-Processing . . . . . 87
5.2.1 Structural Reduction . . . . . . . . . . . . . . . . . . . 87
5.2.2 Content Reduction . . . . . . . . . . . . . . . . . . . . 88
5.3 Remarks on DHGN DPR Scheme . . . . . . . . . . . . . . . 89
III Systems and Tools 91
6 Internet-Scale Applications Development 93
6.1 Distributed Computing Models for IS-PR . . . . . . . . . . . 93
6.1.1 Commodity Grid (CoG) . . . . . . . . . . . . . . . . . 94
6.1.2 Cloud Computing . . . . . . . . . . . . . . . . . . . . 94
6.1.3 Peer-to-Peer(P2P) Computing . . . . . . . . . . . . . 98
6.2 ParallelProgrammingTechniques . . . . . . . . . . . . . . . 100
6.2.1 Message-PassingScheme . . . . . . . . . . . . . . . . . 100
6.2.2 GPU Programming. . . . . . . . . . . . . . . . . . . . 103
6.3 From Coding to Applications . . . . . . . . . . . . . . . . . . 104
IV Implementations and Applications 107
7 Multi-Feature Classifications for Complex Data 109
7.1 Data Features for Pattern Recognition . . . . . . . . . . . . 110
7.2 Distributed Multi-Feature Recognition . . . . . . . . . . . . 111
7.2.1 Conceptual Design and Implementation . . . . . . . . 112
7.2.2 Complexity Estimation . . . . . . . . . . . . . . . . . 113
7.3 Handwritten Object Classification with Multiple Features . . 116
7.3.1 Handwritten Object . . . . . . . . . . . . . . . . . . . 117
7.3.2 Classification Procedures . . . . . . . . . . . . . . . . 118
7.4 Distributed Multi-Feature Recognition Perspective . . . . . . 120
8 Pattern Recognition within Coarse-Grained Networks 121
8.1 Network Granularity Considerations . . . . . . . . . . . . . . 121
8.1.1 DHGN Configurations for Adaptive Granularity . . . 122
8.1.2 DHGN Commodity Grid Framework . . . . . . . . . . 124
8.2 Face Recognition Using the Multi-Feature DPR Approach . 128
8.2.1 Color and Spatio-StructuralFeatures Consideration . 129
8.3 Distributed Data Management within Cloud Computing . . 132
8.3.1 Cloud Data Access Scheme . . . . . . . . . . . . . . . 133
8.3.2 DHGN Approach for Cloud Data Access . . . . . . . . 135
8.4 Adaptive Recognition: A Different Perspective . . . . . . . . 138
