Sensor fusion in smart camera networks for ambient intelligence

220 Pages·2017·21.73 MB·English

Sensor fusion in smart camera networks for ambient
intelligence
Citation for published version (APA):
Maatta, T. T. (2013). Sensor fusion in smart camera networks for ambient intelligence. [Phd Thesis 1 (Research
TU/e / Graduation TU/e), Electrical Engineering]. Technische Universiteit Eindhoven.
https://doi.org/10.6100/IR755363
DOI:
10.6100/IR755363
Document status and date:
Published: 01/01/2013
Document Version:
Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can be
important differences between the submitted version and the official published version of record. People
interested in the research are advised to contact the author for the final version of the publication, or visit the
DOI to the publisher's website.
• The final author version and the galley proof are versions of the publication after peer review.
• The final published version features the final layout of the paper including the volume, issue and page
numbers.
Link to publication
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners
and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.
• You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please
follow below link for the End User Agreement:
www.tue.nl/taverne
Take down policy
If you believe that this document breaches copyright please contact us at:
[email protected]
providing details and we will investigate your claim.
Download date: 27. Feb. 2023
Sensor Fusion in Smart Camera Networks for
Ambient Intelligence
by Tommi M¨a¨att¨a
The work described in this thesis was carried out at the Philips Research Labora-
tories Eindhoven, Eindhoven University of Technology, and Stanford University in
California. This research was commissioned and funded by Philips Research.
AcataloguerecordisavailablefromtheEindhovenUniversityofTechnologyLibrary.
ISBN: 978-90-386-3404-3
NUR: 959
Cover Design: Tommi M¨a¨att¨a
Reproduction: Ipskamp Drukkers, Enschede
Sensor Fusion in Smart Camera Networks for
Ambient Intelligence
PROEFSCHRIFT
ter verkrijging van de graad van doctor aan de
Technische Universiteit Eindhoven, op gezag van de
rector magnificus, prof.dr.ir. C.J. van Duijn, voor een
commissie aangewezen door het College voor
Promoties in het openbaar te verdedigen
op maandag 01 juli 2013 om 14.00 uur
door
Tommi Tapio M¨a¨att¨a
geboren te Sotkamo, Finland
Dit proefschrift is goedgekeurd door de promotor:
prof.dr. H. Corporaal
Copromotoren:
dr.ir. A. H¨arm¨a
en
prof.dr.ir. H. Aghajan
Contents
I Background 1
1 Introduction - Visual Analysis 3
1.1 Enabling Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Vision Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Multi-Sensor Data Fusion . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Targeting Individuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Towards Ambient Intelligence . . . . . . . . . . . . . . . . . . . . . . . 10
1.6 Thesis Overview and Contributions . . . . . . . . . . . . . . . . . . . . 10
2 Vision Networks 13
2.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Network Categorization . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3 Three-Level Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.4 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3 Fusion Levels and Methods for Vision Networks 27
3.1 Process of Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2 Levels of Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3 Methods of Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4 Fusion Architectures for Vision Networks 53
4.1 Architectures for Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . 54
v
vi CONTENTS
4.2 Synchronization for Fusion. . . . . . . . . . . . . . . . . . . . . . . . . 55
4.3 Architecture Models for Fusion . . . . . . . . . . . . . . . . . . . . . . 57
4.4 Part-I Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
II Framework and Experiments 61
5 Vision Fusion Framework 63
5.1 Related Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.2 The Proposed Framework . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.3 Effects from the Vision Network . . . . . . . . . . . . . . . . . . . . . 67
5.4 Effects of the Fusion Architecture . . . . . . . . . . . . . . . . . . . . . 71
5.5 Effects of the Fusion Level . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.6 Effects of the Fusion Method . . . . . . . . . . . . . . . . . . . . . . . 76
5.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6 Remote Visual Communication 79
6.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6.2 Proposed Vision System . . . . . . . . . . . . . . . . . . . . . . . . . . 82
6.3 Application Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.4 Fusion Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7 Recognition of Activities 111
7.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.2 Activity Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.3 Classifiers and Their Structure . . . . . . . . . . . . . . . . . . . . . . 115
7.4 Multi-View Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
7.5 Datasets used in the experiments . . . . . . . . . . . . . . . . . . . . . 119
7.6 Experiments: Feature Selection . . . . . . . . . . . . . . . . . . . . . . 122
7.7 Experiments: Multi-View Scenarios . . . . . . . . . . . . . . . . . . . . 127
7.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
8 Detection of Repetitive Behavior 133
8.1 Introduction and Related Work . . . . . . . . . . . . . . . . . . . . . . 134
8.2 Behavior by Transitions: Action History Matrices . . . . . . . . . . . . 136
8.3 System - Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
CONTENTS vii
8.4 System - Person Tracking . . . . . . . . . . . . . . . . . . . . . . . . . 141
8.5 System - Transition Modeling . . . . . . . . . . . . . . . . . . . . . . . 142
8.6 Experiments - Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
8.7 Experiments - Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
8.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
9 Office Ergonomics 159
9.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
9.2 Vision System for Office Ergonomics . . . . . . . . . . . . . . . . . . . 162
9.3 Dataset used in the Experiments . . . . . . . . . . . . . . . . . . . . . 166
9.4 Experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
9.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
10 General Discussion 179
10.1 Vision Networks - Potential . . . . . . . . . . . . . . . . . . . . . . . . 180
10.2 Emerging Applications - Opportunities . . . . . . . . . . . . . . . . . . 180
10.3 Fusion in Vision Networks . . . . . . . . . . . . . . . . . . . . . . . . . 181
10.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
III Appendices 185
References 199
Glossary 201
Summary 205
Acknowledgments 209
Curriculum Vitae 211
viii CONTENTS
Part I
Background
1