Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications: Dagstuhl 2012 Seminar on Time-of-Flight Imaging and GCPR 2013 Workshop on Imaging New Modalities

324 Pages·2013·31.55 MB·English

by  Damien Lefloch

t Marcin Grzegorzek Christian Theobalt
r
A
- Reinhard Koch Andreas Kolb (Eds.)
e
h
t
-
f
o
y
-e
e
v
t
ar
u
t
SS
Time-of-Flight
0
0 and Depth Imaging
2
8

S
C
N Sensors, Algorithms, and Applications
L
Dagstuhl 2012 Seminar on Time-of-Flight Imaging
and GCPR 2013 Workshop on Imaging New Modalities
123
Lecture Notes in Computer Science 8200
CommencedPublicationin1973
FoundingandFormerSeriesEditors:
GerhardGoos,JurisHartmanis,andJanvanLeeuwen
EditorialBoard
DavidHutchison
LancasterUniversity,UK
TakeoKanade
CarnegieMellonUniversity,Pittsburgh,PA,USA
JosefKittler
UniversityofSurrey,Guildford,UK
JonM.Kleinberg
CornellUniversity,Ithaca,NY,USA
AlfredKobsa
UniversityofCalifornia,Irvine,CA,USA
FriedemannMattern
ETHZurich,Switzerland
JohnC.Mitchell
StanfordUniversity,CA,USA
MoniNaor
WeizmannInstituteofScience,Rehovot,Israel
OscarNierstrasz
UniversityofBern,Switzerland
C.PanduRangan
IndianInstituteofTechnology,Madras,India
BernhardSteffen
TUDortmundUniversity,Germany
MadhuSudan
MicrosoftResearch,Cambridge,MA,USA
DemetriTerzopoulos
UniversityofCalifornia,LosAngeles,CA,USA
DougTygar
UniversityofCalifornia,Berkeley,CA,USA
GerhardWeikum
MaxPlanckInstituteforInformatics,Saarbruecken,Germany
Marcin Grzegorzek Christian Theobalt
Reinhard Koch Andreas Kolb (Eds.)
Time-of-Flight
and Depth Imaging
Sensors,Algorithms,andApplications
Dagstuhl 2012 Seminar on Time-of-Flight Imaging
andGCPR2013WorkshoponImagingNewModalities
1 3
VolumeEditors
MarcinGrzegorzek
UniversityofSiegen,PatternRecognitionGroup
Siegen,Germany
E-mail:[email protected]
ChristianTheobalt
Max-Planck-Institute,Graphics,Vision&VideoGroup
Saarbrücken,Germany
E-mail:[email protected]
ReinhardKoch
UniversityofKiel,MultimediaInformationProcessingGroup
Kiel,Germany
E-mail:[email protected]
AndreasKolb
UniversityofSiegen,ComputerGraphicsandMultimediaSystemsGroup
Siegen,Germany
E-mail:[email protected]
ISSN0302-9743 e-ISSN1611-3349
ISBN978-3-642-44963-5 e-ISBN978-3-642-44964-2
DOI10.1007/978-3-642-44964-2
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2013952935
CRSubjectClassification(1998):I.4,I.5,I.2
LNCSSublibrary:SL6–ImageProcessing,ComputerVision,PatternRecognition,
andGraphics
©Springer-VerlagBerlinHeidelberg2013
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
withreviewsorscholarlyanalysisormaterialsuppliedspecificallyforthepurposeofbeingenteredand
executedonacomputersystem,forexclusiveusebythepurchaserofthework.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheCopyrightLawofthePublisher’slocation,
initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Permissionsforuse
maybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violationsareliabletoprosecution
undertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpublication,
neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforanyerrorsor
omissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespecttothe
materialcontainedherein.
Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Cameras for 3D depth imaging, using either time-of-flight (ToF) or structured
light sensors, have received a lot of attention recently and have matured con-
siderably over the last few years. Presently, these techniques make full-range
3D data available at video frame rates, and thus open the path toward a much
broader application of 3D vision systems.
AseriesofworkshopshavecloselyfollowedthedevelopmentsonToFimaging
over the years, like the Dynamic 3D Vision workshops in conjunction with the
annual conference of the German Association of Pattern Recognition (DAGM)
in 2007 and 2009, as well as the CVPR workshops Time-Flight Camera-Based
Computer Vision (TOF-CV) in 2008 and later years. The advent of the Kinect
depth sensor in 2010 as a versatile and inexpensive depth imaging device has
furtheropenedthefieldfordepth-basedalgorithms.Today,depthimagingwork-
shops can be found at every major computer vision conference.
As a consequence, a prominent selection of leading researchers in the field
of ToF and depth imaging decided to initiate a seminar at Schloss Dagstuhl
- Leibniz Center for Informatics, which was successfully held in October 2012.
All aspects of ToF depth imaging, from sensors and basic foundations, over
algorithmsforlow-levelprocessing,toimportantapplicationsthatexploitdepth
imaging,werediscussed.Theresultsofthis workshoparecompiledinthisstate-
of-the-art survey series on ToF imaging. Eleven chapters, with an average of
25 pages each and over 700 reference citations in total, convey an excellent
overviewover all aspects of ToF foundations, algorithms, and applications. The
survey is divided into three parts. “Part I: Foundations of Depth Imaging”,
discusses in four contributions the basic working principle, sensor calibration,
image enhancement, and the important issue of benchmarking ToF systems.
“Part II: Depth Data Processing and Fusion”, deals with advanced data fusion
combiningToF,colorandstereosensors,thehandlingofmirrorsurfaces,andthe
tracking of deformable scenes. In ”Part III: Human-Centered Depth Imaging”,
the importantapplicationareaof human—machineinterfaces using ToFimages
is surveyed. Human motion analysis from depth images, gesture interfaces, as
wellasfull-bodymotioncaptureisanalyzed.Thissectionconcludeswithasurvey
on range imaging in health care.
As follow-up on recent activities in the field, a workshop on Imaging New
Modalities was held at the German Conference on Pattern Recognition, GCPR
2013, in Saarbru¨cken, Germany. Part IV of this volume comprises the proceed-
ings of this workshop. A state-of-the-art report on the Kinect sensor and its
applications is followedby two reports on local andglobal ToF motioncompen-
sation, picking up a very current aspect closely related to Part I of the survey,
and a novel depth capture system using a plenoptic multi-lens multi-focus cam-
era sensor.
VI Preface
We wouldlike to thank all the people who contributedto the workshopsand
the proceedings at hand. Special thanks go to the authors, the workshop co-
organizers, the sponsors, the supporting organizations, and, last but not least,
to the reviewers and members of the ProgramCommittee.
August 2013 Marcin Grzegorzek
Christian Theobalt
Reinhard Koch
Andreas Kolb
Organization
Organizers of the Dagstuhl Seminar on Time-of-Flight
Imaging: Algorithms, Sensors, and Applications
Christian Theobalt Max Planck Institute, Saarbru¨cken, Germany
James Davis University of California, Santa Cruz, USA
Bernd Ja¨hne University of Heidelberg, Germany
Andreas Kolb University of Siegen, Germany
Ramesh Raskar Media Lab, Massachusetts Institute of
Technology, USA
Reviewers
Damien Lefloch University of Siegen, Germany
Frank Lenzen University of Heidelberg, Germany
Julien Thollot SoftKinetic, Brussels, Belgium
Rahul Nair University of Heidelberg, Germany
Ilya Reshetouski Inria Bordeaux Sud-Ouest, Bordeaux, France
Andreas Jordt University of Kiel, Germany
Mao Ye University of Kentucky, Lexington, USA
Thomas Helten Max Planck Institute, Saarbru¨cken, Germany
Foti Coleca University of Lu¨beck, Germany
Sebastian Bauer University of Erlangen-Nuremberg, Germany
Organizers of the Workshop on Imaging New Modalities
Reinhard Koch University of Kiel, Germany
Andreas Kolb University of Siegen, Germany
Ivo Ihrke Max Planck Institute, Saarbru¨cken, Germany
Elli Angelopoulou University of Erlangen-Nuremberg, Germany
Program Committee
Amit Agraval MERL, USA
Oliver Bimber University of Linz, Austria
Oliver Cossairt North Western University, USA
Joachim Denzler University of Jena, Germany
Jan-Michael Frahm University of North Carolina, USA
Bastian Goldluecke University of Heidelberg, Germany
Fran¸cois Goudail Institute d’Optique, France
Oliver Grau Intel Visual Computing Institute, Germany
VIII Organization
Jinwei Gu University of Rochester, USA
Wolfgang Heidrich University of British Columbia, Canada
Sharham Izadi Microsoft, Cambridge UK
Bernd Ja¨hne University of Heidelberg, Germany
Chandra Kambhamettu University of Delaware, USA
Andreas Koschan University of Tennessee, Knoxville, USA
Marcus Magnor TU Braunschweig, Germany
Thomas Moeslund University of Aalborg, Denmark
Lars Olmor Carl Zeiss AG, Germany
Antonio Robles-Kelly Australian National University, Australia
Jannick Rolland University of Rochester, USA
Yoav Schechner Technion, Israel
Sabine Su¨sstrunk EPFL, Switzerland
Kevin Thompson Synopsys Inc., USA
Kiran Varanasi MPI, Germany
Peter Yuen Cranfield University, UK
Sponsoring Institutions
pmd technologies GmbH, Siegen, Germany
raytrix GmbH, Kiel, Germany
DFG Research Training Group 1564, University of Siegen, Germany
Deutsche AG Mustererkennung e.V.
Table of Contents
Part I: Foundations of Depth Imaging
Technical Foundation and Calibration Methods for Time-of-Flight
Cameras ........................................................ 3
Damien Lefloch, Rahul Nair, Frank Lenzen, Henrik Scha¨fer,
Lee Streeter, Michael J. Cree, Reinhard Koch, and Andreas Kolb
Denoising Strategies for Time-of-Flight Data ........................ 25
Frank Lenzen, Kwang In Kim, Henrik Sch¨afer, Rahul Nair,
Stephan Meister, Florian Becker, Christoph S. Garbe, and
Christian Theobalt
Stabilization of 3D Position Measurement ........................... 46
Julien Thollot, Xavier Baele, and Ilse Ravyse
Ground Truth for Evaluating Time of Flight Imaging................. 52
Rahul Nair, Stephan Meister, Martin Lambers, Michael Balda,
Hannes Hoffmann, Andreas Kolb, Daniel Kondermann, and
Bernd Ja¨hne
Part II: Depth Data Processing and Fusion
Mirrors in Computer Graphics, Computer Vision and Time-of-Flight
Imaging ........................................................ 77
Ilya Reshetouski and Ivo Ihrke
A Survey on Time-of-Flight Stereo Fusion........................... 105
Rahul Nair, Kai Ruhl, Frank Lenzen, Stephan Meister,
Henrik Scha¨fer, Christoph S. Garbe, Martin Eisemann,
Marcus Magnor, and Daniel Kondermann
Reconstruction of Deformation from Depth and Color Video with
Explicit Noise Models ............................................ 128
Andreas Jordt and Reinhard Koch
X Table of Contents
Part III: Human-Centered Depth Imaging
A Survey on Human Motion Analysis from Depth Data............... 149
Mao Ye, Qing Zhang, Liang Wang, Jiejie Zhu, Ruigang Yang, and
Juergen Gall
Full-Body Human Motion Capture from Monocular Depth Images ..... 188
Thomas Helten, Andreas Baak, Meinard Mu¨ller, and
Christian Theobalt
Gesture Interfaces with Depth Sensors.............................. 207
Foti Coleca, Thomas Martinetz, and Erhardt Barth
Real-Time Range Imaging in Health Care: A Survey.................. 228
Sebastian Bauer, Alexander Seitel, Hannes Hoffmann, Tobias Blum,
Jakob Wasza, Michael Balda, Hans-Peter Meinzer, Nassir Navab,
Joachim Hornegger, and Lena Maier-Hein
Part IV: Proceedings of the Workshop on Imaging
New Modalities
A State of the Art Report on Kinect Sensor Setups in Computer
Vision .......................................................... 257
Kai Berger, Stephan Meister, Rahul Nair, and Daniel Kondermann
Real-Time Motion Artifact Compensation for PMD-ToF Images ....... 273
Thomas Hoegg, Damien Lefloch, and Andreas Kolb
Real-Time Image Stabilization for ToF Cameras on Mobile Platforms... 289
Benjamin Langmann, Klaus Hartmann, and Otmar Loffeld
On the Calibration of Focused Plenoptic Cameras.................... 302
Ole Johannsen, Christian Heinze, Bastian Goldluecke, and
Christian Perwaß
Author Index.................................................. 319