Multimedia Fundamentals, Volume 1: Media Coding and Content Processing: Computing, Communications and Applications (IMSC Press Multimedia Series)

291 Pages·2002·1.752 MB·English

by  Ralf Steinmetz, Klara Nahrstedt

MULTIMEDIA
FUNDAMENTALS
VOLUME 1:
Media Coding and Content Processing
IMSC Press Multimedia Series
ANDREW TESCHER, Series Editor, Compression Science Corporation
Advisory Editors
LEONARDO CHIARIGLIONE, CSELT
TARIQ S. DURRANI, University of Strathclyde
JEFF GRALNICK, E-splosion Consulting, LLC
CHRYSOSTOMOS L. “MAX” NIKIAS,University of Southern California
ADAM C. POWELL III, The Freedom Forum
▲ Desktop Digital Video Production
Frederic Jones
▲ Touch in Virtual Environments:
Haptics and the Design of Interactive Systems
Edited by Margaret L. McLaughlin,
João P. Hespanha, and Gaurav S. Sukhatme
▲ The MPEG-4 Book
Edited by Fernando M. B. Pereira and Touradj Ebrahimi
▲ Multimedia Fundamentals, Volume 1:
Media Coding and Content Processing
Ralf Steinmetz and Klara Nahrstedt
▲ Intelligent Systems for Video Analysis and Access Over
the Internet
Wensheng Zhou and C. C. Jay Kuo
The Integrated Media Systems Center (IMSC), a National Science
Foundation Engineering Research Center in the University of
Southern California’s School of Engineering, is a preeminent
multimedia and Internet research center. IMSC seeks to develop
integrated media systems that dramatically transform the way
we work, communicate, learn, teach, and entertain. In an
integrated media system, advanced media technologies
combine, deliver, and transform information in the form of
images, video, audio, animation, graphics, text, and haptics
(touch-related technologies). IMSC Press, in partnership with
Prentice Hall, publishes cutting-edge research on multimedia
and Internet topics. IMSC Press is part of IMSC’s educational
outreach program.
M
ULTIMEDIA
F
UNDAMENTALS
V 1:
OLUME
Media Coding and Content Processing
Ralf Steinmetz
Klara Nahrstedt
PRENTICE HALL PTR
UPPER SADDLE RIVER, NJ 07458
WWW.PHPTR.COM
Library of Congress Cataloging-in-Publication Data
Steinmetz, Ralf.
Multimedia fundamentals / Ralf Steinmetz, Klara Nahrstedt
p. cm.
Includes bibliographical references and index.
Contents: v. 1. Media coding and content processing.
ISBN 0-13-031399-8
1. Multimedia systems. I. Nahrstedt, Klara II. Title.
QA76.575 .S74 2002
006.7—dc21 2001056583
CIP

Editorial/Production Supervision: Nick Radhuber
Acquisitions Editor: Bernard Goodwin
Editorial Assistant: Michelle Vincente
Marketing Manager: Dan DePasquale
Manufacturing Buyer: Alexis Heydt-Long
Cover Design: John Christiana
Cover Design Director: Jerry Votta
© 2002 by Prentice Hall PTR
Prentice-Hall, Inc.
Upper Saddle River, NJ 07458
Prentice Hall books are widely used by corporations and government agencies for training,
marketing, and resale.
The publisher offers discounts on this book when ordered in bulk quantities. For more information,
contact Corporate Sales Department, phone: [phone]; fax: [phone]; email: [email protected]
Or write: Corporate Sales Department, Prentice Hall PTR, One Lake Street, Upper Saddle River, NJ 07458.
Product and company names mentioned herein are the trademarks or registered trademarks
of their respective owners.
All rights reserved. No part of this book may be
reproduced, in any form or by any means, without
permission in writing from the publisher.
Printed in the United States of America
10 9 8 7 6 5 4 3 2 1
ISBN 0-13-031399-8
Pearson Education LTD.
Pearson Education Australia PTY, Limited
Pearson Education Singapore, Pte. Ltd
Pearson Education North Asia Ltd
Pearson Education Canada, Ltd.
Pearson Educación de Mexico, S.A. de C.V.
Pearson Education—Japan
Pearson Education Malaysia, Pte. Ltd
Pearson Education, Upper Saddle River, New Jersey
To our families:
Jane, Catherine, and Rachel Horspool,
and Marie Clare Gorman
for their patience and support
Contents
Preface xv
1 Introduction 1
1.1 Interdisciplinary Aspects of Multimedia..........................................................2
1.2 Contents of This Book......................................................................................3
1.3 Organization of This Book...............................................................................4
1.3.1 Media Characteristics and Coding........................................................5
1.3.2 Media Compression...............................................................................5
1.3.3 Optical Storage......................................................................................6
1.3.4 Content Processing................................................................................6
1.4 Further Reading About Multimedia..................................................................6
2 Media and Data Streams 7
2.1 The Term “Multimedia”...................................................................................7
2.2 The Term “Media”............................................................................................7
2.2.1 Perception Media...................................................................................8
2.2.2 Representation Media............................................................................8
2.2.3 Presentation Media................................................................................8
2.2.4 Storage Media.......................................................................................9
2.2.5 Transmission Media..............................................................................9
vii
viii Contents
2.2.6 Information Exchange Media................................................................9
2.2.7 Presentation Spaces and Presentation Values.......................................9
2.2.8 Presentation Dimensions.....................................................................10
2.3 Key Properties of a Multimedia System.........................................................11
2.3.1 Discrete and Continuous Media..........................................................12
2.3.2 Independent Media..............................................................................12
2.3.3 Computer-Controlled Systems............................................................12
2.3.4 Integration...........................................................................................12
2.3.5 Summary.............................................................................................13
2.4 Characterizing Data Streams..........................................................................13
2.4.1 Asynchronous Transmission Mode.....................................................13
2.4.2 Synchronous Transmission Mode.......................................................14
2.4.3 Isochronous Transmission Mode........................................................14
2.5 Characterizing Continuous Media Data Streams............................................15
2.5.1 Strongly and Weakly Periodic Data Streams......................................15
2.5.2 Variation of the Data Volume of Consecutive Information Units......16
2.5.3 Interrelationship of Consecutive Packets............................................18
2.6 Information Units............................................................................................19
3 Audio Technology 21
3.1 What Is Sound?...............................................................................................21
3.1.1 Frequency............................................................................................22
3.1.2 Amplitude............................................................................................23
3.1.3 Sound Perception and Psychoacoustics...............................................23
3.2 Audio Representation on Computers..............................................................26
3.2.1 Sampling Rate.....................................................................................27
3.2.2 Quantization........................................................................................27
3.3 Three-Dimensional Sound Projection.............................................................28
3.3.1 Spatial Sound.......................................................................................28
3.3.2 Reflection Systems..............................................................................30
3.4 Music and the MIDI Standard........................................................................30
3.4.1 Introduction to MIDI...........................................................................31
3.4.2 MIDI Devices......................................................................................31
3.4.3 The MIDI and SMPTE Timing Standards..........................................32
3.5 Speech Signals................................................................................................32
Contents ix
3.5.1 Human Speech.....................................................................................32
3.5.2 Speech Synthesis.................................................................................33
3.6 Speech Output.................................................................................................33
3.6.1 Reproducible Speech Playout..............................................................34
3.6.2 Sound Concatenation in the Time Range............................................34
3.6.3 Sound Concatenation in the Frequency Range....................................36
3.6.4 Speech Synthesis.................................................................................36
3.7 Speech Input...................................................................................................37
3.7.1 Speech Recognition.............................................................................38
3.8 Speech Transmission......................................................................................40
3.8.1 Pulse Code Modulation.......................................................................40
3.8.2 Source Encoding.................................................................................41
3.8.3 Recognition-Synthesis Methods..........................................................42
3.8.4 Achievable Quality..............................................................................43
4 Graphics and Images 45
4.1 Introduction.....................................................................................................45
4.2 Capturing Graphics and Images......................................................................46
4.2.1 Capturing Real-World Images.............................................................46
4.2.2 Image Formats.....................................................................................48
4.2.3 Creating Graphics................................................................................53
4.2.4 Storing Graphics..................................................................................54
4.3 Computer-Assisted Graphics and Image Processing......................................55
4.3.1 Image Analysis....................................................................................56
4.3.2 Image Synthesis...................................................................................71
4.4 Reconstructing Images....................................................................................72
4.4.1 The Radon Transform.........................................................................73
4.4.2 Stereoscopy.........................................................................................74
4.5 Graphics and Image Output Options..............................................................75
4.5.1 Dithering..............................................................................................76
4.6 Summary and Outlook....................................................................................77
5 Video Technology 79
5.1 Basics..............................................................................................................79