Synthetic Actors: in Computer-Generated 3D Films

131 Pages·1990·5.13 MB·English

by  Prof. Dr. Nadia Magnenat Thalmann

Computer Science Workbench
Editor: Tosiyasu L. Kunii
Computer Science Workbench
N. Magnenat Thalmann, D. Thalmann: Image Synthesis. Theory and
Practice. XV, 400 pp., 223 figs., including 80 in color. 1987
B. A. Barsky: Computer Graphics and Geometric Modeling Using
Beta-splines. IX, 156 pp., 85 figs., including 31 in color. 1987
H. Kitagawa, T. L. Kunii: The Unnormalized Relational Data Model.
For Office Form Processor Design. XIII, 164 pp., 78 figs. 1989
N. Magnenat Thalmann, D. Thalmann: Computer Animation. Theory
and Practice. Second Revised Edition. XIII, 245 pp., 156 figs., including
73 in color. 1990
N. Magnenat Thalmann, D. Thalmann: Synthetic Actors in Computer
Generated 3D Films. X, 129 pp., 133 figs., including 83 in color. 1990
Nadia Magnenat Thalmann
Daniel Thalmann
Synthetic Actors
in Computer-Generated 3D Films
With 133 Figures, Including 83 in Color
Springer-Verlag
Berlin Heidelberg New York London Paris
Tokyo Hong Kong Barcelona
Prof. Dr. Nadia Magnenat Thalmann
MIRALab, CUI
University of Geneva, 12 rue du Lac
CH-1207 Geneva, Switzerland
Prof. Dr. Daniel Thalmann
Computer Graphics Laboratory
Swiss Federal Institute of Technology
CH -1015 Lausanne, Switzerland
Series editor:
Dr. Tosiyasu L. Kunii
Professor and Director
Kunii Laboratory of Computer Science
Department ofInformation Science
Faculty of Science, The University of Tokyo
7-3-1 Hongo, Bunkyo-ku
Tokyo 113, Japan
CR Subject Classification (1987): 1.3
Cover picture: Frame from the film "Flashback" (1990) directed by Nadia Magnenat
Thalmann and Daniel Thalmann. The cloth animation is based on an original algo
rithm of self-collision detection. Software implementation: Benoit Lafleur and Domi
nique Boisvert. Decor design: Arghyro Paouri.
ISBN-13: 978-3-642-75455-5 e-ISBN-13: 978-3-642-75453-1
DOl: 10.1007/978-3-642-75453-1
Library of Congress Cataloging-in-Publication Data
Magnenat Thalmann, Nadia, 1946-
Synthetic actors in computer-generated 3D films 1 Nadia Magnenat
Thalmann, Daniel Thalmann. p. cm. -
(Computer science workbench)
Includes bibliographical references and index.
ISBN-13: 978-3-642-75455-5
1. Computer animation. 2. 3-D films. I. Thalmann, Daniel.
II. Title. III. Series. TR897.5.M333 1990 778.5'3'028566 - dc20 90-9923 CIP
This work is subject to copyright. All rights are reserved, whether the whole or part of the material
is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in other ways, and storage in data banks. Duplica
tion of this publication or parts thereof is only permitted under the provisions of the German
Copyright Law of September 9, 1965, in its current version, and a copyright fee must always be
paid. Violations fall under the prosecution act of the German Copyright Law.
© Springer-Verlag Berlin Heidelberg 1990
Softcover reprint of the hardcover 1s t edition 1990
The use of registered names, trademarks, etc. in this publication does not imply, even in the
absence of a specific statement, that such names are exempt from the relevant protective laws and
regulations and therefore free for general use.
2145/3140 -543210 - Printed on acid-free paper
Preface
This book presents the making of computer-generated films using three-dimensional
a
synthetic actors. It is based mainly on the production of the film Rendez-vous
Montreal, an animated film that uses advanced computer techniques to achieve such
effects as reincarnating film stars Humphrey Bogart and Marilyn Monroe. The main
a
purpose of Rendez-vous Montreal is to show that true synthetic actors can be
created. This fllm represents a technological breakthrough which opens up new vistas
in motion pictures, television, and advertising. With this technique, it will now be
possible to produce short fllms or motion pictures featuring any celebrity in any
situation.
The book explains in detail how such a film can be produced using numerous
drawings and color pictures. In particular, the following topics are presented:
- Preparation of object construction: documentation search, and plaster models
preparation
- Object construction: digitizing and modeling
- Body animation
- Hand animation and object grasping
- Facial animation: expressions, phonemes and emotions
- Choreography: decors, actors, cameras, and lights
- Realism: colors, lights, shading, shadows, and textures
- Image recording: special effects
Finally, an extensive appendix describes in detail the HUMAN FACTORY
system.
The authors are indebted to the designers who have worked on the production of
a
the films Eglantine, Rendez-vous Montreal and Galaxy Sweetheart: Ross Racine,
Alain Caron, Dominique Guy, and Marie-Andree Allaire. They also express their
gratitude to all the computer scientists who participated in the development of the
HUMAN FACTORY system, in particular Mario de Angelis, Richard Laperriere, Eric
Primeau, Denis Rambaud, and Minh Hong Tong.
Nadia Magnenat Thalmann
Daniel Thalmann
Table of Contents
1 . How to Make a Film with Synthetic Actors
1.1 Computer Animation and Synthetic Actors ................................... .
1.1.1 The Role of Synthetic Actors .................................................. . 1
1.1.2 Scenario and Storyboard ....................................................... . 1
1.1.3 Decors and Actors ............................................................... . 2
1.1.4 Motion ............................................................................ . 2
1.1.5 Types of Animation ............................................................. . 2
1.1.6 Cameras and Lights ............................................................. . 3
1.1.7 Choreography and Recording .................................................. . 4
a
1.2 The Film Rendez-vous Montreal ............................................. . 4
1.2.1 TheFilm .......................................................................... . 4
1.2.2 The Scenario ..................................................................... . 4
1.2.3 The Storyboard .................................................................. . 5
a
1.2.4 The History of Rendez-vous Montreal ...................................... . 7
2. Three-Dimensional Construction of Synthetic Actors
2.1 Planning and Preparation of Models for 3D Reconstruction ............... . 9
2.1.1 Planning .......................................................................... . 9
2.1.2 Model and Documentation Search ............................................. . 9
2.1.3 Guidelines for Choosing or Constructing Objects to be Digitized ......... . 11
2.2 How to Enter the Shapes ....................................................... . 12
2.2.1 Input Scanning, Digitizing, and 3D Reconstruction ........................ . 12
2.2.2 3D Reconstruction from 2D Plans or Photographs ......................... . 12
2.2.3 The Steps of Construction by Digitizing ..................................... . 13
2.2.4 Cross-section Methods ......................................................... . 13
2.2.5 Plaster Models Preparation .................................................... . 13
2.2.6 Planar Polygonal Facet Representation ...................................... . 14
2.2.7 Choice of Facets and Vertices ................................................. . 16
2.3 Photographs ..................................................................... . 17
2.3.1 Principles ........................................................................ . 17
2.3.2 Turntable ......................................................................... . 18
2.3.3 Camera ........................................................................... . 18
2.3.4 Choice of Photo Angles ........................................................ . 19
2.3.5 Marking Vertices, Facets, and Edges on Photos ............................ . 19
2.3.6 Step-by-Step 3D Digitizing Procedure ....................................... . 19
2.4 Object Composition ............................................................ . 21
2.4.1 The Composition Operation ................................................... . 21
2.4.2 An Example: The Complete Composition of Marilyn Monroe ............ . 22
VITI Table of Contents
3. Human Prototyping
3.1 Local Transfonnations of Existing Synthetic Actors 25
3.1.1 Introduction ...................................................................... . 25
3.1.2 Local Transfonnations .......................................................... . 25
3.2 Shape Interpolation Between Human Faces .................................. . 26
3.2.1 Introduction ...................................................................... . 26
3.2.2 Grid Generation ................................................................. . 27
3.2.3 Inbetween Calculation .......................................................... . 28
4. Body Animation of Synthetic Actors
4.1 Principles of Body Animation ................................................. . 31
4.2 Skeleton Definition and Positioning .......................................... . 32
4.2.1 Skeleton Definition .............................................................. . 32
4.2.2 Fixed Points ..................................................................... . 33
4.2.3 Actor Sections ................................................................... . 33
4.2.4 Procedure for Skeleton Positioning ........................................... . 34
4.3 Skeleton Animation ............................................................. . 36
4.3.1 Key Positions .................................................................... . 36
4.3.2 Spline Interpolation ............................................................. . 36
4.3.3 Definition and Manipulation of Key Values .................................. . 37
4.3.4 An Example ...................................................................... . 41
5 . Hand Animation, Object Grasping, and Foot Animation
5.1 Hand Skeleton and Surface ..................................................... 45
5.1.1 Hand Skeleton.................................................................... 45
5.1.2 Hand Surface..................................................................... 46
5.2 Hand Animation .... ... .... ... .... ............................ .... ...... .......... 46
5.3 Object Grasping and ManipUlating ............................................. 49
5.3.1 The Action of Grasping an Object.............................................. 49
5.3.2 Determination of Joint Angles for Object Grasping .... ...................... 49
5.3.3 Object Association ...... ...................... ................ .................... 50
5.3.4 A Case Study from Rendez-vous a Montreal ................................. 50
5.3.5 Flower Grasping........ ............................ ............................. 51
5.4 Foot Animation................................................................... 54
6. Facial Animation of Synthetic Actors
6.1 A Three-Level Approach to Facial Animation.... ............................. 55
6.1.1 How to Animate the Face of a Synthetic Actor................................ 55
6.1.2 Parameter, Expression, and Script Levels..................................... 55
6.2 The Facial Parameters. . . .. . .. . . .. . . . . . .. . .. . . .. . .. . . . . . .. . . .. . . . . . . . . . . . . .. . . . . . . 57
6.2.1 The Structure of a Human Face............................. .... ... ... ...... .... 57
6.2.2 A Human Face Model for Animation.. . .. . . .. . ... . ... .. . . .. . .. . . .. . . . . . . . . . .. . . 57
Table of Contents IX
6.2.3 Constraints on Faces of a Synthetic Actor..................................... 57
6.2.4 The Role of Facial Parameters................................ ... . ... ... ... ..... . 58
6.2.5 Installation of the Face of a New Synthetic Actor ............................ 59
6.2.6 Lip Facial Parameters ............................................................ 59
6.3 The Expressions: Phonemes and Emotions ......... ~...................... ... 59
6.3.1 The Expression Level......................................................... ... 59
6.3.2 Phonemes...................................................................... ... 63
6.3.3 Emotions....................... ...................... ....... ................... .. . 64
6.4 The Script Level .................................................................. 64
6.4.1 Scripts for Facial Animation .................................................... 64
6.4.2 Key Values and Inbetweening ............................................... ... 64
6.4.3 The Role of Multiple Tracks .................................................... 65
6.4.4 Speech Timing .................................................................. , . 65
6.4.5 The Use of Splines.............................................. .................. 65
a
6.5 A Script Example from the Film Rendez-vous Montreal .................. 66
6.6 Interpolating Facial Parameters and Expressions Between
Two Different Human Faces ................................................... . 66
6.6.1 Shape Level ...................................................................... . 66
6.6.2 Parameter Level ................................................................. . 66
6.6.3 Expression Level ................................................................ . 69
6.6.4 Script Level ...................................................................... . 69
7. Color, Reflectance, Transparency, and Texture
7.1 Colors ............................................................................. . 71
7.1.1 Drawing Style ........................................................................ . 71
7.1.2 How to Color a Synthetic Actor ............................................... . 71
7.1.3 The RGB System ..................................................................... . 71
7.1.4 The HLS System ................................................................ . 72
7.2 Reflectance ....................................................................... . 74
7.2.1 Definition ......................................................................... . 74
7.2.2 A User Interface.for Reflectance .............................................. . 74
7.2.3 Reflectance and Highlight for Synthetic Actors ............................. . 74
7.3 Transparency and Textures .......................................................... . 75
7.3.1 Transparency '" .................................................................... . 75
7.3.2 Textures .......................................................................... . 77
8. Cameras, Lights, and Shadows
8.1 Cameras .......................................................................... . 79
8.1.1 The Animation Environment ................................................... . 79
8.1.2 Synthetic Cameras ................................................................... . 79
8.1.3 Eye, Interest Point, Viewing Angle, and Viewport ......................... . 79
8.2 Lights ............................................................................. . 80
8.2.1 Introduction to lliumination .................................................... . 80
8.2.2 Surface Shading ................................................................. . 81
8.2.3 How to Select Light Parameters ............................................... . 83
8.3 Shadows .......................................................................... . 84
X Table of Contents
9. Choreography
9.1 Decors and Actors ............................................................... . 87
9.1.1 Scene and Script. ................................................................ . 87
9.1.2 Decors ............................................................................ . 87
9.1.3 Non-Human Actors ............................................................. . 87
9.2 Cameras and Lights ............................................................. . 89
9.2.1 The Role of Synthetic Cameras in Computer-Animated Films .......... . 89
9.2.2 Characteristics of an Animated Camera ....................................... . 89
9.2.3 Wipe Effects and Multiple Synthetic Movie Camera Effects ............... . 90
9.2.4 Animation of Lights and Spots ................................................ . 92
9.3 Camera and Light Paths ........................................................ . 92
9.3.1 Introduction ...................................................................... . 92
9.3.2 Paths Defined by a Law ........................................................ . 92
9.3.3 Camera Assigned to an Actor .................................................. . 93
9.3.4 Graphical Input of Path ......................................................... . 93
9.3.5 How to Design a Camera Spline .............................................. . 93
9.3.6 Spline for the Camera Motion in the Film Eglantine ........................ . 94
Appendix: The HUMAN FACTORY Software
A.l The Architecture of the HUMAN FACfORY System....................... 99
A.I.1 Introduction..... .... .. .. ... ..... . .. ... . . .. . . .. ... . . .. . . ... .. . . .. . . .. . ... . .. . . .. .. . 99
A.l.2 Interfaces for the HUMAN FACfORY Programs ................ ........... 100
A.I. 3 The Hierarchical Structure ...................................................... 101
A.2 The DIGffiD Program.......................................................... 101
A.3 The SABRINA System.......................................................... 102
A.4 The BODY_PARTS System ................................................... . 105
A.4.1 Active Figure and Cameras .................................................... . 107
A.4.2 Current Vectors and Sets ....................................................... . 107
A.5 The BODY_MOVING System ................................................ . 107
A.5.1 An Overview of the BODY_MOVING System ............................. . 107
A.5.2 Definition of Key Values ....................................................... . 109
A.5.3 Manipulation of Key Values ................................................... . 111
A.5.4 A User Interface for Object Grasping ......................................... . 113
A.6 The FACE_MOVING System ................................................. . 114
A.6.1 An Overview of the FACE_MOVING System .............................. . 114
A.6.2 Lip Facial Parameters .......................................................... . 115
A.7 The MIRANIM System ........................................................ . 117
A.8 The DISPLAY Program ........................................................ . 120
Bibliography....... ... .... .... ... ........... .... ............................... ..... ..... 123
Subject Index.. .... ....... .............. .... ........................................ ..... 127
1. How to Make a Film with Synthetic Actors
1.1 Computer Animation and Synthetic Actors
1.1.1 The Role of Synthetic Actors
It is difficult to imagine a live action movie without humans. However, for a decade,
three-dimensional computer-animated films have been appearing and human characters
were not present. These films seemed to present worlds without real life, just flying
logos or balls, sometimes robots, puppets or cartoon characters.
Synthetic actors bring to these computer-generated films a new dimension. From an
artistic point of view, they allow the creation of new stories with existing people such
as cinema stars, or historical or political personalities. Fictitious persons may live
alongside existing or dead ones. In the near future, it will be possible to recreate any
film with any existing artiste, even from different times. For example, nothing may
prevent us recreating West Side Story with Greta Garbo and Michael Jackson.
From a scientific point of view, synthetic actors may be used, for example, in
medicine, to study the mechanisms of organs, in psychology, to recreate the behavior
of people, in engineering, to study people using new machines, and in any visual
simulation.
1.1.2 Scenario and Storyboard
The first step in the creation of a computer-animated film is the selection of the story.
This story could be an existing one or one specially designed for the film. Two extreme
approaches are possible in the design of a scenario for a computer-animated film:
1. The scenario is created without thinking about how to animate by computer.
Computer programs are then developed or modified to produce the corresponding
film.
2. The scenario is created based on the capabilities of the computer and the software
used. .
The first approach does not limit the artist at all; however, it may lead to unsolvable
difficulties and even force those involved to reconsider the initial scenario. The second
approach is much more secure; however, it is more likely to lead to a purely technical
product, with limited artistic merits.