Table Of ContentAMSTERDAM· BOSTO:-.l • HEIDELBERG · LONDON
NEW YORK · OXFORD · PARIS, SAN DIEGO
SAN FRANCISCO, SINGAPORE· SYDNEY · TOKYO
ELSEVIER Morgan K.mfm.lnn IS .,n l1TlJlrHll of Lis('\'It'r
Acquisitions Editor: TIffany Gasbarrini
Publishing Services Manager: Gcorge Morrison
Project Manager: M6nica Gonzalez de Mcndoz.l
Assistant Editor. Matt Cater
COl'er Design: Joonnc l31ank
Cover Imagt'S: Sec page xii for full credits
Morgan Kaufmann Publishers is an imprint of Elsevier
30 Corporate Dri\'e, Suite 400, Burlington, MA 01803, USA
This book is printed on acid-ffL'C paper.
© 2008 Elsevier, Inc. All rights rt'SCn'ed.
Designations used by companies to distinguish their products arc often claimed ;,s
trademarks or registerL'Ci tr"dcm;,rks. In all instances in which Morgan Kaufm,mn
Publishers is ;,w;,re of;, cI;'im, the product n"mes ;,ppear in initi;,l capital or all
capitallellers. Readers, however, should contaclthe appropriate companies for more
complete informiltion regilrding tr;,dem;,rks and registr;,tion.
No p"rt of Ihis public;,tion may be reproduced, stored in a retriev,,1 system, or transmitted in
any form or by any means--ck'Clronic, me<:hanical, photocopying, sc.lIlning, or
otherwise-without prior written permission of the publisher.
Permissions may be sought dilX.'Ctly from Elsevier's Science & TI..'Chnology I~ights
Department in Oxford, UK: phone: (+4-1) 18658-13830, fax: (+-14) 1865853333,
E-mail: [email protected]. You may also complete your request online via
the Elsevier homepage (http://dscvier.rom), by selecting "Support & Contact"
then "Copyright and Permission" and then "Obtaining Permissions."
library of Congress Cataloging-in-Publication Doltol
Brinkmnnn, I~on.
The art and science of digital composiling / Ron Brinkmmm.
p.cm.
Includes bibliographic;,1 references and index.
ISBN-13: 978-0-12-370638-6 (pbk.: alk. paper) 1. Computer graphics. 2. Imng..: processing-Digital
techniques. I. TItle.
T385.13752008
718.5 '235028.5----dc22
2007053015
ISBN: 978-0-12-370638-6
For information on all Morgan Kaufmann publications,
"isit our \Ncb site at www.mkp.comorww\\. .b ooks.elscl.ier.com
08 09 10 11 12 13 1098765-132
Printed in China
Working together to grow
libraries in developing countries
I I
w. .... w.dscvicr.com www.boobid.org www.sabrc.org
III I rodllcl iOIl 10 I he Secolld Ed i I iOIl .................................................................................. xiii
Chapter 1 IlIlrodllcliolllo Digilnl COll/posilillg ............................................................ 1
Definition ....................................................................................................................................2
I-listorical Perspective ............................................................................................................... 3
Terlninology ............................................................................................................................... 7
Organization of the Book ....................................................................................................... 11
Chapter 2 Lenmillg 10 Sec ........................................................................................... ·15
Judging Color, Brightness, and Contrast ............................................................................. 16
Light and Shado\v ...................................................................................................................1 9
The Canleril ..............................................................................................................................2 7
1~t'::()/lIlitlll Lill,ils ............................................................................................................... 27
Foclf.--lIlld Deplh (~r Field ................................................................................................... 30
Lt'J/~ F/l1rt'~ t1J/d OIlier Lt'II:' /\l"lijt1(/:' ............................................................................... .32
MolicJ/1 1311/,. ....................................................................................................................... 39
Depth, Distill1ce, ilnd Perspective ........................................................................................ .-1-1
Pt'r~Jlc'Cl ii'I' t11l11 Ih c' CIIIIII'/"ll .............................................................................................. ..J2
01'1'111 CIfI'S .........................................................................................................................- 15
Visuill "Correctness" ...............................................................................................................5 1
Chapter 3 TI,e Digilnl !!epreselliniioll "!Visllnl lll!onllnlioll ................................... .53
Ilnilge Ceneriltion .................................................................................................................... 53
Pixels, Components, ilnd Chilnnels ...................................................................................... 55
Spatial Resolution ................................................................................................................... 59
Bit Depth ................................................................................................................................... 60
Nonn<1lized V<1lues ..................................................................................................................6 3
Beyond I3lilck and White ........................................................................................................M
Flo<1lin~-point ilnd High Dynilmic I~angc Im<1gl'ry (J-IDRI) ............................................. 7"\
Thl' \-ISV Color Represl'nliltion .............................................................................................7 -1
The YUV Color Rcprcscntiltion .............................................................................................7 7
Inlagc Input Devices ............................................................................................................... 80
Digit<1llnlilgc File Fornlilts ....................................................................................................8 1
File FOrl11ilt FC<1turcs ...............................................................................................................8 2
VI1,.iI1Vlt' Bil DI'Jllh:' ............................................................................................................ 82
Dijfi'/"('I// Spl1lil1J J~t':'tl/lltinlf:' ............................................................................................. 82
COIIlI'/"(':':'iol, ....................................................................................................................... 82
((lIIllIlell! Ilfjorllll1titll/ ill a I-leader .................................................................................... B2
Additiollalllllage C/ulI/lld:, ............................................................................................... 82
COlltellts
VI
Vendor-specific File Format Implementations .................................................................... 83
COin pression ............................................................................................................................ 83
RIiI/-felIgtlt ElIcodillg ........................................................................................................ 83
Lossy Conipressioll .............................................................................................................8 6
CllrDllta Subsailiplillg ......................................................................................................... 87
Image Sequellce COlllpressioll ............................................................................................9 0
Choosing a File Format .......................................................................................................... 90
Nonlinear Color Encoding ..................................................................................................... 91
Chapter 4 Basic Image Manipulation ......................................................................... 93
Tenninology ............................................................................................................................. 94
Color Manipulations ............................................................................................................... 97
I~GB Mliitiply .................................................................................................................... 97
AM.................................................... ..%
Ganlllia Correctioll ...........................................................................................................1 00
Illvert ................................................................................................................................1 02
COlltrast .......................................................................................... ................................ 103
Chal/llel Szvappillg .................................. ........................................................................ 106
HSV Malliplf/atiolls ........................................................................................................1 07
Look-up Table Malliplllal iOlls .......................................................................................... 108
Expressioll Langllage ....................................................................................................... 11 0
Spatial Filters ......................................................................................................................... 113
COllvolves ......................................................................................................................... 113
B/llfrillg ........................................................................................................................... 116
Sharpell ............................................................................................................................ 119
Mediall Filter ................................................................................................................... 124
Geometric Transformations ................................................................................................. 125
Pall/li/lg ............................................................................................................................ 126
Rotatioll .................................. ................. ....................................................................... 129
Senle .................................................................................................................................1 30
3D Tralls!onlls ................................................................................................................ .133
Wnrpillg ........................................................................................................................... 135
Expressioll Language ....................................................................................................... 138
FilteriJJg Algorilltllls ........................................................................................................ 139
Motioll Blur ..................................................................................................................... 144
Chapter 5 Basic III/age COli/positing ......................................................................... 149
Multisource Operators ......................................................................................................... 149
Add .............................................................................................................. .149
SlIbtraet ............................................................................................................................ 151
M~....]~
The Matte Image .................................................................................................................... 154
The Integrated Matte Channel ............................................................................................ 157
Over ................................................................................................................................. 158
Additional Operators .......................................................................................................... 160
MII/lip/y ..................................................................................... ..................................... 162
COl/lei/Is VII
Sen'ell ............................................................................................................................... 162
Maximlllll nl/d MiJlil1llllll ................................................................................................ 166
III ...................................................................................................................................... 168
01/1. .................................................................................................................................. 169
A lop .................................................................................................................................. 170
Masks ...................................................................................................................................... 172
Compositing with Prcmultiplied Images .......................................................................... 174
Color Correclillg muf COII/Villillg Prcllllflliplierl J/IIn~es ................................................. J78
Llllllii/Osily nlld Ihe IlIInge- Mnlte Re/ntio/lSllip .............................................................. 181
Morphing ................................................................................................................................1 83
Chapter 6 Matte Creatioll Dlld MDllipl/latioll .......................................................... 189
Rotoscoping ........................................................................................................................... 192
Teclllliqllt·s ........................................................................................................................ 195
Molioll BIIII' ..................................................................................................................... 203
Lill,itnl iOIl5 ....................................................................................................................... 206
Procedural Matte Extraction ................................................................................................ 206
Kcyillg Based 0/1 Lllllliwlllce. ........................................................................................... 207
Keyillg Bnsed C"roll/illallCf ........................................................................................ 21 0
OJI
Tlte Color 0iffacl/cC Melltod ........................................................................................... 214
0iffacl/c£, Mal IiI/g ........................................................................................................... 218
Specinlized Kt'yilIS Softwnre ............................................................................................ 2·/9
Matte Manipulations ............................................................................................................ 222
Checkillg Ille Solirlity of YOllr Mnt Ie ................................................................................ 223
Garvage Mattes ................................................................................................................ 223
Erlge Mattes ..................................................................................................................... 227
COIIIVillillg Mnttes ...........................................................................................................2 28
11Ilage ProcessiJ/g 011 Mattes ............................................................................................ 228
Chapter 7 Tillie alld Telllporal MDllipl/latiolls .......................................................... 233
Apparent Motiol1 ................................................................................................................... 233
Tel1''Iporal Resolution ............................................................................................................. 234
Temporal Artifacts ................................................................................................................. 235
Changing the Lcngth or Timing of a Sequence ................................................................ 236
Keyframing ............................................................................................................................2 44
Chapter 8 11lIage Trackillg and Stabilization ............................................................ 249
Tracking an Element into a Plate ........................................................................................ 250
ChoosiJlg tf,e Fentllre to Track .......................................................................................... 252
Lilllitillg ti,e Searcll Arm ................................................................................................. 254
Hun1an Intervention .............................................................................................................2 55
UsiJlg Trnckillg Ct,rves MallllaJ/y .................................................................................. .256
Tracking Multiple Points ...................................................................................................... 257
Stabilizing a Plate .................................................................................................................. 258
Can1cra Tracking ...................................................................................................................2 62
VIII COlllellis
Chapter 9 IlIlelfnce IlIlerncliolls ............................................................................... 263
Workflow ................................................................................................................................ 264
The Evolution of Inter<lctivity ............................................................................................. 266
Methods of Representing the Composiling Process ........................................................ 268
Layer Lists. .......................................................................................................................2 69
Trees ................................................................................................................................. 270
COIl/pressed Bmllel,es ....................................................................................................... 273
Tilnelines ................................................................................................................................ 275
Curve Editors ......................................................................................................................... 277
Working \,vith Proxy lInages ................................................................................................ 281
Image Viewing and Analysis Tools .................................................................................... 282
IlIlnge Vieluers ..................................................................................................................2 83
Pixel or l~egioJ/nllllforlllalioJ/ Tools ................................................................................ 285
Histograllls ....................................................................................................................... 287
Chapter 10 Fillll Fonllnls: Medin, Resolulioll, nlld Aspecl Rnlios ........................... 291
Aspect Ratio ........................................................................................................................... 293
NOllsf/lfnre Pixels ............................................................................................................. 294
Decidillg 011 a l~eso{lIlioll Wllell YOII Have nil Aspect I\nlio ........................................... 296
Format Conversion Pipeiine ............................................................................................... .297
A Fonl/at COl/versiol/ Exalllple. ....................................................................................... 299
FillTl FOrnl<lts ..........................................................................................................................3 02
COIlIllIOII 35 111111 FOl'lIIats .................................................................................................3 03
16,1111/ FOl'll1nts ................................................................................................................ 305
Specialized FilII/ FOl'lllals .................................................................................................3 06
Video FOrnl<lts ........................................................................................................................ 308
Fields ............................................................................................................................... .308
Color I\I'SOllllioll .............................................................................................................. 311
Calli II In .............................................................................................................................3 12
COII/IIIOII Video FOl'/lInts. .................................................................................................. 3·{2
Other Forlnats ........................................................................................................................ 314
Working with Nonsqu<lre Pixels ........................................................................................ .314
Converting Clnd Combining Formats ................................................................................. 319
COJ/pa lillg iJetwt'eII Fillll mid Video ............................................................................... .323
Chapter 11 Qllnlily nlld Efficiellcy ............................................................................. 327
Quality .................................................................................................................................... 327
Efficiency ................................................................................................................................ 328
Production Methodologies .................................................................................................. 328
Mininlizing DClta Loss .......................................................................................................... 329
Internal Software Accuracy ................................................................................................. 333
Consolidating Operations .................................................................................................... 336
Region of Interest .................................................................................................................. 340
Working in a Networked Environment ............................................................................ .341
Disk Usage ............................................................................................................................. 341
Preconlpositing ...................................................................................................................... 342
COJ/ICJ/ts IX
Chapter 12 erenli"g Elellle"'s ................................................................................. .345
Lighting .................................................................................................................................. 346
Interactive Lighting .............................................................................................................. 348
Matched Can1cras ................................................................................................................. 349
The Reference Stand-in ......................................................................................................... 351
Clcan Plates ............................................................................................................................ 353
Film Stock ............................................................................................................................... 353
Filters ..................................................................................................................................... .356
Choosing a Fonnat ................................................................................................................ 356
Lighting and Shooting with B1uescrccns .......................................................................... .358
Bluescreen versus Greenscreen ........................................................................................... 364
Shooting Order ......................................................................................................................3 66
Chapter 13 Addiliollnlllllegmiioll Tecillliqlles ........................................................ .367
Scene Continuity ................................................................................................................... 369
Color and Lighting ............................................................................................................... .369
Ligilli"g ........................................................................................................................... .370
Lighl WrnflPi"g ............................................................................................................... .371
Shadows ........................................................................................................................... 376
Digilal Color Malehi"g ................................................................................................... 379
Spil/ S"ppressio" ............................................................................................................. 382
Afl110spllere ...................................................................................................................... 388
Can1era Characteristics ................................................ ,. .. ,. .. ,. ............................................... 389
Call/era Misl1lalches .........................................................................................................3 89
Call1era Movelllt!J/fs ......................................................................................................... 393
Scale alld Pel'spccl iv(' ...................................... ,. ............................................................... 394
Foe"s ............................................................................................................................... .395
Mofioll BIIII' ..................................................................................................................... 397
~.~m ...................................................................................................................... .3M
FilllI el'aill allri SCIISOl' Noise ..........................................................................................3 99
Chapter 14 Advallced alld Relaled Topics ................................................................ .405
The Digital Rcprescntation of Images, Revisited ............................................................ .405
High DylJamic Rnllge Imngillg ....................................................................................... .406
Color Reprodllclioll .......................................................................................................... 414
WOl'killg wilh Limited Bit Depth .................................................................................... .417
COIICI,/sioIIS. ..................................................................................................................... 428
3D Compositing .................................................................................................................... 428
Working with CG Elen1cnts .................................................................................................4 35
2-Depll' COlllposilillg ..................................................................................................... .437
Mlllti-pass l~ellde";'lg ...................................................................................................... 448
Related 2D Disciplines .........................................................................................................4 55
Digilal paillli"S .............................................................................................................. .456
Editi"g ............................................................................................................................ .457
The Digifnllllfenllediafe. ................................................................................................. 458
Cover/Interior Image Credits
Stills from Tile Goldell Compass courtesy of New Line Productions, Inc. Photo by Digital Domain.
© MMVII New Line Productions, Inc. All rights reserved.
Stills from I, /~obol courtesy of Twentieth Century Fox. © 2004 Twentieth Century Fox. All rights
reserved.
Stills from KillS KOIIS courtesy of Univcrsill Studios. © 2005 Univcrsnl Studios. All rights reserved.
Images from LA'sl courtesy Touchstone Television. ® Touchstone Television.
Tlu: Lord of /f,e I~i"gs: Tile r~elllrl1 of ti,e Killg photos courtesy of New Line Productions, Inc. Copyright
MMIlI, New Line Productions, Inc. TM The Saul Z<1cnlz Company d/b/., Talkien Enterprises under
license to New Line Productions, Inc. All rights reserved. Photo by Pierre Vinet.
Hubble Space Telescope images courtesy of NASA, ESA, M. Robberto (Space Telescope Science
Institute/ESA) and the Hubble Space Telescope Orion Treasury Project Team.
Stills from Sill Cily courtesy of Miramax Films © Mir<1max Films. All rights reserved.
Stills from Sky ( nplnill nud tI,C World of Tomorrow courtesy of Paramount Pictures Corp. © PnrnlnOllnt
Pictures Corp. All rights reserved.
Spider-Ma,,1,l2 lighting pass imnges courtesy of Sony Pictures Imageworks Inc. © 2oo-l Sony Pictures
Inc. All rights reserved
Ulldcnllorld: Evollilioll imnges courtesy of Sony Pictures Inc. © 2006 Sony Pictures Inc. All rights
reserved.
Wnllncc & Gromit: Cllrse of tile Were-mbbit imnges courtesy of Anrdman Anirn<1tions Ltd. © and
'".\1,
Anrdmnn Animntions Ltd. 2005. All Rights Reserved.
Stars Wars: Rcvcllse of thc Sith stills courtesy of LUCASFI LM LTD, © LUCASFI LM LTD.
Images from Speed courtesy Twentieth Century Fox Film Corporation. © 1994 by Twentieth Ccntury
Fox Film Corporation. All rights reserved.
Images from /IIdcpC/ldcl/ce Dny courtcsy Twentieth Century Fox Film Corporation. Image from
flldepcmiellce Dny © 1996 by Twentieth Century Fox Film Corporntion. All rights reserved.
Photos from the motion picture Tf,e i'rillce of EgyptTM& © 1998 Drenm Works L.L.c., reprinted with
permission of DrcnmWorks Animntion.
Images from 1/1(lII;C courtesy Twentieth Century Fox Film Corporation. © '1997 by Twentieth Century
Fox Film Corporation. All rights reserved.
Images from A Gellt!clI/tII"s Duel courtesy Blur Studio. © 2007 by Blur Studio. All rights reserved.
Images of BnltleSlar Galactica courtesy of Universal Studios Licensing L.L.L.P.
The perrnission of Foster's Group Limited to reproducc materinl from TI1C Carlton Draught "Big Ad" is
gratefully acknowledged.
Imnges of Cllrollielcs of Nan/ia: TIle Lioll, Tile Witel" (1IId TI,e Wnrdrobe courtesy of Disney Enterprises,
Inc and W. . lden Medi<1, LLC. © 2005 Disney Enterprises, Inc. and W. . ldcn Mcdia, LLC.
Images of TI,e iucredibics courtesy of Disney Enterprises, Inc/Pixar. © 2004 Disney Entcrprises,
Inc./Pixnr
I n the years since the publication of the first edition of this book (1999), there has
been a continued growth and flourishing of digital compositing in the worlds of
film, television, gaming, and online content.
But even though a great deal of new sciel1ce has been added to the toolbox ilvailable to
digital artists, the fundamentals of compositing and, more importantly, the fundamen
tals of the artistry behind the process of digital compositing, has remained the same.
And so, as with the first edition, this new edition is still intended to be a foundation
work-something that will give the reader a basic knowledge of digital compositing's
underpinnings rather than concentrate on a particular piece of software. New tools
will continue to become available that extend the artist's reach and (hopefully) make
certain things easier, but the fundamentals are just as important today as they have
been in the past.
What's New
If you already own or have read the first edition of this book, you're probably won
dering what, specifically, is new for this edition. My original goal was to simply
update the chapter that contains the case studies-to cover a number of more recent
works and thus have the opportunity to discuss newer tools and techniques \vithin
the context of those case studies.
But as I was dealing with the acquisition of those various properties (a process that
ultimately took well over a year, all told-traversing the bureaucracies of major movie
studios is definitely not recommended for the impatient!) I found myself continually
coming across areas that I felt could use a bit of an update in the rnain body of the text
as well. Although the book was originally written in a fashion that I'd hoped would be
reasonably obsolescence-proof, technology doesn't stand still and there were a number
of areas that I realized could usc hlrther elaboration and some updating.
The new subtitle-Tec!miqlles for Visual Effects, Allil1lafioll mid Motion Graphics-is of
course partially a way to make this book easier to find when one is searching for
information on the web. But it also reflects the fact that this edition is even more
focused on proViding a set of practical, real-world concepts for working with images.
Technology continues to advance but ultimately one needs to be able to apply that
technology to solve a problern. And technology is ultimately (and even etymologi
cally) all about knowledge of techniques.
In addition, the first edition was published at a time when color imagery still added a
significant expense to a book. As such there were a number of concepts that I wasn't
