Table Of ContentCompositing Visual Effects
Essentials for the Aspiring Artist
Steve Wright
AMSTERDAM • BOSTON • HEIDELBERG • LONDON • NEW YORK • OXFORD
PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO
Focal Press is an imprint of Elsevier
Focal Press is an imprint of Elsevier
The Boulevard, Langford Lane, Kidlington, Oxford, OX5 1GB, UK
225 Wyman Street, Waltham, MA 02451, USA
First edition 2008
Second edition 2011
Copyright © 2011 Elsevier Inc. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic
or mechanical, including photocopying, recording, or any information storage and retrieval system,
without permission in writing from the publisher. Details on how to seek permission, further infor-
mation about the Publisher’s permissions policies and our arrangement with organizations such as
the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website:
www.elsevier.com/permissions
This book and the individual contributions contained in it are protected under copyright by the Publisher
(other than as may be noted herein).
Notices
Knowledge and best practice in this field are constantly changing. As new research and experience
broaden our understanding, changes in research methods, professional practices, or medical treatment may
become necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and
using any information, methods, compounds, or experiments described herein. In using such informa-
tion or methods they should be mindful of their own safety and the safety of others, including parties for
whom they have a professional responsibility.
To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any
liability for any injury and/or damage to persons or property as a matter of products liability, negligence or
otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the
material herein.
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Library of Congress Control Number: 2011927783
ISBN: 978-0-240-81781-1
For information on all Focal Press publications
visit our website at www.focalpress.com
Printed and bound in China
11 12 13 14 10 9 8 7 6 5 4 3 2 1
Digital compositing is a key component of today’s visual effects, which create fantastic
and exciting images for audiences everywhere. There are now many schools, universities,
training institutes, and online training services attracting an ever-growing number of new
students eager to join this exciting branch of the entertainment industry. Beyond formal
training institutions, it is also one of those rare fields in which an enterprising person can
actually train themselves, then go out and get a seriously paying job. You don’t need a degree
from any particular institute. You only need to demonstrate that you can do the work. The
problem is it can be difficult to get started because so much of the training available out
there makes assumptions about the background of the student. This book does not.
This is an introductory book on digital compositing for visual effects. It makes no
assumptions about the readers’ background and is written for those that do not yet
have a deep exposure to digital compositing or visual effects, but would like to quickly
and painlessly come up to speed. It is digital compositing lite, readable by all, lavished
with hundreds of film shots, figures, illustrations and diagrams to help tell the story
for the visual reader. For those that already have a background in digital compositing,
I would recommend my more advanced book on the subject, Digital Compositing for
Film and Video, by Steve Wright. Those that would benefit most from this book include:
• Digital compositing or visual effects students new to the industry
• Someone considering digital compositing as a career and would like to see what
it is all about
• Entertainment industry professionals that would like to understand digital com-
positing, such as producers, directors, editors, colorists, and post-production
supervisors
• The curious, the interested, the explorer.
While carefully written not to confuse or intimidate the newcomer, it is also packed
full of information and techniques specifically designed to be useful to those new to
compositing and visual effects. The reader will gain a valuable vocabulary and a basic
understanding of the full range of visual effects, in which digital compositing plays
xi
xii Introduction
a key role, all the while avoiding the muck and mire of math. The first chapter will
take you on an inspirational tour of the scope and magnitude of digital compositing
for visual effects and provide a solid overview of the kinds of digital effects routinely
done today. See how CGI (computer generated image) is composited with live action,
how set extensions are done, and what a match-move shot is. Following that, each of
the key applications of purely digital compositing are revealed including bluescreen
compositing, bullet time shots, motion tracking, and rotoscoping, to name but a few.
The second chapter offers a basic education on digital images in order to reveal
the key concepts and terms used in the subsequent chapters. Each chapter focuses on
one major application of digital compositing in the entertainment industry today and
introduces the fundamental concepts and processes behind them. This includes the
many ways to composite CGI, bluescreen compositing, key frame animation, creating
masks, working with digital keyers, and many, many more. But most importantly, the
art of digital compositing—making your shots look not just photorealistic, but cool.
Special attention is paid to defining new terminology and telling a clear story from the
ground up with the only requirement being that you have read the previous chapters.
The entertainment industry professional that has no intention of becoming a digital
compositor, but would like to better understand the process, will find it laid out in a
way that is extremely accessible and informative. Learn what is easy and hard, pos-
sible and impossible, and what to expect when working on a job that entails digital
compositing. There are tips on when not to use the new low-end DV video cameras
and how they damage the quality of the visual effects shot. There are even tips for the
client such as guidelines on how to shoot the quality bluescreen or greenscreen to get
the best results at compositing time.
Enjoy.
Steve Wright is a 20-year veteran of digital effects with 70 broadcast television
commercials and over 70 feature film credits. He now teaches VFX compositing,
Nuke and Shake at visual effects facilities around the world, as well as speaks,
writes, lectures, publishes, and develops training material. To learn more about
Steve Wright you can visit his web site at:
www.vfxio.com
A book like this is immensely visual and requires a great many pictures. While I do
have a large library of 35mm film scans and HDTV elements to tap into, it was not
enough for a book of this scope. I am grateful to the following groups and individuals
who contributed images that enliven the pages of this book.
A special heartfelt thank you goes to the amazing Sylvia Binsfeld of DreamWeaver
Films and her fabulous 35mm short film Dorme that I had the privilege and pleasure
of compositing for her. Her contribution of high-resolution 35mm film scans, delight-
ful CGI elements, and gorgeous digital matte paintings grace many pages of this book
including the cover (www.dormefilm.com).
A very special thanks to Jonathan Karafin who was my stellar subject matter expert
on stereo conversion. His insights and depth of knowledge from working in the stereo
conversion industry virtually from its inception were invaluable to making this section
accurate and totally authentic.
Many thanks to Alex Lindsay and Christopher Marler of PixelCorps for contribut-
ing high definition bluescreen and greenscreen footage shot with their fabulous Sony
HDC-950 HD color video camera direct to hard drive (www.pixelcorps.com).
A special thank you to Tom Vincze, a terrific CGI artist, who created multipass CGI,
sims, and particle system examples specifically for this book (www.thomasvincze.com).
Many thanks to Greystone Films and the Mount Vernon Ladies’ Association for
providing high-resolution 35mm film scans from their location-based film We Fight to
be Free, a docu-drama about the early heroic life of George Washington.
My thanks to Pavel Dvorak, a photographer and friend, who contributed many
high-quality digital still photos from his collection (www.absolutpavel.com).
Thanks also to Kodak for their generous contribution of scanned 35mm film ele-
ments from their corporate film The Lightning Project, courtesy © Eastman Kodak
Company (www.kodak.com).
I would also like to thank the very surprising Ruairi Robinson for contributing key
shots from his spectacular visual effects film Silent City. His examples of match move
and set extension were valuable additions to this book (www.ruairirobinson.com).
xiii
xiv Acknowledgements
Thanks to the Foundry, a world leader in providing plug-ins to the visual effects
community, for providing the crowd duplication example (www.thefoundry.co.uk).
Thanks go to Eric Deinzer (www.ericdeinzer.com) and his charming wife Mariana
Acuna (www.pixelsbyme.com) for their excellent advice on the stereo conversion
section.
A special shout-out to Mauricio Monroy who advised me on the gritty details of
stereo conversion and reviewed my methods for accuracy (www.mmvfx.com).
The chapter on the Life of a VFX Shot would not have been possible without
the contribution of stories and photographs from Sylvia Binsfeld, cited above. Many
thanks to Svetlana Cvetko the DP for Dorme, who contributed her stories and pictures
(www.svetlana-dp.com). I am pleased to also thank Nicholas Barnes, the 3D animator
for Dorme, for his contributions to the stories and pictures about his role in creating
the CGI for the VFX shot in Dorme.
And, finally, a very special thanks to my darling wife, Diane Wright, for she has
been a great supporter and contributor to the entire effort on this book. She not only
graciously tolerated the amount of time spent writing it, but also reviewed, critiqued,
formatted, and contributed graphics ([email protected]).
Digital compositing is an essential part of visual effects,
which are everywhere in the entertainment industry
today—in feature films, television commercials, and
even many TV shows. And it’s growing. Even a non-
effects film will have visual effects. It might be a love
story or a comedy, but there will always be something
that needs to be added or removed from the picture
to tell the story. And that is the short description of what visual effects are all about—
adding elements to a picture that are not there, or removing something that you don’t
want to be there. And digital compositing plays a key role in all visual effects.
The things that are added to the picture can come
from practically any source today. We might be add-
ing an actor or a model from a piece of film or video
tape. Or perhaps the mission is to add a spaceship or
dinosaur that was created entirely in a computer, so it
is referred to as a computer generated image (CGI).
Maybe the element to be added is a matte painting
done in Adobe Photoshop®. Some elements might
even be created by the compositor himself.
It is the digital compositor that takes these dis-
parate elements, no matter how they were created,
and blends them together artistically into a seam-
less, photorealistic whole. The digital compositor’s
mission is to make them appear as if they were
all shot together at the same time under the same
lights with the same camera, and then give the
shot its final artistic polish with superb color correction. This is a non-trivial accom-
plishment artistically, and there are a variety of technical challenges that have to be
met along the way. Digital compositing is both a technical and an artistic challenge.
1
2 Compositing Visual Effects
The compositor is first and foremost an artist, and
works with other artists such as matte painters, color-
ists, CGI artists, and art directors as a member of a
visual effects team. This team must coordinate their
efforts to produce technically sophisticated and artis-
tically pleasing effects shots. The great irony here is
that if we all do our jobs right, nobody can tell what we did because the visual effects
blend seamlessly with the rest of the movie. If we don’t, the viewer is pulled out of the
movie experience and starts thinking, “Hey, look at those cheesy effects!”
So what is the difference between visual effects
and special effects? Visual effects are the creation or
modification of images, whereas special effects are
things done on the set which are then photographed,
such as pyrotechnics or miniatures. In other words,
visual effects specifically manipulate images. And
since manipulating images is best done with a com-
puter, they are the tool of choice, and this is why the
job is known as digital compositing.
I mentioned earlier that digital compositing is
growing. There are two primary reasons for this. First
is the steady increase in the use of CGI for visual
effects, and every CGI element needs to be compos-
ited. The reason CGI is on the upswing is because of
the steady improvement in technology which means
that CGI can solve more visual problems every year,
thus increasing the demand by movie makers for ever-more spectacular effects for
their movies (or TV shows or television commercials). Furthermore, as the hardware
gets cheaper and faster, and the software becomes
more capable, it tends to lower the cost of creating
CGI. However, any theoretical cost savings here are
quickly overwhelmed by the insatiable appetite for
more spectacular, complex, and expensive visual
effects. In other words, the creative demands continu-
ously expand to fill the capabilities of the technology.
The second reason for the increase in digital compositing is that the compositing
software and hardware technologies are also advancing on their own track, separately
from CGI. This means that visual effects shots can
be done faster, more cost-effectively, and with higher
quality. There has also been a general rise in the aware-
ness of the film makers to what can be done with digi-
tal compositing, which makes them more sophisticated
users. As a result, they demand ever-more effects be
done for their movies.
Visual Effects Today 3
In this chapter we take a wide-ranging tour of the
incredible scope and scale of digital compositing in visual
effects today. The first section describes digital compos-
iting with CGI. A lot of our work is to composite CGI,
so it warrants its own section. The second section reveals
the many types of amazing visual effects that are done
strictly with modern compositing software—no CGI
required. The third section takes a brief look at the two types of compositing programs,
node based and timeline based. From this point on I will refer to film in these examples,
but I am also including video. It is just a bit too cumbersome to continuously refer to film
(and videotape) or filming (and videotaping) or cinematographers (and videographers).
1.1 DIGITAL COMPOSITING WITH CGI
Whenever someone makes a CGI element, someone else has to composite it. In this
section we will look at three general areas where CGI elements are composited. First
up is straightforward CGI compositing where a CGI object has been created and
needs to be composited into the scene. Next we will take a look at set extension, a rap-
idly expanding technique in film making. Last we will look at the match move, where
separate programs are used to analyze the live action and provide terrain and camera
move data for the CGI programs.
1.1.1 CGI Compositing
By far the most common application of digital compositing is to composite CGI.
Whether it is for a $100,000 dollar commercial or a $100 million dollar movie, the
CGI is created in a computer and composited over some kind of a background image.
The background image is very often live action, meaning it was shot on film or video,
but it too could be CGI that was also created in a computer, or it might be a digital
matte painting. Regardless of where the background comes from, the digital compositor
puts it all together and gives it the final touch of photorealism. Figure 1-1 illustrates a
basic CGI composite where the jet fighter was created in the computer, the background
is live action footage, and the final composite puts the jet fighter into the background.
CGI object Live action background CGI composited over
background
Figure 1-1 A CGI object composited over a live action background
4 Compositing Visual Effects
Today, CGI goes far beyond jet fighters and dinosaurs. Recent advances in the
technology have made it possible to create a very wide range of incredibly realistic
synthetic objects for compositing. Beyond the obvious things such as cars, airplanes,
and rampaging beasts, CGI has mastered the ability to create photorealistic hair, skin,
cloth, clouds, fog, fire, and even water. It has even become common practice to use a
“digital double” when you want the movie star to execute a daring stunt that is beyond
even the expert stunt double. It will not be long before the first “cyber thespian” (an
all-CGI character) stars in what would otherwise be a live action movie. However,
after all the spaceships, beasts, water, and fire have been rendered, somebody has to
composite them all together. And that somebody is the dauntless digital compositor.
1.1.2 Set Extension
If you wanted your actors to be seen standing in front of the Imperial Palace on the
planet Mungo you may not want to spend the money to build the entire exterior of
the Imperial Palace. Better to build a small piece of it on a movie set, place the talent
in it, and then extend the set later using CGI when you need a wide shot. For this little
piece of digital magic the film is digitized into the computer and the CGI folks build
a 3D model of the Imperial Palace and line it up with the original film footage and its
camera angle. There might even be a camera move in the live action which the CGI
artists will carefully track and match. But eventually the live action and the CGI set
extension must be composited and color corrected to match perfectly.
Original photography Set extension Resulting composite
Figure 1-2 Set extension*
Figure 1-2 is a classic example of set extension that demonstrates how it can even
be applied to an exterior location shot. The original photography captures the future
soldier walking along an otherwise small and uninteresting rubble pit. The middle
picture shows the set extension element that was an all-CGI city in the distance with
smoke generated by a particle system and a matte painting for the sky. The resulting
composite shows all of the elements composited together and color corrected to blend
properly.
*Courtesy of Ruairi Robinson’s Silent City.
