Table Of ContentActa Numerica 1992
Managing Editor
A. Iserles
DAMTP, University of Cambridge, Silver Street
Cambridge CB3 9EW, England
Editorial Board
C. de Booi; University of Wisconsin, Madison, USA
F. Brezzi, Instituto di Analisi Numerica del CNR, Italy
J.C. Butchei; University of Auckland, New Zealand
P.G. Ciarlet, Universiti Paris VI, France
G.H. Golub, Stanford University, USA
H.B. Keller California Institute of Technology, USA
H.-O. Kreiss, University of California, Los Ang/eks, USA
K.W. Morton, University of Oxford, England
MJ.D. Powell, University of Cambridge, England
R. Temam, Universiti Paris Sud, France
umerica
1992
CAMBRIDGE
UNIVERSITY PRESS
Published by the Press Syndicate of the University of Cambridge
The Pitt Building, Trumpington Street, Cambridge CB2 1RP
40 West 20th Street, New York, NY 10011-4211, USA
10 Stamford Road, Oakleigh, Victoria 3166, Australia
© Cambridge University Press 1992
First published 1992
Printed in the United States of America
A catalog record for this book is available from the British Library
ISBN 0-521-41026-6 hardback
ISSN 0962-4929
Contents
Wavelets 1
Ronald A. DeVore and Bradley J. Lucier
Iterative solution of linear systems 57
R. W. Freund, G.H. Golub and N.M. Nachtigal
Problems with different time scales 101
Heinz-Otto Kreiss
Numerical methods for differential algebraic equations 141
Roswitha Mdrz
Theory of algorithms for unconstrained optimization 199
J. Nocedal
Symplectic integrators for Hamiltonian problems:
an overview 243
J.M. Sanz-Serna
Error analysis of boundary integral methods 287
Ian H. Sloan
Interior methods for constrained optimization 341
Margaret H. Wright
Preface
In these days and age, when the sheer number of publications in numeri-
cal mathematics increases so rapidly, it is surely necessary to provide valid
justification to a new publication. The reason for Ada Numerica is, para-
doxically, to counteract the information explosion by presenting selected and
important developments in numerical mathematics and scientific computa-
tion on an annual basis.
Each year, the Editorial Board of Ada Numerica poses itself the question
'what are recent significant developments in our subject, developments that
are important enough to merit interest by the numerical community as a
whole'. Having selected a shortlist of topics, we ask named individuals to
write survey papers. The purpose of the exercise being to disseminate knowl-
edge outside restricted professional boundaries, the authors are expected to
pitch their exposition so that it can be understood and appreciated by all
practitioners of the numerical art, and not just by workers in a narrow sub-
discipline. We are guided in our choice of authors both by their contribution
to the underlying topic and by their track record as expositors.
Numerical analysts, like other professionals in a competitive world, are
busy with their own research, academic and administrative duties. It is
difficult (and sometimes impossible) to keep up with developments outside
one's narrow experience. This, we believe, is an unhealthy and undesir-
able situation, not only because of broader cultural considerations but also
since developments in different parts of numerical mathematics frequently
impinge upon each other. We hope that Ada Numerica will play a role in
bridging gaps and presenting many new and exciting ideas - algorithms and
mathematical analysis alike - to a wider audience.
Ada Numerica (1991), pp. 1-56
Wavelets*
Ronald A. DeVore
Department of Mathematics
University of South Carolina,
Columbia, SC 29208 USA
E-maU: [email protected]
Bradley J. Lucier
Department of Mathematics
Purdue University,
West Lafayette, IN 47907 USA
E-mail: [email protected]
CONTENTS
1 Introduction 1
2 The Haar wavelets 4
3 The construction of wavelets 13
4 Fast wavelet transforms 37
5 Smoothness spaces and wavelet coefficients 40
6 Applications 44
References 54
1. Introduction
The subject of 'wavelets' is expanding at such a tremendous rate that it is
impossible to give, within these few pages, a complete introduction to all
aspects of its theory. We hope, however, to allow the reader to become suf-
ficiently acquainted with the subject to understand, in part, the enthusiasm
of its proponents toward its potential application to various numerical prob-
lems. Furthermore, we hope that our exposition can guide the reader who
wishes to make more serious excursions into the subject. Our viewpoint is
biased by our experience in approximation theory and data compression; we
warn the reader that there are other viewpoints that are either not repre-
sented here or discussed only briefly. For example, orthogonal wavelets were
developed primarily in the context of signal processing, an application upon
* This work was supported in part by the National Science Foundation (grants DMS-
8922154 and DMS-9006219), the Air Force Office of Scientific Research (contract 89-
0455-DEF), the Office of Naval Research (contracts N00014-90-1343, N00014-91-J-1152,
and N00014-91-J-1076), the Defense Advanced Research Projects Agency (AFOSR con-
tract 90-0323), and the Army High Performance Computing Research Center.
R. A. DEVORE AND B. J. LUCIER
4>
-1 0 j - 1 j + 1
_ 2*
2* 2*
Fig. 1. An example of functions <f> and ^(2*- — j).
which we touch only indirectly. However, there are several good expositions
(e.g. Daubechies (1990) and Rioul and Vetterli (1991)) of this application.
A discussion of wavelet decompositions in the context of Littlewood-Paley
theory can be found in the monograph of Frazier et al. (1991). We shall also
not attempt to give a complete discussion of the history of wavelets. Histor-
ical accounts can be found in the book of Meyer (1990) and the introduction
of the article of Daubechies (1990). We shall try to give sufficient historical
commentary in the course of our presentation to provide some feeling for
the subject's development.
The term 'wavelet' (originally called wavelet of constant shape) was intro-
duced by J. Morlet. It denotes a uni-variate function ty (multi-variate wave-
lets exist as well and will be discussed subsequently), defined on R, which,
when subjected to the fundamental operations of shifts (i.e. translation by
integers) and dyadic dilation, yields an orthogonal basis of L2(R). That is,
the functions V'j.fc := 2*/2V>(2*- — j), j,fc6Z, form a complete orthonormal
system for L2OR). In this work, we shall call such a function an orthogo-
nal wavelet, since there are many generalizations of wavelets that drop the
requirement of orthogonality. The Haar function H := X[o,i/2) ~ X[i/2,i)>
which will be discussed in more detail in the section that follows, is the sim-
plest example of an orthogonal wavelet. Orthogonal wavelets with higher
smoothness (and even compact support) can also be constructed. But before
considering that and other questions, we wish first to motivate the desire
for such wavelets.
We view a wavelet ip as a 'bump' (and think of it as having compact
support, though it need not). Dilation squeezes or expands the bump and
translation shifts it (see Figure 1). Thus, tpj^ is a scaled version of xp centred
at the dyadic integer j2~k. If k is large positive, then tpj^ is a bump with
small support; if k is large negative, the support of ipj^ is large.
WAVELETS 3
The requirement that the set {V"j,fc}j,fcez forms an orthonormal system
means that any function / G L2O&) can be represented as a series
/ =
with {/, g) := J fgdx the usual inner product of two L2W functions. We
R
view (1.1) as building up the function / from the bumps ipj,k- Bumps
corresponding to small values of k contribute to the broad resolution of /;
those corresponding to large values of k give finer detail.
The decomposition (1.1) is analogous to the Fourier decomposition of a
function / G Z<2(T) in terms of the exponential functions e^ :=eik', but there
are important differences. The exponential functions e*; have global support.
Thus, all terms in the Fourier decomposition contribute to the value of /
at a point x. On the other hand, wavelets are usually either of compact
support or fall off exponentially at infinity. Thus, only the terms in (1.1)
corresponding to Vj,fc with j2~k near x make a large contribution at x. The
representation (1.1) is in this sense local. Of course, exponential functions
have other important properties; for example, they are eigenfunctions for
differentiation. Many wavelets have a corresponding property captured in
the 'refinement equation' for the function <j> from which the wavelet ijj is
derived, as discussed in Section 3.1.
Another important property of wavelet decompositions not present di-
rectly in the Fourier decomposition is that the coefficients in wavelet de-
compositions usually encode all information needed to tell whether / is in a
smoothness space, such as the Sobolev and Besov spaces. For example, if ip
is smooth enough, then a function / is in the Lipschitz space Lip(o!, Loc(R)),
0 < a < 1, if and only if
sup2*(a+2)|(/,t/',fc)| (1.2)
: i
is finite, and (1.2) is an equivalent semi-norm for Lip(a, Loo(R)).
All this would be of little more than theoretical interest if it were not for
the fact that one can efficiently compute wavelet coefficients and reconstruct
functions from these coefficients. Such algorithms, known as 'fast wavelet
transforms' are the analogue of the Fast Fourier Transform and follow simply
from the refinement equation mentioned earlier.
In many numerical applications, the orthogonality of the translated di-
lates tpjk is not vital. There are many variants of wavelets, such as the
t
pre-wavelets proposed by Battle (1987) and the ^-transform of Frazier and
Jawerth (1990), that do not require orthogonality. Typically, for a given
function ip, one wants the translated dilates V'j.fc) h & € Z, to form a stable
basis (also called a Riesz basis) for Z,2(K). This means that each / € L2W
has a unique series decomposition in terms of the Vj.fc, and that the £2
