Approximation and Interpolation

105 Pages·2017·1.14 MB·English

Chapter 2
Approximation and Interpolation
Thepresentchapterisbasicallyconcernedwiththeapproximationoffunctions.The
functionsin questionmaybefunctionsdefinedona continuum–typicallya finite
interval–orfunctionsdefinedonlyonafinitesetofpoints.Thefirstinstancearises,
forexample,inthecontextofspecialfunctions(elementaryortranscendental)that
onewishestoevaluateasapartofasubroutine.Sinceanysuchevaluationmustbe
reducedtoafinitenumberofarithmeticoperations,wemustultimatelyapproximate
thefunctionbymeansofa polynomialorarationalfunction.Thesecondinstance
is frequently encountered in the physical sciences when measurements are taken
of a certain physical quantity as a function of some other physical quantity (such
as time). In either case one wants to approximate the given function “as well as
possible”intermsofothersimplerfunctions.
The general scheme of approximation can be described as follows. We are
giventhe functionf to be approximated,alongwith a class ˆ of“approximating
functions”' anda“norm”k (cid:2) kmeasuringtheoverallmagnitudeoffunctions.We
arelookingforanapproximation'O 2ˆoff suchthat
kf (cid:3)'Ok(cid:4)kf (cid:3)'k forall ' 2ˆ: (2.1)
Thefunction'O is calledthe bestapproximationto f fromthe classˆ, relativeto
thenormk (cid:2) k.
The class ˆ is called a (real) linear space if with any two functions ' ,
1
' 2 ˆ it also contains ' C' and c' for any c 2 R, hence also any (finite)
2 1 2 1
linear combination of functions ' 2 ˆ. Given n “basis functions” (cid:2) 2 ˆ,
i j
j D 1;2;:::;n,wecandefinealinearspaceoffinitedimensionnby
8 9
< Xn =
ˆDˆ D ' W '.t/D c (cid:2) .t/; c 2R : (2.2)
n : j j j ;
jD1
Examplesoflinearspaces˚. 1.ˆDP :polynomialsofdegree(cid:4)m.Abasisfor
m
P is, forexample,(cid:2) .t/ D tj(cid:2)1, j D 1;2;:::;mC1, so thatn D m C 1.
m j
Polynomials are the most frequently used “general-purpose” approximants for
W.Gautschi,NumericalAnalysis,DOI10.1007/978-0-8176-8259-0 2, 55
©SpringerScience+BusinessMedia,LLC1997,2012
56 2 ApproximationandInterpolation
dealing with functions on bounded domains (finite intervals or finite sets of
points). One reason is Weierstrass’s theorem, which states that any continuous
functioncanbe approximatedona finite intervalasclosely as onewishesby a
polynomialofsufficientlyhighdegree.
2. ˆ DSk.(cid:3)/:(polynomial)splinefunctionsofdegreemandsmoothnessclassk
m
onthesubdivision
(cid:3)W aDt1 <t2 <t3 <(cid:2)(cid:2)(cid:2)<tN(cid:2)1 <tN Db
of the interval [a;b]. These are piecewise polynomials of degree (cid:4) m pieced
togetheratthe“joints” t2;:::;tN(cid:2)1 in sucha way thatallderivativesupto and
includingthektharecontinuousonthewholeinterval[a;b],includingthejoints:
ˇ
Sk.(cid:3)/Dfs 2CkŒa;b(cid:4)W sˇ 2P ; i D1;2;:::;N (cid:3)1g:
m Œti;tiC1(cid:4) m
We assume here 0 (cid:4) k < m; otherwise, we are back to polynomialsP (see
m
Ex.68).Wesetk D (cid:3)1ifweallowdiscontinuitiesatthejoints.Thedimension
ofSk.(cid:3)/isnD.m(cid:3)k/(cid:2).N (cid:3)2/CmC1(seeEx.71),buttofindabasisisa
m
nontrivialtask;formD1,seeSect.2.3.2.
3. ˆ D T Œ0;2(cid:2)(cid:4): trigonometricpolynomialsof degree (cid:4) m on [0, 2(cid:2)]. These
m
arelinearcombinationsofthebasicharmonicsuptoandincludingthemthone,
thatis,
(cid:2) .t/Dcos .k(cid:3)1/t; k D1;2;:::;mC1I
k
(cid:2)mC1Ck.t/Dsinkt; k D1;2;:::;m;
where now n D 2m C 1. Such approximants are a natural choice when the
functionf to be approximatedis periodicwith period2(cid:2). (If f hasperiod p,
onemakesapreliminarychangeofvariablest 7!t (cid:2)p=2(cid:2).)
4. ˆD En:exponentialsums.Forgivendistinct˛j >0,onetakes(cid:2)j.t/ De(cid:2)˛jt,
j D 1;2;:::;n. Exponential sums are often employed on the half-infinite
intervalRC:0(cid:4)t <1,especiallyifoneknowsthatf decaysexponentiallyas
t !1.
Notethattheimportantclassofrationalfunctions,
ˆDR Df' W ' Dp=q; p 2P ; q 2P g;
r;s r s
isnotalinearspace.(Whynot?)
Possible choices of norm – both for continuous and discrete functions –
and the type of approximation they generate are summarized in Table 2.1. The
continuouscase involvesaninterval[a;b] anda “weightfunction”w.t/ (possibly
w.t/(cid:5)1)definedon[a;b]andpositiveexceptforisolatedzeros.Thediscretecase
involvesa setofN distinctpointst ,t ;:::;t alongwithpositiveweightfactors
1 2 N
2 ApproximationandInterpolation 57
Table2.1 Typesofapproximationandassociatednorms
Continuousnorm Approximation Discretenorm
kuk1D maxju.t/j L1 kuk1 D max ju.ti/j
a(cid:2)t(cid:2)b 1(cid:2)i(cid:2)N
Uniform
Chebyshev
Z
b XN
kuk D ju.t/jdt L kuk D ju.t /j
1 1 1 i
aZ iD1
b XN
kuk D ju.t/jw.t/dt WeightedL kuk D wju.t /j
1;w 1 1;w i i
aZ b !12 i DX1N !12
kuk D ju.t/j2w.t/dt WeightedL kuk D wju.t /j2
2;w 2 2;w i i
a iD1
Leastsquares
w ;w ;:::;w (possibly all equal to 1). The interval [a;b] may be unboundedif
1 2 N
the weightfunctionwis such thatthe integralextendedover[a;b],whichdefines
thenorm,makessense.
Hence,wemaytakeanyoneofthenormsinTable2.1andcombineitwithanyof
theprecedinglinearspacesˆtoarriveatameaningfulbestapproximationproblem
(2.1).Inthecontinuouscase,thegivenfunctionf,andthefunctions' oftheclass
ˆ,ofcourse,mustbedefinedon[a;b]andsuchthatthenormkf (cid:3)'kmakessense.
Likewise,f and' mustbedefinedatthepointst inthediscretecase.
i
Notethatifthebestapproximant'O inthediscretecaseissuchthatkf (cid:3)'OkD0,
then 'O.t / D f.t / for i D 1;2;:::;N. We then say that 'O interpolates f at
i i
the points t and we refer to this kind of approximationproblem as an interpola-
i
tionproblem.
Thesimplestapproximationproblemsaretheleastsquaresproblemandthein-
terpolationproblem,andtheeasiestspaceˆtoworkwiththespaceofpolynomials
of givendegree.These are indeedthe problemswe concentrateon in this chapter.
In the case of the least squares problem,however,we admit generallinear spaces
ˆ of approximants,and also in the case of the interpolation problem, we include
polynomialsplinesinadditiontostraightpolynomials.
Beforewestartwiththeleastsquaresproblem,weintroduceanotationaldevice
that allows us to treat the continuous and the discrete case simultaneously. We
define,inthecontinuouscase,
8
ˆˆˆˆˆˆ<Z0 tif t <a .whenever (cid:3)1<a/;
w.(cid:6)/d(cid:6) if a (cid:4)t (cid:4)b;
(cid:5).t/D (2.3)
ˆˆˆˆˆˆ:Zab
w.(cid:6)/d(cid:6) if t >b .whenever b <1/:
a
58 2 ApproximationandInterpolation
Thenwecanwrite,forany(say,continuous)functionu,
Z Z
b
u.t/d(cid:5).t/D u.t/w.t/dt; (2.4)
R a
since d(cid:5).t/ (cid:5) 0 “outside” [a;b], and d(cid:5).t/ D w.t/dt inside. We call d(cid:5) a
continuous(positive)measure.Thediscretemeasure(alsocalled“Diracmeasure”)
associatedwiththepointsetft ;t ;:::;t gisameasured(cid:5)thatisnonzeroonlyat
1 2 N
thepointst andhasthevaluew there.Thus,inthiscase,
i i
Z
XN
u.t/d(cid:5).t/D w u.t /: (2.5)
i i
R
iD1
(AmoreprecisedefinitioncanbegivenintermsofStieltjesintegrals,ifwedefine
(cid:5).t/tobeastepfunctionhavingjumpw att .)Inparticular,wecandefinetheL
i i 2
normas
(cid:2)Z (cid:3)
1
2
kuk D ju.t/j2d(cid:5).t/ ; (2.6)
2;d(cid:5)
R
andobtainthecontinuousorthediscretenormdependingonwhether(cid:5)istakento
beasin(2.3),orastepfunction,asin(2.5).
We callthesupportofd(cid:5)–anddenoteitbysuppd(cid:5)–theinterval[a;b]inthe
continuouscase (assuming w positive on [a;b] exceptfor isolated zeros), and the
setft ;t ;:::;t ginthediscretecase.Wesaythatthesetoffunctions(cid:2) .t/in(2.2)
1 2 N j
islinearlyindependentonthesupportofd(cid:5)if
Xn
c (cid:2) .t/(cid:5)0 forall t 2suppd(cid:5) implies c Dc D(cid:2)(cid:2)(cid:2)Dc D0: (2.7)
j j 1 2 n
jD1
ExampleW thepowers(cid:2) .t/Dtj(cid:2)1,j D1;2;:::;n.
j
Xn
Here cj(cid:2)j.t/ D pn(cid:2)1.t/isapolynomialofdegree(cid:4) n(cid:3)1.Suppose,first,
jD1
that suppd(cid:5) = [a;b]. Then the identity in (2.7) says that pn(cid:2)1.t/ (cid:5) 0 on [a;b].
Clearly, this implies c D c D (cid:2)(cid:2)(cid:2) D c D 0, so that the powers are linearly
1 2 n
independentonsuppd(cid:5)= [a;b].If,on theotherhand,suppd(cid:5) D ft ;t ;:::;t g,
1 2 N
thenthepremisein(2.7)saysthatpn(cid:2)1.ti/ D 0,i D 1;2;:::;N;thatis,pn(cid:2)1 has
NQdistinctzerosti. Thisimpliespn(cid:2)1 (cid:5) 0 onlyifN (cid:6) n. Otherwise, pn(cid:2)1.t/ D
NiD1.t (cid:3)ti/ 2 Pn(cid:2)1 wouldsatisfy pn(cid:2)1.ti/ D 0, i D 1;2;:::;N, withoutbeing
identicallyzero.Thus,wehavelinearindependenceonsuppd(cid:5)Dft ;t ;:::;t gif
1 2 N
andonlyifN (cid:6)n.
2.1 LeastSquaresApproximation 59
2.1 LeastSquares Approximation
Wespecializethebestapproximationproblem(2.1)bytakingasnormtheL norm
2
(cid:2)Z (cid:3)
1
2
kuk D ju.t/j2d(cid:5).t/ ; (2.8)
2;d(cid:5)
R
whered(cid:5)iseitheracontinuousmeasure(cf.(2.3))oradiscretemeasure(cf.(2.5)),
andbyusingapproximants' fromann-dimensionallinearspace
8 9
< Xn =
ˆDˆ D ' W '.t/D c (cid:2) .t/; c 2R : (2.9)
n : j j j ;
jD1
Herethebasisfunctions(cid:2) areassumedlinearlyindependentonsuppd(cid:5)(cf.(2.7)).
j
Wefurthermoreassume,ofcourse,thattheintegralin(2.8)ismeaningfulwhenever
uD(cid:2) oruDf,thegivenfunctiontobeapproximated.
j
The solution of the least squares problem is most easily expressed in terms of
orthogonalsystems(cid:2) relativetoanappropriateinnerproduct.Wethereforebegin
j
withadiscussionofinnerproducts.
2.1.1 InnerProducts
Givenacontinuousordiscretemeasured(cid:5),asintroducedearlier,andgivenanytwo
functionsu;vhavingafinitenorm(2.8),wecandefinetheinnerproduct
Z
.u;v/D u.t/v.t/d(cid:5).t/: (2.10)
R
(Schwarz’sinequalityj.u;v/j(cid:4)kuk (cid:2)kvk ,cf.Ex.6,tellsusthattheintegral
2;d(cid:5) 2;d(cid:5)
in (2.10)is welldefined.)Theinnerproduct(2.10)hasthe followingobvious(but
useful)properties:
1. symmetry:.u;v/D.v;u/;
2. homogeneity:.˛u;v/D˛.u;v/,˛ 2R;
3. additivity:.uCv;w/D.u;w/C.v;w/;and
4. positive definiteness:.u;u/ (cid:6) 0, with equalityholdingif andonlyif u (cid:5) 0 on
suppd(cid:5).
Homogeneityandadditivitytogethergivelinearity,
.˛ u C˛ u ;v/D˛ .u ;v/C˛ .u ;v/ (2.11)
1 1 2 2 1 1 2 2
60 2 ApproximationandInterpolation
Fig.2.1 Orthogonalvectorsandtheirsum
in the first variableand, by symmetry,also in the second. Moreover,(2.11)easily
extendstolinearcombinationsofarbitraryfinitelength.Notealsothat
kuk2 D.u;u/: (2.12)
2;d(cid:5)
Wesaythatuandvareorthogonalif
.u;v/D0: (2.13)
Thisisalwaystriviallytrueifeitheruorvvanishesidenticallyonsuppd(cid:5).
Itisnowasimpleexercise,forexample,toprovetheTheoremofPythagoras:
if .u;v/D0; then kuCvk2 Dkuk2Ckvk2; (2.14)
where k (cid:2) k D k (cid:2) k . (From now on we use this abbreviated notation for the
2;d(cid:5)
norm.)Indeed,
kuCvk2 D.uCv;uCv/D.u;u/C.u;v/C.v;u/C.v;v/
Dkuk2C2.u;v/Ckvk2 Dkuk2Ckvk2;
where the first equality is a definition, the second follows from additivity, the
third from symmetry, and the last from orthogonality. Interpreting functions u;v
as “vectors,”we can picture the configurationof u;v (orthogonal)and uCv as in
Fig.2.1.
Moregenerally,wemayconsideranorthogonalsystemsfu gn :
k kD1
.u ;u / D0 if i ¤j; u 6(cid:5)0 onsuppd(cid:5)I
i j k
i;j D1;2;:::;nI k D1;2;:::;n: (2.15)
2.1 LeastSquaresApproximation 61
ForsuchasystemwehavetheGeneralizedTheoremofPythagoras,
ˇˇ ˇˇ
ˇˇˇˇXn ˇˇˇˇ2 Xn
ˇˇ ˛ u ˇˇ D j˛ j2ku k2: (2.16)
ˇˇ k kˇˇ k k
kD1 kD1
Theproofisessentiallythesameasbefore.Animportantconsequenceof(2.16)is
thateveryorthogonalsystemislinearlyindependentonthesupportofd(cid:5).Indeed,
if the left-hand side of (2.16)vanishes, then so does the right-handside, and this,
sinceku k2 >0byassumption,implies˛ D˛ D(cid:2)(cid:2)(cid:2)D˛ D0.
k 1 2 n
2.1.2 TheNormal Equations
We are now in a position to solve the least squares approximation problem. By
(2.12),wecanwritetheL error,orratheritssquare,intheform:
2
E2Œ'(cid:4)WDk'(cid:3)fk2 D.'(cid:3)f;'(cid:3)f/D.';'/(cid:3)2.';f/C.f;f/:
Inserting' herefrom(2.9)gives
0 1 0 1
Z 2 Z Z
Xn Xn
E2Œ'(cid:4)D @ c (cid:2) .t/Ad(cid:5).t/(cid:3)2 @ c (cid:2) .t/Af.t/d(cid:5).t/C f2.t/d(cid:5).t/:
j j j j
R R R
jD1 jD1
(2.17)
The squared L error, therefore, is a quadratic function of the coefficients c ,
2 1
c ;:::;c of'.TheproblemofbestL approximationthusamountstominimizing
2 n 2
a quadratic function of n variables. This is a standard problem of calculus and is
solvedbysettingallpartialderivativesequaltozero.Thisyieldsasystemoflinear
algebraic equations. Indeed, differentiating partially with respect to c under the
i
integralsignin(2.17)gives
0 1
Z Z
Xn
@ E2Œ'(cid:4)D2 @ c (cid:2) .t/A(cid:2) .t/d(cid:5).t/(cid:3)2 (cid:2) .t/f.t/d(cid:5).t/;
j j i i
@ci R jD1 R
and setting this equal to zero, interchanging integration and summation in the
process,weget
Xn
.(cid:2) ;(cid:2) /c D.(cid:2) ;f/; i D1;2;:::;n: (2.18)
i j j i
jD1
These are called thenormalequationsforthe least squaresproblem.Theyform a
linearsystemoftheform
Ac Db; (2.19)
wherethematrixAandthevectorbhaveelements
62 2 ApproximationandInterpolation
A DŒa (cid:4); a D.(cid:2) ;(cid:2) /I bDŒb (cid:4); b D.(cid:2) ;f/: (2.20)
ij ij i j i i i
Bysymmetryoftheinnerproduct,Aisasymmetricmatrix.Moreover,Aispositive
definite;thatis,
Xn Xn
xTAx D a x x >0 if x ¤Œ0;0;:::;0(cid:4)T: (2.21)
ij i j
iD1jD1
Thequadraticfunctionin(2.21)iscalledaquadraticform(sinceitishomogeneous
of degree 2). Positive definiteness of A thus says that the quadratic form whose
coefficients are the elements of A is always nonnegative, and zero only if all
variablesx vanish.
i
To prove(2.21),allwe haveto dois insertthe definitionofthe a anduse the
ij
elementaryproperties1–4oftheinnerproduct:
(cid:4) (cid:4)
Xn Xn Xn Xn (cid:4)(cid:4)Xn (cid:4)(cid:4)2
xTAx D x x .(cid:2) ;(cid:2) /D .x (cid:2) ;x (cid:2) /D(cid:4) x (cid:2) (cid:4) :
i j i j i i j j (cid:4) i i(cid:4)
iD1jD1 iD1jD1 iD1
P
Thisclearlyisnonnegative.Itiszeroonlyif n x (cid:2) (cid:5)0onsuppd(cid:5),which,by
iD1 i i
theassumptionoflinearindependenceofthe(cid:2) ,impliesx Dx D(cid:2)(cid:2)(cid:2)Dx D0.
i 1 2 n
Now itisa well-knownfactof linearalgebrathatasymmetricpositivedefinite
matrixA isnonsingular.Indeed,itsdeterminant,aswellasallitsleadingprincipal
minordeterminants,arestrictlypositive.Itfollowsthatthesystem(2.18)ofnormal
equations has a unique solution. Does this solution correspond to a minimum of
EŒ'(cid:4)in(2.17)?Calculustellsusthatforthistobethecase,theHessianmatrixH D
Œ@2E2=@c @c (cid:4) has to be positive definite. But H D 2A, since E2 is a quadratic
i j
function.Therefore,H,withA,isindeedpositivedefinite,andthesolutionofthe
normalequationsgives us the desired minimum. The least squaresapproximation
problemthushasauniquesolution,givenby
Xn
'O.t/D cO (cid:2) .t/; (2.22)
j j
jD1
wherecO DŒcO ;cO ;:::;cO (cid:4)T isthesolutionvectorofthenormalequation(2.18).
1 2 n
Thiscompletelysettlestheleastsquaresapproximationproblemintheory.How
aboutinpractice?
Assumingageneralsetof(linearlyindependent)basisfunctions,wecanseethe
followingpossibledifficulties.
1. The system (2.18) may be ill-conditioned. A simple example is provided by
suppd(cid:5) D Œ0;1(cid:4), d(cid:5).t/ D dt on [0,1], and (cid:2) .t/ D tj(cid:2)1, j D 1;2;:::;n.
j
Then
Z
1 1
.(cid:2) ;(cid:2) /D tiCj(cid:2)2dt D ; i;j D1;2;:::;nI
i j
i Cj (cid:3)1
0
2.1 LeastSquaresApproximation 63
thatis,thematrixAin(2.18)ispreciselytheHilbertmatrix(cf.Chap.1,(1.60)).
The resulting severe ill-conditioning of the normal equations in this example
is entirely due to an unfortunatechoice of basis functions– the powers. These
becomealmostlinearlydependent,moresothelargertheRexponent(cf.Ex.38).
Another source of degradationlies in the elementb D 1(cid:2) .t/f.t/dt of the
j 0 j
right-hand vector b in (2.18). When j is large, the power (cid:2) D tj(cid:2)1 behaves
j
verymuchlikeadiscontinuousfunctionon[0,1]:itispracticallyzeroformuch
of the interval until it shoots up to the value 1 at the right endpoint. This has
the unfortunate consequence that a good deal of information about f is lost
whenoneformstheintegraldefiningb .Apolynomial(cid:2) thatoscillatesrapidly
j j
on [0,1] would seem to be preferable from this point of view, since it would
“engage”thefunctionf morevigorouslyoveralloftheinterval[0,1].
2. The seconddisadvantageis the factthat all coefficientscO in (2.22)dependon
j
n; that is, cO D cO.n/, j D 1;2;:::;n. Increasing n, for example, will give an
j j
enlargedsystemofnormalequationswithacompletelynewsolutionvector.We
refertothisasthenonpermanenceofthecoefficientscO .
j
Bothdefects1and2canbeeliminated(oratleastattenuatedinthecaseof1)
inonestroke:selectforthebasisfunctions(cid:2) anorthogonalsystem,
j
.(cid:2) ;(cid:2) /D0 if i ¤jI .(cid:2) ;(cid:2) /Dk(cid:2) k2 >0: (2.23)
i j j j j
Then the system of normalequationsbecomesdiagonaland is solved immedi-
atelyby
.(cid:2) ;f/
j
cO D ; j D1;2;:::;n: (2.24)
j
.(cid:2) ;(cid:2) /
j j
Clearly, each of these coefficients cO is independent of n, and once com-
j
puted, remains the same for any larger n. We now have permanence of
the coefficients. Also, we do not have to go through the trouble of solv-
ing a linear system of equations, but instead can use the formula (2.24)
directly. This does not mean that there are no numerical problems associ-
ated with (2.24). Indeed, it is typical that the denominators k(cid:2) k2 in (2.24)
j
decrease rapidly with increasing j, whereas the integrand in the numera-
tor (or the individual terms in the case of a discrete inner product) are of
the same magnitude as f. Yet the coefficients cO also are expected to de-
j
crease rapidly. Therefore, cancellation errors must occur when one computes
the inner productin the numerator.The cancellationproblem can be alleviated
somewhatbycomputingcO inthealternativeform
j
!
Xj(cid:2)1
1
cO D f (cid:3) cO (cid:2) ;(cid:2) ; j D1;2;:::;n; (2.25)
j k k j
.(cid:2) ;(cid:2) /
j j kD1
where the empty sum (when j D 1) is taken to be zero, as usual. Clearly, by
orthogonality of the (cid:2) , (2.25) is equivalent to (2.24) mathematically, but not
j
necessarilynumerically.
64 2 ApproximationandInterpolation
AnalgorithmforcomputingcO from(2.25),andatthesametime'O.t/,isas
j
follows:
s D0;
0
for j D1;2;:::;n do
2
1
4cOj D k(cid:2) k2 .f (cid:3)sj(cid:2)1;(cid:2)j/
j
sj Dsj(cid:2)1CcOj(cid:2)j.t/:
ThisproducesthecoefficientscO ;cO ;:::;cO aswellas'O.t/Ds .
1 2 n n
Any system f(cid:2)O g that is linearly independent on the support of d(cid:5) can be
j
orthogonalized (with respect to the measure d(cid:5)) by a device known as the
Gram1–Schmidt2procedure.Onetakes
(cid:2) D(cid:2)O
1 1
and,forj D2;3;::: ,recursivelyforms
Xj(cid:2)1
.(cid:2)O ;(cid:2) /
j k
(cid:2) D(cid:2)O (cid:3) c (cid:2) ; c D :
j j k k k
.(cid:2) ;(cid:2) /
kD1 k k
Theneach(cid:2) sodeterminedisorthogonaltoallprecedingones.
j
2.1.3 Least SquaresError;Convergence
We have seen in Sect. 2.1.2 that if the class ˆ D ˆ consists of n functions (cid:2) ,
n j
j D 1;2;:::;n,thatarelinearlyindependentonthesupportofsomemeasured(cid:5),
thentheleastsquaresproblemforthismeasure,
minkf (cid:3)'k Dkf (cid:3)'Ok ; (2.26)
2;d(cid:5) 2;d(cid:5)
'2ˆn
1Jo´rgen Pedersen Gram (1850–1916) was a farmer’s son who studied at the University of
Copenhagen. After graduation, he entered an insurance company as computer assistant and,
moving up the ranks, eventually became itsdirector. He was interested in series expansions of
specialfunctionsandalsocontributedtoChebyshevandleastsquaresapproximation.The“Gram
determinant”wasintroducedbyhiminconnectionwithhisstudyoflinearindependence.
2Erhard Schmidt (1876–1959), astudent of Hilbert,became aprominent member of theBerlin
SchoolofMathematics,wherehefoundedtheInstituteofAppliedMathematics.Heisconsidered
one of the originators of Functional Analysis, having contributed substantially to the theory of
Hilbertspaces.Hisworkonlinearandnonlinearintegralequationsisoflastinginterest,asishis
contributiontolinearalgebraicsystemsofinfinitedimension.Heisalsoknownforhisproofofthe
Jordancurvetheorem.Hisprocedureoforthogonalizationwaspublishedin1907andtodayalso
carriesthenameofGram.Itwasknown,however,alreadytoLaplace.