Uniform in bandwidth consistency of local polynomial regression function estimators

0.15 MB·English

by  Julia Dony

Uniform in bandwidth consistency of local polynomial
regression function estimators
Julia Dony
Departement Wiskunde, Vrije Universiteit Brussel
6
Uwe Einmahl
0
0 Departement Wiskunde, Vrije Universiteit Brussel
2
David M. Mason
n
a Statistics Program, University of Delaware
J

3 Abstract: We generalize a method for proving uniform in bandwidth con-
2
sistency results for kernel type estimators developed by the two last named

] authors. Such results are shown to be useful in establishing consistency of
T
S local polynomialestimatorsoftheregressionfunction.
.
h Keywords: consistentestimators,kernels, density,regression function,local
t
a polynomialapproach
m
[

1 Introduction
1
v
8
Let X,X ,X ,... be i.i.d. IRd (d 1) valued random variables and assume that the
4 1 2
≥
5 common distribution function of these variables has a Lebesgue density function, which
1
we shall denote by f . A kernel K will be any measurable function which satisfies the
0 X
6 followingconditions:
0
/
h (K.i) K(s)ds = 1, and
at ZIRd
m
: (K.ii) K := sup K(x) = κ < .
Xiv k k∞ x∈IRd| | ∞
The kernel density estimator of f based upon the sample X ,...,X and bandwidth
r X 1 n
a 0 < h < 1 is
1 n x X
f (x) := K − i , x IRd.
n,h nh h1/d ∈
i=1 (cid:18) (cid:19)
X
It is well known that if obne chooses a suitable bandwidth sequence h 0 and the den-
n
→
sity f is continuous, one obtains a strongly consistent estimator f := f of f , i.e.
X n n,hn X
onehaswithprobability1,f (x) f (x),x IRd. Itisalsonatural toinvestigateother
n X
→ ∈ b b
modes of convergence, for instance uniform convergence and to ask what convergence
b
rates are feasible.
For proving such results, one usually writes the difference f (x) f (x) as the sum
n X
−
ofaprobabilistictermf (x) IEf (x)andadeterministictermIEf (x) f (x),theso-
n n n X
− b −
called bias. The order of the bias depends on smoothness properties of f only, whereas
X
b b b
thefirst(random)termcanbestudiedviaempiricalprocesstechniquesashasbeenpointed
out by[14], [15], [16]and[12], amongotherauthors.
[9](seealso[2]forthe1-dimensionalcase)haveshownthatifK isa“regular”kernel,
the density function f is bounded and h satisfies the regularity conditions h 0,
X n n
ց
h /h isbounded,and
n 2n
log(1/h )/loglogn andnh /logn ,
n n
→ ∞ → ∞
onehas withprobability1,
f IEf = O logh /nh , (1.1)
n n n n
k − k∞ | |
(cid:18)q (cid:19)
b b
where denotesthesupremumnormonIRd.Moreover,thisratecannotbeimproved.
||·||∞
Interestingly one does not need continuity of f for this result. (Continuity of f is of
X X
course needed for controlling the bias.) Recently, [6] have provided a “uniform in h”
versionofthisresult,thatis, theyhaveprovedthat
√nh f IEf
limsup sup k n,h − n,hk∞ =: K(c) < . (1.2)
n→∞ clongn≤h≤1 |logbhn|∨logblogn ∞
q
This result implies that if one chooses the bandwidth depending on the data and/or the
locationx, as isusuallydoneinpractice, onehasthesameorderofconvergenceas inthe
case ofadeterministicbandwidthsequence.
Now let Y,Y ,Y ,... be a sequence of r-dimensional random vectors (r 1) so that
1 2
≥
the random vectors (X,Y),(X ,Y ),... are i.i.d. with common joint Lebesgue density
1 1
function f. In this case it is also of great interest to estimate IE[ψ(Y) X = x], where
|
ψ : IRr IR is a suitable mapping. A possible kernel type estimator which reduces to
→
theclassical Nadaraya-Watson estimatorifr = 1,ψ(y) = y,isgivenby
n ψ(Y )K((x X )/h )
m (x,ψ) = i=1 i − i n . (1.3)
n n K((x X )/h )
P i=1 − i n
Likewise by setting in tche 1-dimensionPal case for t IR, ψ (y) = I (y),y IR, we
t ] ,t]
∈ −∞ ∈
obtainthekernel estimatoroftheconditionalempirical function
F(t x) := IP Y t X = x
| { ≤ | }
givenby
n 1(Y t)K((x X )/h )
F (t x) := i=1 i ≤ − i n .
n | n K((x X )/h )
P i=1 − i n
This kernel estimatorbis called the conditional empirical distribution function and was
P
first extensively studied by [17]. Exact convergence rates uniformly on compact subsets
of IRd have been obtained for both Nadaraya-Watson type estimators as in (1.3) and the
conditional empirical distribution function by [5] in the case of deterministic bandwidth
sequences. Recently, [6] have established uniform in bandwidth results for these esti-
mators which are of a similar type as result (1.2). The proof of these results requires
establishinga suitableversionofaresultoftype(1.2) forprocesses oftheform
1 n x X x X
i i
ϕ(Y )K − IE ϕ(Y)K − ,
nh i h1/d − h1/d
i=1(cid:26) (cid:18) (cid:19) (cid:20) (cid:18) (cid:19)(cid:21)(cid:27)
X
where x I (I a compact subset of IRd or I = IRd) and ϕ Φ, where Φ is a suitable
∈ ∈
class offunctions.
For certain applications, however, this class of processes could be too small. One of
the purposes of this paper is to establish such uniform in bandwidth consistency results
for a larger class of processes. As an application of our results, we shall prove uniform
in bandwidth consistency of local polynomial regression estimators. Such estimators are
generalizations of the classic Nadaraya-Watson estimator (see, especially, [7] and [19]).
In Section 2 we will state two general consistency results, one of which will be proved
in Section 3. In Section 4 we treat the local polynomial regression estimators. In an
appendix wegathertogethersomefacts needed inourproofs.
2 General consistency results
Weshallbeginbystatingaresultprovedin[6],whichwillbeinstrumentalinestablishing
uniforminbandwidthconsistencyoflocalpolynomialregressionfunctionestimators. Let
ΦdenoteaclassofmeasurablefunctionsonIRr withafinitevaluedmeasurableenvelope
functionF,
F(y) sup ϕ(y) , y IRr. (2.1)
≥ | | ∈
ϕ Φ
∈
Further assume that Φ is pointwise measurable and satisfies (A.2) in the Appendix with
replaced byΦ.(Forthedefinitionofpointwisemeasurablealsorefer totheAppendix.)
G
Considerthefollowingclassoffunctions
= K((x )/h1/d) : h > 0,x IRd , (2.2)
K −· ∈
n o
and assume that is pointwise measurable and satisfies (A.2) with replaced by .
K G K
Introducetheclassofcontinuousfunctionson acompactsubsetJ ofIRd indexedby Φ:
:= c : ϕ Φ .
ϕ
C { ∈ }
Weshallalwaysassumethattheclass isrelativelycompactwithrespecttothesup-norm
C
topology, which by the Arzela-Ascoli theorem is equivalent to being uniformly bounded
and uniformlyequicontinuous.
For any ϕ Φ and continuous functions c on a compact subset J of IRd, set for
ϕ
∈
x J,
∈
n x X
i
η (x) = c (x)ϕ(Y )K − ,
ϕ,n,h ϕ i
h1/d
i=1 (cid:18) (cid:19)
X
where K is a kernel with support contained in [ 1/2,1/2]d such that (K.i) and (K.ii)
−
hold. The following result was proved in [6], where it is stated as Proposition 2. (
k·kI
denotes thesupremumnormon I.)
Theorem1. LetI beacompactsubsetofIRd suchthatJ = Iη,forsome0 < η < 1. Also
assumethat
f iscontinuousand strictlypositiveon J. (2.3)
FurtherassumethattheenvelopefunctionF of theclassΦ satisfies
M > 0 : F(Y)1 X J M, a.s. (2.4)
∃ { ∈ } ≤
or forsomep > 2
α := supIE[Fp(Y) X = z] < . (2.5)
| ∞
z J
∈
Then wehaveforanyc > 0 and0 < h < (2η)d,with probability1,
0
sup η IEη
limsup sup ϕ∈Φk ϕ,n,h − ϕ,n,hkI =: Q(c) < , (2.6)
n→∞ c(logn/n)γ≤h≤h0 nh(|logh|∨loglogn) ∞
q
where γ = 1 in theboundedcase(2.4)and γ = 1 2/punderassumption(2.5).
−
The next result generalizes Theorem 1 in the bounded case. Its proof is illustrative
of how that of Theorem 1 goes using an empirical process approach based upon an in-
equality of Talagrand coupled with a moment bound for the supremum of the empirical
process. Thesebasictoolsare statedintheAppendix.
In the following, denotes the supremum norm on IRd or IRd+r, whichever
|| · ||∞
is appropriate. Let denote a class of measurable real valued functions g of (u,t)
G ∈
IRd IRr = IRd+r. Weshallassumethat satisfies:
× G
(G.i)sup g =: κ < ;
g∈G || ||∞ ∞
(G.ii)sup g2(x,y)dxdy =: L < .
g∈G IRd+r ∞
Denoteby , thReclassoffunctionsof(s,t) IRd+r formed from as follows:
FG ∈ G
= g(z sλ,t) : λ 1,z IRd and g .
FG − ≥ ∈ ∈ G
n o
Weshallalsoassumethattheclassoffunctions satisfiesthefollowinguniformentropy
FG
condition:
(F.i)forsomeC > 0 andν > 0,N(ǫ, ) C ǫ ν0,0 < ǫ < 1.
0 0 0 −
FG ≤
Finally, to avoidusingouter probabilitymeasures in all ofourstatements,we imposethe
measurabilityassumption:
(F.ii) is apointwisemeasurableclass.
FG
(For the definitions of pointwise measurable and of N(ǫ, ) see the Appendix below,
FG
where weuseκas ourenvelopefunction.)
Foranyg and 0 < h < 1 define,
∈ G
n x X
g (x) := (nh) 1 g − i,Y , x IRd.
n,h − h1/d i ∈
i=1 (cid:18) (cid:19)
X
Theorem2. Assuming(G.i),(G.ii),(F.i),(F.ii),andf (thejointdensityof(X,Y))bounded,
we haveforc > 0 and0 < h < 1,
0
√nh g IEg
n,h n,h
limsup sup sup k − k =: G(c) < . (2.7)
∞
n→∞ clongn≤h≤h0 g∈G |logh|∨loglogn ∞
q
Remark. Theorem 2 is still valid for r = 0. In this case, g : IRd IRd and condition
→
(G.ii)shouldberead as sup g2(x)dx =: L < .
g∈G IRd ∞
R
3 Proof of Theorem 2
Let α be the empirical process based on the sample (X ,Y ),...,(X ,Y ), i.e. if ϕ :
n 1 1 n n
IRd IRr IR, wehave
× →
n
α (ϕ) = (ϕ(X ,Y ) IEϕ(X,Y))/√n.
n i i
−
i=1
X
Noticethatin thisnotation
1
g (x) IEg (x) = α g((x )/h1/d, ) , x IRd
n,h n,h n
− h√n −· · ∈
(cid:16) (cid:17)
so weget that
sup √nhkgn,h −IEgn,hk = sup sup √nαn g hx1−/d·,· ,
∞ (cid:12) (cid:16) (cid:16) (cid:17)(cid:17)(cid:12)
g logh loglogn g x IRd n(cid:12)h( logh loglog(cid:12)n)
∈G | |∨ ∈G ∈ (cid:12) | |∨ (cid:12)
q q
where g((x )/h1/d, ) denotes the function (s,t) g((x s)/h1/d,t). We first note
− · · → −
that by(G.ii)and theassumptionthat f < ,
|| ||∞ ∞
x X x s
IE g2 − ,Y = h h 1g2 − ,t f (s,t)dsdt
−
(cid:20) (cid:18) h1/d (cid:19)(cid:21) ZIRd ZIRr (cid:18) h1/d (cid:19)
h f L.
≤ || ||∞
Set forj 0and c > 0,
≥
h := 2jclogn /n
j,n
(cid:16) (cid:17)
and
= g((x )/h1/d, ) : g ,h h h , x IRd .
j,n j,n j+1,n
F −· · ∈ G ≤ ≤ ∈
n o
Clearly forh h h ,
j,n j+1,n
≤ ≤
x X
IE g2 − ,Y 2h f L =: D h =: σ2 .
h1/d ≤ j,n|| ||∞ 0 j,n j,n
(cid:20) (cid:18) (cid:19)(cid:21)
We shalluse PropositionA.1 in theAppendix to bound IE n ǫ ϕ(X ,Y ) . To
k i=1 i i i kFj,n
that end we note that each satisfies (A.1) of the proposition with G = β = κ and
Fj,n P
(A.3)withσ2 = σ2 . Further, since ,weseeby(F.i) thateach also fulfills
j,n Fj,n ⊂ FG Fj,n
(A.2). Finally(A.4)holdsforlargeenoughnandallj 0. NowbyapplyingProposition
≥
A.1 weget foralllarge enoughnand j 0,
≥
n
IE ǫ ϕ(X ,Y ) D nh log(D h ) , (3.1)
k i i i kFj,n ≤ 1 j,n| 2 j,n |
Xi=1 q
for someD > 0 and D > 0. Let forlargeenoughn
1 2
l := max j : h 2h ,
n j,n 0
{ ≤ }
then alittlecalculationshowsthat
log nh0
l clogn . (3.2)
n ∼ l(cid:16)og2 (cid:17)
Fork 1, setn = 2k, and let
k
≥
c := n h ( logD h loglogn ), j 0.
j,k k j,nk | 2 j,nk|∨ k ≥
q
ApplyingInequalityA.1in theAppendixwith
M = κ and σ2 = σ2 D h ,
G Fj,nk ≤ 0 j,nk
we getforany t > 0,
IP max √nα A (D c +t)
(cid:26)nk−1≤n≤nk|| n||Fj,nk ≥ 1 1 j,k (cid:27)
2 exp A t2/(D n h ) +exp( A t/κ) .
≤ − 2 0 k j,nk − 2
Set foranyρ > 1,j h0 and(cid:16)k 1, (cid:17) i
≥ ≥
p (ρ) := IP max √nα A (D +ρ)c .
j,k (cid:26)nk−1≤n≤nk|| n||Fj,nk ≥ 1 1 j,k(cid:27)
As wehavec / n h √loglogn , wereadily obtainforj 0,
j,k k j,nk ≥ k ≥
q
ρ2A √cρA
2 2
p (ρ) 2 exp loglogn +exp logn loglogn ,
j,k k k k
≤ " − D0 ! − κ q !#
which forγ = A2 √cA2 implies
D0 ∧ κ
p (ρ) 4exp( ργloglogn ).
j,k k
≤ −
Thus
lnk−1
Pk(ρ) := pj,k(ρ) ≤ 4lnk (lognk)−ργ,
j=0
X
which by(3.2), forall largek and largeenoughρ > 1
ργ 1
1 −
Pk(ρ) 8(lognk)1−ργ = 8 k−2.
≤ klog2! ≤
Noticethatby definitionofl ,for largek
n
2h = h 2h ,
lnk,nk lnk+1,nk ≥ 0
which impliesthat wehaveforn n n
k 1 k
− ≤ ≤
clogn clogn
k
,h ,h .
" n 0# ⊂ " nk lnk,nk#
Thus foralllarge enoughk andn n n ,
k 1 k
− ≤ ≤
A (ρ) :=
k
√nh g IEg
n,h n,h
nk−m1≤anx≤nksgu∈Gp clongsnu≤ph≤h0 |lokgh|∨−loglogkn∞ > 2A1(D1 +ρ)
q
 lnk−1 
max √nα A (D +ρ)c .
⊂ j[=0 (cid:26)nk−1≤n≤nk|| n||Fj,nk ≥ 1 1 j,k(cid:27)
It followsnowforlargeenoughρthat
IP A (ρ) P (ρ) k 2,
k k −
{ } ≤ ≤
which bytheBorel-Cantelli lemmaimpliesourtheorem. 2
4 Application to local polynomial regression function es-
timators
In this section we shall always assume that the assumptions of Theorem 1 hold (in par-
ticular,that K hassupportcontainedin[ 1/2,1/2])and I isa fixedcompact intervalin
−
IR. Weshallalsoassumethat K 0.
≥
4.1 Estimating the regression function by local polynomials
Let (X,Y),(X ,Y ),...,(X ,Y ) bei.i.d. 2-dimensionalrandom vectorsandwrite
1 1 n n
g(x) := IE[Y X = x]
||
for the regression function. Suppose that g(x) is (p+1) times differentiable on J = Iη,
thenwecanapproximateg(x)locallyaroundx I byapolynomialoforderp(Taylor):
0
∈
g(p)(x )
g(x) g(x )+g (x )(x x )+...+ 0 (x x )p.
0 ′ 0 0 0
≈ − p! −
Then considertheweightedleast-squares regressionproblem(WLS)
1 n p x X
argmin [Y β (X x )j]2K 0 − i . (4.1)
βββ IRp+1 nh i − j i − 0 h
∈ i=1 j=0 (cid:18) (cid:19)
X X
It is clear that if βββˆˆˆ IRp+1 is the solution of the WLS problem in (4.1), we obtain an
∈
estimatorgˆ(p)(x ) of g(x ) by takingitbe βˆ , thefirst componentofβββˆˆˆ. At thesametime
n,h 0 0 0
we obtain estimators of the derivativesof the regression function up to order p. To solve
(4.1), first notethatitcan bewrittenin amatrixnotation:
argmin (YYY XXX βββ)tWWW (YYY XXX βββ), (4.2)
βββ IRp+1 − x0 x0 − x0
∈
whereWWWx0 = (nh)−1diag K x0−hXi ∈ IRn×n, andXXXx0 ∈ IRn×(p+1),YYY ∈ IRn×1 and
βββ IR(p+1) 1 are defined as(cid:16) (cid:16) (cid:17)(cid:17)
×
∈
1 (X x ) (X x )p Y β
1 0 1 0 1 0
. −. ··· −. . .
XXXx0 :=  .. .. .. , YYY :=  .. , βββ :=  .. .
1 (Xn x0) (Xn x0)p Yn βp
 − ··· −     
     
If weset
1 n p x X
L(x ) := [Y β (X x )j]2K 0 − i ,
0 i j i 0
nh − − h
i=1 j=0 (cid:18) (cid:19)
X X
it isnottoo difficulttoseethatfork = 0,...,p, thepartial derivativescan bewrittenas
∂L(x )
0 = 2(YYY XXX βββ)tWWW XXX eeet,
∂β − − x0 x0 x0 k
k
where eee is the k-th unit vector in IRp+1. So by setting the partial derivatives equal to
k
zero, weobtainthatthesolutionβββˆˆˆoftheWLSproblem(4.1)mustsatisfy
YYYtWWW XXX =βββˆˆˆtXXXt WWW XXX .
x0 x0 x0 x0 x0
Assumingthat
:=XXXt WWW XXX ,
Sx0 x0 x0 x0
is invertible,wecan computethesolutionby
βββˆˆˆ = XXXt WWW XXX −1XXXt WWW YYY.
x0 x0 x0 x0 x0 x0
(cid:16) (cid:17)
We shall show that asymptotically the inverse matrix of always exists. To see this,
Sx0
considerfor0 j 2p thefunctions
≤ ≤
H(j)(u) := ( u)jK(u).
−
Since we assume K to be bounded with support contained in [ 1/2,1/2], we see that
−
each H(j) L (IR)andhassupportcontainedin[ 1/2,1/2]. Nowforeachj 0define
1
∈ − ≥
theboundedfunction
φ (u) = ( u)j1 u [ 1/2,1/2] .
j
− { ∈ − }
Since thisfunctionisofboundedvariation,theclass
φ ((x )/h) : h > 0,x IR
j
{ −· ∈ }
satisfies (A.2). (See Lemma22 of [11].) Thus the class , as defined in (2.2) is assumed
K
to bepointwisemeasurableand satisfies(A.2). By LemmaA.1 intheAppendix,foreach
j = 0,...,2p,theclass
:= H(j)((x )/h) : h > 0,x IR
j
G −· ∈
n o
also fulfills (A.2). Moreover, it is easily checked that each is pointwise measurable.
j
G
HencetheassumptionsofTheorem2holdandwecaninferthatforeach0 j 2p,and
≤ ≤
sequence a satisfying
n
a 0 and na /logn , (4.3)
n n
ց → ∞
we have
sup sup H(j)(x ) IEH(j)(x ) 0, a.s., (4.4)
n,h 0 − n,h 0 −→
x0∈Ian≤h≤h0(cid:12) (cid:12)
(cid:12) (cid:12)
where (cid:12) (cid:12)
1 n x X
H(j)(x ) := H(j) 0 − i .
n,h 0 nh h
i=1 (cid:18) (cid:19)
X
Noticethat
1 x t
IEH(j)(x ) = H(j) 0 − f(t)dt =: f H(j)(x ),
n,h 0 h h ∗ h 0
ZIR (cid:18) (cid:19)
and sincef iscontinuouson J = Iη withI beingacompactinterval,wecan useLemma
A.2 intheAppendixtoget thatas h 0,
ց
sup IEH(j)(x ) f(x ) ( u)jK(u)du 0. (4.5)
n,h 0 − 0 − −→
x0 I(cid:12) ZIR (cid:12)
∈ (cid:12) (cid:12)
(cid:12) (cid:12)
Hence,itfollowsimme(cid:12)diatelyby(4.4)and(4.5),thatuniformly(cid:12) inx I andfora < b
0 n n
∈
with a satisfying(4.3) and b 0,
n n
ց
sup H(j)(x ) f(x ) ( u)jK(u)du 0, a.s. (4.6)
n,h 0 − 0 − −→
an h bn(cid:12) ZIR (cid:12)
≤ ≤ (cid:12) (cid:12)
(cid:12) (cid:12)
Next consider the Hilb(cid:12)ert space (IR,Kdλ) consisting of(cid:12)all the measurable functions
L
φ : IR IR such that
→
φ2(u)K(u)du < .
∞
ZIR
As usual, φ = φ if (φ φ )2(u)K(u)du = 0; that is, each φ (IR,Kdλ) repre-
1 2 IR 1 − 2 ∈ L
sents an equivalenceclass offunctions. Nowlet
R
p,p
G := ( u)j+kK(u)du ,
−
(cid:18)ZIR (cid:19)j=0,k=0
then G istheGramian matrixofthesetoffunctions ϕ : ϕ (x) = ( x)j,j = 0,...,p
j j
{ − }
and thesefunctionsbelong to (IR,Kdλ) sinceK has compact support. It is knownthat
L
G is nonsingular if the functions are linearly independent. Hence, in our case, G will
always be invertible. (Here we use K 0 and 0 < K(u)du < .) To see that
≥ IR ∞
R
is invertible as well, recall that the function M detM with M (IR) is
Sx0 → ∈ Mp+1
continuous,andthat by(4.4) and(4.5),with probabilityone,thecomponentsof
(j+k) p,p
A := H (x ) ,
x0 n,h 0 j=0,k=0
(cid:16) (cid:17)
converge uniformly in x I and a h b with b 0 to those of f(x )G. Hence,
0 n n n 0
∈ ≤ ≤ ց
since we assume f to be strictly positive on J = Iη, for n large enough, uniformly in
x I, wehavedetA > 0. Nowlet := diag 1,h,...,hp ,notethat
0 ∈ x0 Hp { }
= A ,
Sx0 Hp x0Hp
and observe that det = hp(p+1)detA , so for n large enough, uniformly in x I
Sx0 x0 0 ∈
and a h b , will have a positivedeterminant, showingthat asymptotically,
n ≤ ≤ n Sx0 Sx0
is nonsingularand invertible.
Fromtheaboveitfollowsthatwithprobabilityone,foralllargen,uniformlyinx I
0
∈
and a h b , thelocal polynomialregressionestimatorofg(x ) isgivenby
n n 0
≤ ≤
gˆ(p)(x ) =eee 1XXXt WWW YYY.
n,h 0 1Sx−0 x0 x0
Thedifficultyistodetermine 1 explicitly,especiallywhenpbecomeslarge. Moreover,
Sx−0
it is not possible to find a nice general formula for gˆ(p)(x ), since the calculation of 1
n,h 0 Sx−0
and gˆ(p)(x ) becomes more complex as p increases. However, we shall see in the next
n,h 0
section thatgˆ(p)(x ) can beeasilycomputedforp = 0,1,2.
n,h 0
4.2 Uniform in bandwidth consistency
We shall now discuss uniform in bandwidth consistency of gˆ(p) on a compact interval I.
n,h
Define thefunctions
1 n X x j x X
˜ i i
f (x) := − K − , j = 0,...,2p,
n,h,j
nh h h
i=1(cid:18) (cid:19) (cid:18) (cid:19)
X
1 n X x j x X
i i
r˜ (x) := Y − K − , j = 0,...,p.
n,h,j i
nh h h
i=1 (cid:18) (cid:19) (cid:18) (cid:19)
X
By Theorem2,
√nh f˜ IEf˜
n,h,j n,h,j
limsup sup max − I < , a.s.
n→∞ clongn≤h≤h00≤j≤2p |l(cid:13)(cid:13)(cid:13)ogh|∨loglogn(cid:13)(cid:13)(cid:13) ∞
q
and by Theorem1 withobviousidentificationsandK replaced byH(j),
r˜ IEr˜
limsup sup max k n,h,j − n,h,jkI < , a.s.
n→∞ c(logn/n)γ≤h≤h00≤j≤p nh(|logh|∨loglogn) ∞
q