Conditional least squares estimation in nonstationary nonlinear stochastic regression models

0.35 MB·English

by  Christine Jacob

TheAnnalsofStatistics
2010,Vol.38,No.1,566–597
DOI:10.1214/09-AOS733
(cid:13)c InstituteofMathematicalStatistics,2010
CONDITIONAL LEAST SQUARES ESTIMATION IN
NONSTATIONARY NONLINEAR STOCHASTIC
0
REGRESSION MODELS
1
0
2 By Christine Jacob1

n
National Agronomical Research Institute (INRA)
a
J
Let {Zn} be a real nonstationary stochastic process such that
13 E(Zn|Fn−1)a<.s.∞andE(Zn2|Fn−1)a<.s.∞,where{Fn}isanincreas-
ingsequenceofσ-algebras.AssumingthatE(Zn|Fn−1)=gn(θ0,ν0)=
] gn(1)(θ0)+gn(2)(θ0,ν0), θ0∈Rp, p<∞, ν0∈Rq and q≤∞, we study
ST theasymptoticpropertiesofθbn:=argminθPnk=1(Zk−gk(θ,νb))2λ−k1,
where λk is Fk−1-measurable, ν={νk} is a sequence of estimations
h. of ν0, gn(θ,ν) is Lipschitz in θbandbgn(2)(θ0,ν)−gn(2)(θ,ν) is asymp-
b b b
at toticallynegligiblerelativetogn(1)(θ0)−gn(1)(θ).Wefirstgeneralizeto
m thisnonlinearstochasticmodelthenecessaryandsufficientcondition
[ obtained for the strong consistency of {θbn} in the linear model. For
that,weproveastronglawoflargenumbersforaclassofsubmartin-

1 gales. Again using this strong law, we derive the general conditions
v leading to the asymptotic distribution of θbn. We illustrate the the-
2 oretical results with examples of branching processes, and extension
0 to quasi-likelihood estimators is also considered.
1
2
1. 1. Introduction. Let{Zn}n∈N beanobservedone-dimensionalrealstochas-
00 tkincopwrnocpeassradmeefitneerd(θon,νa)p,rθobabΘilityRspp,ac0e<(Ωp,<F,P,θ0ν,ν0) depenRdqe,n0t onqa un-,
0 0 0 0
1 ∈ ⊂ ∞ ∈N ⊂ ≤ ≤∞
and assumed to satisfy
:
v
Xi MZ: ∀n Eθ0,ν0(Zn|Fn−1) = gn(θ0,ν0)=gn(1)(θ0)+gn(2)(θ0,ν0),
a.s.
r E (Z2 ) < ,
a θ0,ν0 n|Fn−1 ∞
Received June 2008; revised July 2009.
1Supportedin part by theFrench cooperation network ECONET.
AMS 2000 subject classifications. Primary 62M10, 62J02, 62F12, 62M05, 62M09,
62P05, 62P10; secondary 60G46, 60F15.
Key words and phrases. Stochasticnonlinearregression,heteroscedasticity,nonstation-
ary process, time series, branching process, conditional least squares estimator, quasi-
likelihood estimator, consistency, asymptotic distribution, martingale difference, sub-
martingale, polymerase chain reaction.
This is an electronic reprint of the original article published by the
Institute of Mathematical Statistics in The Annals of Statistics,
2010,Vol. 38, No. 1, 566–597. This reprint differs from the original in pagination
and typographic detail.
1
2 C. JACOB
where is an increasing sequence of σ-algebras depending only on ob-
n
{F }
served processes, θ is a unknown parameter that we want to estimate, ν is
0 0
(2)
a nuisanceparameter defined,when q= ,by ν = ν with g (θ ,ν )=
0 0n n 0 0
∞ { }
(2) (1)
g (θ ,ν ), g (θ ) is the -measurable parametric part of the model
n 0 0n n 0 n−1
F
(2)
that may be nonlinear in θ , and g (θ ,ν ) is -measurable and such
0 n 0 0 n−1
F
(2) (2) (1)
that g (θ ,ν ) g (θ,ν )isasymptotically negligiblerelative to g (θ )
n 0 0 n 0 n 0
− −
(1)
g (θ) (definition given in Section 4). The simplest example of asymptotic
n
(2)
negligibility is when g (θ ,ν ) is independent of θ . The case q=0 is de-
n 0 0 0
(2)
finedbyg (θ ,ν )=0,foralln,andcorrespondstotheclassicalparametric
n 0 0
setting.
Examples of models M are nonlinear regression models with random
Z
covariates and heteroscedastic variances, stochastic dynamical models in
discrete time, nonlinear time series model (TARMA, SETAR, bilinear pro-
cesses), financial models (ARCH, GARCH and others) and branching pro-
cesses, provided that the first two conditional moments at each n of all of
these processes are finite. This means, in particular, that processes with a
heavy-tailed distribution(see[10]forsuchanexample) donotbelongtothis
class. However, a solution when E (Z )a<.s. with σ2(θ ,ν )a=.s.
θ0,ν0 n|Fn−1 ∞ n 0 0
, where σ2(θ ,ν ) := E ([Z E (Z )]2 ), could be to
∞ n 0 0 θ0,ν0 n − θ0,ν0 n|Fn−1 |Fn−1
deal with the truncated process Z :=Z 1 , where lim I a=.s.R since
n n {Zn∈In} n n
by defining η :=Z E (Z ) and g (θ ,ν ):=E (Z ),
n n − θ0,ν0 n|Fn−1 n 0 0 θ0,ν0 n|Fn−1
(1) e (2)
we can then define g (θ )=E (Z ) and g (θ ,ν )= E
(Z 1 e ).e n 0 eθ0,ν0 n|Fn−1e n 0 0 −e θ0,ν0 ×
n {Zn∈/In}|Fn−1
We consider the class of weighted CLSE (conditional least squares esti-
e e
mators) of θ in the approximate model g (θ ,ν) , where ν = ν , ν
0 k 0 k≤n n n
{ } { }
being any estimation of ν based on observations up to n. We will consider
0
two different settings:
b b b b
A1: n g (θ,ν)=g (θ,ν ) or g (θ,ν)=g (θ,ν );
∀ n n n n n n0
A2: n, k n g (θ,ν)=g (θ,ν ).
k k n
∀ ∀ ≤ b b b b
Such an estimator is defined by
b b
n
(1.1) θn:=argminSn|νb(θ), Sn|νb(θ):= (Zk−gk(θ,ν))2λ−k1,
θ∈Θ
k=1
X
where λb is an -measurable variable independent ofb(θ ,ν ). When
k k−1 0 0
F
g (θ,ν) has a first derivative g′(θ,ν) in θ, (1.1) implies that θ is an es-
k k n
timating equations estimator (EEE), that is,
b b b
(1.2) Qn|νb(θn; Zk,ak(θ) )=0p×1, ak(θ) k−1-measurable ,
{ } { F }
b
ESTIMATIONIN STOCHASTIC REGRESSION 3
n
(1.3) Qn|νb(θ; Zk,ak(θ) ):= (Zk gk(θ,ν))ak(θ),
{ } − −
k=1
X
where, here, a (θ)=g′(θ,ν)λ−1 for all k. b
k k k
In the classical parametric setting q=0, the weighted CLSE’s and, more
generally, the EEE’s, are well studied and are known to have interesting
b
properties. These estimators are robust to the form of distribution of the
respective residuals Z g (θ ,ν ) since they require at most the knowl-
n n 0 0
{ − }
edge of the first two conditional moments of the process at each time n and
their computation may beachieved, even in thecase of complex or unknown
likelihoods. When σ2(θ ,ν ) is an explicit function of (θ ,ν ), such an es-
n 0 0 0 0
timator may be used to derive the empirical distribution of the estimated
residuals [Z g (θ ,ν)][σ (θ ,ν)]−1 , thereby allowing a re-estimation of
k k n k n
{ − }
θ by a maximum likelihood estimator (MLE) when the theoretical distri-
0
bution may be modbelebd by abfubnction of θ0, provided that the estimator
is close enough to θ and is therefore strongly consistent [26]. In the par-
0
ticular setting σ2(θ ,ν )=σ2(θ ,ν )λ , where λ is -measurable and
n 0 0 0 0 n n Fn−1
independent of (θ ,ν ), the optimal CLSE of θ , from the point-of-view of
0 0 0
the asymptotic variance, is obtained by making the errors of the model sta-
tionary, that is, by minimizing n (Z g (θ,ν ))2λ−1, and is equal to
k=1 k − k 0 k
the optimal EEE, called the quasi-likelihood estimator (QLE) (see [15] for
P
the optimality of the convergence rate in the branching process setting and
[7] for the QLE). In the general case, if g (θ,ν ) has a first derivative in θ,
n 0
then a possible estimator is obtained by replacing σ2(θ ,ν ) by σ2(θ,ν) in
k 0 0 k
Qn|νb(θ;{Zkσk−1(θ0,ν0),gk′(θ,ν)σk−1(θ0,ν0)}). When q=0, the obtained esti-
mator is the QLE. This estimator is optimal from its asymptotic varibance
point-of-view within the class of estimators which solve (1.2) and (1.3), [7],
b
and is moreover equal to the maximum likelihood estimator (MLE) when
the conditional distribution of Z belongs to an exponential family at each
n
n [32]. Another possible estimator is the weighted CLSE defined by (1.1),
where λ is a sequence of -measurable estimators of σ2(θ ,ν ) up
{ k} Fk−1 { k 0 0 }
to a multiplicative constant. Since
σ2(θ ,ν ) n (Z g (θ,ν))2λ−1 σ2(θ ,ν )
min k 0 0 < k=1 k− k k <max k 0 0 ,
k≤n λk nk=P1(Zk−gk(θ,ν))2σk−2(θ0,ν0) k≤n λk
b
if λ is such that P
n
{ } b
a.s. a.s.
(1.4) 0 < limσ2(θ ,ν )λ−1 limσ2(θ ,ν )λ−1 < ,
n n 0 0 n ≤ n n 0 0 n ∞
then the asymptotic behavior of n (Z g (θ,ν))2λ−1, and therefore
k=1 k − k k
of its argmin, should be close to that of n (Z g (θ,ν))2σ−2(θ ,ν )
P k=1 k − k k 0 0
b
P
b
4 C. JACOB
and of its argmin. Finally, if F denotes the set of -measurable vari-
n−1 n−1
F
ables, since E (Z ) is thebestpredictor of Z based on F in the
θ0,ν0 n|Fn−1 n n−1
least squares sense because Ebθ0,ν0(Zn|Fn−1) = argming∈Fbn−1Eθ0,ν0((Zn −
g)2λ−1 ), provided that λ is an -measurable variablbe indepen-
n |Fn−1 n Fn−1
dent of g [9], a weighted CLSE should easily be strongly consistent. Since
we are particularly interested in such a property, which is necessary when
accurate knowledge of the true parameter is required, we will focus here on
the asymptotic properties (strong consistency, asymptotic distribution), as
n , of the weighted CLSE solution of (1.1) in the general setting M
Z
→∞
with the weakest possible assumptions on the process behavior, condition-
ally on ν . However, the results could easily be generalized to the QLE
{ }
when a primitive of the estimating functions exists (see Section 9).
From nbow on, to simplify notation when studying θn which solve (1.1),
{ }
−1/2
we will use the normalized process Y :=Z λ and denote by M the as-
n n n Y
sumptions concerning Y when Z verifies M . Mbore precisely, defining
n n Z
{ } { }
(1) (1) −1/2 (2) (2) −1/2
f (θ ):=g (θ )λ and f (θ ,ν ):=g (θ ,ν )λ , we have
n 0 n 0 n n 0 0 n 0 0 n
M : n E (Y ) = f (θ ,ν )=f(1)(θ )+f(2)(θ ,ν ),
Y ∀ θ0,ν0 n|Fn−1 n 0 0 n 0 n 0 0
a.s.
E (Y2 ) <
θ0,ν0 n|Fn−1 ∞
and η :=Y E (Y ) is a martingale difference. Equations (1.1)
n n− θ0,ν0 n|Fn−1
and (1.4) are now written, respectively,
n
2
(1.5) θn=argminSn|νb(θ), Sn|νb(θ):= (Yk fk(θ,ν)) ,
−
θ∈Θ
k=1
X
b a.s. a.s. b
(1.6) 0<limσ2 limσ2 < , σ2 :=E(η2 ).
n n ≤ n n ∞ n n|Fn−1
Among published works on the estimator consistency in M , only the case
Y
q=0 is considered and two large classes of proofs exist. One class is based
on the stationarity and ergodicity assumptions of the process [26], on the
strongest assumption of independence of the errors (classical regression) or
on the explicit expression of the estimator according to the process together
with the knowledge of its asymptotic behavior. It is, in particular, the case
of a branching process when the corresponding model is linear in θ [8,
0
35]. The other approach is based on the (much more general) martingale
difference property of η . Here, we are interested in this second class,
n
{ }
which is particularly usefulfor processes. When dealing with the parametric
linear model f (θ )=θTW , where W is either a deterministic vector and
n 0 0 n n
a.s.
η are i.i.d. [20] or W is stochastic with p=1 [21], then lim θ = θ if
n n n n 0
{ }
b
ESTIMATIONIN STOCHASTIC REGRESSION 5
and only if
n
(1.7) limλ W WT a=.s. ,
n min( k k) ∞
k=1
X
λ n W WT being the smallest eigenvalue of n W WT. Defin-
min{ k=1 k k} k=1 k k
ing
P P
n
(1.8) D (θ):= [d (θ)]2, d (θ):=f (θ ) f (θ),
n k k k 0 k
−
k=1
X
a.s.
(1.7) is equivalent to lim D (θ) = for all θ =θ . This quantity is the
n n 0
∞ 6
identifiability criterion of θ in the model. It is interesting to observe that
0
a.s.
θ cannotbeconsistent,orevenweaklyconsistent,ontheset lim D (θ) <
n n n
{ } {
.
∞}
bHowever, in the general nonlinear stochastic setting M with q=0, un-
Y
der some Lipschitz property of the model, all published theorems of con-
a.s.
sistency require, besides the condition lim σ2 < and a condition of the
a.s. n n ∞
type lim D (θ ) = for some sequence θ Θ θ (depending on the
n n n n 0
∞ { }∈ \
author), additional conditions concerning some rate of convergence to
∞
of D () . Meoreover, these conditions diffeer from one author to another
n
{ · }
([1, 16, 18, 21–23, 31, 33, 36]; see [31] or [14] for some examples of models
that do not verify these additional conditions).
Here, we generalize the necessary and sufficient condition (1.7) to our
generalnonstationarynonlinearstochasticmodelM with0 q .When
Y
≤ ≤∞
q=0, we prove the strong consistency of θ on the set
n
{ }
(1.9) LIP ( f (θ) ) SI ( D (θ) ) VAR ( σ2,d (θ),D (θ) ),
θ { k } ∩ θ { n } ∩ b θ { k k k }
where, in the following, “ δ>0” means “ δ>0 small enough” and
∀ ∀
LIP ( f (θ) ) is the set of trajectories satisfying the following Lipschitz
θ k
• { }
condition: for all k, there exists a nonnegative -measurable function
k−1
F
g and a function h():R+ R+ with lim h(x)=0 such that for all
k xց0
· → a.s.
θ Θ,θ Θ, f (θ ) f (θ ) h( θ θ )g , where is any norm
1 2 k 1 k 2 1 2 k
∈ ∈ | − | ≤ k − k k·k
in Rp;
a.s.
VAR ( σ2,d (θ),D (θ) )thatgeneralizes lim σ2 < (provedinSection
• θ { k k k } n n ∞
5) is the set
∞
a.s.
(1.10) δ>0, sup σ2[d (θ)]2[D (θ)]−2 < ;
∀ k k k ∞
( kθ−θ0k≥δk=1 )
X
6 C. JACOB
SI ( D (θ) ) concerns the identifiability condition generalizing (1.7):
θ n
• { }
δ>0 inf D (θ) is -measurable,
n n−1
∀ kθ−θ0k≥δ F
a.s.
lim inf D (θ) = .
n
n kθ−θ0k≥δ ∞
The same terms LIP (), VAR (), SI () will indicate both the set of tra-
θ θ θ
· · ·
jectories and the corresponding conditions verified by these sets. The result
is then generalized to the setting 0 q , replacing, in each condition of
≤ ≤∞
(1)
(1.9), f (θ)by f (θ).Thisconsistency resultisduetoanoriginal SLLNSM
k k
(strong law of large numbers for submartingales). In addition, we show that
the asymptotic distribution of the CLSE is easily derived from a classi-
cal CLT (central limit theorem), thanks to this SLLNSM. Therefore, this
SLLNSM is the key result of this work.
The paper is organized in the following way. In Section 2, we give some
examples of processes Z satisfying M . We deal with the consistency of
n Z
{ }
θ in Section 3 when q=0, and in Section 4 in the more general setting
n
{ }
0 q . This result is obtained thanks to an SLLNSM that is proved in
≤ ≤∞
Sbection 5 using submartingale properties [9], analytical lemmas and Wu’s
lemma concerning the consistency of estimators minimizing a contrast [34].
The consistency result obtained in the general setting 0 q shows
≤ ≤∞
the robustness of this property with respect to the chosen model since, if
θ is strongly consistent in a given model, then it is strongly consistent
n
{ }
in every model “close,” from the identifiability point-of-view, to this given
mbodel.
In Section 6, we give general conditions for obtaining the asymptotic dis-
tributionof θ fromtheclassical CLTformartingales orforrandomsums.
n
{ }
As in the classical nonlinear deterministic regression model [34], the proof is
based on thebTaylor series expansion of ∂Sn|νb(θ)/∂θ, where the convergence
to 0 of the remaining term of the Taylor series is a direct consequence of the
SLLNSM.
In Section 7, we estimate the part of ν involved in the asymptotic dis-
0
tribution of θ and give conditions for its consistency.
n
{ }
InSection8,wegivesomeexamplesinthesingle-typebranchingprocesses
field. These pbrocesses model population dynamics. The population size N
n
at n is defined by N = Nn−1X , where the offspring sizes X , given
n i=1 n,i { n,i}i
, are i.i.d. with mean m (F ) and variance σ2 (F ), F
Fn−1 P θ0,ν0 n−1 θ0,ν0 n−1 n−1
denoting the set of random variables involved in , and being
n−1 n−1
F F
generated by N and possibly environmental processes until n. Es-
k k≤n−1
{ }
timation in this field is well understood in the framework of a BGW (Bi-
enaym´e–Galton–Watson) process m (F )=m , σ2 (F )=σ2 or a
θ0,ν0 n−1 0 θ0,ν0 n−1 0
derived process (BGW with immigration, controlled branching processes),
ESTIMATIONIN STOCHASTIC REGRESSION 7
assuming a linear model in θ , and are predominantly based on the observa-
0
tionofthesizeN ofthepopulationateach timen.Theestimators aremost
n
often moment estimators, MLE, CLSE, QLE. The asymptotic properties as
n are derived on the nonextinction set from the explicit expression
→∞
for the estimator according to N using the asymptotic behavior of the
n
{ }
process, when suitably normalized. An overview of the references may be
found in [8, 35], or in [25] for additional references for multitype processes.
Much more difficult is the study in the nonlinear case when there is
no explicit expression for the estimators. The MLE is generally not eas-
ily computed because of large combinatorial terms. However, if we write
Y :=N N−1/2=m (F )N1/2 +η , where η =N−1/2 Nn−1(X
n n n−1 θ0,ν0 n−1 n−1 n n n−1 i=1 n,i−
m (F )), then η is approximately normally distributed on the nonex-
θ0,ν0 n−1 n P
tinction set as soon as n is large enough.Consequently, theCLSEwith λ
n
{ }
satisfying (1.4) is approximately equal to the MLE on this set.
Therefore,we began to studytheCLSEof θ in size-dependentbranching
0
processes [15, 24] and in regenerative branching processes [16]. The results
presented in this paper improve on, and generalize, these results. The first
example is a supercritical single-type BGW process, where the offspring
mean m is estimated. We give the asymptotic properties of m . The re-
0 n
sults are well known [3, 8], but the indirect form of proof given here for the
consistency does not require accurate knowledge of the asymptotic behav-
b
ior of the process, as is the case in the classical direct proof based on the
analytical expression of the estimator. We then deal with some near-critical
size-dependent branching processes which model the amplification process
in the polymerase chain reaction setting [27]. In this setting, we prove the
strong consistency and asymptotic distribution of the CLSE of the parame-
ters of the replication probability for each model. The more complex model
contains an explosive part tending to , a persistent bounded part and a
∞
transient part. In this model, despite these three different kinds of behavior,
we prove the consistency and the asymptotic normality of the CLSE of a
bidimensional parameter with components belonging to the persistent part
and the transient one.
Finally, in Section 9, we extend the consistency conditions to estimators
solving (1.2), (1.3).
In the following sections, we will simply write E() for E ().
· θ0,ν0 ·
2. Examples of models. Here, we give some examples of classical models
satisfying M .
Z
2.1. Regression models. TheobservedvariableZ isexplainedbyapara-
n
metric regression function g (θ ) of a random (or deterministic) vector of
n 0
observed covariates or coprocesses X and the residuals Z g (θ )
k k≤n n n 0
{ } { − }
8 C. JACOB
are assumed independent with finite variances. The time regression model
with d=1 and X =n is a simple example.
n,l
The general class M allows the extension of these models to g (θ ,ν )
Z n 0 0
and nonindependent residuals such as the following example: Z =g (θ )+
n n 0
ǫ ǫ . Sincewe may write ǫ as a function of Z g (θ ) n−1,ǫ , this
n−1 n n−1 {{ k− k 0 }k=1 0}
model belongs to the M class.
Z
2.2. Financial time series. The most well known of these are the ARCH
model introduced by Engle [5] and the more general GARCH models. Let
ξ =s (θ )U , wherethe U are i.i.d. (0,1), s (θ ) 0, is generated
n n 0 n n n 0 n−1
by ξ2 ands2(θ )is{ } -measurablewiths2(θ≥)=αF+ q α ξ2 +
{ k}k≤n−1 n 0 Fn−1 n 0 j=1 j n−j
p β s2 (θ).Theprocess s (θ ) iscalledvolatility.ThenE(ξ2 )=
j=1 j n−j { n 0 } P n|Fn−1
s2(θ ) and ξ follows a GARCH(p,q) model. If ξ is observed, then Z
Pn 0 { n} { n} n
is defined in the following way: Z =ξ2 =s2(θ )+s2(θ )(U2 1), implying
n n n 0 n 0 n−
that g (θ )=s2(θ ), σ (θ ,ν )=s2(θ )[E((U2 1)2 )]1/2. More gener-
n 0 n 0 n 0 0 n 0 n− |Fn−1
ally, ξ may be nondirectly observed [14, 26].
n
{ }
2.3. Linear or nonlinear time series models that may depend on nonsta-
tionary exogenous inputs u , such as ARMAX models. We define e :=
n n
{ }
Z g (θ ), where g (θ)= p α Z + q β e + b γ u .
n − n 0 n k=1 k n−k k=1 k n−k k=0 k n−k
Here, is generated by Z , u .
n−1 k k≤n−1 k k≤n
F {P} { } P P
2.4. Single-type discrete-time branching processes. This class of models
is described in Sections 1 and 8.
2.5. Modelswithobservationerrors. Inpractice,themodelmaybeZth=
n
gth(θ )+e , Z =Zth+u , where Z is the observation of the theoretical
n 0 n n n n n
unknown variable Zth, u is the observation error with a unknown distri-
n n
bution and gth(θ )=E(Zth th ). Then, assuming that E(u )=0,
n 0 n |Fn−1 n|Fn−1
we have g (θ ,ν )=E(gth(θ ) ). For example, consider the BGW pro-
n 0 0 n 0 |Fn−1
cess Nth with N =Nth+u , fth(m)=m(Nth )1/2.Then,usingthe first
{ n } n n n n n−1
order Taylor series expansion, we get
f (m ,ν ):=E(fth(θ ) )
n 0 0 n 0 |Fn−1
=mE([N u ]1/2 )
n−1 n−1 n−1
− |F
=mN1/2 +mO(E(u2 )N−3/2),
n−1 n−1|Fn−1 n−1
where ν = mE(u2 ) , f(2)(m,ν)=mO(E(u2 )N−3/2). The
{ n−1|Fn−1 } n n−1|Fn−1 n−1
(2)
asymptotic negligibility of f (m,ν) is obtained on the nonextinction set
n
{ } a.s.
for any bounded sequence ν since lim N = on this set.
n n
∞
b
b
ESTIMATIONIN STOCHASTIC REGRESSION 9
2.6. Multivariate stochastic regression models. Estimation in this field
may also be expressed as a finite fixed set of one-dimensional models be-
longing to one of the previous types. Let us assume that Z Rd with
k
∈
E(Z )=g (θ ,ν ) and let Σ be a known -measurable positive
n n−1 n 0 0 n n−1
|F F
definiteestimation oftheconditional variance-covariance matrixof Z upto
n
someunknownmultiplicative constant.Thenθn=argminθ∈ΘSn|νb(θ),where
n
Sn|νb(θ):= (Zk−gk(θ,ν))TΣ−kb1(Zk−gk(θ,ν)).
k=1
X
Since Σ is positive definite, we may wbrite Σ =U Λ U−1b, where U is an
k k k k k k
orthogonal matrix and Λ is the diagonal matrix of the eigenvalues of Σ .
k k
Therefore, writing Y =Λ−1/2U−1Z , where Y =(Y ,...,Y )T, we get
k k k k k k,1 k,d
f (θ ,ν)=Λ−1/2U−1g (θ ,ν), Y E(Y ) is a martingale difference
k 0 k k k 0 k− k|Fk−1
and
b b d n
2
(2.1) Sn|νb(θ)= (Yk,j fk,j(θ,ν)) .
−
j=1k=1
XX
b
3. Consistency in MY with q =0. In this section, we generalize (1.7)
(1)
to the setting of the model M under q=0. So, we have d (θ)=d (θ). Let
Y k k
us assume that θ Θ, where Θ is an open set and Θ is compact. From now
0
∈
on (in all sections), let γ Bc := γ: γ γ δ , where γ may be any
{ ∈ δ} { k − 0k≥ }
subset of (θ,ν).
Proposition 3.1. Let Ω Ω be defined by
∞
⊂
Ω =LIP ( f (θ) ) SI ( D (θ) ) VAR ( σ2,d (θ),D (θ) ).
∞ θ { k } ∩ θ { n } ∩ θ { k k k }
a.s.
Let us assume that P(Ω )>0. Then lim θ = θ on Ω .
∞ n n 0 ∞
Remark 3.1. In the linear model, f (bθ)=θTW with lim σ2 a<.s. ,
Ω is reduced to lim λ n W WnT a=.s. tnhanks to Pnropnositi∞on
∞ { n min{ k=1 k k} ∞}
5.2.
P
Proof of Proposition 3.1. We use Wu’s lemma ([34], see Lemma
A.1) and Wu’s decomposition based on Y f (θ)= η +d (θ) [34], im-
k k k k
plying S (θ) S (θ )=D (θ)+2L (θ), whe−re L (θ)= n η d (θ) and,
n − n 0 n n n k=1 k k
consequently,
P
(3.1) inf S (θ) S (θ ) inf D (θ) 1 2 sup L (θ)[D (θ)]−1 .
n n 0 n n n
θ∈Bδc − ≥θ∈Bδc h − θ∈Bδc| | i
10 C. JACOB
The proof then follows directly from the SLLNSM (Proposition 5.1) applied
to d (θ)=f (θ ) f (θ) and Θ=Bc. (cid:3)
k k 0 − k δ
Let us now assume that θ e=argmin n (Y f (θ))2, where h
h,n θ∈Θ k=h+1 k− k
may depend on n (e.g., n h is constant for all n) and let L (θ) L (θ)=:
n h
− P −
L (θ) and D (θ) D (θ)b=:D (θ). Let us define
h,n n h h,n
−
a.s.
RAT (D (θ)[D (θ)]−1 ):= lim sup D (θ)[D (θ)]−1 < .
θ n h,n n h,n
} n θ∈Bc ∞
n δ o
Proposition 3.2. Let Ω Ω be defined by
∞
⊂
Ω =LIP ( f (θ) ) SI ( D (θ) ) VAR ( σ2,d (θ),D (θ) )
∞ θ { k } ∩ θ { n } ∩ θ { k k k }
RAT ( D (θ)[D (θ)]−1 ).
θ n h,n
∩ { }
a.s.
Let us assume that P(Ω )>0. Then lim θ = θ on Ω .
∞ n h,n 0 ∞
Proof. When hisfixed,weareinthesbettingof Proposition3.1.There-
fore, let us assume that h as n . According to (3.1) written with
→∞ →∞
S (θ)insteadofS (θ),itissufficienttoprovethatlim sup L (θ)[D (θ)]−1=
h,n n n θ h,n h,n
a.s.
0 when limninfθ∈BcDh,n(θ) = . For that, we use Proposition 5.1 with
δ ∞
L (θ) L (θ) D (θ) L (θ) D (θ)
h,n n n h h
= .
(cid:3)
D (θ) D (θ)D (θ) − D (θ)D (θ)
h,n n h,n h h,n
Let us now assume, as in the last item of Section 2, the multidimensional
case Z Rd. TheCLSE of θ is then given by (2.1) and, more generally, by
n 0
∈
d n
2
θ =argmin (Y f (θ)) .
h,n k,j k,j
−
θ∈Θ
j=1k=h+1
X X
b
Let D (θ)= n (f (θ) f (θ ))2 and D (θ):=D (θ).
h,n,j k=h+1 k,j − k,j 0 n,j 0,n,j
P
Proposition 3.3. Let Ω Ω be defined by
∞
⊂
Ω = LIP ( f (θ) ) SI ( D (θ) )
∞ θ k,j θ n,j
{ { } ∩ { }
j
\
VAR ( σ2 ,d (θ),D (θ) ) RAT ( D (θ)[D (θ)]−1 ) .
∩ θ { k,j k,j k,j } ∩ θ { n,j h,n,j } }
a.s.
Let us assume that P(Ω )>0. Then lim θ = θ on Ω .
∞ n h,n 0 ∞
TheprooffollowsdirectlyfromPropositiobn3.2appliedtoeachj=1,...,d.