Electronic Journal of Statistics

Vol. 6 (2012) 1307–1357

ISSN: 1935-7524
DOI: 10.1214/12-EJS713

Consistency of the posterior distribution

and MLE for piecewise linear regression
Tristan Launay and Anne Philippe∗
Laboratoire de Mathématiques Jean Leray
2 Rue de la Houssinière – BP 92208
44322 Nantes Cedex 3, France
e-mail: [email protected]; [email protected]

Sophie Lamarche
Electricité de France R&D
1 Avenue du Général de Gaulle
92141 Clamart Cedex, France

Abstract: We prove the weak consistency of the posterior distribution

and that of the Bayes estimator for a two-phase piecewise linear regression
model where the break-point is unknown. We also establish a Bernstein-
von Mises theorem for this non regular model. The non differentiability
of the likelihood of the model with regard to the break-point parameter
induces technical difficulties that we overcome by creating a regularised
version of the problem at hand. We first recover the strong consistency of
the quantities of interest for the regularised version, using results about
the MLE, and we then prove that the regularised version and the original
version of the problem share the same asymptotic properties.

Keywords and phrases: Bayesian asymptotic, Bernstein-von Mises theo-

rem, maximum-likelihood estimation, non-regular model, piecewise regres-
sion.

Received March 2012.

1. Introduction

We consider a continuous segmented regression model with 2 phases, one of

them (the rightmost) being zero. Let u be the unknown breakpoint and γ ∈ R
be the unknown regression coefficient of the non zero phase. The observations
X1:n = (X1 , . . . , Xn ) depend on an exogenous variable that we denote t1:n =
(t1 , . . . , tn ) via the model given for i = 1, . . . , n by
Xi = µ(η, ti ) + ξi := γ · (ti − u)1[ti , +∞[ (u) + ξi , (1.1)
where (ξi )i∈N is a sequence of independent and identically distributed (i.i.d.)
random variables with a common centered Gaussian distribution of unknown
variance σ 2 , N (0, σ 2 ), and where 1A denotes the indicator function of a set A.
Such a model is for instance used in practice to estimate and predict the
heating part of the electricity demand in France. See Bruhns et al. (2005) for
∗ Corresponding author.
1307
1308 T. Launay et al.

the definition of the complete model and Launay et al. (2012) for a Bayesian
approach. In this particular case, u corresponds to the heating threshold above
which the temperatures t1:n do not have any effect over the electricity load, and
γ corresponds to the heating gradient i.e. the strength of the described heating
effect.
The work presented in this paper is most notably inspired by the results
developed in Ghosh et al. (2006) and Feder (1975).
Feder proved the weak consistency of the least squares estimator in segmented
regression problems with a known finite number of phases under the hypotheses
of his Theorem 3.10 and some additional assumptions disseminated throughout
his paper, amongst which we find that the empirical cumulative distribution
functions of the temperatures at the n-th step tn1 , . . . , tnn are required to con-
verge to a cumulative distribution function, say Fn converges to F , which is of
course to be compared to our own Assumption (A1). Feder also derived the
asymptotic distribution of the least squares estimator under the same set of
assumptions. Unfortunately there are a few typographical errors in his paper
(most notably resulting in the disappearance of σ02 from the asymptotic vari-
ance matrix in his main theorems), and he also did not include σ bn2 in his study
of the asymptotic distribution.
The asymptotic behaviour of the posterior distribution is a central question
that has already been raised in the past. For example, Ghosh et al. worked out
the limit of the posterior distribution in a general and regular enough i.i.d.
setup. In particular they manage to derive the asymptotic normality of the
posterior distribution under third-order differentiability conditions. There are
also a number of works dealing with some kind of non regularity, like these
of Sareen (2003) which consider data the support of which depends on the
parameters to be estimated, or those of Ibragimov and Has’minskii (1981) which
offer the limiting behaviour of the likelihood ratio for a wide range of i.i.d. models
whose likelihood may present different types of singularity. Unfortunately, the
heating part model presented here does not fall into any of these already studied
categories.
In this paper, we show that the results of Ghosh et al. can be extended to a
non i.i.d. two-phase regression model. We do so by using the original idea found
in Sylwester (1965)1 : we introduce a new, regularised version of the problem
called pseudo-problem, later reprised by Feder. The pseudo-problem consists
in removing a fraction of the observations in the neighbourhood of the true
parameter to obtain a differentiable likelihood function. We first recover the
results of Ghosh et al. for this pseudo-problem and then extend these results
to the (full) problem by showing that the estimates for the problem and the
pseudo-problem have the same asymptotic behaviour.
From this point on, we shall denote the parameters θ = (γ, u, σ 2 ) = (η, σ 2 )
and θ0 will denote the true value of θ. We may also occasionally refer to the
intercept of the model as β = −γu. The log-likelihood of the n first observations
1 Sylwester indeed considers the same model as we do here, however his asymptotic results

are false due to an incorrect reparametrisation of the problem and an error in the proof of his
Theorem 3.5.
 Asymptotic results for piecewise linear regression 1309

X1:n of the model will be denoted

n
X
l1:n (X1:n |θ) = li (Xi |θ) (1.2)
i=1
n
n
X 1
2
=− log 2πσ 2 − 2
Xi − γ · (ti − u)1[ti , +∞[ (u) , (1.3)
2 i=1
2σ

where li (Xi |θ) designates the log-likelihood of the i-th observation Xi , i.e.
1
1
2
li (X1:n |θ) = − log 2πσ 2 − 2 Xi − γ · (ti − u)1[ti , +∞[ (u) . (1.4)
2 2σ
Notice that we do not mention explicitly the link between the likelihood l and
the sequence of temperatures (tn )n∈N in these notations, so as to keep them as
minimal as possible. The least square estimator θbn of θ being also the maximum
likelihood estimator of the model, we refer to it as the MLE.
Throughout the rest of this paper we work under the following assumptions
Assumption (A1). The sequence of temperatures (exogenous variable) (tn )n∈N
belongs to a compact set [u, u] and the sequence of the empirical cumulative
distribution functions (Fn )n∈N of (t1 , . . . , tn ), defined by
n
1X
Fn (u) = 1[t , +∞[ (u),
n i=1 i

converges pointwise to a function F where F is a cumulative distribution func-

tion itself, which is continuously differentiable over [u, u].
Remark 1. Due to a counterpart to Dini’s Theorem (see Theorem 7.1 taken
from Polya and Szegö, 2004, (p81)), Fn converges to F uniformly over [u, u].
Remark 2. Let h be a continuous, bounded function on [u, u]. As an immediate
consequence of this assumption, for any interval I ⊂ [u, u], we have, as n −
→ +∞
n Z Z Z
1X
h(ti )1I (ti ) = h(t) dFn (t) −
→ h(t) dF (t) = h(t)f (t) dt,
n i=1 I I I

the convergence holding true by definition of the convergence of probability

measures (see Billingsley, 1999, pages 14–16). In particular, for I = [u, u] and
I =] − ∞, u] we get, as n −→ +∞
n Z n Z
1X u
1X u
h(ti ) −
→ h(t)f (t) dt, h(ti )1[ti , +∞[ (u) −
→ h(t)f (t) dt.
n i=1 u n i=1 u

Remark 3. It is a general enough assumption which encompasses both the

common cases of i.i.d. continuous random variables and periodic (non random)
variables under a continous (e.g. Gaussian) noise.
1310 T. Launay et al.

Assumption (A2). θ0 ∈ Θ, where the parameter space Θ is defined (for iden-

tifiability) as
Θ = R∗ ×]u, u[×R∗+ ,
where R∗ = {x ∈ R , x 6= 0} and R∗+ = {x ∈ R , x > 0}.
Assumption (A3). f = F ′ does not vanish (i.e. is positive) on ]u, u[.
Assumption (A4). There exists K ⊂ Θ a compact subset of the parameter
space Θ such that θbn ∈ K for any n large enough.

The paper is organised as follows. In Section 2, we present the Bayesian con-

sistency (the proofs involved there rely on the asymptotic distribution of the
MLE) and introduce the concept of pseudo-problem. In Section 3, we prove
that the MLE for the full problem is strongly consistent. In Section 4 we derive
the asymptotic distribution of the MLE using the results of Section 3: to do so,
we first derive the asymptotic distribution of the MLE for the pseudo-problem
and then show that the MLEs for the pseudo-problem and the problem share
the same asymptotic distribution. We discuss these results in Section 5. The ex-
tensive proofs of the main results are found in Section 6 while the most technical
results are pushed back into Section 7 at the end of this paper.
Notations. Whenever mentioned, the O and o notations will be used to des-
ignate a.s. O and a.s. o respectively, unless there are indexed with P as in OP
and oP , in which case they will designate O and o in probability respectively.
Hereafter we will use the notation Ac for the complement of the set A and
B(x, r) for the open ball of radius r centred at x i.e. B(x, r) = {x′ , kx′ −xk < r}.

2. Bayesian consistency

In this Section, we show that the posterior distribution of θ given (X1 , . . . , Xn )

asymptotically favours any neighbourhood of θ0 as long as the prior distribution
itself charges a (possibly different) neighbourhood of θ0 (see Theorem 2.1). We
then present in Theorem 2.2 the main result of this paper i.e. the convergence
of posterior distribution with suitable normalisation to a Gaussian distribution.

2.1. Consistency and asymptotic normality of the posterior

distribution

Theorem 2.1. Let π(·) be a prior distribution on θ, continuous and positive on

a neighbourhood of θ0 and let U be a neighbourhood of θ0 , then under Assump-
tions (A1)–(A4), as n −→ +∞,
Z
a.s.
π(θ|X1:n ) dθ −−→ 1. (2.1)
U

Proof for Theorem 2.1. The proof is very similar to the one given in Ghosh and
Ramamoorthi (2003) for a model with i.i.d. observations. Let δ > 0 small enough
 Asymptotic results for piecewise linear regression 1311

so that B(θ0 , δ) ⊂ U . Since

Z
1
π(θ|X1:n ) dθ = R
U c π(θ) exp[l 1:n (X 1:n |θ) − l1:n (X1:n |θ0 )] dθ
1 + RU
U
π(θ) exp[l 1:n (X 1:n |θ) − l1:n (X1:n |θ0 )] dθ
1
6 R
B c (θ0 ,δ) π(θ) exp[l 1:n (X 1:n |θ) − l1:n (X1:n |θ0 )] dθ
1+ R
B(θ0 ,δ) π(θ) exp[l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] dθ

it will suffice to show that

R
c π(θ) exp[l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] dθ a.s.
RB (θ0 ,δ) −−→ 0. (2.2)
B(θ0 ,δ) π(θ) exp[l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] dθ

To prove (2.2) we adequately majorate its numerator and minorate its denom-
inator. The majoration mainly relies on Proposition 7.11 while the minoration
is derived without any major difficulties. The comprehensive proof of (2.2) can
be found in Section 6.1 on page 1319.
Let θ ∈ Θ, we now define I(θ), the asymptotic Fisher Information matrix
I(θ) of the model, as the symmetric matrix given by
 Z u Z u 
σ −2 (t − u)2 dF (t) −σ −2 γ (t − u) dF (t) 0
 u uZ 
 u 
 −2 2 
I(θ) =  σ γ 1 dF (t) 0  . (2.3)
 
 u 
1 −4
σ
2
It is obviously positive and definite since all its principal minor determinants
are positive. The proof of the fact that it is indeed the limiting matrix of the
Fisher Information matrix of the model is deferred to Lemma 7.10.
Theorem 2.2. Let π(·) be a prior distribution on θ, continuous and positive at
θ0 , and let k0 ∈ N such that
Z
kθkk0 π(θ) dθ < +∞,
Θ
and denote
1
t = n 2 (θ − θbn ), (2.4)
en (·|X1:n ) the posterior density of t given X1:n , then under Assumptions (A1)–
and π
(A4), for any 0 6 k 6 k0 , as n − → +∞,
Z  
 3 1 1 ′  P
ktkk eπn (t|X1:n ) − (2π)− 2 |I(θ0 )| 2 e− 2 t I(θ0 )t  dt −
→ 0, (2.5)
R3

where I(θ) is defined in (2.3) and θ0 the true value of the parameter.
The proof Theorem 2.2 relies on the consistency of the pseudo-problem, first
introduced in Sylwester (1965), that we define in the next few paragraphs.
1312 T. Launay et al.

2.2. Pseudo-problem

The major challenge in proving Theorem 2.2 is that the typical arguments usu-
ally used to derive the asymptotic behaviour of the posterior distribution (see
Ghosh et al., 2006, for example) do not directly apply here. The proof provided
by Ghosh et al. requires a Taylor expansion of the likelihood of the model up to
the third order at the MLE, and the likelihood of the model we consider here at
the n-th step is very obviously not continuously differentiable w.r.t. u in each
observed temperature ti , i = 1, . . . , n. Note that the problem only grows worse
as the number of observations increases.
To overcome this difficulty we follow the original idea first introduced in
Sylwester (1965), and later used again in Feder (1975): we introduce a pseudo-
problem for which we are able to recover the classical results and show that
the differences between the estimates for the problem and the pseudo-problem
are, in a sense, negligeable. The pseudo-problem is obtained by deleting all the
observations within intervals Dn of respective sizes dn centred around u0 . The
intervals Dn are defined as
 
dn dn
Dn = u 0 − , u0 + ,
2 2
and their sizes dn are chosen such that as n −
→ +∞
1
dn −
→ 0, n− 2 (log n) · d−1
n −→ 0. (2.6)
This new problem is called pseudo-problem because the value of u0 is unknown
and we therefore cannot in practice delete these observations. Note that the
actual choice of the sequence (dn )n∈N does not influence the rest of the results
in any way, as long as it satisfies to conditions (2.6). It thus does not matter at
1
all whether one chooses (for instance) dn = n− 4 or dn = log−1 n.
Let us denote n∗∗ the number of observations deleted from the original prob-
lem, and n∗ = n−n∗∗ the sample size of the pseudo-problem. Generally speaking,
quantities annotated with a single asterisk ∗ will refer to the pseudo-problem.
∗
l1:n (X1:n |θ) will thus designate the likelihood of the pseudo-problem i.e. (rein-
dexing observations whenever necessary)
∗
n
∗ n∗
X 1
2
l1:n (X1:n |θ) = − log 2πσ 2 − 2
Xi − γ · (ti − u)1[ti , +∞[ (u) .
2 i=1
2σ
(2.7)
On one hand, from an asymptotic point of view, the removal of those n∗∗
observations should not have any kind of impact on the distribution theory.
The intuitive idea is that deleting n∗∗ observations takes away only a fraction
n∗∗ /n of the information which asymptotically approaches zero as will be shown
below. The first condition (2.6) seems only a natural requirement if we ever
hope to prove that the MLE for the problem and the pseudo-problem behave
asymptotically in a similar manner (we will show they do in Theorem 4.2, see
equation (4.1)).
 Asymptotic results for piecewise linear regression 1313

On the other hand, assuming the MLE is consistent (we will show it is, in
Theorem 3.3) and assuming that the sizes dn are carefully chosen so that the
sequence (b un )n∈N falls into the designed sequence of intervals (Dn )n∈N (see
Proposition 4.1, whose proof the second condition (2.6) is tailored for), these
regions will provide open neighbourhoods of the MLE over which the likelihood
of the pseudo-problem will be differentiable. The pseudo-problem can therefore
be thought of as a locally regularised version of the problem (locally because we
are only interested in the differentiability of the likelihood over a neighbourhood
of the MLE). We should thus be able to retrieve the usual results for the pseudo-
problem with a bit of work. It will be shown that this is indeed the case (see
Theorem 2.3).
If the sequence (dn )n∈N satisfies to conditions (2.6), then as n − → +∞,
n∗∗ n∗
→ 0,
− → 1.
−
n n
Using the uniform convergence of Fn to F over any compact subset (see
Assumption (A1), and its Remark 1), we indeed find via a Taylor-Lagrange
approximation
   
n∗∗ dn dn
= Fn u0 + − Fn u0 −
n 2 2
   
dn dn
= F u0 + − F u0 − + o(1)
2 2
= dn · f (un ) + o(1),
where un ∈ Dn , so that in the end, since un −
→ u0 and f is continuous and
positive at u0 , we have a.s.
n∗∗
= dn · (f (u0 ) + o(1)) + o(1) −
→ 0.
n
We now recover the asymptotic normality of the posterior distribution for
the pseudo problem.
Theorem 2.3. Let π(·) be a prior distribution on θ, continuous and positive at
θ0 , and let k0 ∈ N such that
Z
kθkk0 π(θ) dθ < +∞.
Θ

and denote
1
t∗ = n 2 (θ − θbn∗ ), (2.8)
and πen∗ (·|X1:n ) the posterior density of t∗ given X1:n , then under Assump-
tions (A1)–(A4) and conditions (2.6), for any 0 6 k 6 k0 , as n − → +∞,
Z  
 ∗ 3 1 1 ′  a.s.
ktkk eπn (t|X1:n ) − (2π)− 2 |I(θ0 )| 2 e− 2 t I(θ0 )t  dt −−→ 0, (2.9)
R3

where I(θ) is defined in (2.3).

1314 T. Launay et al.

Proof of Theorem 2.3. The extensive proof, to be found in Section 6.1, was in-
spired by that of Theorem 4.2 in Ghosh et al. (2006) which deals with the case
where the observations X1 , . . . , Xn are independent and identically distributed
and where the (univariate) log-likelihood is differentiable in a fixed small neigh-
bourhood of θ0 . We tweaked the original proof of Ghosh et al. so that we could
deal with independent but not identically distributed observations and a (mul-
tivariate) log-likelihood that is guaranteed differentiable only on a decreasing
small neighbourhood of θ0 .

2.3. From the pseudo-problem to the original problem

We now give a short proof of Theorem 2.2. As we previously announced, it relies

upon its counterpart for the pseudo-problem, i.e. Theorem 2.3.
Proof of Theorem 2.2. Recalling the definition of t and t∗ given in (2.4) and
(2.8) we observe that
1
t = t∗ + n 2 (θbn∗ − θbn ).

Thus the posterior distribution of t∗ and that of t, given X1:n are linked together
via

π en∗ (t − αn |X1:n )
en (t|X1:n ) = π (2.10)

where
1
αn = n 2 (θbn∗ − θbn ).

Relationship (2.10) allows us to write

Z  
 3 1 1 ′ 
ktkk π en (t|X1:n ) − (2π)− 2 |I(θ0 )| 2 e− 2 t I(θ0 )t  dt
R 3
Z  
 ∗ 3 1 1 ′ 
= ktkk e πn (t − αn |X1:n ) − (2π)− 2 |I(θ0 )| 2 e− 2 t I(θ0 )t  dt
R3
Z  
 ∗ 3 1 1 ′ 
= kt + αn kk e πn (t|X1:n ) − (2π)− 2 |I(θ0 )| 2 e− 2 (t+αn ) I(θ0 )(t+αn )  dt
3
ZR  
 ∗ 3 1 1 ′ 
6 kt + αn kk e πn (t|X1:n ) − (2π)− 2 |I(θ0 )| 2 e− 2 t I(θ0 )t  dt
R 3
Z  1 
3 1  ′ 1 ′ 
+ (2π)− 2 |I(θ0 )| 2 kt + αn kk e− 2 (t+αn ) I(θ0 )(t+αn ) − e− 2 t I(θ0 )t  dt
R3

Theorem 2.3 ensures that the first integral on the right hand side of this last
inequality goes to zero in probability. It therefore suffices to show that the second
integral goes to zero in probability to end the proof, i.e. that as n − → +∞
Z  1 
 ′ 1 ′  P
kt + αn kk e− 2 (t+αn ) I(θ0 )(t+αn ) − e− 2 t I(θ0 )t  dt −
→ 0. (2.11)
R3
 Asymptotic results for piecewise linear regression 1315

P
But the proof of (2.11) is straightforward knowing that αn −
→ 0 (see (4.1)) and
using dominated convergence.
As an immediate consequence of Theorem 2.2 we want to mention the weak
consistency of the Bayes estimator.
Corollary 2.4. Let π(·) a prior distribution on θ, continuous and positive at
θ0 , such that
Z
kθkπ(θ) dθ < +∞,
Θ

and denote
Z
θen = θπn (θ|X1:n ) dθ,
Θ

the Bayes estimator of θ in the problem. Then under Assumptions (A1)–(A4),

as n −
→ +∞,
1 P
n 2 (θen − θbn ) −
→ 0.

Proof of Corollary 2.4. By definition,

Z
e
θn = θπn (θ|X1:n ) dθ
Θ

and this allows us to write

Z
1 1
n (θen − θbn ) =
2 n 2 (θ − θbn )πn (θ|X1:n ) dθ
ZΘ
P
= πn (t|X1:n ) dt −
te → 0,
R3

the last convergence being a direct consequence of Theorem 2.2 with k0 = 1.

Observe that, under conditions (2.6), the same arguments naturally apply to
the pseudo-problem and lead to a strong consistency (a.s. convergence) of its
associated Bayes estimator due to Theorem 2.3, thus recovering the results of
Ghosh et al. (2006) for the regularised version of the problem.

3. Strong consistency of the MLE

In this Section we prove the strong consistency of the MLE over any compact
set including the true parameter (see Theorem 3.1). It is a prerequisite for a
more accurate version of the strong consistency (see Theorem 3.3) which lies at
the heart of the proof of Theorem 2.3.
Theorem 3.1. Under Assumptions (A1)–(A4), we have a.s., as n −
→ +∞,

kθbn − θ0 k = o(1).
1316 T. Launay et al.

Proof of Theorem 3.1. Recall that K is a compact subset of Θ, such that θbn ∈ K
for any n large enough. We denote
l1:n (X1:n |S) = sup l1:n (X1:n |θ), for any S ⊂ K,
θ∈S
Kn (a) = {θ ∈ Θ, l1:n (X1:n |θ) > log a + l1:n (X1:n |K)} , for any a ∈]0, 1[.
All we need to prove is that
!
∃a ∈]0, 1[, P lim sup kθ − θ0 k = 0 = 1. (3.1)
n−
→+∞ θ∈Kn (a)

since for any n large enough we have θbn ∈ Kn (a) for any a ∈]0, 1[. We control the
likelihood upon the complement of a small ball in K and prove the contrapositive
of (3.1) using compacity arguments. The extensive proof of (3.1) is to be found
in Section 6.2.
We strengthen the result of Theorem 3.1 by giving a rate of convergence for
the MLE (see Theorem 3.3). This requires a rate of convergence for the image
of the MLE through the regression function of the model, that we give in the
Proposition 3.2 below.
Proposition 3.2. Under Assumptions (A1)–(A4), as n − → +∞, a.s., for any
open interval I ⊂ [u, u],
 1 
min |µ(bηn , ti ) − µ(η0 , ti )| = O n− 2 log n .
ti ∈I, i6n

Proof of Proposition 3.2. The proof is given in Section 6.2.

Theorem 3.3. Under Assumptions (A1)–(A4), we have a.s., as n −
→ +∞,
 1 
kθbn − θ0 k = O n− 2 log n . (3.2)

Proof of Theorem 3.3. We show that a.s. (3.2) holds for each coordinate of θbn −
θ0 . The calculations for the variance σ 2 are pushed back into Section 6.2. We
now prove the result for the parameters γ and u. It is more convenient to use
a reparametrisation of the model in terms of slope γ and intercept β where
β = −γu.
Slope γ and intercept β. Let V1 and V2 be two non empty open intervals
of ]u, u0 [ such that their closures V1 and V2 do not overlap. For any (t1 , t2 ) ∈
V1 × V2 , define M (t1 , t2 ) the obviously invertible matrix
 
1 t1
M (t1 , t2 ) = ,
1 t2
and observe that for any τ = (β, γ),
 
µ(η, t1 )
M (t1 , t2 )τ = .
µ(η, t2 )
 Asymptotic results for piecewise linear regression 1317

Observe that by some basic linear algebra tricks we are able to write for any
(t1 , t2 ) ∈ V1 × V2

τn − τ0 k∞ = kM (t1 , t2 )−1 M (t1 , t2 )(b

kb τn − τ0 )k∞
6 k|M (t1 , t2 )−1 k|∞ · kM (t1 , t2 )b
τn − M (t1 , t2 )τ0 k∞
|t2 | + |t1 | + 2
6 · kM (t1 , t2 )b
τn − M (t1 , t2 )τ0 k∞ .
|t2 − t1 |
Thus, using the equivalence of norms and a simple domination of the first term
of the product in the inequality above, we find that there exists a constant
C ∈ R∗+ , such that for any (t1 , t2 ) ∈ V1 × V2

kb
τn − τ0 k 6 C · kM (t1 , t2 )b
τn − M (t1 , t2 )τ0 k,

i.e.
" 2
# 12
X
2
kb
τn − τ0 k 6 C · ηn , ti ) − µ(η0 , ti ))
(µ(b . (3.3)
i=1

Taking advantage of Proposition 3.2, we are able to exhibit two sequences of

points (t1,n )n∈N in V1 and (t2,n )n∈N in V2 such that a.s., for i = 1, 2
 1 
ηn , ti,n ) − µ(η0 , ti,n )| = O n− 2 log n .
|µ(b (3.4)

Combining (3.3) and (3.4) together (using ti = ti,n for every n), it is now trivial
to see that a.s.
 1 
τn − τ0 k = O n− 2 log n ,
kb

which immediately implies the result for the γ and β components of θ.

Break-point u. Recalling that u = −βγ −1 and thanks to the result we just
proved, we find that a.s.
h  1 i h  1 i−1
bn = −βbn γ
u bn−1 = − β0 + O n− 2 log n γ0 + O n− 2 log n
 1   1 
= −β0 γ0−1 + O n− 2 log n = u0 + O n− 2 log n .

4. Asymptotic distribution of the MLE

In this Section we derive the asymptotic distribution of the MLE for the pseudo-
problem (see Proposition 4.1) and then show that the MLE of pseudo-problem
and that of the problem share the same asymptotic distribution (see Theo-
rem 4.2).
1318 T. Launay et al.

Proposition 4.1. Under Assumptions (A1)–(A4) and conditions (2.6), as

n−
→ +∞
 

1 d
n 2 θbn∗ − θ0 −→ N 0, I(θ0 )−1 ,

where the asymptotic Fisher Information Matrix I(·) is defined in (2.3).

Proof of Theorem 4.1. The proof is divided in two steps. We first show that
the likelihood of the pseudo-problem is a.s. differentiable in a neighbourhood of
the MLE θbn∗ for N large enough. We then recover the asymptotic distribution
of the MLE following the usual scheme of proof, with a Taylor expansion of
the likelihood of the pseudo-problem around the true parameter. The details of
these two steps are given in Section 6.3.
Theorem 4.2. Under Assumptions (A1)–(A4) and conditions (2.6), as n −
→
+ ∞,
 

1 d
n 2 θbn − θ0 −→ N 0, I(θ0 )−1 ,

where the asymptotic Fisher Information Matrix I(·) is defined in (2.3).

Proof of Theorem 4.2. It is a direct consequence of Proposition 4.1 as soon as
we show that as n −
→ +∞
 1
θbn − θbn∗ = oP n− 2 . (4.1)

To prove (4.1), we study each coordinate separately. For γ and u, we apply

Lemmas 4.12 and 4.16 found in Feder (1975) with a slight modification: the rate
of convergence dn he uses may differ from ours but it suffices to formally replace
1
(log log n) 2 by (log n) all throughout his paper and the proofs he provides go
through without any other change. We thus get
 1  1
γ γn∗ = oP n− 2 ,
bn − b b∗n = oP n− 2 .
bn − u
u (4.2)

It now remains to show that

 1
σ bn2∗ = oP n− 2 .
bn2 − σ (4.3)

To do so, we use (4.2) and the decomposition (6.40)

n n n
1X 2 2X 1X 2
bn2 =
σ νi (b
ηn ) + νi (b
ηn )ξi + ξ ,
n i=1 n i=1 n i=1 i

ηn ) = γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ

where νi (b bn · (ti − u
bn )1[ti , +∞[ (b
un ). The details
of this are available in Section 6.3.

5. Discussion

In this Section, we summarise the results presented in this paper. The con-
sistency of the posterior distribution for a piecewise linear regression model is
 Asymptotic results for piecewise linear regression 1319

derived as well as its asymptotic normality with suitable normalisation. The

proofs of these convergence results rely on the convergence of the MLE which
is also proved here. In order to obtain all the asymptotic results, a regularised
version of the problem at hand, called pseudo-problem, is first studied and the
difference between this pseudo-problem and the (full) problem is then shown to
be asymptotically negligeable.
The trick of deleting observations in a diminishing neighbourhood of the
true parameter, originally found in Sylwester (1965) allows the likelihood of the
pseudo-problem to be differentiated at the MLE, once the MLE is shown to
asymptotically belong to that neighbourhood (this requires at least a small con-
trol of the rate of convergence of the MLE). This is the key argument needed to
derive the asymptotic distribution of the MLE through the usual Taylor expan-
sion of the likelihood at the MLE. Extending the results of Ghosh et al. (2006) to
a non i.i.d. setup, the asymptotic normality of the posterior distribution for the
pseudo-problem is then recovered from that of the MLE, and passes on almost
naturally to the (full) problem.
The asymptotic normality of the MLE and the posterior distribution are
proved in this paper in a non i.i.d. setup with a non continuously differentiable
likelihood. In both cases we obtain the same √ asymptotic results as for an i.i.d.
regular model: the rate of convergence is n and the limiting distribution is
Gaussian (see Ghosh et al., 2006; Lehmann, 2004). For the piecewise linear re-
gression model, the exogenous variable t1:n does not appear in the expression of
the rate of convergence as opposed to what is known for the usual linear regres-
sion model (see Lehmann, 2004): this is due to our own Assumption (A1) which
implies that t′1:n t1:n is equivalent
√ to n. Note that for a simple linear regression
model, we also obtain the rate n under Assumption (A1). In the litterature,
several papers already highlighted the fact that the rate of convergence and the
limiting distribution (when it exists) may be different for non regular models
in the sense that the likelihood is either non continuous, or non continuously
differentiable, or admits singularities (see Dacunha-Castelle, 1978; Ghosh et al.,
1994; Ghosal and Samanta, 1995; Ibragimov and Has’minskii, 1981). For the
piecewise regression model, the likelihood is continuous but non continuously
differentiable on a countable set√(but the left and right derivatives exist and are
finite): the rate of convergence n is not so surprising in our case, because this
rate was already obtained for a univariate i.i.d. model the likelihood of has the
same non regularity at a single point. In that case, the rate of convergence of
the MLE is shown to be n (see Dacunha-Castelle, 1978, for instance).

6. Extensive proofs

6.1. Proofs of Section 2

Proof of Theorem 2.1. To prove (2.2), we proceed as announced and deal with
numerator and denominator in turn.
Majoration. From Proposition 7.11 with ρn = 1, for any given ǫ > 0, we can
1320 T. Launay et al.

choose δ > 0 small enough so that a.s. for any n large enough
1
sup [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] 6 −ǫ.
θ∈B c (θ0 ,δ) n

We thus obtain a.s. for any n large enough

Z
06 π(θ) exp[l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] dθ
B c (θ0 ,δ)
Z
−nǫ
6e π(θ) dθ. (6.1)
B c (θ0 ,δ)

Minoration. Define θn ∈ B(θ0 , δ) such that

1 1
inf [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] = [l1:n (X1:n |θn ) − l1:n (X1:n |θ0 )]
θ∈B(θ0 ,δ) n n

It is possible to define such a θn because B(θ0 , δ) is a compact subset of Θ for

δ > 0 small enough and l1:n (X1:n |·) is continuous as a function of θ. Let now
  n
σ02 σ2 1 1X
bn (θ) = 2
− 1 − log 02 + · [µ(η0 , ti ) − µ(η, ti )]2 . (6.2)
σ σ σ 2 n i=1

Recalling the definition of the log-likehood given in (1.2) and replacing Xi by

its expression given in (1.1) we find via straightforward algebra
  n
!
2 σ02 1 1 1X 2
[l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] = log 2 + − 2 ξ
n σ σ2 σ0 n i=1 i
n n
1 X 2 2 1X
− [µ(η0 , ti ) − µ(η, ti )] − 2 [µ(η0 , ti ) − µ(η, ti )]ξi
nσ 2 i=1 σ n i=1
  n
!
σ02 1 1 1X 2 2 2
= log 2 + − 2 ξ − σ0 + σ0
σ σ2 σ0 n i=1 i
n n
1 X 2 2 1X
− [µ(η ,
0 i t ) − µ(η, t i )] − [µ(η0 , ti ) − µ(η, ti )]ξi
nσ 2 i=1 σ 2 n i=1
  n
!
σ02 σ02 σ02 − σ 2 1 X 2 2
= log 2 + 1 − 2 + ξ − σ0
σ σ σ 2 σ02 n i=1 i
n n
1 X 2 2 1X
− [µ(η ,
0 it ) − µ(η, t i )] − [µ(η0 , ti ) − µ(η, ti )]ξi (6.3)
nσ 2 i=1 σ 2 n i=1
n
! n
σ02 − σ 2 1 X 2 2 2 1X
= −bn (θ) + ξi − σ 0 − [µ(η0 , ti ) − µ(η, ti )]ξi . (6.4)
σ 2 σ02 n i=1 σ 2 n i=1
 Asymptotic results for piecewise linear regression 1321

It is now easy to see that

2 2
inf [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] = [l1:n (X1:n |θn ) − l1:n (X1:n |θ0 )]
n
θ∈B(θ0 ,δ) n
n
! n
σ02 − σn2 1 X 2 2 2 1X
= −bn (θn ) + ξ − σ 0 − [µ(η0 , ti ) − µ(ηn , ti )]ξi
σn2 σ02 n i=1 i σn2 n i=1
" n
! n
#
1 σ02 − σn2 1 X 2 2 2X
= −bn (θn ) + 2 ξ − σ0 − [µ(η0 , ti ) − µ(ηn , ti )]ξi
σn σ02 n i=1 i n i=1
1
= −bn (θn ) + Rn
σn2
  n
σ02 σ02 1 1X 2 1
= log 2 + 1 − 2 − 2 · [µ(η0 , ti ) − µ(ηn , ti )] + 2 Rn
σn σn σn n i=1 σn
a.s.
where Rn −−→ 0 because of the Law of Large Numbers and Lemma 7.6. Thanks
to Lemma 7.3 we thus find that there exists C ∈ R∗+ such that
 
2 σ2 σ2
inf [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] > log 20 + 1 − 20
θ∈B(θ0 ,δ) n σn σn
1

− 2 Ckθn − θ0 k2 − Rn
σn
We now choose κ > 0 and δ > 0 small enough so that
 
2 σ02 σ02
σn log 2 + 1 − 2 > −κ, (6.5)
σn σn
2
3(κ + Cδ ) 1
− 2 > − ǫ. (6.6)
2(σ0 − δ) 2
Thanks to (6.5) and the definition of θn , we can now write that
2 1

inf [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] > − 2 κ + Ckθn − θ0 k2 − Rn
θ∈B(θ0 ,δ) n σn
1

> − 2 κ + Cδ 2 − Rn .
σn
Since for any n large enough
1

|Rn | 6 κ + Cδ 2 ,
2
we find via (6.6) that for any n large enough
2 3

inf [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] > − 2 κ + Cδ 2
θ∈B(θ0 ,δ) n 2σn
3(κ + Cδ 2 ) 1
>− > − ǫ.
2(σ02 − δ) 2
1322 T. Launay et al.

We just proved that for any ǫ > 0, we have a.s. for any n large enough
2 1
0> inf [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] > − ǫ,
θ∈B(θ0 ,δ) n 2
which immediately implies
Z Z
1
π(θ) exp[l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] dθ > e− 2 nǫ π(θ) dθ. (6.7)
B(θ0 ,δ) B(θ0 ,δ)

Conclusion. Let now ǫ > 0 and δ > 0 small enough so that a.s. for any n
large enough (6.1) and (6.7) both hold. We have a.s. for any n large enough
R R
B c (θ0 ,δ) π(θ) exp[l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] dθ B c (θ0 ,δ) π(θ) dθ − 1 nǫ
R 6 R e 2 − → 0,
B(θ0 ,δ)
π(θ) exp[l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] dθ B(θ0 ,δ)
π(θ) dθ

which ends the proof.

Proof of Theorem 2.3. Because the posterior distribution of θ in the pseudo-
problem, πn∗ (·|X1:n ), can be written as

πn∗ (θ|X1:n ) ∝ π(θ) exp[l1:n

∗
(X1:n |θ)],

the posterior density of t∗ = n 2 (θ − θbn∗ ) ∈ R3 can be written as

1

1 1
en∗ (t|X1:n ) = Cn−1 π(θbn∗ + n− 2 t) exp[l1:n
π ∗
(X1:n |θbn∗ + n− 2 t) − l1:n
∗
(X1:n |θbn∗ )]

where
Z
1 1
Cn = π(θbn∗ + n− 2 t) exp[l1:n
∗
(X1:n |θbn∗ + n− 2 t) − l1:n
∗
(X1:n |θbn∗ )] dt. (6.8)
R3

Denoting
1 1
gn (t) = π(θbn∗ + n− 2 t) exp[l1:n
∗
(X1:n |θbn∗ + n∗− 2 t) − l1:n
∗
(X1:n |θbn∗ )]
1 ′
− π(θ0 )e− 2 t I(θ0 )t , (6.9)
to prove (2.9) it suffices to show that for any 0 6 k 6 k0 ,
Z
a.s.
ktkk |gn (t)| dt −−→ 0. (6.10)
R3
a.s. 3 1
Indeed, if (6.10) holds, Cn −−→ π(θ0 )(2π) 2 |I(θ0 )|− 2 (k = 0) and therefore, the
integral in (2.9) which is dominated by
Z
Cn−1 ktkk |gn (t)| dt
R3
Z  
 1 ′ 1 1 1 ′ 
+ ktkk Cn−1 π(θ0 )e− 2 t I(θ0 )t − (2π)− 2 |I(θ0 )| 2 e− 2 t I(θ0 )t  dt
R3

also goes to zero a.s.

 Asymptotic results for piecewise linear regression 1323

Let 0 < δ to be chosen later, and let 0 6 k 6 k0 . To show (6.10), we break

R3 into two regions
1 1
T1 (δ) = B c (0, δn 2 dn ) = {t : ktk > δn 2 dn }
1 1
T2 (δ) = B(0, δn 2 dn ) = {t : ktk < δn 2 dn }

and show that for i = 1, 2

Z
a.s.
ktkk |gn (t)| dt −−→ 0. (6.11)
Ti (δ)

R
Proof for i = 1. Note that T1 (δ) ktkk |gn (t)| is dominated by
Z
1 1
ktkk π(θbn∗ + n 2 t) exp[l1:n
∗
(X1:n |θbn∗ + n− 2 t) − l1:n
∗
(X1:n |θbn∗ )] dt
T1 (δ)
Z
1 ′
+ ktkk π(θ0 )e− 2 t I(θ0 )t dt.
T1 (δ)

The second integral trivially goes to zero. For the first integral, we observe that
it can be rewritten as
Z
1 k
n 2 n 2 kθ − θbn∗ kk π(θ) exp[l1:n
∗ ∗
(X1:n |θ) − l1:n (X1:n |θbn∗ )] dθ.
B c (θbn
∗ ,δd )
n

The strong consistency of θbn∗ (see Theorem 3.3) implies that a.s., for any n large
enough
1
kθbn∗ − θ0 k < δdn .
2

From this, we deduce that a.s., for any n large enough, B c (θbn∗ , δdn ) ⊂ B c (θ0 , 21 δdn )
and thus that the first integral is dominated by
Z
k+1
n 2 kθ − θbn∗ kk π(θ) exp[l1:n
∗ ∗
(X1:n |θ) − l1:n (X1:n |θbn∗ )] dθ.
B c (θ0 , 21 δdn )

Recalling that n∗ ∼ n, Proposition 7.11 with ρn = dn implies that there a.s.

exists ǫ > 0 such that for any n large enough and any θ ∈ B c (θ0 , 12 δdn ) we have
∗
l1:n ∗
(X1:n |θ) − l1:n (X1:n |θbn∗ ) 6 −ǫnd2n .

It follows, using (2.6) that, a.s. for any n large enough the first integral is
dominated by
Z
k+1 k+1
n 2 exp(−ǫnd2n ) kθ − θbn∗ kk π(θ) dt = n 2 exp(−ǫnd2n ) · O(1)
Θ
k+1
6n 2 n−ǫ log n · O(1) −
→ 0,
1324 T. Launay et al.

since by (2.6) we find that nd2n > (log n)2 for any n large enough. Hence (6.11)
holds for i = 1.
Proof for i = 2. We first recall the multivariate Taylor expansion for a
function g (k+1)-times continuously differentiable within a neighbourhood of
y ∈ Rn . With the usual differential calculus notations
X ∂αg
Dα g(y) · h(α) = (y) · hi1 · · · hiα
∂i1 · · · ∂iα
16i1 ,...,iα 6n

we have
Xk
1 α
g(x) = D g(y) · (x − y)(α) + Rk+1 (x) (6.12)
α=0
α!

where
Z 1
1
Rk+1 (x) = (1 − s)k Dk+1 g(y + s(x − y)) · (x − y)(k+1) ds. (6.13)
(k + 1)! 0

Before expanding the log-likelihood over T2 (δ) in a such a way, we first have
to make sure it is differentiable over the correct domain. Indeed, the strong
consistency of θbn∗ (see Theorem 3.3) implies that a.s., whatever δ0 > 0, for n
large enough,

kθbn∗ − θ0 k < δ0 dn .

For δ chosen small enough, since t ∈ T2 (δ) implies

kθ − θbn∗ k < δdn

it follows from the triangle inequality that a.s. for n large enough,

kθ − θ0 k < (δ0 + δ)dn < dn .

A.s. for any n large enough, t ∈ T2 (δ) hence implies θ ∈ B(θ0 , (δ + δ0 )dn ). We
∗
choose δ0 and δ small enough so that δ + δ0 < 1. This way, θ 7→ l1:n (X1:n |θ) is
guaranteed to be infinitely continuously differentiable over B(θ0 , (δ + δ0 )dn ) ⊂
B(θ0 , dn ).
Now expanding the log-likelihood in a Taylor series for any n large enough,
and taking advantage of the fact that l1:n ∗
(X1:n |θbn∗ ) = 0, we define B1:n ∗
(·) the
symmetric matrix defined for u ∈ Dn by
 2∗ 
∂ l1:n (X1:n |θ) ∂ 2 l1:n
∗
(X1:n |θ) ∂ 2 l1:n ∗
(X1:n |θ)
 ∂γ∂γ ∂γ∂u ∂γ∂σ 2 
 2 ∗ 2 ∗

∗  ∂ l (X 1:n |θ) ∂ l (X 1:n |θ) 
B1:n (θ) = −  1:n 1:n  . (6.14)
 ∂u∂u ∂u∂σ 2 
 ∂ 2 l∗ (X |θ)  1:n 1:n
∂σ 2 ∂σ 2
 Asymptotic results for piecewise linear regression 1325

and write that

1  
∗
l1:n ∗
(X1:n |θ) − l1:n (X1:n |θbn∗ ) = − (θ − θbn∗ )′ B1:n
∗
(θbn∗ ) (θ − θbn∗ )
2
+ R3,n (θ) (6.15)

where
Z 1
1
R3,n (θ) = (1 − s)2 D3 l1:n
∗
(X1:n |θbn∗ + s(θ − θbn∗ )) · (θ − θbn∗ )(3) ds. (6.16)
3! 0

∗
Lemma 7.12 allows us to write that a.s. there exists a constant C ∈ R+ such
that for any n large enough, for any t ∈ T2 (δ)

1 1  
∗
l1:n (X1:n |θbn∗ + n− 2 t) − l1:n
∗
(X1:n |θbn∗ ) = − t′ n−1 B1:n
∗
(θbn∗ ) t + Sn (t) (6.17)
2
where
1
|Sn (t)| 6 Cn− 2 · ktk3 . (6.18)
a.s.
From (6.18), we obtain that for any t ∈ T2 (δ), Sn (t) −−→ 0. Because of
a.s.
Lemma 7.10, we have n−1 B1:n
∗
(θbn∗ ) −−→ I(θ0 ), and it follows immediately that
for any t ∈ T2 (δ),
a.s.
gn (t) −−→ 0,

and thus that

a.s.
ktkk gn (t) −−→ 0.

From (6.18) we also obtain

|Sn (t)| 6 Cδdn ktk2 .

Lemma 7.10, combined with (2.6), (6.17) and the positivity of I(θ0 ), ensures
that a.s. for any n large enough

1 ′  −1 ∗ b∗ 
|Sn (t)| 6 t n B1:n (θn ) t,
4
so that from (6.17), a.s. for any n large enough

(X1:n |θbn∗ )] 6 e− 4 t (n (θbn ))t

1 1 ′ −1 ∗ ∗ 1 ′
∗
exp[l1:n (X1:n |θbn∗ + n− 2 t) − l1:n
∗ B1:n
6 e− 8 t I(θ0 )t .
(6.19)

Therefore, for n large enough, ktkk |gn (t)| is dominated by an integrable function
on the set T2 (δ) and (6.11) holds for i = 2 which completes the proof.
1326 T. Launay et al.

6.2. Proofs of Section 3

Proof of Theorem 3.1. From (6.4), it is easy to see that

2 2
[l1:n (X1:n |θ) − l1:n (X1:n |K)] 6 [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )]
n n !
n
σ02 − σ 2 1 X 2 2
6 −bn (θ) + ξ − σ0
σ 2 σ02 n i=1 i
n
2 1X
− [µ(η0 , ti ) − µ(η, ti )]ξi .
σ 2 n i=1

For any θ′ ∈ Θ and r > 0, let B(θ′ , r) = {θ, ; kθ′ − θk1 < r}. It is now obvious
that
2
[l1:n (X1:n |B(θ′ , r)) − l1:n (X1:n |K)]
n
 2   n 
 σ0 − σ 2   1 X 
2
6 sup {−bn (θ)} + sup  2 2  ·    2
ξi − σ0 
θ∈B(θ ′ ,r) θ∈B(θ ′ ,r) σ σ0 n 
i=1
  ( n )
2 1 X 
 
+ sup 2
· sup  [µ(η0 , ti ) − µ(η, ti )]ξi  . (6.20)
θ∈B(θ ,r)
′ σ θ∈B(θ ,r)
′  n 
i=1

Lemma 7.6 now ensures that

 
1 Xn 
  a.s.
sup  [µ(η0 , ti ) − µ(η, ti )]ξi  −−→ 0,

θ∈B(θ ′ ,r) n i=1 
 2

and σ 2 being bounded away from 0 ensures the boundedness of sup σ2
θ∈B(θ ′ ,r)
which implies
  ( n )
2 1 X 
  a.s.
sup · sup  [µ(η0 , ti ) − µ(η, ti )]ξi  −−→ 0.
θ∈B(θ ′ ,r) σ2 θ∈B(θ ′ ,r)  n i=1 

Since σ 2 is bounded away from 0, taking advantage of the Strong Law of Large
Numbers, we also obtain
 2   n 
 σ − σ02   1 X 
2  a.s.

sup  2 2  ·   2
ξi − σ0  −−→ 0.
θ∈B(θ ′ ,r) σ σ0 n 
i=1

We may thus rewrite (6.20) as

2
[l1:n (X1:n |B(θ′ , r)) − l1:n (X1:n |K)] 6 sup {−bn (θ)} + Rn , (6.21)
n θ∈B(θ ′ ,r)

a.s.
where Rn −−→ 0.
 Asymptotic results for piecewise linear regression 1327

Assume now that θ′ 6= θ0 , then we have

sup |bn (θ) − b(θ′ )| 6 sup |bn (θ) − bn (θ′ )| + |bn (θ′ ) − b(θ′ )|. (6.22)
θ∈B(θ ′ ,r) θ∈B(θ ′ ,r)

Lemma 7.5 (see (7.9)) ensures the existence of a r small enough, say r = r(θ′ ),
such that
1 ′
sup |bn (θ) − bn (θ′ )| 6 b(θ ), (6.23)
θ∈B(θ ′ ,r(θ ′ )) 4

uniformly in n. For n large enough, that same Lemma 7.5 (see (7.10)) also
guarantees that
1 ′
|bn (θ′ ) − b(θ′ )| 6 b(θ ). (6.24)
4
Adding inequalities (6.23) and (6.24) together and combining the result with
(6.22), we deduce that for any n large enough
1 ′
sup |bn (θ) − b(θ′ )| 6 b(θ ),
θ∈B(θ ′,r(θ ′ )) 2

i.e.
1
sup {−bn (θ)} 6 − b(θ′ ),
θ∈B(θ ′ ,r(θ ′ )) 2

which finally gives together with (6.21)

 
1 1
∀θ′ 6= θ0 , P lim sup [l1:n (X1:n |B(θ′ , r(θ′ ))) − l1:n (X1:n |K)] 6 − b(θ′ ) = 1.
→+∞ n
n− 4
(6.25)

Since Lemma 7.5 ensures that b(θ′ ) > 0 for any θ′ 6= θ0 , the previous statement
implies ∀θ′ 6= θ0 ,

P (∃n(θ′ ) ∈ N, ∀n > n(θ′ ), l1:n (X1:n |B(θ′ , r(θ′ ))) − l1:n (X1:n |K) < −1) = 1.
(6.26)

For a given δ > 0, let us now define K(δ) = K \ B(θ0 , δ). K(δ) is obviously a
compact set since K itself is a compact set. By compacity, from the covering
[
B(θ′ , r(θ′ )) ⊃ K(δ),
θ ′ ∈K(δ)

there exists a finite subcovering, i.e.

m(δ)
[
∃m(δ) ∈ N, B(θj′ , r(θj′ )) ⊃ K(δ).
j=1
1328 T. Launay et al.

In particular, (6.26) holds for θ′ = θj′ , j = 1, . . . , m(δ). Let us define

n0 (δ) = max n(θj′ ).

j=1,...,m(δ)

We may now write:

∀δ > 0, ∃n0 (δ) ∈ N, ∃m(δ) ∈ N, ∀j = 1, . . . , m(δ),


P ∀n > n0 (δ), l1:n (X1:n |B(θj′ , r(θj′ ))) − l1:n (X1:n |K) < −1 = 1,
which we turn into:
∀δ > 0, ∃n0 (δ) ∈ N, ∃m(δ) ∈ N,


P ∀n > n0 (δ), ∀j = 1, . . . , m(δ), l1:n (X1:n |B(θj′ , r(θj′ ))) − l1:n (X1:n |K) < −1 = 1,
thanks to the finiteness of m(δ), and finally into
∀δ > 0, ∃n0 (δ) ∈ N, P (∀n > n0 (δ), l1:n (X1:n |K(δ)) − l1:n (X1:n |K) < −1) = 1,
because of the covering
m(δ)
[
B(θj′ , r(θj′ )) ⊃ K(δ).
j=1

Let us now sum up what we have obtained so far. We proved that:

∀δ > 0, ∃n0 (δ) ∈ N,


P if ∀n > n0 (δ), l1:n (X1:n |θ) − l1:n (X1:n |K) > log e−1 , then θ 6∈ K(δ) = 1,
i.e. ∃a = e−1 ∈]0, 1[, ∀δ > 0, ∃n0 (δ) ∈ N,
P (if ∀n > n0 (δ), θ ∈ Kn (a), then kθ − θ0 k1 < δ) = 1,
that is to say
!
∃a ∈]0, 1[, P lim sup kθ − θ0 k1 = 0 = 1.
n−
→+∞ θ∈Kn (a)

Proof of Proposition 3.2. In this proof k · k will refer to the usual Euclidean
norm. Reindexing whenever necessary, we also assume that the observations ti
are ordered, and we denote
t = (t1 , . . . , tn ), X = (X1 , . . . , Xn ), µ0 = (µ(η0 , t1 ), . . . , µ(η0 , tn )),

n
1X
N0,n = sup{i, ti < u0 } = 1[t , +∞[ (u0 ),
i6n n i=1 i
n
1X
Nn = sup{i, ti < u
bn } = 1[t , +∞[ (b
un ),
i6n n i=1 i
 Asymptotic results for piecewise linear regression 1329

 (0, . . . , 0, β0 + γ0 tNn +1 , . . . , β0 + γ0 tN0,n , 0, . . . , 0), if Nn < N0,n
ζ= (0, . . . , 0), if Nn = N0,n ,

(0, . . . , 0, β0 + γ0 tN0,n +1 , . . . , β0 + γ0 tNn , 0, . . . , 0), if Nn > N0,n

Let G be the linear space spanned by the 2 linearly independent n-vectors

v1 = (1, . . . , 1, 0, . . . , 0) v2 = (t1 , . . . , tNn , 0, . . . , 0)

(both of which have their last n − Nn coordinates valued to zero), and denote
Q the orthogonal projection onto G.
Let G + denote the linear space spanned by v1 , v2 and µ0 and denote Q+ the
orthogonal projection onto G + . Observe that G + is also spanned by v1 , v2 and ζ.
Finally, denote µ∗ the orthogonal projection of X onto G + and µ b the closest
point to X in G + satisfying the continuity assumption of the model, i.e.

µ∗ = Q+ X, µ
b = (µ(b
ηn , t1 ), . . . , µ(b
ηn , tn )).

We have

kX − µ∗ k2 + kµ∗ − µ
b||2 = kX − µ
bk2 6 kX − µ0 k2 ,
kX − µ0 k2 − kµ∗ − µ0 k2 + kµ∗ − µ
b||2 6 kX − µ0 k2 ,
kµ∗ − µ0 k2 − 2 hµ∗ − µ0 , µ µ − µ0 k2 6 kµ∗ − µ0 k2 .
b − µ0 i + kb

Thus

µ − µ0 k2 6 2 hµ∗ − µ0 , µ
kb b − µ0 i 6 2kµ∗ − µ0 k · kb
µ − µ0 k,

which leads to

µ − µ0 k 6 2kµ∗ − µ0 k 6 2kQ+ ξk.

kb

Our aim is to show that a.s.

kQ+ ξk = O (log n) . (6.27)

If (6.27) held, then we would have a.s. kb

µ − µ0 k = O (log n) i.e. a.s.
n
X

ηn , ti − µ(η0 , ti ))2 = O log2 n .
(µ(b
i=1

Hence, a.s. for any open interval I ⊂ [u, u] we would have

n
X

ηn , ti − µ(η0 , ti ))2 1I (ti ) = O log2 n .
(µ(b
i=1

This would immediately imply the desired result, i.e. that a.s.
 1 
min |µ(b ηn , ti ) − µ(η0 , ti )| = O n− 2 log n ,
ti ∈I, i6n
1330 T. Launay et al.

since a.s.
n

X
O log2 n = ηn , ti − µ(η0 , ti ))2 1I (ti )
(µ(b
i=1
n
2 1X
>n· min |µ(b
ηn , ti ) − µ(η0 , ti )| · 1I (ti ),
ti ∈I, i6n n i=1

where (see Assumption (A1))

n Z Z Z
1X
1I (ti ) = dFn (t) −
→ dF (t) = f (t) dt > 0.
n i=1 I I I

Let us now prove that (6.27) indeed holds. We consider the two following
mutually exclusive situations.
Situation A: ζ = (0, . . . , 0). In this situation

kQ+ ξk = kQξk, (6.28)

and Cochran’s theorem guarantees that kQξk2 ∼ χ2 (2) for n > 2. Hence, via
Corollary 7.7, a.s.

kQξk = O (log n) , (6.29)

and (6.27) follows from (6.28) and (6.29).

Situation B: ζ 6= (0, . . . , 0). Since

|hζ, ξi|
∼ N (0, σ02 ),
kζk

we also have, via Lemma 7.7, a.s.

|hζ, ξi|
= O (log n) . (6.30)
kζk

Notice that (6.27) follows from (6.29) and (6.30) if we manage to show that a.s.
 
|hζ, ξi|
kQ+ ξk 6 O(1) · kQξk + . (6.31)
kζk

It thus now suffices to prove that a.s., for any g ∈ G

|hζ, gi| = kζk kgk · o(1), (6.32)

where the o(1) mentioned in (6.32) is uniform in g over G (i.e. a.s. ζ is asymp-
totically uniformly orthogonal to G), for (6.31) is a direct consequence of (6.32)
and Lemma 6.1 whose proof is found in Feder (1975).
 Asymptotic results for piecewise linear regression 1331

Lemma 6.1. Let X and Y be two linear subspaces of an inner product space E.
If there exists α < 1 such that
∀(x, y) ∈ X × Y, |hx, yi| 6 αkxk kyk,
then
kx + yk 6 (1 − α)−1 (kx∗ k + ky ∗ k),
where x∗ (resp. y ∗ ) is the orthogonal projection of x + y onto X (resp. Y).
Observe that, as a consequence of Assumption (A1) and Theorem 7.1, the
three following convergences are uniform in u over [u, u] for k = 0, 1, 2,
n Z u Z u Z u
1X k
ti 1[ti , +∞[ (u) = tk dFn (t) −
→ tk dF (t) = tk f (t) dt. (6.33)
n i=1 u u u

We have a.s., for any g(φ) = (cos φ)v1 + (sin φ)v2 ∈ G, with φ ∈ [0, 2π]
 
X N0,n 
 Nn X 

|hζ, g(φ)i| =  (β0 + γ0 ti )(cos φ + ti sin φ) − (β0 + γ0 ti )(cos φ + ti sin φ)
 i=1 i=1 
 
X N0,n 
 Nn X 

6 (max(|u|, |u|) + 1) ·  |β0 + γ0 ti | − |β0 + γ0 ti |
 i=1 i=1 
6 (max(|u|, |u|) + 1) · kζk1
1 1
6 (max(|u|, |u|) + 1) · kζk · n 2 |Nn − N0,n | 2
 n  21
 X
1 1 1X
n 

6 (max(|u|, |u|)+1) · kζk · n 2  1[ti , +∞[ (b
un )− 1[ti , +∞[ (u0 ) ,
n n 
i=1 i=1

i.e. we have a.s. for any φ ∈ [0, 2π]

1
|hζ, g(φ)i| = n 2 kζk · o(1), (6.34)
a.s.
thanks to the strong consistency u bn −−→ u0 (see Theorem 3.1) and the uniform
convergence mentioned in (6.33) with (k = 0). Observe that the o(1) mentioned
in (6.34) is uniform in φ over [0, 2π]. We also have a.s. for any φ ∈ [0, 2π]
n
1 1X
kg(φ)k2 = (cos φ + ti sin φ)2 1[ti , +∞[ (b
un )
n n i=1
n n
1X 1X
= un ) cos2 φ + 2
1[ti , +∞[ (b ti 1[ti , +∞[ (b
un ) cos φ sin φ
n i=1 n i=1
n
1X 2
+ un ) sin2 φ
t 1[t , +∞[ (b
n i=1 i i
Z u0 Z u0 Z u0
a.s.
−−→ cos2 φ f (t) dt + cos φ sin φ 2tf (t) dt + sin2 φ t2 f (t) dt,
u u u
1332 T. Launay et al.

a.s.
once again making use of the strong consistency u bn −−→ u0 (see Theorem 3.1)
and taking advantage of all three uniform convergences mentioned in (6.33). We
thus obviously have a.s., uniformly in φ over [0, 2π]
Z u0
1
kg(φ)k2 −→ (cos φ + t sin φ)2 f (t) dt. (6.35)
n u

The limit in (6.35) is a positive and continuous function of φ, and is hence

bounded, i.e. there exists m > 0 such that we have a.s.
1
kg(φ)k2 > m + o(1), (6.36)
n

i.e.

1 1
= O(n− 2 ), (6.37)
kg(φ)k
1
where the o(1) mentioned in (6.36) and the O(n− 2 ) mentioned in (6.37) are
uniform in φ over [0, 2π].
Combining (6.34) and (6.37) together, we have a.s. for any φ ∈ [0, 2π]

|hζ, g(φ)i| = kζk kg(φ)k · o(1), (6.38)

where the o(1) mentioned in (6.38) is uniform in φ over [0, 2π].

Hence, we have a.s, for any r ∈ R∗+ , and any φ ∈ [0, 2π], now denoting
g(φ) = (r cos φ)v1 + (r sin φ)v2 and applying (6.38) to r−1 g(φ)


|hζ, g(φ)i| = r  ζ, r−1 g(φ)  = r · kζk kr−1 g(φ)k · o(1) = kζk kg(φ)k · o(1),

where the o(1) mentioned is uniform in φ over [0, 2π] and does not depend on r.
We immediately deduce that a.s. (6.32) holds i.e. a.s. ζ is asymptotically
uniformly orthogonal to G, which completes the proof.

6.3. Proofs of Section 4

Proof of Proposition 4.1. We proceed as announced.

Step 1. We first prove that a.s.

b∗n ∈ Dn .
∃N ∈ N, ∀n > N, u

Let us notice that anything proven for the problem remains valid for the pseudo-
problem. Because n∗ ∼ n, we have a.s., thanks to Theorem 3.3 and conditions
(2.6), as n −
→ +∞
1
n 2 (log−1 n) · (b
u∗n − u0 ) = O(1),
1
n 2 (log−1 n) · dn −
→ +∞,
 Asymptotic results for piecewise linear regression 1333

and thus deduce from the ratio of these two quantities that

b∗n − u0 a.s.
u
−−→ 0,
dn
and this directly implies the desired result.
Step 2. Let A∗1:n (·) be the column vector defined for u ∈ Dn by
 ∗    
∂l1:n (X1:n |θ)  ∂l1:n
∗
(X1:n |θ)  ∂l1:n
∗
(X1:n |θ) 
A∗1:n (θ) =  ,  ,  . (6.39)
∂γ θ ∂u θ ∂σ 2 θ

Step 1 allows us to expand a.s. A∗1:n (θbn∗ ) around θ0 using a Taylor-Lagrange

approximation
 
0 = A∗1:n (θbn∗ ) = A∗1:n (θ0 ) − B1:n
∗
(θen ) θbn∗ − θ0 ,

where θen is a point between θbn∗ and θ0 (see (6.14) for the definitions of B1:n
∗
),
and rewrite it as a.s.
1 ∗ e  
∗ 12 b∗ 1

∗
B 1:n (θ n ) · n θ n − θ 0 = n∗− 2 A∗1:n (θ0 ).
n
Since θbn∗ −
→ θ0 , we also have θen − → θ0 and using both Lemmas 7.9 and 7.10 we
immediately find that as n − → +∞
 
1 d
I(θ0 ) · n∗ 2 θbn∗ − θ0 −→ N (0, I(θ0 )) ,

which means, remembering both that n∗ ∼ n and that I(θ0 ) is positive definite
and thus invertible that as n −→ +∞
 

1 d
n 2 θbn∗ − θ0 −→ N 0, I(θ0 )−1 .

Proof of Theorem 3.3. We now prove that

 1 
σn2 − σ02 k = O n− 2 log n .
kb

Variance of noise σ 2 . Observe that

n
1 X
2
bn2 =
σ Xi − γ
bn ti − u
bn )1[ti , +∞[ (b
un
n i=1
n
1 X 2
= γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − b
γn · (ti − u
bn )1[ti , +∞[ (b
un ) + ξi
n i=1
n n n
1X 2 2X 1X 2
= νi (b
ηn ) + νi (b
ηn )ξi + ξ , (6.40)
n i=1 n i=1 n i=1 i
1334 T. Launay et al.

where we denote for i = 1, . . . , n,

νi (η) = γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u)1[ti , +∞[ (u). (6.41)

We have
 
ηn )| = sup γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − b
sup |νi (b γn · (ti − u un )
bn )1[ti , +∞[ (b
i∈N i∈N
 
bn | · sup (ti − u0 )1[t , +∞[ (u0 )
6 |γ0 − γ i
i∈N
 
γn | · sup (ti − u0 )1[ti , +∞[ (u0 ) − (ti − u
+ |b un )
bn )1[ti , +∞[ (b
i∈N
= O (γ0 − γ
bn ) + |b
γn | O (u0 − u
bn ) , (6.42)

using straightforward dominations and Lemma 7.2, so that in the end, thanks
to the previous results we have a.s.
 1 
ηn )| = O n− 2 log n .
sup |νi (b (6.43)
i∈N

It is thus easy to see that a.s.

1X 2
n

 1 
ηn ) = O n−1 log2 n = O n− 2 log n ,
νi (b (6.44)
n i=1

and also that, via Corollary 7.7, a.s.

! 21
2X
n
2
n
X  1 
νi (b
ηn )ξi = νi2 (b
ηn ) · O(log n) = O n− 2 log n . (6.45)
n i=1 n i=1

From the Law of the Iterated Logarithm (see Breiman, 1992, Chapter 13, page
291) we have a.s.

1X 2
n

 1 1
  1 
ξi − σ02 = O n− 2 (log log n) 2 = O n− 2 log n (6.46)
n i=1

and the desired result follows from (6.44), (6.45) and (6.46) put together into
(6.40).
Proof of Theorem 4.2. To finish the proof, we need to show (4.3) i.e. that
 1
bn2 − σ
σ bn2∗ = oP n− 2 .

We use the decomposition (6.40)

n n n
1X 2 2X 1X 2
bn2
σ = νi (b
ηn ) + νi (b
ηn )ξi + ξ ,
n i=1 n i=1 n i=1 i

ηn ) = γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ

where νi (b bn · (ti − u
bn )1[ti , +∞[ (b
un ).
 Asymptotic results for piecewise linear regression 1335

Having proved in Proposition 4.1 that

 1  1
bn∗ − γ0 = OP n− 2 ,
γ b∗n − u0 = OP n− 2
u

we add these relationships to those from (4.2) and find that

 1  1
bn − γ0 = OP n− 2 ,
γ bn − u0 = OP n− 2 .
u (6.47)

We now use (6.47) together with (6.42), we are able to write

 1
ηn )| = OP n− 2 .
sup |νi (b (6.48)
i∈N

It is hence easy to see that

1X 2
n

 1
ηn ) = OP n−1 = oP n− 2 ,
νi (b
n i=1

and also that

! 12
2X
n
2
n
X  1
νi (b
ηn )ξi = νi2 (b
ηn ) · OP (1) = oP n− 2 ,
n i=1 n i=1

which once both substituted into (6.40) yield

n
1X 2  1
bn2 =
σ ξi + oP n− 2 .
n i=1

What was done above with the problem and σbn2 can be done with the pseudo-
2∗
problem and σ
bn without any kind of modification so that
n
1 X 2
∗
 1
bn2∗ =
σ ∗
ξi + oP n− 2 .
n i=1

We observe that
1X 2
n n∗
1 X 2  1
bn2
σ − bn2∗
σ = ξi − ∗ ξi + oP n− 2
n i=1 n i=1
  X n∗ Xn  1
1 1 1
= − ∗ · ξi2 + · ξi2 + oP n− 2
n n i=1
n i=n∗ +1
! !
n∗ − n 1 X 2
n∗
n − n∗ 1 Xn  1
2
= · ξ + · ξ + o P n− 2
n n∗ i=1 i n n − n∗ i=n∗ +1 i
n∗ − n  2   n − n∗   1

· σ02 + OP (n − n∗ )− 2
1
= · σ0 + OP n∗− 2 +
n   n
− 12
+ oP n ,
1336 T. Launay et al.

using the Central Limit Theorem, and in the end we get

n∗ − n   n − n∗    1
1 −1
bn2 − σ
σ bn2∗ = · OP n∗− 2 + · OP (n − n∗ ) 2 + oP n− 2
n n
 1  1 !  1
−2 − 21 n − n∗ 2
= o(1) · OP n + n · OP + o P n− 2
n
 1 1
 1  1
= oP n− 2 + n− 2 · OP (o(1)) + oP n− 2 = oP n− 2 .

7. Technical results

Theorem 7.1 (Polya’s Theorem). Let (gn )n∈N be a sequence of non decreasing
(or non increasing) functions defined over I = [a, b] ⊂ R. If gn converges point-
wise to g (i.e. gn (x) −
→ g(x) as n −
→ +∞, for any x ∈ I) and g is continuous
then

sup |gn (x) − g(x)| −−−−−→ 0.

x∈I n−
→+∞

Proof of Lemma 7.1. Assume the functions gn are non decreasing over I (if not,
consider their opposites −gn ). g is continuous over I and thus bounded since
I is compact. g is also non decreasing over I as the limit of a sequence of non
decreasing functions. Let ǫ > 0 and k > g(b)−g(a)
ǫ such that

∃a = a0 < . . . < ak = b ∈ I k+1 , ∀i = 0, . . . , k − 1, g(ai+1 ) − g(ai ) < ǫ.

Now let x ∈ I and let i ∈ N such that ai 6 x 6 ai+1 . Since gn and g are non
decreasing, we find that

gn (x) − g(x) 6 gn (ai+1 ) − g(ai ) 6 gn (ai+1 ) − g(ai+1 ) + ǫ,

gn (x) − g(x) > gn (ai ) − g(ai+1 ) > gn (ai ) − g(ai ) − ǫ.

The pointwise convergence of gn to g and the finiteness of k together ensure

that

∃N0 ∈ N, ∀n > N0 , ∀i = 0, . . . , k, |gn (ai ) − g(ai )| < ǫ,

which implies with both of the inequations mentioned above that

∃N0 ∈ N, ∀n > N0 , ∀x ∈ I, |gn (x) − g(x)| < ǫ.

Lemma 7.2. Let k ∈ N∗ , there exists a constant C ∈ R∗+ such that for any
(u, u′ ) ∈ [u, u]2

sup |(t − u′ )k 1[t, +∞[ (u′ ) − (t − u)k 1[t, +∞[ (u)| = C|u − u′ |. (7.1)
t∈[u, u]
 Asymptotic results for piecewise linear regression 1337

Proof of Lemma 7.2. For any (u, u′ ) ∈ [u, u]2 we have

sup |(t − u′ )k 1[t, +∞[ (u′ ) − (t − u)k 1[t, +∞[ (u)|

t∈[u, u]

6 sup {|(t − u′ )k − (t − u)k |1[t, +∞[ (u′ )}

t∈[u, u]

+ sup {|t − u|k |1[t, +∞[ (u′ ) − 1[t, +∞[ (u)|}. (7.2)
t∈[u, u]

The mean value theorem guarantees that there exists v between u and u′ such
that

(t − u′ )k − (t − u)k = −k(t − v)k−1 (u′ − u).

We thus have

sup {|(t − u′ )k − (t − u)k |1[t, +∞[ (u′ )} 6 sup |(t − u′ )k − (t − u)k |

t∈[u, u] t∈[u, u]

6 k|u − u|k−1 |u − u′ |. (7.3)

Because |t − u| 6 |u′ − u| whenever |1[t, +∞[ (u′ ) − 1[t, +∞[ (u)| 6= 0, we also find
that

sup {|t − u|k |1[t, +∞[ (u′ ) − 1[t, +∞[ (u)|} 6 |u − u′ |k 6 |u − u′ ||u − u|k−1 .
t∈[u, u]
(7.4)

And now (7.1) is a simple consequence of (7.2), (7.3) and (7.4).

Lemma 7.3. For any η ′ ∈ R × [u, u], there exists C ∈ R∗+ such that for any
η ∈ R × [u, u]

sup |µ(η, t) − µ(η ′ , t)| 6 Ckη − η ′ k. (7.5)

t∈[u, u]

Proof of Lemma 7.3. We have indeed

sup |µ(η, t) − µ(η ′ , t)| = sup |γ · (t − u)1[t, +∞[ (u) − γ ′ (t − u′ )1[t, +∞[ (u′ )|
t∈[u, u] t∈[u, u]
′
6 sup |[γ − γ ](t − u)1[t, +∞[ (u)|
t∈[u, u]

+ sup |γ ′ [(t − u)1[t, +∞[ (u) − (t − u′ )1[t, +∞[ (u′ )]|

t∈[u, u]

6 |γ − γ ′ | · sup |t − u|
t∈[u, u]

+ |γ | · sup |(t − u)1[t, +∞[ (u) − (t − u′ )1[t, +∞[ (u′ )|

′
t∈[u, u]

6 |γ − γ | · |u − u| + |γ ′ | · sup |(t − u)1[t, +∞[ (u) − (t − u′ )1[t, +∞[ (u′ )|.

′
t∈[u, u]

And now (7.5) is a simple consequence of Lemma 7.2.

1338 T. Launay et al.

Lemma 7.4. Let A ⊂ R × [u, u] be a bounded set. Then,

∀ǫ > 0, ∃m(ǫ) ∈ N, ∃η1 , . . . , ηm(ǫ) ∈ A,
∀η, η ′ ∈ A, ∃j, j ′ ∈ {1, . . . , m(ǫ)},
   
sup  µ(η, t) − µ(η ′ , t) − µ(ηj , t) − µ(ηj ′ , t)  < ǫ,
t∈[u, u]

Proof of Lemma 7.4. It suffices to prove the following claim

∀ǫ > 0, ∃m(ǫ) ∈ N, ∃η1 , . . . , ηm(ǫ) ∈ A,

∀η ∈ A, ∃j ∈ {1, . . . , m(ǫ)}, sup |µ(η, t) − µ(ηj , t)| < ǫ.
t∈[u, u]

and then use the triangle inequality. To see that the claim holds, it suffices,
thanks to Lemma 7.3, to exhibit a finite and tight enough grid of A such that
any point in A lies close enough to a point of the grid. The existence of such a
grid is obviously guaranteed since A ⊂ R2 is bounded.
Lemma 7.5. Recall the definition of bn given in (6.2). Let
  Z u
σ02 σ02 1 2
b(θ) = − 1 − log + 2 [µ(η0 , t) − µ(η, t)] f (t) dt. (7.6)
σ2 σ2 σ u

Then, under Assumptions (A1)–(A4),

bn (θ) > 0. (7.7)

b(θ) > 0, with equality if and only if θ = θ0 . (7.8)
bn (θ′ ) −
→ bn (θ), uniformly in n, as θ′ −
→ θ. (7.9)
bn (θ) −
→ b(θ), as n −
→ +∞. (7.10)

Proof of Lemma 7.5. We will prove each claim separately.

Proof of (7.7). That bn (θ) > 0 is trivial since the first term in (6.2) is non
negative (having x − 1 − log x > 0 with equality only if x = 1), and the second
term in (6.2) is obviously non negative too.
Proof of (7.8). That b(θ) > 0 is again easy enough to prove, both terms in
(7.6) being trivially non negative. If θ 6= θ0 then either σ 2 6= σ02 which implies
the first term is positive, or µ(η0 , ·) 6= µ(η, ·) which implies the second term is
positive (since f is assumed positive on [u, u]). Hence if θ 6= θ0 then b(θ) > 0.
That θ = θ0 implies b(θ) = 0 is of course straightforward.
Proof of (7.9). We first observe that
 
1 X n
1X
n 
 ′ 2 2
 (µ(η0 , ti ) − µ(η , ti )) − (µ(η0 , ti ) − µ(η, ti )) 
n n i=1 
i=1
 n 
1 X 
 ′ ′ 
= [2µ(η0 , ti ) − µ(η , ti ) − µ(η, ti )] · [µ(η, ti ) − µ(η , ti )]
n 
i=1
 Asymptotic results for piecewise linear regression 1339

n
1X
6 |2µ(η0 , ti ) − µ(η ′ , ti ) − µ(η, ti )| · |µ(η, ti ) − µ(η ′ , ti )|
n i=1
!
6 sup |µ(η0 , t)−µ(η, t)|+ sup |µ(η ′ , t)−µ(η, t)| · sup |µ(η ′ , t)−µ(η, t)| .
t∈[u, u] t∈[u, u] t∈[u, u]
(7.11)

As θ′ −
→ θ, the convergence of the first term of bn to the first term of b is
obviously uniform in n since this part of bn does not involve n at all. As θ′ −
→ θ,
via Lemma 7.3, we also obtain

sup |µ(η ′ , t) − µ(η, t)| −

→ 0,
t∈[u, u]

which ensures that the second part of (6.2) converges uniformly in n thanks
to (7.11).
Proof of (7.10). Thanks to Assumption (A1), it is easy to see that
n Z u
1X 2 2
[µ(η0 , ti ) − µ(η, ti )] = [µ(η0 , t) − µ(η, t)] dFn (t)
n i=1 u
Z u
2
→
− [µ(η0 , t) − µ(η, t)] dF (t)
Z uu
2
= [µ(η0 , t) − µ(η, t)] f (t) dt.
u

Lemma 7.6. Let A ⊂ R × [u, u] be a bounded set, and let η0 ∈ A, then under
Assumptions (A1)–(A4),
 n 
1 X 
  a.s.
sup  [µ(η0 , ti ) − µ(η, ti )]ξi  −−→ 0.
η∈A  n i=1 

Proof of Lemma 7.6. Let ǫ > 0, η ∈ A, and apply Lemma 7.4 to get the corre-
sponding m(ǫ) ∈ N, {η1 , . . . , ηm(ǫ) } ⊂ A, j, j ′ ∈ {1, . . . , m(ǫ)}. We can decom-
pose the som of the form
n n
1X 1X
[µ(η0 , ti ) − µ(η, ti )]ξi = [µ(ηj , ti ) − µ(ηj ′ , ti )]ξi
n i=1 n i=1
n
1X
+ {[µ(η0 , ti ) − µ(η, ti )] − [µ(ηj , ti ) − µ(ηj ′ , ti )]} ξi
n i=1

with the triangle inequality we obtain

n
1X
[µ(η0 , ti ) − µ(η, ti )]ξi
n i=1
( n
) n
1X 1X
6 sup [µ(ηj , ti ) − µ(ηj ′ , ti )]ξi +ǫ· |ξi | .
(j,j ′ )∈{1,...,m(ǫ)} n i=1 n i=1
1340 T. Launay et al.

Hence
( n
)
1X
sup [µ(η0 , ti ) − µ(η, ti )]ξi
η∈A n i=1
( n ) n
1X 1X
6 sup [µ(ηj , ti ) − µ(ηj , ti )]ξi + ǫ ·
′ |ξi | . (7.12)
(j,j ′ )∈{1,...,m(ǫ)} n i=1 n i=1

Let us now recall Kolmogorov’s criterion, a proof of which is available in Section

17 of Loève (1991) on pages 250–251. This criterion guarantees that for any
sequence (Yi )i∈N of independent random variables and any numerical sequence
(bi )i∈N such that
+∞
X Var Yi
< +∞, bn −
→ +∞,
i=1
b2i

we have
Pn
i=1 (Yi − EYi ) a.s.
−−→ 0.
bn
For each couple (j, j ′ ) ∈ {1, . . . , m(ǫ)}, Kolmogorov’s criterion ensures that
n
1X a.s.
[µ(ηj , ti ) − µ(ηj ′ , ti )]ξi −−→ 0,
n i=1

for the coefficients [µ(ηj , ti ) − µ(ηj ′ , ti )] are obviously bounded, and it suffices
to pick Yi = [µ(ηj , ti ) − µ(ηj ′ , ti )]ξi and bi = i. Having only a finite number of
couples (j, j ′ ) ∈ {1, . . . , m(ǫ)}2 to consider allows us to write
n
1X a.s.
sup [µ(ηj , ti ) − µ(ηj ′ , ti )]ξi −−→ 0. (7.13)
(j,j )∈{1,...,m(ǫ)}
′ n i=1

By (7.13), the first term on the right hand side of (7.12) converges almost surely
to zero. The Strong Law of Large Numbers ensures that the second term on the
1
right hand side of (7.12) converges almost surely to ǫ · (2π −1 σ 2 ) 2 , and the
result follows, since all the work done above for (ξn )n∈N can be done again for
(−ξn )n∈N .
Lemma 7.7. Let (Zi )i∈N be a sequence of independent identically distributed
random variables such that for all i ∈ N, either Zi ∼ N (0, σ 2 ) with σ 2 > 0, or
Zi ∼ χ2 (k) with k > 0. Then a.s., as n −
→ +∞

Zn = O(log n).

Proof of Lemma 7.7. Denote Yn = Zn when the random variables are Gaussian,
and Yn = Zn /5 when the random variables considered are chi-squared (so that
Ee2Y1 and Ee−2Y1 are both finite). We will show that a.s. Yn = O(log n).
 Asymptotic results for piecewise linear regression 1341

For any ǫ > 0, from Markov’s inequality we get:

P n−1 |eYn | > ǫ = P n−2 e2Yn > ǫ2 6 ǫ−2 n−2 Ee2Y1 .
From there it is easy to see that for any ǫ > 0 we have
+∞
X
π2
P n−1 |eYn | > ǫ = ǫ−2 Ee2Y1 < ∞,
n=1
6

which directly implies via Borel-Cantelli’s Lemma (see for example Billingsley,
1995, Section 4, page 59) that a.s.
eYn = o(n).
In particular, a.s. for any n large enough,
Yn 6 log n.
What was done with (Yn )n∈N can be done again with (−Yn )n∈N so that in the
end we have a.s for any n large enough,
− log n 6 Yn 6 log n.

Lemma 7.8. Under Assumptions (A1)–(A4), for any η0 ∈ R × [u, u], there
exists C ∈ R∗+ such that for any n large enough, and for any η
n
X 2
n−1 [µ(η0 , ti ) − µ(η, ti )] > Ckη − η0 k2 .
i=1

Proof of Lemma 7.8. We have already almost proved this result in (3.3) (see
Theorem 3.3). There is however a small difficulty since the majoration was
obtained for τ = (β, γ) and not η = (γ, u).
Let V1 and V2 two non empty open intervals of ]u, u0 [ such that their closures
V1 and V2 are do not overlap. We have
n
X n
X
n−1 [µ(η0 , ti ) − µ(η, ti )]2 > n−1 [µ(η0 , ti ) − µ(η, ti )]2 1V1 (ti )
i=1 i=1
n
!
X 2
+ [µ(η0 , ti ) − µ(η, ti )] 1V2 (ti ) .
i=1

Using the same arguments we used to prove (3.3), we find that there exists
C ∈ R∗+ such that (remembering the definition of the intercept β of the model)
n n n
!
X 2
X X
n−1 [µ(η0 , ti )−µ(η, ti )] > min n−1 1V1 (ti ), n−1 1V2 (ti ) · C|γ −γ0 |2 ,
i=1 i=1 i=1
n n n
!
X 2
X X
−1 −1 −1
n [µ(η0 , ti )−µ(η, ti )] > min n 1V1 (ti ), n 1V2 (ti ) · C|β −β0 |2 ,
i=1 i=1 i=1
1342 T. Launay et al.

and since for j = 1, 2 we have

n
X Z
−1
n 1Vj (ti ) −
→ f (t) dt > 0,
i=1 Vj

there exists C ∈ R∗+ such that for any n large enough

n
X 2
n−1 [µ(η0 , ti ) − µ(η, ti )] > C|γ − γ0 |2 ,
i=1
n
X
n−1 [µ(η0 , ti ) − µ(η, ti )]2 > C|β − β0 |2 .
i=1

Notice now that

|u − u0 | = |γ0−1 β0 − γ −1 β|
= |γ0−1 ||β0 − γ0 γ −1 β|

6 |γ0−1 | |β0 − β| + |β − γ0 γ −1 β|

6 |γ0−1 | |β0 − β| + |γ −1 β| |γ − γ0 |
6 |γ0−1 |(|β0 − β| + |u| |γ − γ0 |)
6 |γ0−1 |(1 + max(|u|, |u|)) · max(|β0 − β|, |γ − γ0 |).

From here, since u ∈ [u, u] is bounded, it is straightforward that there exists

C ∈ R∗+ such that for any n large enough
n
X 2
n−1 [µ(η0 , ti ) − µ(η, ti )] > C|γ − γ0 |2 ,
i=1
n
X
n−1 [µ(η0 , ti ) − µ(η, ti )]2 > C|u − u0 |2 ,
i=1

which ends the proof.

Lemma 7.9. Recall the definition of A∗1:n given in (6.39). Under Assump-
tions (A1)–(A4) and conditions (2.6), as n −
→ +∞
1 d
n− 2 A∗1:n (θ0 ) −
→ N (0, I(θ0 )) . (7.14)

Proof of Lemma 7.9. We will show that any linear combination of the coordi-
nates of A1:n (θ0 ) is asymptotically normal using Lyapounov’s Theorem. Let
α ∈ R3 , kαk 6= 0, so that differential calculus allows us to write
  
∂l∗ (X1:n |θ)  ∗
∂l1:n (X1:n |θ)  ∗
∂l1:n (X1:n |θ) 
hα, A∗1:n (θ0 )i = α1 · 1:n  +α2 ·  +α3 · 
∂γ θ0 ∂u θ0 ∂σ 2 θ0
 Asymptotic results for piecewise linear regression 1343

∗
n
1 X 
= α1 · 2 (ti − u0 )1[ti , +∞[ (u0 ) · ξi
σ0 i=1
n∗ n∗  
γ0 X   1 X 1 2
− α2 · 2 1[ti , +∞[ (u0 ) · ξi + α3 · 2 ·ξ −1
σ0 i=1 2σ0 i=1 σ02 i
∗
n
X
= σ0−2 Zi ,
i=1

where we denote, for i = 1, . . . , n∗

n o
Zi = (ti − u0 )1[ti , +∞[ (u0 ) · α1 − γ0 1[ti , +∞[ (u0 ) · α2 · ξi
1 
+ α3 · σ0−2 · ξi2 − 1 . (7.15)
2
Since for i = 1, . . . , n∗ E[ξi ] = 0 and E[ξi2 ] = σ 2 , we deduce that E [Zi ] = 0, and
hence that E [hα, A∗1:n (θ0 )i] = 0.
Let us now find the expression of Var hα, A∗1:n (θ0 )i. Because ξi and ξj are
independent when i 6= j, so are Zi and Zj and we hence write

Var hα, A∗1:n (θ0 )i

∗
n
X
= σ0−4 Var Zi
i=1
n h
X
∗
i2
= σ0−4 (ti − u0 )1[ti , +∞[ (u0 ) · α1 − γ0 1[ti , +∞[ (u0 ) · α2 · Var ξi
i=1

1 2  −2 2 
+ α3 · Var σ0 ξi − 1 ,
4
  
because Cov ξi , σ0−2 ξi2 − 1 = 0, and we finally get

Var hα, A∗1:n (θ0 )i

Xn∗ h i2 
−4 2 1 2
= σ0 (ti − u0 )1[ti , +∞[ (u0 ) · α1 − γ0 1[ti , +∞[ (u0 ) · α2 · σ0 + α3 · 2 .
i=1
4

We can hence write

n∗−1 Var hα, A∗1:n (θ0 )i

1 Xh i2 1
n∗
= σ0−2 (t i − u 0 )1 [ti , +∞[ (u 0 ) · α1 − γ 0 1 [ti , +∞[ (u 0 ) · α2 + σ0−4 α23
n∗ i=1 2
( n∗ ) ( n∗ )
2 −2 1 X 2 2 −2 2 1 X
= α1 · σ0 (ti − u0 ) 1[ti , +∞[ (u0 ) + α2 · σ0 γ0 1[t , +∞[ (u0 )
n∗ i=1 n∗ i=1 i
1344 T. Launay et al.
( n∗
)
1 X 1
− 2α1 α2 · σ0−2 γ0 (ti − u0 )1[ti , +∞[ (u0 ) + α23 · σ0−4
n∗ i=1 2

= hα, I1:n (θ0 )αi ,

where we denote
 ∗ ∗

n n
1 X 1 X
σ −2
(ti − u)2 1[ti , +∞[ (u) −2
−σ γ ∗ (ti − u)1[ti , +∞[ (u) 0 

 n∗ i=1 n i=1 

n∗
I1:n (θ) = 
∗
−2 2 1 .
 X 
σ γ ∗ 1[t , +∞[ (u) 0 
n i=1 i

 
 1 −4
σ
2
(7.16)

Remark that, by virtue of Assumption (A1), it is easy to check that for any
θ∈Θ
∗
I1:n (θ) −
→ I(θ), (7.17)
∗
and observe that just like I(θ), I1:n (θ) is positive definite, since all its principal
minor determinants are positive.
Let us now check that the random variables Zi meet Lyapounov’s Theorem
(see Billingsley, 1995, page 362) requirements before wrapping up this proof.
The random variables Zi are independent and trivially L2 . We denote Vn∗2 =
Pn∗
i=1 Var Zi and claim that Lyapounov’s condition holds, that is

n ∗  
X  Zi − EZi 2+δ
∃δ > 0, 
E  = o(1).
i=1
V∗ n

Indeed we have (δ = 1)
n∗   n∗  
X  Zi − EZi 3 X  Zi 3
E   = E  
i=1
Vn∗  i=1
V ∗ 
n
∗
n
n∗ 1 X 3
= 3 · ∗
E |Zi |
Var 2 hα, A∗1:n (θ0 )i n i=1
∗
n
1 1 X 3
= 3 · ∗
E |Zi | .
n∗ 2
1
∗
hα, I1:n (θ0 )αi 2 n i=1

1
The first term of this last product is O(n∗− 2 ) thanks to (7.17), and recalling
the definition of Zi from (7.15), there is no difficulty in showing that the last
Pn∗ 3
term of the product, namely n1∗ i=1 E |Zi | converges to a finite limit. Indeed
we find, using trivial dominations and Assumption (A1) once again,
 Asymptotic results for piecewise linear regression 1345


|Zi | =  (ti − u0 )1[ti , +∞[ (u0 ) · α1 − γ0 1[ti , +∞[ (u0 ) · α2 · ξi
3


1  −2 2 3
+ α3 · σ0 · ξi − 1 
2
  
  1  3
E|Zi | 6 (ti − u0 )1[ti , +∞[ (u0 ) · α1 − γ0 1[ti , +∞[ (u0 ) · α2 +  α3 
3   
2
 −2 2 
3
× E |ξi | + σ0 · ξi − 1

and finally
n n 
 1  3
 
1X 1 X 
E|Zi |3 6 (ti − u0 )1[ti , +∞[ (u0 ) · α1 − γ0 1[ti , +∞[ (u0 ) · α2  +  α3 
n i=1 n i=1 2
 −2 2 
3
× E |ξi | + σ0 · ξi − 1
n 
 1  3
 
1 X 
6 O(1) · (ti −u0 )1[ti , +∞[ (u0 ) · α1 −γ0 1[ti , +∞[ (u0 ) · α2  +  α3 
n i=1 2
6 O(1).
Lyapounov’s Theorem thus applies here and leads to
∗
n
X Zi − EZi d
−
→ N (0, 1),
i=1
Vn∗

i.e. multiplying numerator and denominator by σ0−2 we get

hα, A∗1:n (θ0 )i d
1 −
→ N (0, 1),
Var 2 hα, A∗1:n (θ0 )i
that is
hα, A∗1:n (θ0 )i d
1 1 −
→ N (0, 1),
n∗ 2 ∗ (θ )αi 2
hα, I1:n 0

and because of (7.17) we can also write,

hα, A∗1:n (θ0 )i d
1 1 −
→ N (0, 1),
n∗ 2 hα, I(θ0 )αi 2

which, remembering that a.s. n∗ ∼ n, is equivalent to (7.14).

∗
Lemma 7.10. Recall the definition of B1:n given in (6.14). Under Assump-
tions (A1)–(A4) and conditions (2.6), as n −
→ +∞,
1 ∗ a.s.
B (θ0 ) −−→ I(θ0 ), as n −
→ +∞. (7.18)
n 1:n
1 ∗ a.s.
B (θ) −−→ I(θ0 ), as θ −→ θ0 and n − → +∞. (7.19)
n 1:n
where the asymptotic Fisher Information Matrix I(·) is defined in (2.3).
1346 T. Launay et al.

Proof of Lemma 7.10. We will prove each claim separately.

Proof of (7.18). Differential calculus provides the following expressions for
1 ∗
the coefficients of ∗ B1:n (θ) = Bn∗ .
n
∗
n
1 X
(B )
n∗ 11 =σ −2
(ti − u)2 1[ti , +∞[ (u),
n i=1
∗

∗
n
1 X 
(Bn∗ )12 = σ −2 ξi + γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − 2γ · (ti − u) 1[ti , +∞[ (u),
n i=1
∗

∗
n
1 X 
(Bn∗ )13 = σ −4 ξi + γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u) (ti − u)1[ti , +∞[ (u),
n i=1
∗

∗
n
1 X
(Bn∗ )22 = σ −2 γ 2 1[t , +∞[ (u),
n∗ i=1 i
∗
n
1 X 
(Bn∗ )23 = −σ −4 γ ξi + γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u) 1[ti , +∞[ (u),
n∗ i=1
1
(Bn∗ )33 = − σ −4
2
∗
n
1 X 2
+ σ −6 ξi + γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u)1[ti , +∞[ (u) .
n∗ i=1

The convergence we claim is then a direct consequence of Assumption (A1) and

the fact that n∗ ∼ n and, depending on the coefficients, either the Strong Law
of Large Numbers or Kolmogorov’s criterion. Notice that
1 ∗ ∗ a.s.
∗
B1:n (θ0 ) − I1:n (θ0 ) −−→ 0,
n
∗
where I1:n is defined in (7.16).
Proof of (7.19). We will show that in fact, as n − → +∞ and θ −
→ θ0 ,
∗ 1 ∗ 1 ∗ a.s.
C1:n B (θ0 ) − ∗ B1:n
(θ) = (θ) −−→ 0,
n∗ 1:n n
which will end the proof since n∗ ∼ n. We will consider each coefficient of C1:n
∗
(θ)
in turn, making use of Assumption (A1) once again and apply repeatedly the
Strong Law of Large Numbers and Kolmogorov’s criterion as well as Lemma
7.2, whenever needed.
n ∗ n ∗
1 X 1 X
∗
C1:n (θ)11 = σ0−2 (ti − u0 )2 1[ti , +∞[ (u0 ) − σ −2 ∗ (ti − u)2 1[ti , +∞[ (u)
n i=1
∗ n i=1
∗
n

1 X
= σ0−2 − σ −2 · ∗ (ti − u0 )2 1[ti , +∞[ (u0 )
n i=1
∗ ∗ !
n n
1 X 2 1 X
+σ −2
· (t i − u0 ) 1 [t i , +∞[ (u0 ) − (ti − u)2 1[ti , +∞[ (u)
n∗ i=1 n∗ i=1
= o(1) · O(1) + O(1) · O (u − u0 ) −
→ 0.
 Asymptotic results for piecewise linear regression 1347

then last equality holding true because of Lemma 7.2.

∗ ∗
n n
1 X −2 2 1
X
∗
C1:n (θ)22 = σ0−2 γ02 1 [t , +∞[ (u0 ) − σ γ 1[t , +∞[ (u)
n∗ i=1 i n∗ i=1 i
∗
n

1 X
= σ0−2 γ02 − σ −2 γ 2 · ∗ 1[t , +∞[ (u0 )
n i=1 i
n∗ n∗
" #
1 X 1 X
+ σ −2 γ 2 · 1 [t , +∞[ (u 0 ) − 1 [t , +∞[ (u)
n∗ i=1 i n∗ i=1 i

= o(1) · O(1) + O(1) · {Fn∗ (u0 ) − F (u0 )} + {F (u0 ) − F (u)}

+ {F (u) − Fn∗ (u)}
= o(1) + O(1) · [o(1) + o(1) + o(1)] −
→ 0,

the last equality holding true because of the uniform convergence of Fn∗ to
F over any compact subset such as [u, u] (see Assumption (A1), and its Re-
mark 1).
∗
n
1 1 X
∗
C1:n (θ)33 = σ −4 − σ −6 ∗ [ξi + γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u)1[ti , +∞[ (u)]2
2 n i=1
n∗
!
1 −4 −6 1
X 2
− σ − σ0 ∗ ξi
2 0 n i=1
∗
n
1 −4 1 X 2
= (σ − σ0−4 ) − (σ −6 − σ0−6 ) · ∗ ξi
2 n i=1
∗
n
1 X 
− σ −6 γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u)1[ti , +∞[ (u) ξi
n i=1
∗

∗
n
1 X 2
− σ −6 γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u)1[ti , +∞[ (u)
n i=1
∗

∗
n
1 X 2 a.s.
= o(1) + o(1) · ξi + o(1) + o(1) −−→ 0,
n i=1
∗

where the two last o(1) are direct consequences of Lemmas 7.3 and 7.6. Those
same Lemmas used together with Lemma 7.2, the Strong Law of Large Numbers
as well as the well-known Cauchy-Schwarz inequality imply that a.s.
∗
n
∗ 1 X 
C1:n (θ)23 = σ −4 γ ξi + γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u) 1[ti , +∞[ (u)
n i=1
∗

∗
n
1 X
− σ0−4 γ0 ξi 1[ti , +∞[ (u0 )
n∗ i=1
∗
n
1 X  −4 
= σ γ1[ti , +∞[ (u) − σ0−4 γ0 1[ti , +∞[ (u0 ) ξi
n i=1
∗
1348 T. Launay et al.

∗
n
1 X 
+ γ0 · (ti − u0 )1[ti, +∞[ (u0 ) − γ · (ti − u) σ −4 γ1[ti , +∞[ (u)
n i=1
∗

a.s.
= o(1) + o(1) −−→ 0,

and also that a.s.

∗
n
∗ 1 X 
C1:n (θ)13 = σ0−4 (ti − u0 )1[ti , +∞[ (u0 ) ξi
n∗ i=1
∗
n
1 X 
− σ −4 ξi + γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − γ · (ti − u)
n i=1
∗

× (ti − u)1[ti , +∞[ (u)

n∗
1 X  
= σ0−4 (ti − u0 )1[ti , +∞[ (u0 ) − σ −4 (ti − u)1[ti , +∞[ (u) ξi
n∗ i=1
∗
n
−4 1 X 
−σ γ0 · (ti −u0 )1[ti , +∞[ (u0 )−γ · (ti − u) (ti − u)1[ti , +∞[ (u)
n∗ i=1
a.s.
= o(1) + o(1) −−→ 0.

and finally that a.s.

∗
n
∗ 1 X
C1:n (θ)12 = σ0−2 [ξi − γ0 · (ti − u0 )] 1[ti , +∞[ (u0 )
n∗ i=1
∗
n
1 X 
− σ −2 ξi + γ0 · (ti − u0 )1[ti , +∞[ (u0 ) − 2γ · (ti − u) 1[ti , +∞[ (u)
n i=1
∗

∗
n
1 X  
= ξi · σ0−2 1[ti , +∞[ (u0 ) − σ −2 1[ti , +∞[ (u)
n i=1
∗

∗
n
1 X  −2
+ −σ0 γ0 · (ti −u0 )1[ti , +∞[ (u0 )−σ −2 (γ0 · (ti − u0 )1[ti, +∞[ (u0 )
n i=1
∗


−2γ · (ti − u)1[ti , +∞[ (u))
a.s.
= o(1) + o(1) −−→ 0.

Proposition 7.11. Let 0 < δ, and let (ρn )n∈N be a positive sequence such that,
as n −
→ +∞

ρn = O(1) (7.20)
− 12
n (log n) · ρ−1
n →0
− (7.21)

and denote

B c (θ0 , δρn ) = {θ ∈ Θ, kθ − θ0 k > δρn } ,

 Asymptotic results for piecewise linear regression 1349

Then, under Assumptions (A1)–(A4), a.s., there exists ǫ > 0 such that, for any
n large enough
1
sup [l (X1:n |θ) − l1:n (X1:n |θbn )] 6 −ǫ.
2 1:n
(7.22)
θ∈B c (θ 0 ,δρn )
nρ n
1
sup [l (X1:n |θ) − l1:n (X1:n |θ0 )] 6 −ǫ.
2 1:n
(7.23)
θ∈B c (θ0 ,δρn ) nρn

Proof of Proposition 7.11. This proposition is to be compared to the regularity

condition imposed in Ghosh et al. (2006) (see their condition (A4) in Chapter
4). The aim of this proposition is to show that our model satisfies to a somewhat
stronger version of that condition.
Let 0 < δ. Notice first that, similarly to what was done in (6.20), we are able
to deduce that a.s.
2 2
[l1:n (X1:n |θ) − l1:n (X1:n |θbn )] 6 [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] =: in (θ).
n n
(7.24)

where in is defined over R × [u, u] × R∗+ ⊃ Θ ⊃ B c (θ0 , δρn ) by

in (θ) (7.25)
n
X n
X
σ02 1 1
= log +1+ (ξi2 − σ02 ) − [ξi + µ(η0 , ti ) − µ(η, ti )]2 . (7.26)
σ2 nσ02 i=1
nσ 2 i=1
n
σ02 σ02 1 X 2
= log +1− + (ξ − σ02 )
σ2 σ2 nσ02 i=1 i
n
1 X
− 2
[ξi + µ(η0 , ti ) − µ(η, ti )]2 − σ02 . (7.27)
nσ i=1

From (7.24) it is clear that we need only prove (7.23) to end the proof.
The rest of this proof is divided into 6 major steps. Step 1 shows that for a
given n the supremum considered is reached on a point θn . Step 2 and 3 focus
on obtaining useful majorations of the supremum. Step 4 is dedicated to proving
that the sequence θn admits an accumulation point (the coordinates of which
satisfy to some conditions), while step 5 makes use of this last fact to effectively
dominate the supremum. Step 6 wraps up the proof.
Step 1. We first show that a.s. for any n there exists θn ∈ R × [u, u] × R∗+
such that kθn − θ0 k > δρn and

in (θn ) = sup in (θ). (7.28)

Θ∈B c (θ0 ,δρn )

Let n ∈ N and let (θn,k )k∈N be a sequence of points in B c (θ0 , δρn ) such that

lim in (θn,k ) = sup in (θ).

k−
→+∞ Θ∈B c (θ0 ,δρn )
1350 T. Launay et al.

2
From (7.26) it is obvious that σn,k is bounded: if it was not, we would be able
2
to extract a subsequence such that σn,k j
would go to +∞ and thus in (θn,kj )
would go to −∞. For the very same reason, γn,k too is bounded. Recalling that
un,k is bounded too by definition, we now see that there exists a subsequence
(θn,kj )j∈N in B c (θ0 , δρn ) and a point θn in B c (θ0 , δρn ) (i.e. in R × [u, u] × R+ ,
and such that kθn − θ0 k > δρn ) such that (θn,kj )j∈N −−−−−→ θn .
j−→+∞
Finally from (7.26) again it is easy to see that σn2 > 0 for if it was not
in (θn,kj ) would go to −∞ once again, unless (by continuity of µ with regard to
η) ξi + µ(η0 , ti ) − µ(ηn , ti ) = 0 for all i 6 n which a.s. does not happen.
Step 2. From the previous step and the continuity of in with regard to θ we
are able to write
2
sup [l1:n (X1:n |θ) − l1:n (X1:n |θ0 )] = in (θn ). (7.29)
Θ∈B c (θ 0 ,δρn )
n

where (θn )n∈N is the sequence defined in Step 1. We now derive a convenient
majoration of in (θn ). Expanding from (7.27) we get
in (θn )
n n
σ2 σ02
 
1 X 2 1 X
= log 02 + 1 − (ξi − σ02 ) − [ξi + µ(η0 , ti ) − µ(η, ti )]2 − σ02

+ 2
σn σn2 nσ0 i=1 2
nσ i=1
n n
σ2 σ02 σn2 − σ02 X 2
 
2 1 X
= log 02 + 1 − + (ξ i − σ 0 ) − [µ(η0 , ti ) − µ(ηn , ti )]2
σn σn2 nσ02 σn2 i=1 nσn2 i=1
n
2 X
− [µ(η0 , ti ) − µ(ηn , ti )]ξi
nσn2 i=1

Thanks to Lemma 7.8, we know that there exists C1 ∈ R∗+ such that
  n
σ02 σ02 σn2 − σ02 X 2
in (θn ) 6 log 2 + 1 − 2 + (ξ − σ02 )
σn σn nσ02 σn2 i=1 i
n
1 2 2 X
− C1 kηn − η0 k − [µ(η0 , ti ) − µ(ηn , ti )]ξi
σn2 nσn2 i=1

From there, the Law of the Iterated Logarithm and a factorisation of the last
term together with Corollary 7.7 lead to:
 
σ02 σ02 1 1
in (θn ) 6 log 2 + 1 − 2 + 2 |σn2 − σ02 |R1,n − 2 C1 kηn − η0 k2
σn σn σn σn
n
! 12
1 X
2
+ [µ(η0 , ti ) − µ(ηn , ti )] R2,n
nσn2 i=1
1 1
where a.s. R1,n = O(n− 2 (log log n) 2 ) and R2,n = O (log n). Lemma 7.3 ensures
there exists C2 ∈ R∗+ such that
 Asymptotic results for piecewise linear regression 1351
 
σ02 σ02 1
in (θn ) 6 log 2 + 1 − 2 + 2 |σn2 − σ02 |R1,n
σn σn σn
1 1 1
− 2 C1 kηn − η0 k2 + C2 n 2 kηn − η0 kR2,n
σn nσn2
We thus deduce that there exists C ∈ R∗+ such that:
 
σ2 σ2 1 1
in (θn ) 6 log 20 + 1 − 20 − 2 Ckηn − η0 k2 + 2 kθn − θ0 kRn (7.30)
σn σn σn σn
1
where a.s. Rn = O(n− 2 log n).
Notice in particular that, due to (7.21), Rn = o(ρn ).
Step 3. We obtain two majorations, (7.32) and (7.33), that we will make use
of in the coming steps. Using a conversion of θ = (γ, u, σ 2 ) into the spherical
coordinate system we write θn as
θn = (rn cos ψn cos φn , rn sin ψn cos φn , rn sin φn ),
where
(rn , ψn , φn ) ∈ R∗+ × [0, 2π]×]0, π[,
and deduce from (7.30) that
 
σ02 σ02 cos2 φn 1
in (θn ) 6 log +1− − Crn + Rn (7.31)
rn sin φn rn sin φn sin φn sin φn
 
σ02 σ02 1  
6 log +1− + Rn − Crn cos2 φn . (7.32)
rn sin φn rn sin φn sin φn
From (7.31) we also get the following majoration
 
σ02 σ02 1
in (θn ) 6 log +1− + Rn . (7.33)
rn sin φn rn sin φn sin φn
Step 4. We show that the sequence (θn )n∈N we built, converges to a finite
limit θ∞ (extracting a subsequence if necessary). Extracting a subsequence if
necessary, we can assume that (ψn , φn ) −→ (ψ∞ , φ∞ ) ∈ [0, 2π] × [0, π]. We
consider the two following mutually exclusive situations.
Situation A: φ∞ = 0 mod π. In this situation, there exists ǫ > 0 such that
for any n large enough,
 
  Rn
Rn − Crn cos2 φn = − C cos2 φn rn
rn
6 −ǫrn ,
because a.s. Rn = o(rn ) (since Rn = o(ρn ) and rn 6 ρn ). Used together with
(7.32), this leads to
 
σ02 σ02 rn
in (θn ) 6 − 1 − log −ǫ ,
rn sin φn rn sin φn sin φn
for any n large enough and hence in (θn ) −
→ −∞ whether rn goes to zero or not.
1352 T. Launay et al.

Situation B: φ∞ 6= 0 mod π. In this situation, from (7.33), we see that

rn −→ 0 and rn −→ +∞ both lead to in (θn ) − → −∞.
Observing that in (θ) converges a.s. to a finite value for any θ ∈ Θ as n −
→ +∞,
→∞ in (θn ) = −∞ is not possible by construction of the sequence
we see that limn−
θn , and deduce that, extracting a subsequence if necessary, there exists

(r∞ , ψ∞ , φ∞ ) ∈ R∗+ × [0, 2π]×]0, π[,

2
such that limn−
→+∞ θn = θ∞ . Notice that in particular, σ∞ > 0.
Step 5. We will now end the proof by showing that there exists ǫ > 0 such
that for any n large enough

in (θn ) 6 −ǫρ2n . (7.34)

We consider the two following mutually exclusive situations.

2
Situation A: σ∞ 6= σ02 . In this situation, from (7.30) we get
 
σ02 σ02 1
in (θn ) 6 log 2 + 1 − 2 + 2 kθn − θ0 k Rn
σn σn σn

and the right-hand side converges to

 
σ2 σ2
log 20 + 1 − 20 < 0.
σ∞ σ∞

There hence exists ǫ > 0 such that for any n large enough

in (θn ) 6 −ǫ.

Since ρn = O(1) by (7.20), (7.34) is a direct consequence of this.

2
Situation B: σ∞ = σ02 . In this situation, recalling that for any x > 0

(x − 1)2 (x − 1)3
log x + 1 − x 6 − + ,
2 3
we deduce from (7.30) that for any n large enough
 2  3
1 σ02 1 σ02 1 1
in (θn ) 6 − 2
−1 + 2
− 1 − 2 Ckηn − η0 k2 + 2 kθn − θ0 k Rn
2 σn 3 σn σn σn
 2 2   2  
σ0 1 σ0 1 1 1
6 −1 −1 − − 2 Ckηn − η0 k2 + 2 kθn − θ0 k Rn
σn2 3 σn2 2 σn σn
 2
1 σ02 1 1
6− 2
− 1 − 2 Ckηn − η0 k2 + 2 kθn − θ0 k Rn
4 σn σn σn
1   2 
6 2 −c (σ0 − σn2 )2 − kηn − η0 k2 + kθn − θ0 kRn
σn
 Asymptotic results for piecewise linear regression 1353

where c = min(1/4, C) > 0. It follows that for any n large enough

1

in (θn ) 6 −ckθn − θ0 k2 + kθn − θ0 kRn
σn2
1
6 2 kθn − θ0 k (Rn − ckθn − θ0 k) .
σn
Thus, for any n large enough
 
1 kθn − θ0 k Rn kθn − θ0 k
in (θn ) 6 2 −c ρ2n .
σn ρn ρn ρn
Recalling that
kθn − θ0 k > δρn ,
Rn = o(ρn ),
σn2 − 2
→ σ∞ >0
we obtain for any n large enough,
 
1 kθn − θ0 k δ cδ 2 cδ 2
in (θn ) 6 2 −c ρ2n 6 − 2 ρ2n 6 − 2 ρ2n .
σn ρn 2 2σn 3σ∞
cδ 2
Hence (7.34) holds in this situation too: it suffices to take ǫ = 2
.
3σ∞
We just proved that (7.34) holds in both cases considered.
Step 6. (7.23) is a consequence of (7.29) and (7.34).

Lemma 7.12. Let 0 < δ < 1 then under Assumptions (A1)–(A4) and condi-
∗
tions (2.6), a.s. there exists a constant C ∈ R+ such that for any n large enough
and for any 1 6 i1 , i2 , i3 6 3
 
 1 ∂ 3 l1:n
∗
(X1:n |θ) 

 n ∂i ∂i ∂i 6C (7.35)
1 2 3

for any θ ∈ B(θ0 , δdn ).

Proof of Lemma 7.12. Let 0 < δ < 1. We will prove (7.35) stands true for any
∗
1 6 i1 , i2 , i3 6 3. First notice that for n large enough, θ 7→ l1:n (X1:n |θ) is
indeed infinitely continuously differentiable over B(θ0 , δdn ) by definition of the
pseudo-problem. Any θ subsequently considered within this proof is assumed to
belong to B(θ0 , δdn ). Any convergence subsequently mentioned within this proof
is uniform in θ for θ ∈ B(θ0 , δdn ) for any n large enough thanks to Theorem 7.1
and Lemma 7.6.
Proof of (7.35) for β = (3, 0, 0).
1 ∂ 3 l1:n
∗
(X1:n |θ)
= 0.
n (∂γ)3
1354 T. Launay et al.

Proof of (7.35) for β = (2, 1, 0).

   n∗ 
 1 ∂ 3 l1:n
∗
(X |θ)  2 1 X 
 1:n =  
 n (∂γ)2 ∂u  σ 2  n  (t i − u)1 ]t , +∞[ (u) 
i

i=1
Z u 
2   2
−−−−−→ 4  (t − u)f (t) dt 6 2 |u − u|.
n−→+∞ σ u σ

Proof of (7.35) for β = (2, 0, 1).

   
 1 ∂ 3 l1:n
∗
(X |θ)  1 1 Xn∗ 
 1:n =  2 
 n (∂γ)2 ∂σ 2  σ 4  n  (t i − u) 1 ]t , +∞[ (u)
i

i=1
Z u 
1   1
−−−−−→ 4  (t − u)2 f (t) dt 6 4 |u − u|2 .
n−→+∞ σ u σ

Proof of (7.35) for β = (1, 2, 0).

   
 1 ∂ 3 l1:n
∗
(X |θ)  2  1Xn∗ 
 1:n   
 n ∂γ · (∂u)2  = σ 2 γ n 1]ti , +∞[ (u)

i=1
 Z u 
2   2
−−−−−→ 2 γ f (t) dt 6 2 |γ|.
n−
→+∞ σ u σ

Proof of (7.35) for β = (1, 1, 1).

 
 1 ∂ 3 l1:n
∗
(X1:n |θ) 

 n ∂γ∂u∂σ 2 
 
1  1 X 
n∗

= 4 (Xi − 2γ · (ti − u))1]ti , +∞[ (u)
σ  n i=1 
 
1  1 X  
n∗
 
= 4 ξi + γ0 · (ti − u0 )1]ti , +∞[ (u0 ) − 2γ · (ti − u) 1]ti , +∞[ (u)
σ  n i=1 
Z Z u 
a.s. 1  min(u,u0 ) 

−−−−−→ 4  γ0 · (t − u0 )f (t) dt − 2 γ · (t − u)f (t) dt
n−
→+∞ σ  u u 

And this limit is bounded by σ34 |u − u|(|γ| + |γ0 |).

Proof of (7.35) for β = (1, 0, 2).
 
 1 ∂ 3 l1:n
∗
(X1:n |θ) 

 n ∂γ · (∂σ 2 )2 
 n∗ 
2  1 X   

= 6 xi − γ · (ti − u)1]ti , +∞[ (u) (ti − u)1]ti , +∞[ (u)
σ  n i=1 
 Asymptotic results for piecewise linear regression 1355
 n∗ 
2 1 X  
 
= 6  ξi + γ0 · (ti − u0 )1]ti , +∞[ (u0 ) − γ · (ti − u) (ti − u)1]ti , +∞[ (u)
σ n 
i=1
Z Z u 
a.s. 2  min(u,u0 ) 

−−−−−→ 6  γ0 · (t − u0 )(t − u)f (t) dt − γ · (t − u)2 f (t) dt
n−
→+∞ σ  u u 

And this limit is bounded by σ46 |u − u|2 (|γ| + |γ0 |).

Proof of (7.35) for β = (0, 3, 0).
 
 1 ∂ 3 l1:n
∗
(X1:n |θ) 

n (∂u)3  = 0.

Proof of (7.35) for β = (0, 2, 1).

   
 1 ∂ 3 l1:n
∗
(X |θ)  1 1 X
n∗ 
 1:n  2  
 n (∂u)2 ∂σ 2  = σ 4 γ  n 1]ti , +∞[ (u)

i=1
Z u 
1   1
−−−−−→ 4 γ 2  f (t) dt 6 4 γ 2 .
n−
→+∞ σ u σ

Proof of (7.35) for β = (0, 1, 2).

 
 1 ∂ 3 l1:n
∗
(X1:n |θ) 

 n ∂u(∂σ 2 )2 
 
2  1 X  
n∗
 
= 6 xi − γ · (ti − u)1]ti , +∞[ (u) γ1]ti , +∞[ (u)

σ n i=1 
 
2  1 X  
n∗
 
= 6 ξi + γ0 · (ti − u0 )1]ti , +∞[ (u0 ) − γ · (ti − u) γ1]ti , +∞[ (u)
σ  n i=1 
Z u0 Z u 
a.s. 2  
−−−−−→ 6  γγ0 · (t − u0 )f (t) dt − γ 2 (t − u)f (t) dt
n−
→+∞ σ u u

And this limit is bounded by σ26 |u − u|(|γ 2 | + |γ0 γ|).

Proof of (7.35) for β = (0, 0, 3).
 
 1 ∂ 3 l1:n
∗
(X1:n |θ) 

n (∂σ 2 )3 
 
1  2 
n∗
3 1 X
= 6 −1 + 2 xi − γ · (ti − u)1]ti , +∞[ (u) 
σ  σ n i=1 
 
1  2 
∗
n
3 1 X
= 6 −1 + 2 ξi + γ · (ti − u0 )1]ti , +∞[ (u0 ) − γ · (ti − u)1]ti , +∞[ (u) 
σ  σ n i=1 
1356 T. Launay et al.
  Z u0
a.s. 1  2 2
−−−−−→ 8 −σ + 3 σ0 + γ02 (t − u0 )2 f (t) dt
n−
→+∞ σ u
Z min(u,u0 ) Z !
u 

−2 γγ0 · (t − u0 )(t − u)f (t) dt + γ 2 (t − u)2 f (t) dt 
u u 
 
And this limit is bounded by σ18 3σ02 + σ 2 + (|γ| + |γ0 |)2 (u − u)2 .
(7.35) is thus a direct consequence of both the uniform convergences men-
tioned above and the trivial majoration of all the limits involved by a fixed
constant C for any n large enough.

Acknowledgements

The authors would like to thank the referees and associate editor for their con-
structive comments.

References

Billingsley, P. (1995). Probability and Measure. Wiley, 3rd edition.

MR1324786
Billingsley, P. (1999). Convergence of Probability Measures. Wiley, 2nd
edition. MR1700749
Breiman, L. (1992). Probability. SIAM. MR1163370
Bruhns, A., Deurveilher, G., and Roy, J. (2005). A non-linear regres-
sion model for mid-term load forecasting and improvements in seasonnality.
Proceedings of the 15th Power Systems Computation Conference 2005, Liege
Belgium.
Dacunha-Castelle, D. (1978). Vitesse de convergence pour certains
problèmes statistiques. In École d’Été de Probabilités de Saint-Flour, VII
(Saint-Flour, 1977), volume 678 of Lecture Notes in Math., pages 1–172.
Springer, Berlin. MR0518733
Feder, P. I. (1975). On asymptotic distribution theory in segmented regression
problems – identified case. The Annals of Statistics, 3(1):49–83. MR0378267
Ghosal, S. and Samanta, T. (1995). Asymptotic behaviour of Bayes esti-
mates and posterior distributions in multiparameter nonregular cases. Math.
Methods Statist., 4(4):361–388. MR1372011
Ghosh, J. K., Delampady, M., and Samanta, T. (2006). An Introduction
to Bayesian Analysis, Theory and Methods. Springer. MR2247439
Ghosh, J. K., Ghosal, S., and Samanta, T. (1994). Stability and conver-
gence of the posterior in non-regular problems. In Statistical decision theory
and related topics, V (West Lafayette, IN, 1992), pages 183–199. Springer,
New York. MR1286304
Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics.
Springer. MR1992245
Ibragimov, I. and Has’minskii, R. (1981). Statistical Estimation Asymptotic
Theory. Springer. MR0620321
 Asymptotic results for piecewise linear regression 1357

Launay, T., Philippe, A., and Lamarche, S. (2012). Construction of an

informative hierarchical prior distribution. application to electricity load fore-
casting. Preprint. arXiv:1109.4533.
Lehmann, E. L. (2004). Elements of large-sample theory. Springer Texts in
Statistics. Springer-Verlag, New York. MR1663158
Loève, M. (1991). Proability Theory I. Springer, 4th edition.
Polya, G. and Szegö, G. (2004). Problems and Theorems in Analysis I.
Springer.
Sareen, S. (2003). Reference bayesian inference in non-regular models. Journal
of Econometrics, 113:265–288. MR1963197
Sylwester, D. L. (1965). On maximum likelihood estimation for two-phase
linear regression. Technical Report 11, Department of Statistics, Stanford
University. MR2615310