Robust Linear Calibration

CHRISTOS P. KITSOS and CHRISTINE H. MÜLLER Athens University of Business and Economics, Freie Universität Berlin

March 1995

Summary. We regard the simple linear calibration problem where only the response y of the regression line y = β_{0} + β_{1}t is observed with errors. The experimental conditions t are observed without error. For the errors of the observations y we assume that there may be some gross errors providing outlying observations. This situation can be modeled by a conditionally contaminated regression model. In this model the classical calibration estimator based on the least squares estimator has an unbounded asymptotic bias. Therefore we introduce calibration estimators based on robust one-step-M-estimators which have a bounded asymptotic bias. For this class of estimators we discuss two problems: The optimal estimators and their corresponding optimal designs. We derive the locally optimal solutions and show that the maximin efficient designs for non-robust estimation and robust estimation coincide.

AMS 1991 subject classifications: 62J05, 62F35, 62G20, 62K05

Key words: Linear calibration, classical estimator, conditional contamination, robust estimation, one-step-M-estimator, local optimality, optimal design, asymptotic efficiency, maximin efficiency.

1 Introduction

In the linear calibration set-up the simple linear regression model

Y (t) = β_{0} + β_{1}t + Z = x(t)′β + Z with t ∈ IR

is assumed where β = (β_{0}, β_{1})′ is unknown and x(t) = (1, t)′ is the known regression function. Given a favorite value y of Y (t), the corresponding experimental condition t_{y} should be estimated so as to reproduce y if there is no error, Z = 0. In particular, if the expected value of the error Z is equal to zero, the experimental condition t_{y} should be that condition which provides E(Y (t_{y})) = y. Hence, the interest is focused on the non-linear aspect

φ(β) := φ_{y}(β) := t_{y} = \frac{y − β_{0}}{β_{1}}

of the linear regression model. This non-linear aspect can be estimated by repeating the linear regression experiment at given experimental conditions t_{1N}, ..., t_{NN} ∈ T := [−1, 1] and estimating φ(β) from the observations Y_{nN} given by

Y_{nN} = β_{0} + β_{1}t_{nN} + Z_{nN}.

In absolute calibration problems (see for example Buonaccorsi (1986), Osborne (1991) and Kitsos (1992)) it is assumed that the errors Z_{1N},...,Z_{NN} are independent and identically distributed with expectation E(Z_{nN}) = 0 and that the experimental conditions t_{nN} are without errors. For these problems the so-called ”classical estimator” φ(β̂_{N}^{∗}) based on the least squares estimator β̂_{N}^{∗} for β should be used (see Eisenhart (1939), Osborne (1991)). But, if with some probability, at a given experimental condition t_{nN}, an outlying observation caused by a gross error may occur, then the expected value of Z_{nN} is not any more zero. Then a conditionally contaminated linear regression model is more adequate. In such a model it is assumed that the error variables Z_{1N}, ..., Z_{NN} are independent and distributed according to a contaminated normal distribution where the contamination may be different for different experimental conditions. I.e.

\frac{1}{σ} Z_{nN} ∼ Q_{nN}(dz) := (1 − N^{−1/2}R ϵ(t_{nN})) n_{(0,1)}(dz) (1.1)

+ N^{−1/2}R ϵ(t_{nN}) g(z, t_{nN}) n_{(0,1)}(dz)

where n_{(0,1)} is the standard normal distribution, R > 0, \frac{1}{N} ∑_{n=1}^{N} ϵ(t_{nN}) ≤ 1 for almost all N ∈ IN and ∫ g(z, t) n_{(0,1)}(dz) = 1, g(z, t) ≥ 0 for all z ∈ IR, t ∈ T. Thereby the Markov kernel g(·, t)n_{(0,1)} models the form of contamination, and ϵ(t) ≥ 0 the amount of contamination. The set P_{R} of all sequences (Q^{N} := ⨂_{n=1}^{N} Q_{nN})_{N∈IN} defines a contamination neighbourhood around the classical model (P^{N})_{N∈IN} := ((n_{(0,1)})^{N})_{N∈IN}. If (Z_{1N}, ..., Z_{NN}) is distributed according to Q^{N} and β is the true parameter vector, then we denote by Q_{β}^{N} the distribution of (Y_{1N}, ..., Y_{NN}), in particular by P_{β}^{N} the distribution of (Y_{1N}, ..., Y_{NN}) for the classical model.

Estimating in this model the non-linear aspect φ(β) by the estimator φ(β̂_{N}^{∗}), where β̂_{N}^{∗} is the least squares estimator, an unbounded asymptotic bias appears. I.e. \sqrt{N}(φ(β̂_{N}^{∗}) − φ(β)) is under (Q_{β}^{N})_{N∈IN} asymptotically normally distributed with mean (bias) b_{β}((Q^{N})_{N∈IN}), where sup{b_{β}((Q^{N})_{N∈IN}); (Q^{N})_{N∈IN} ∈ P_{R}} = ∞. But robust estimators have a bounded asymptotic bias, whence the estimator φ(β̂_{N}^{∗}) is not robust. Therefore in Section 2 robust estimators for φ(β) are proposed for which the asymptotic bias is bounded. In Section 3 several strategies are discussed for obtaining optimal robust estimators and optimal designs for robust estimation. In particular a maximin efficiency criterion for robust estimation is proposed. For this criterion locally optimal robust estimators and locally optimal designs for robust estimation are defined as those estimators and designs which minimize the asymptotic variance under the side condition that the asymptotic bias is bounded by some local bound b(β) < ∞. Then the efficiency of a given robust estimator at a given design is measured by comparing its asymptotic variance with the asymptotic variance of the locally optimal robust estimator at the locally optimal design. This efficiency criterion is similar to the classical efficiency criterion for non-robust estimation of a non-linear aspect (see Silvey (1980)), where a given design is compared with the locally optimal design. In Section 4 the locally optimal robust estimators at the locally optimal designs are derived. In Section 5 the locally optimal solutions are used to show that the maximin efficient design for robust estimation coincides with the maximin efficient design for non-robust estimation which was derived by Kitsos (1992). All proofs are given in Section 6.

2 Estimators based on robust estimators for β

In a conditionally contaminated linear calibration model the non-linear aspect φ(β) should be estimated with a robust estimator. Such an estimator can be of the form φ(β̂_{N}) where β̂_{N} is an optimal robust one-step-M-estimator for β. A one-step-M-estimators for β with score function ψ : IR × T → IR^{2} has the form

β̂_{N}(y_{N}, d_{N}) = β̂_{N}^{0}(y_{N}, d_{N}) + \frac{1}{N} ∑_{n=1}^{N} ψ ( \frac{y_{nN} −x(t_{nN})′_{β_{N}^{0}} (y_{N} ,d_{N})}{_{σ_{N}}}, t_{nN} ) σ̂_{N}

where d_{N} := (t_{1N}, ..., t_{NN}) and y_{N} := (y_{1N}, ..., y_{NN}). The inital estimator β̂_{N}^{0} may be the least squares estimator or some robust estimator for β, for example a M-estimator. The variance estimator σ̂_{N} may be the mean squared residuals or some robust estimator for the variance, for example Huber’s Proposal 2 (Huber (1964)).

If d_{N} = (t_{1N}, ..., t_{NN}) is the design for the calibration experiment then we can define a corresponding design measure by

δ_{N} := \frac{1}{N} ∑_{n=1}^{N} e_{t_{nN}}

where e_{t} denotes the Dirac measure on t. Now, assume that a design measure δ and a finite set T_{0} ⊂ T exist, so that the supports of δ and all δ_{N} are included in T_{0} and that for all t ∈ T_{0} we have

lim_{N→∞} δ_{N}(t) = δ(t).

Moreover assume that the score function ψ satisfies

(2.1) ψ(z, t) = ψ_{0}(z, t) + c(t)sgn(z), where for all t ∈ T_{0}

ψ_{0}(·, t) is antisymmetric, continuous and there exists l_{1}(t), ..., l_{L}(t) so that
ψ_{0}(·, t) has bounded and continuous derivatives on IR \ {l_{1}(t), ..., l_{L}(t)},

and

∫ ψ(z, t)x(t)′zP (dz)δ(dt) = E_{p×p} (2.2)

where E_{p×p} denotes the identity matrix.

Then under some tightness conditions on the initial and variance estimator the onestep-M-estimator β̂_{N} is asymptotically normally distributed for all contaminated error distributions (see Müller (1994b)), i.e.

L(\sqrt{N}(β̂_{N} − β)|Q_{β}^{N}) (2.3)

⟶^{N→∞} N (b(ψ, (Q^{N})_{N∈IN}), σ^{2} ∫ ψ ψ′ d(P ⊗δ) )
for all (Q^{N})_{N∈IN} ∈ P_{R} and all β ∈ IR^{2}

with maximum asymptotic bias

sup{|b(ψ, (Q^{N})_{N∈IN})|; (Q^{N})_{N∈IN} ∈ P_{R}} = R σ ess sup_{P ⊗δ} |ψ|.

The following theorem shows that an estimator φ(β̂_{N}) based on an asymptotically normally distributed one-step-M-estimator β̂_{N} is also asymptotically normally distributed. For that we have

B := {β = (β_{0}, β_{1})′ ∈ IR^{2}; β_{1} ≠ 0},

and

φ̇_{β} := \frac{∂}{∂β} φ(β) = \frac{−1}{β_{1}} (1, φ(β)).

Lemma 2.1 If β̂_{N} is an asymptotically normally distributed one-step-M-estimator for β, then

\sqrt{N}(φ(β̂_{N}) − φ(β)) − \sqrt{N}(φ̇_{β}β̂_{N} − φ̇_{β}β) ⟶^{N→∞} 0

in probability for (P_{β}^{N})_{N∈IN} for all β ∈ B.

Using this lemma we get the following theorem.

Theorem 2.1 If β̂_{N} is an asymptotically normally distributed one-step-M-estimator for β with score function ψ, then also φ(β̂_{N}) is asymptotically normally distributed for all contaminated error distributions and all β ∈ B, i.e.

L(\sqrt{N}(φ(β̂_{N}) − φ(β))|Q_{β}^{N}) (2.4)

⟶^{N→∞} N (φ̇_{β} b(ψ, (Q^{N})_{N∈IN}), σ^{2} φ̇_{β} ∫ ψψ′ d(P ⊗δ) φ̇′_{β})
for all (Q^{N})_{N∈IN} ∈ P_{R} and all β ∈ B,

with maximum asymptotic bias

sup{|φ̇_{β} b(ψ, (Q^{N})_{N∈IN})|; (Q^{N})_{N∈IN} ∈ P_{R}} = R σ ess sup_{P ⊗δ} |φ̇_{β} ψ| for all β ∈ B.

Without loss of generality we now can assume that the variance σ^{2} as well as the radius R of the contamination neighbourhood P_{R} are equal to 1.

Recall that the least squares estimator behaves asymptotically like a one-step-M-estimator with score function

ψ_{∞}(z, t) := I(δ)^{−1}x(t) z

where I(δ) := ∫ x(t) x(t)′ δ(dt). Hence the maximum asymptotic bias of an estimator for φ(β) which is based on the least squares estimator is equal to ess sup_{P ⊗δ} |φ̇_{β}ψ_{∞}| = ∞. For robust estimators the maximum asymptotic bias should be less than infinity. But this is only satisfied if the score function of the underlying one-step-M-estimator for β is bounded, i.e. if the underlying one-step-M-estimator is a robust estimator for β.

To get ”good” robust estimators for φ(β) we can base the estimator φ(β̂_{N}) on an optimal robust one-step-M-estimator β̂_{N} for β. This will not provide an optimal robust estimator for φ(β) but it will provide a reasonable choice of a robust estimator for φ(β) because a globally optimal robust estimator for φ(β) does not exist (see Section 3). Optimal robust estimators β̂_{N}^{b}, say, for β are those estimators which minimize the trace of the asymptotic covariance matrix under the side condition that the maximum asymptotic bias is bounded by some bias bound b. Because only the score functions of the one-step-M-estimators determine the asymptotic behaviour, the optimal robust estimators are given by their score function. A general characteriztion of these optimal score functions is given in Krasker (1980). For designed experiments Kurotschka and Müller (1992) gave a more explicit characterization of these score functions. In particular if we use the design

δ_{p} := p e_{1} + (1 − p) e_{−1}, with p ∈ (0, 1),

then the optimal score function ψ_{b,p} for robust estimation of β with bias bound b has the form

ψ_{b,p}(z, t) = \{ \begin{matrix}sgn(z) \frac{b}{\sqrt{2}} (1, t)′ & for \frac{\sqrt{π}}{2 δ_{p}(\{t\})} = b, \\ sgn(z) \frac{min\{|z|,b y_{b,p}(t)\}}{\sqrt{2} y_{b,p}(t)} (1, t)′ & for \frac{\sqrt{π}}{2 δ_{p}(\{t\})} < b, \end{matrix}

for t ∈ \{1, −1\} where y_{b,p}(t) is the positive solution of y = δ_{p}(\{t\})\sqrt{2}(2Φ(b y) − 1). This optimal score function exists if and only if b ≥ max\{\frac{1}{\sqrt{2}p}, \frac{1}{\sqrt{2}(1−p)}\}\sqrt{\frac{π}{2}}. Note that for deriving the optimal score function according to Theorem 2 of Kurotschka and Müller (1992) we have to calculate

I(δ_{p})^{−1} x(t) = \{ \begin{matrix}\frac{1}{2 p} (1, 1)′ & for t = 1 \\ \frac{1}{2 (1−p)} (1, −1)′ & for t = −1 \end{matrix} . (2.5)

The following lemma gives the maximum asymptotic bias and the asymptotic variance of the estimators based on optimal robust estimators β̂_{N}^{b} at δ_{p}. For that set

v_{b}(a^{2}) := \{ \begin{matrix}\frac{π}{2} & for a = \frac{1}{b} \sqrt{\frac{π}{2}} \\ \frac{g(b f(a,b)) a^{2}}{f(a,b)^{2}} & for a > \frac{1}{b} \sqrt{\frac{π}{2}} \end{matrix}

where f(a, b) is an implicit function given by f(a, b) = (2Φ(b f(a, b)) − 1) a and g(v) := ∫ min{|z|, v}^{2}P (dz).

Lemma 2.2 The maximum asymptotic bias of the estimator φ(β̂_{N}^{b}) for φ(β) is equal to

(2.6) b(β, p) := ess sup_{P ⊗δ_{p}} |φ̇_{β} ψ_{b,p}| = max \{\frac{|1 + φ(β)|}{|β_{1}|\sqrt{2}}, \frac{|1 − φ(β)|}{|β_{1}|\sqrt{2}}\} b

which is independent of p. Its asymptotic variance is equal to

φ̇_{β} ∫ ψ_{b,p} ψ′_{b,p} d(P ⊗δ_{p}) φ̇′_{β} (2.7)

= \frac{(1 + φ(β))^{2}}{β_{1}^{2}4p} v_{b}(2p^{2}) + \frac{(1 − φ(β))^{2}}{β_{1}^{2}4(1 − p)} v_{b}(2(1 − p)^{2}).

3 Optimal robust estimators and designs

The asymptotic normality (2.4) of estimators of the form φ(β̂_{N}) shows that the maximum asymptotic bias ess sup_{P ⊗δ} |φ̇_{β} ψ| and the asymptotic variance

V (ψ, δ, β) := φ̇_{β} ∫ ψψ′ d(P ⊗δ) φ̇′_{β}

depend on β. If there is no bias bound for the maximum asymptotic bias then the matrix ∫ ψ ψ′ d(P⊗δ) is minimized uniformly by ψ_{∞} within all score functions ψ satisfying condition (2.1) and (2.2). I.e. the variance V (ψ_{∞}, δ, β) is the minimum for all β so that the estimator based on the least squares estimator is asymptotically optimal. (Recall from Section 2 the comment that the least squares estimator behaves asymptotically like an one-step-M-estimator with score function ψ_{∞}.) But if the maximum asymptotic bias is bounded by some bias bound b, then in principle it is impossible to minimize ∫ ψ ψ′ d(P⊗δ) uniformly, so that an optimal robust estimator cannot be found independently of β. Also optimal designs δ cannot be found independently of β, and this problem already occurs for classical non-robust estimation. All non-linear optimal experimental design theory suffer from this dependence on β so that different strategies for non-robust estimation were investigated to overcome this problem, see Ford et al. (1989) and for calibration Osborne (1991).

One strategy of avoiding dependence on β is the so-called maximin efficiency approach which was used for non-robust non-linear estimation problems by Silvey (1980) and Kitsos (1986). We extend this approach and define it as follows: If ψ_{β} is the score function of the locally optimal estimator for φ(β) and δ_{β} is the locally optimal design, then the efficiency of a given estimator for φ(β) with score function ψ at a given design δ is

E_{β}(ψ, δ) := \frac{V (ψ_{β}, δ_{β}, β)}{V (ψ, δ, β)} ∈ [0, 1].

Then, a design δ and an estimator with score function ψ and bias bound b is called maximin efficient for robust estimation of φ(β) with bias bound b if their minimum efficiency is greater than the minimum efficiency of any other design and estimator with bias bound b, i.e. which solve

(3.1) max{min_{β∈B} E_{β}(ψ, δ); δ ∈ Δ, ψ ∈ Ψ_{δ} with ess sup_{P⊗δ} |ψ| ≤ b}.

Thereby Δ is some given set of design measures, for example Δ = {δ; supp(δ) ⊂ [−1, 1]}, the set of all designs with support included in the intervall [−1, 1], and

Ψ_{δ} := {ψ : IR × T → IR^{2}; ψ satisfies condition (2.1) and (2.2)}

is the set of score functions providing asymptotically normally distributed one-step-M-estimators for β.

For deriving maximin efficient solutions for robust estimation a first step is to determine the locally optimal robust estimators and designs.

4 Locally optimal robust estimators and designs

Lemma 2.1 shows that locally an estimator φ(β̂_{N}) based on an asymptotically normally distributed one-step-M-estimator β̂_{N} behaves asymptotically like an one-step-M-estimators for the linear aspect γ(β̃) = φ̇_{β}β̃. Thereby a one-step-M-estimator γ̂_{N}, say, for a linear aspect γ(β) = Cβ, C ∈ IR^{s×2}, is defined as (see Müller (1994b))

γ̂_{N}(y_{N}, d_{N}) = Cβ̂_{N}^{0}(y_{N}, d_{N}) + \frac{1}{N} ∑_{n=1}^{N} ψ ( \frac{y_{nN} −x(t_{nN})′_{β_{N}^{0}} (y_{N} ,d_{N})}{_{σ_{N}}}, t_{nN} ) σ̂_{N}

where ψ now lies in

Ψ_{δ}(C) := {ψ : IR × T → IR^{s}; ψ satisfies condition 2.1 and
∫ ψ(z, t)x(t)′zP (dz)δ(dt) = C}.

Hence locally optimal robust estimators are the optimal robust estimators for the particular linear aspect γ(β̃) = φ̇_{β}β̃. Therefore locally optimal designs δ_{β} and locally optimal estimators ψ_{β} for robust estimation with local bias bound b(β) in Δ are defined as solutions of the following minimization problem

(4.1) min{V (ψ, δ, β); δ ∈ Δ, ψ : IR × T → IR^{2} satisfies

φ̇_{β} ψ ∈ Ψ_{δ}(φ̇_{β}) and ess sup_{P⊗δ} |φ̇_{β}ψ| ≤ b(β)}.

Note that if ψ lies in Ψ_{δ} then φ̇_{β} ψ lies in Ψ_{δ}(φ̇_{β}) but that φ̇_{β}ψ ∈ Ψ_{δ}(φ̇_{β}) does not imply ψ ∈ Ψ_{δ}. For example φ̇_{β} β may be identifiable at δ while the whole parameter vector β is not identifiable at δ.

Now, if there is no local bias bound b(β) then the least squares estimator is the locally optimal estimator for all β ∈ B and the design

δ_{β} := p_{β}e_{1} + (1 − p_{β})e_{−1},

with

p_{β} := \{ \begin{matrix}\frac{1}{2} (φ(β) + 1) & for |φ(β)| ≤ 1 \\ \frac{1}{2 φ(β)} (φ(β) + 1) & for |φ(β)| > 1 \end{matrix} ,

is the locally optimal one in Δ = {δ; supp(δ) ∈ [−1, 1]}. This shows the following theorem which is due to Kitsos (1992).

Theorem 4.1 If b(β) = ∞ and Δ = {δ; supp(δ) ∈ [−1, 1]} then (δ_{β}, ψ_{∞}) is locally optimal, i.e. a solution of (4.1).

Note if the whole parameter vector β is not identifiable at δ_{β} which is the case for |φ(β)| = 1 then ψ_{∞} will be defined by ψ_{∞}(z, t) = I(δ_{β})^{−}x(t) z where M^{−} denotes a generalized inverse of a matrix M.

If b(β) < ∞ then the score functions of the locally optimal robust estimators have the following form

(4.2) ψ_{β,b(β)}(z, t) := \{ \begin{matrix}I(δ_{β})^{−}x(t) \frac{sgn(z)}{|φ̇_{β} I(δ_{β})^{−}x(t)|} b(β) \\ for b(β) = \sqrt{φ̇_{β} I(δ_{β})^{−} φ̇′_{β}} \sqrt{\frac{π}{2}} \\ I(δ_{β})^{−}x(t)\frac{sgn(z) min\{|z|,b(β) y_{b,β}\}}{|φ̇_{β} I(δ_{β})^{−}x(t)| y_{b,β}} \\ for b(β) > \sqrt{φ̇_{β} I(δ_{β})^{−} φ̇′_{β}} \sqrt{\frac{π}{2}} \end{matrix}

where y_{b,β} is the positive solution of

y = (2Φ(b(β) y) − 1) · (\sqrt{φ̇_{β} I(δ_{β})^{−} φ̇′_{β}} )^{−1} .

Note that |φ̇_{β} I(δ_{β})^{−}x(t)|^{2} = φ̇_{β} I(δ_{β})^{−} φ̇′_{β} for t ∈ \{−1, 1\}, φ̇_{β} I(δ_{β})^{−} φ̇′_{β} = \frac{1}{β_{1}^{2}} for |φ(β)| ≤

1 and φ̇_{β} I(δ_{β})^{−} φ̇′_{β} = \frac{φ(β)^{2}}{β_{1}^{2}} for |φ(β)| > 1 (see Kitsos (1992)).

Theorem 4.2 If \sqrt{φ̇_{β} I(δ_{β})^{−} φ̇′_{β}} \sqrt{\frac{π}{2}} ≤ b(β) < ∞ and Δ = \{δ; supp(δ) ∈ [−1, 1]\} then (δ_{β}, ψ_{β,b(β)}) is locally optimal, i.e. a solution of (4.1).

Theorem 4.2 and Theorem 4.1 show that the locally optimal designs for robust estimation coincide with the locally optimal designs for non-robust estimation. Moreover they show that all locally optimal designs have a support included in the two-point set {−1, 1}.

5 Maximin efficient designs

For the calibration problem Kitsos (1992) showed that δ_{\frac{1}{2}} (i.e. the D- and A-optimal design for non-robust estimation of β) is the maximin efficient design in Δ := {δ; supp(δ) ⊂ [0, 1]\} for classical non-robust estimation of φ(β). We are now going to show that δ_{\frac{1}{2}} is also maximin efficient for robust estimation.

If φ(β̂_{N}) is a robust estimator at a given design δ which is for all β ∈ B asymptotically normally distributed with asymptotic variance V (ψ, δ, β) and maximum asymptotic bias b(β, ψ, δ) := ess sup_{P ⊗δ} |φ̇_{β} ψ| we get the efficiency of this estimator at the design δ by comparing it with the locally optimal robust one-step-M-estimators at the locally optimal design δ_{β}. Thereby for the locally optimal estimators we should use the same local bias bound b(β) which is attained by the estimator φ(β̂_{N}) at δ to make fair comparisons. I.e. we set for the local bias bound b(β) = b(β, ψ, δ). Then the asymptotic efficiency measure is

E_{β}(ψ, δ) := \frac{V (ψ_{β,b(β)}, δ_{β}, β)}{V (ψ, δ, β)}

where ψ_{β,b(β)} is the score function of the locally optimal one-step-M-estimator at δ_{β} with local bias bound b(β) = b(β, ψ, δ).

In Section 4 it was shown that all locally optimal designs in Δ = {δ; supp(δ) ⊂ [−1, 1]} are concentrated on the two-point set {−1, 1}. Hence we can restrict our attention to the set

Δ^{0} := {δ; supp(δ) ⊂ {−1, 1}} = {δ_{p}; p ∈ (0, 1)}

so that the maximin efficiency criterion of Section 3 becomes

max{min_{β∈B} E_{β}(ψ, δ); δ ∈ Δ^{0}, ψ ∈ Ψ_{δ} with ess sup_{P ⊗δ} |ψ| ≤ b}.

Moreover if we restrict ourselves only to estimators of the form φ(β̂_{N}^{b}), which are based on optimal robust one-step-M-estimators for β with bias bound b, which have the score functions ψ_{b,p} (see Section 2), then the maximin efficiency criterion can be reduced to a pure design problem of the following form

max{min_{β∈B} E_{β}(ψ_{b,p}, δ_{p}); p ∈ (0, 1)}. (5.1)

Designs which solve this problem will be called maximin efficient designs for robust estimation with bias bound b. For b = ∞ this maximin efficiency criterion coincides with the classical maximin efficiency criterion for designs for non-robust estimation, as was regarded in Silvey (1980).

Recall that at the design δ_{p} the optimal robust estimator β̂_{N}^{b} for β with bias bound b only exists if b ≥ max\{\frac{1}{\sqrt{2}p}, \frac{1}{\sqrt{2}(1−p)}\}\sqrt{\frac{π}{2}} (see Section 2). Therefore, for deriving maximin efficient designs for robust estimation with bias bound b, we have to restrict Δ^{0} to \{δ_{p}; p ∈ [0, 1] with max\{\frac{1}{\sqrt{2}p}, \frac{1}{\sqrt{2}(1−p)}\}\sqrt{\frac{π}{2}} ≤ b\} or to define E_{β}(ψ_{b,p}, δ_{p}) = 0 for

max\{\frac{1}{\sqrt{2}p}, \frac{1}{\sqrt{2}(1−p)}\}\sqrt{\frac{π}{2}} > b, which means that there is no efficiency if there is no robust estimator for β with bias bound b. We think that defining E_{β}(ψ_{b,p}, δ_{p}) = 0 for max\{\frac{1}{\sqrt{2}p}, \frac{1}{\sqrt{2}(1−p)}\}\sqrt{\frac{π}{2}} > b is the best way for deriving the maximin efficient design for robust estimation.

The following lemma shows that for b ≥ max\{\frac{1}{\sqrt{2}p}, \frac{1}{\sqrt{2}(1−p)}\}\sqrt{\frac{π}{2}} always b(β, p) :=

b(β, ψ_{b,p}, δ_{p}) ≥ \sqrt{φ̇_{β} I(δ_{β})^{−1} φ̇′_{β}} \sqrt{\frac{π}{2}} is satisfied. Hence in this case always the locally optimal robust estimator at δ_{β} exists and the efficiency E_{β}(ψ_{b,p}, δ_{p}) is well-defined.

Lemma 5.1 If b ≥ max\{\frac{1}{\sqrt{2}p}, \frac{1}{\sqrt{2}(1−p)}\}\sqrt{\frac{π}{2}} then for all β ∈ B at δ_{β} the locally optimal robust estimator for φ(β) with local bias bound b(β, p) exists and its asymptotic variance satisfies

V (ψ_{β,b(β)}, δ_{β}, β) ≥ \{ \begin{matrix}\frac{1}{β_{1}^{2}} v_{b}(2) & for |φ(β)| ≤ 1 \\ \frac{φ(β)^{2}}{β_{1}^{2}} v_{b}(2) & for |φ(β)| > 1 \end{matrix} ,

with equality if and only if |φ(β)| = 1.

Theorem 5.1 If b ≥ \sqrt{π} then the design δ_{\frac{1}{2}} is maximin efficient for robust estimation with bias bound b, i.e. a solution of (5.1), and the maximin efficiency is equal to

E(b) := \frac{v_{b}(2)}{2 v_{b}(\frac{1}{2})}.

Figure 1: Maximin efficiencies E(b) for robust estimation with bias bound b versus the bias bound b

Theorem 5.1 shows that the maximin efficient designs for robust estimation are always independent of the bias bound b. Moreover they coincide with the maximin efficient design for non-robust estimation which was derived in Kitsos (1992). Thereby, the derivation of the maximin design in Kitsos (1992) is a special case of the derivation of maximin design for robust estimation (see the proof of Theorem 5.1) because b = ∞ provides the nonrobust case. In particular for b = ∞ we have v_{b}(a) = 1 for all a > 0, so that the maximin efficiency for non-robust estimation is equal to 0.5. For the minimum bias bound b = \sqrt{π} the maximin efficiency is equal to 0.319 and increases for b → ∞ quickly to the maximin efficiency for non-robust estimation as Figure 1 shows.

Note also that the maximin efficient design for robust estimation of φ(β) is also the optimal design for robust estimation of the whole parameter vector β. Thereby the bias bound b for robust estimation of β should also satisfy b ≥ \sqrt{π}. See Müller (1994a).

6 Proofs

Proof of Lemma 2.1

The non-linear function φ is continously differentiable on the set B. Hence the asymptotic normality of β̂_{N} implies for all β ∈ B

\sqrt{N}(φ(β̂_{N}) − φ(β)) − \sqrt{N}(φ̇_{β}β̂_{N} − φ̇_{β}β)
= \sqrt{N}(|β̂_{N} − β|)\frac{φ(β̂_{N}) − φ(β) − φ̇_{β}(β̂_{N} − β)}{|β̂_{N} − β|} ⟶^{N→∞} 0.□

Proof of Theorem 2.1

The assertion follows from the asymptotic normality of β̂_{N} given in (2.3) and Lemma 2.1. □

Proof of Lemma 2.2

The maximum asymptotic bias and the asymptotic variance follow from the special form of ψ_{b,p}. Recall R = 1 = σ. □

Proof of Theorem 4.1

The assertion follows from a result in Kitsos (1992) who showed that the design δ_{β} is locally optimal for classical estimators based on the least squares estimator which asymptotically behaves like an one-step-M-estimator with score function ψ_{∞}. Note that the local optimality of δ_{β} can be also seen by Elfving’s (1952) theorem about c-optimal designs (c = φ̇_{β}). The optimal weights of δ_{β} can be also deduced from a result of Pukelsheim and Torsney (1991). For the optimality of the score function ψ_{∞}, in particular in the case where the whole parameter vector β is not identifiable at δ_{β}, see also the proof of Theorem 4.2. □

Proof of Theorem 4.2

The assertions follow from the results in Müller (1994a) about optimal designs for robust estimation of linear aspects. In particular there was shown that for robust estimation of a linear aspect γ(β) = Cβ with bias bound b always (for all bias bounds b) at the A-optimal design the trace of the asymptotic covariance matrix is minimized. Recall that the A-optimal design for estimating γ(β) = Cβ is that design which minimize tr C I(δ)^{−}C′. For estimating the special linear aspect γ(β̃) = φ̇_{β}β̃ the A-optimal design coincides with the c-optimal design where c = φ̇_{β}. Hence the design δ_{β} is optimal (see the proof of Theorem 4.1). Moreover from Lemma 1 in Müller (1994a) it follows that φ̇_{β}ψ_{β,b(β)} is the score function of the optimal robust estimator for γ(β̃) = φ̇_{β}β̃ with bias bound b(β). Because of ψ_{β,∞} = ψ_{∞} we get besides the optimality of ψ_{β,b(β)} also the optimality of ψ_{∞} as a special case (b(β) = ∞). □

Proof of Lemma 5.1 Because of

b(β) = b(β, p) := ess sup_{P ⊗δ_{p}} |φ̇_{β} ψ_{b,p}| = max \{\frac{|1 + φ(β)|}{|β_{1}|\sqrt{2}}, \frac{|1 − φ(β)|}{|β_{1}|\sqrt{2}}\} b

(see Lemma 2.2) and because of

φ̇_{β} I(δ_{β})^{−} φ̇′_{β} = \{ \begin{matrix}\frac{1}{β_{1}^{2}} & for |φ(β)| ≤ 1, \\ \frac{φ(β)^{2}}{β_{1}^{2}} & for |φ(β)| > 1, \end{matrix}

(see Kitsos (1992)), b ≥ max\{\frac{1}{\sqrt{2}p}, \frac{1}{\sqrt{2}(1−p)}\}\sqrt{\frac{π}{2}} ≥ \sqrt{π} implies

b(β) ≥ max \{\frac{|1 + φ(β)|}{|β_{1}|\sqrt{2}}, \frac{|1 − φ(β)|}{|β_{1}|\sqrt{2}}\} \sqrt{π}
≥ max \{\frac{1}{|β_{1}|}, \frac{|φ(β)|}{|β_{1}|}\} \sqrt{\frac{π}{2}} ≥ \sqrt{φ̇_{β} I(δ_{β})^{−1} φ̇′_{β}} \sqrt{\frac{π}{2}} .

Hence, locally optimal robust estimators with local bias bound b(β) at δ_{β} always exist (see Lemma 1 in Müller (1994a)). Their score functions satisfy (see (4.2) and (2.5))

φ̇_{β} ψ_{β,b(β)}(z, t) := \{ \begin{matrix}−sgn(\frac{1+t φ(β)}{β_{1}}) sgn(z) b(β) \\ for b(β) = \sqrt{φ̇_{β} I(δ_{β})^{−1} φ̇′_{β}} \sqrt{\frac{π}{2}} \\ −sgn(\frac{1+t φ(β)}{β_{1}}) sgn(z) \frac{min\{|z|,b(β) y_{b,β}\}}{y_{b,β}} \\ for b(β) > \sqrt{φ̇_{β} I(δ_{β})^{−1} φ̇′_{β}} \sqrt{\frac{π}{2}} \end{matrix} .

Here y_{b,β} are the positive solutions of

y = (2Φ(b(β) y) − 1) · \{ \begin{matrix}|β_{1}| & for |φ(β)| ≤ 1 \\ |\frac{β_{1}}{φ(β)}| & for |φ(β)| > 1 \end{matrix} ,

i.e. y_{b,β} = f(a, b(β)), with a = |β_{1}| for φ(β) ≤ 1 and a = |\frac{β_{1}}{φ(β)}| for φ(β) > 1.

The definition of f(a, b) shows that f(a, \frac{b}{a})/a is independent of a so that we have v_{\frac{b}{a}} (a^{2}) = v_{b}(1) for all b ≥ \sqrt{\frac{π}{2}} and a > 0. In particular, we have v_{\frac{b\sqrt{2}}{a}} (a^{2}) = v_{b}(2) for all b ≥ \sqrt{\frac{π}{2}} and a > 0. Moreover, v_{b}(a) as function of b is strictly decreasing in b (see Müller (1994a)), and

b(β) ≤ \{ \begin{matrix}\frac{1}{|β_{1}|} b\sqrt{2} & for |φ(β)| ≤ 1 \\ \frac{|φ(β)|}{|β_{1}|} b\sqrt{2} & for |φ(β)| > 1 \end{matrix} ,

with equality if and only if |φ(β)| = 1. Thus, we get for the asymptotic variances of the locally optimal robust estimators

φ̇_{β} ∫ ψ_{β,b(β)}ψ′_{β,b(β)} d(P ⊗δ_{β}) φ̇′_{β}
= \{ \begin{matrix}\frac{1}{β_{1}^{2}} v_{b(β)}(β_{1}^{2}) & for |φ(β)| ≤ 1 \\ \frac{φ(β)^{2}}{β_{1}^{2}} v_{b(β)}(\frac{β_{1}^{2}}{φ(β)^{2}} ) & for |φ(β)| > 1 \end{matrix}

≥ \{ \begin{matrix}\frac{1}{β_{1}^{2}} v_{\frac{b\sqrt{2}}{|β_{1}|}} (β_{1}^{2}) & for |φ(β)| ≤ 1 \\ \frac{φ(β)^{2}}{β_{1}^{2}} v_{\frac{|φ(β)|b\sqrt{2}}{|β_{1}|}} (\frac{β_{1}^{2}}{φ(β)^{2}} ) & for |φ(β)| > 1 \end{matrix}

= \{ \begin{matrix}\frac{1}{β_{1}^{2}} v_{b}(2) & for |φ(β)| ≤ 1 \\ \frac{φ(β)^{2}}{β_{1}^{2}} v_{b}(2) & for |φ(β)| > 1 \end{matrix} ,

with equality if and only if |φ(β)| = 1. □

Proof of Theorem 5.1

Set

v_{1} := \frac{v_{b}(2p^{2})}{v_{b}(2)} and v_{−1} := \frac{v_{b}(2(1 − p)^{2})}{v_{b}(2)}.

Then we obtain with (2.7) in Lemma 2.2 and (5.2) in Lemma 5.1 for the inverse of the relative efficiency of the robust estimator φ(β̂_{N}) at the design δ_{p}

E_{β}(ψ_{b,p}, δ_{p})^{−1} = \frac{φ̇_{β} ∫ ψ_{b,p}ψ′_{b,p} d(P ⊗ δ_{p}) φ̇′_{β}}{φ̇_{β} ∫ ψ_{β,b(β)}ψ′_{β,b(β)} d(P ⊗δ_{β}) φ̇′_{β}} (6.1)

= \{ \begin{matrix}\frac{(1+φ(β))^{2}}{4 p} v_{1} + \frac{(1−φ(β))^{2}}{4 (1−p)} v_{−1} & for |φ(β)| ≤ 1 \\ \frac{1}{φ(β)^{2}} [\frac{(1+φ(β))^{2}}{4 p} v_{1} + \frac{(1−φ(β))^{2}}{4 (1−p)} v_{−1}] & for |φ(β)| > 1 \end{matrix}

≤ max \{\frac{v_{1}}{p} , \frac{v_{−1}}{1 − p}\} · \frac{1}{4} · \{ \begin{matrix}(1 + φ(β))^{2} + (1 − φ(β))^{2} & for |φ(β)| ≤ 1 \\ \frac{1}{φ(β)^{2}} [(1 + φ(β))^{2} + (1 − φ(β))^{2}] & for |φ(β)| > 1 \end{matrix}
≤ max \{\frac{v_{1}}{p} , \frac{v_{−1}}{1 − p}\} ,

with equality if and only if φ(β) = 1 and \frac{v_{1}}{p} ≥ \frac{v_{−1}}{(1−p)}, or φ(β) = −1 and \frac{v_{1}}{p} ≤ \frac{v_{−1}}{(1−p)}. Because v_{b}(a) is strictly decreasing in a (see Müller (1994a)) the following inequalities hold:

\frac{v_{1}}{p} = \frac{v_{b}(2p^{2})}{p v_{b}(2)} ≥ \frac{2 v_{b}(\frac{1}{2})}{v_{b}(2)} if and only if p ≤ \frac{1}{2},

and

\frac{v_{−1}}{1 − p} = \frac{v_{b}(2(1 − p)^{2})}{(1 − p) v_{b}(2)} ≤ \frac{2 v_{b}(\frac{1}{2})}{v_{b}(2)} if and only if p ≥ \frac{1}{2}.

Hence, max\{\frac{v_{1}}{p} , \frac{v_{−1}}{1−p}\} attains its minimum at p = \frac{1}{2}. This yields

δ_{\frac{1}{2}} = arg max \{min_{β∈B} E_{β}(ψ_{b,p}, δ_{p}); p ∈ (0, 1)\} ,

and

max \{min_{β∈B} E_{β}(ψ_{b,p}, δ_{p}); p ∈ (0, 1)\} = \frac{v_{b}(2)}{2 v_{b}(\frac{1}{2})}.

Note that v_{1} = 1 and v_{−1} = 1 for b = ∞, so that the valuation in (6.1) generalizes the valuation in Kitsos (1992). □

References

[1] BUONACCORSI, J.P. (1986). Design considerations for calibration. Technometrics 28, 149-155.

[2] EISENHART, C. (1939). The interpretation of certain regression methods and their use in biological and industrial research. Ann. Math. Statist. 10, 162-186.

[3] ELFVING, G. (1952). Optimum allocation in linear regression theory. Ann. Math. Statist. 23, 255-262.

[4] FORD, I., KITSOS, C.P. and TITTERINGTON, D.M. (1989). Recent advances in nonlinear experimental design. Technometrics 13, 49-60.

[5] HUBER, P.J. (1964). Robust estimation of a location parameter. Ann. Math. Statist. 35, 73-101.

[6] KITSOS, C.P. (1986). Design and inference for nonlinear problems. Ph.D. thesis, University of Glasgow, U.K.

[7] KITSOS, C.P. (1992). Quasi-sequential procedures for the calibration problem. In COMPSTAT 1992, Vol. 2, Y. Dodge and J. Whittaker (eds.), 227-231, Physica-Verlag.

[8] KRASKER, W.S. (1980). Estimation in linear regression models with disparate data points. Econometrica 48, 1333-1346.

[9] KUROTSCHKA, V. and MÜLLER, Ch.H. (1992). Optimum robust estimation of linear aspects in conditionally contaminated linear models. Ann. Statist. 20, 331-350.

[10] MÜLLER, Ch.H. (1994a). Optimal designs for robust estimation in conditionally contaminated linear models. J. Statist. Plann. Inference. 38, 125-140.

[11] MÜLLER, Ch.H. (1994b). One-step-M-estimators in conditionally contaminated linear models. Stat. Decis. 12, 331-342.

[12] OSBORNE, C. (1991). Statistical calibration: A review. Int. Statist. Rev. 59, 309-336.

[13] PUKELSHEIM, F. and TORSNEY, B. (1991). Optimal weights for experimental designs on linearly independent support points. Ann. Statist. 19, 1614-1625.

[14] SILVEY, S.D. (1980). Optimal Design. Chapman and Hall, London.

CHRISTOS P. KITSOS  CHRISTINE H. MÜLLER
Athens University of Business and Economics  Freie Universität Berlin
Department of Statistics  1. Mathematisches Institut
Patission St. 76  Arnimallee 2-6
104 34 Athens  D-14195 Berlin-Dahlem
Greece  Germany