E-optimum Designs in Linear Models with Both Qualitative and Quantitative Factors of Influence

Rainer Schwabe^{∗}

Freie Universität Berlin, 1. Mathematisches Institut, Arnimallee 2-6, 14 195 Berlin, Germany

Abstract

In most experimental situations the outcome is influenced by various factors and different interaction structures. For a variety of simple but practically useful examples we investigate the generation of E-optimum designs, i. e. designs which minimize the maximum variance for all normalized estimates.

1 Introduction

We consider situations in which the outcome of an experiment depends on two or more factors of influence. In particular, the response function associated with the experiment can be linearly parametrized, i. e. the observation Y (t) = ∑_{i=1}^{p} a_{i}(t)β_{i} + Z can be represented as a linear combination of the known regression functions a_{1}, ..., a_{p} with unknown coefficients β_{1}, ..., β_{p}, where t = (t_{1}, ..., t_{K})′ is the setting of the K factors of influence and Z denotes random noise.

For N different experiments which can be realized according to the linear model Y_{n}(t^{(n)}) = a(t^{(n)})′β + Z_{n} the experimental condition t^{(n)} ∈ T can be chosen by the experimenter according to the design d = (t^{(1)}, ..., t^{(N)}), where replicates may occur.

Under the usual assumptions of zero mean and uncorrelated equally dispersed errors the model is completely described by its response function E(Y (t)) = a(t)′β, t ∈ T. The quality of a design is measured by its information matrix I(d) = \frac{1}{N} ∑_{n=1}^{N} a(t^{(n)})a(t^{(n)})′ because the covariance matrix Cov(β̂_{d}) of the least squares estimator β̂_{d} is proportional to the inverse I(d)^{−1} of the information matrix (in case I(d) is regular).

To simplify the optimization problem we make use of the notion of (generalized) designs δ on T and the associated information matrices I(δ) = ∫_{T} aa′ dδ. For a design d of size N consider the corresponding generalized design δ_{d} which assigns equal weight \frac{1}{N} to t^{(1)}, ..., t^{(N)}. Then I(δ_{d}) = I(d) which shows that the generalization is straightforward. For further readings on the use of generalized designs we refer to Fedorov (1972), Kiefer (1974) or Bandemer et al. (1977).

Since, in general, it is impossible to find a design which maximizes the information matrix uniformly we have to investigate some real valued criterion function. In the present note we consider the minimax criterion of minimizing the maximum variance for all normalized estimates of linear combinations of the unknown parameters. By Rayleigh’s formula this is equivalent to the formulation: A design δ^{∗} is E-optimum if it minimizes λ_{max}(C(δ)), where C(δ) is the generalized covariance matrix, C(δ_{d}) = Cov(β̂_{d}), and λ_{max} denotes the largest eigenvalue. In case of a minimal parametrization, i. e. the regression functions are linearly independent on T, the E-optimality can also be expressed as the maximization of the smallest eigenvalue λ_{min}(I(δ)) of the information matrix. In contrast to other commonly used criteria the E-criterion is not differentiable, in general, and the construction of E-optimum designs is more difficult (see Kiefer, 1974).

∗research supported by grant Ku719/2-2 of the Deutsche Forschungsgemeinschaft

In the present note we investigate situations in which product designs are E-optimum. Note that for finitely supported marginal designs δ_{1} and δ_{2} which assign weights w′_{m} and w′′_{n} to experimental conditions t_{1}^{(m)} ∈ T_{1} and t_{2}^{(n)} ∈ T_{2}, respectively, the product design δ_{1} ⊗ δ_{2} is defined as that design which assigns weights w_{m,n} = w′_{m}w′′_{n} to each level combination t^{(m,n)} = (t_{1}^{(m)}, t_{2}^{(n)}) ∈ T = T_{1} × T_{2}.

In general, product designs are E-optimum in models with product type interaction structures and their marginals are the E-optimum designs in the corresponding single factor models (Section 2). Subsequently those models are investigated in which, at least, one of the factors is qualitative, i. e. this factor may be adjusted to, at most, finitely many different levels. By invariance considerations also there E-optimum product designs can be obtained. However, the E-optimality of the marginals is preserved only in some particular cases. Finally, some results are presented for a variety of more complicated interaction structures.

2 Complete product-type interaction

Given the single factor models E(Y_{1}(t_{1})) = f_{1}(t_{1})′β^{(1)}, t_{1} ∈ T_{1}, and E(Y_{2}(t_{2})) = f_{2}(t_{2})′β^{(2)}, t_{2} ∈ T_{2}, the general two-factor model with complete product-type interaction is defined by

E(Y (t_{1}, t_{2})) = ∑_{i=1}^{p_{1}} ∑_{j=1}^{p_{2}} f_{1i}(t_{1})f_{2j}(t_{2})β_{ij}, t_{1} ∈ T_{1} × T_{2}.  (1)

By means of the equivalence theorem for E-optimality (see Pukelsheim, 1993, or Ermakov and Melas, 1995) the following general result can be obtained (Schwabe, 1996, Theorem 4.10, see also Rafajłowicz and Myszka, 1988, for the case of simple maximum eigenvalues) which can immediately be extended to an arbitrary number of K different factors,

Theorem 1. The product design δ^{∗} = δ_{1}^{∗}⊗δ_{2}^{∗} is E-optimum in the model with complete product-type interactions if the marginal designs δ_{1}^{∗} and δ_{2}^{∗} are E-optimum in the associated single factor models.

3 One-way layout with an additional factor: no general mean

In the following we consider models in which one of the factors is qualitative and is described by a one-way layout E(Y_{1}(i)) = α_{i}. For the two-factor additive model

E(Y (i, x)) = α_{i} + f(x)′β, i = 1, ..., I, x ∈ X  (2)

the corresponding second marginal model E(Y_{2}(x)) = μ_{0}+f(x)′β includes a constant term μ_{0}. By invariance considerations with respect to permutations of the treatment levels i of the first factor it can be seen that every design δ is dominated by its symmetrization \overline{δ}_{1} ⊗ δ_{2}, where \overline{δ}_{1} is the uniform design on the levels of the first factor, which assigns equal weights \frac{1}{I} to each level, and δ_{2} is the second marginal of δ, δ_{2}(x) = ∑_{i=1}^{I} δ(i, x) (see e. g. Schwabe, 1996, Subsection 6.2). It remains to find an optimum marginal design δ_{2}.

Theorem 2. The product design δ^{∗} = \overline{δ}_{1} ⊗ δ_{2}^{∗} is E-optimum in the additive model without general mean if the marginal design δ_{2}^{∗} is E-optimum in the transformed second marginal model E(Y_{2}(x)) = μ_{0} + \sqrt{I}f(x)′β, x ∈ X.

The proof is based on the fact that the eigenvalues of the information matrix for the two-factor model coincide with that for the transformed marginal model up to a factor \frac{1}{I}. For the particular case that in the second marginal model the E-optimum design δ_{2}^{∗} is centered, i. e. ∫ f dδ_{2}^{∗} = 0, this design δ_{2}^{∗} constitutes an E-optimum product design \overline{δ}_{1} ⊗ δ_{2}^{∗} (Schwabe, 1995).

Example 1. Two-way layout: E(Y (i, j)) = α_{i} + β_{j}, i = 1, ..., I, j = 1, ..., J, with a control level J for the second factor, i. e. β_{J} = 0. Then the product design \overline{δ}_{1} ⊗ δ_{2}^{∗} is E-optimum which assigns equal weights 2/(I^{2} + 4I(J − 1)) to all treatment combinations (i, j), j ≠ J, and weights (I + 2(J − 1))/(I^{2} + 4I(J − 1)) to the treatment-control combinations (i, J).

4 One-way layout with an additional factor: with general mean

The two-factor additive model of the previous section can also be written as

E(Y (i, x)) = μ_{0} + α_{i} + f(x)′β, i = 1, ..., I, x ∈ X  (3)

with a general mean μ_{0}. The identifiability of the parameters will be ensured by imposing an additional condition ∑_{i=1}^{I} α_{i} = 0 such that the parameters α_{i} now represent the effects of level i compared to their mean. Again, by invariance considerations every design δ is dominated by its symmetrization \overline{δ}_{1} ⊗ δ_{2} (Schwabe, 1996, Example 6.9) and it remains to find an optimum marginal design δ_{2}.

Theorem 3. The product design δ^{∗} = \overline{δ}_{1} ⊗δ_{2}^{∗} is E-optimum in the additive model with general mean if the marginal design δ_{2}^{∗} is E-optimum in the second single factor model.

The proof is based on the block diagonal structure of the generalized covariance matrix where the blocks are related to the single factors.

5 One-way layout combined with regression: various interactions

In this section we confine to the broad class of models in which one of the factors is qualitative and follows a one-way layout and the second factor is described by a polynomial regression of degrees 1 or 2 (linear or quadratic). Also in this setting every design δ is dominated by its symmetrization \overline{δ}_{1} ⊗ δ_{2} in case of a general symmetric parametrization

E(Y (i, x)) = f_{0}(x)′β^{(0)} + f_{1}(x)′β^{(i)}, i = 1, ..., I, x ∈ X .  (4)

In regression models usually an interval X = [b_{1}, b_{2}] is assumed for the marginal design region.

Example 2. One-way layout combined with linear regression: All possible kinds of interaction of one qualitative factor with additional linear regression are listed in Table 1. In all these situations there is a (product) design \overline{δ}_{1} ⊗ δ_{2}^{∗} which is E-optimum. By a majorization argument the optimum marginal design δ_{2}^{∗} is concentrated on the two endpoints b_{1} and b_{2}.

In case of a symmetric interval X = [−b, b] the E-optimum design δ_{2}^{∗} in the marginal model which assigns equal weights \frac{1}{2} to both endpoints yields a block diagonal structure of the information matrix I(\overline{δ}_{1} ⊗ δ_{2}^{∗}) and, hence, the resulting design \overline{δ}_{1} ⊗ δ_{2}^{∗} is E-optimum for all of the present models.

For an interval X = [0, b] the optimum weights w_{0}^{∗} are given in Table 1, together with the corresponding maximum eigenvalues of the (generalized) covariance

\begin{matrix}E(Y (i, x)) & w_{0}^{∗} & λ_{max} \\ β_{0} + β_{1}x & \frac{b^{2} + 2}{b^{2} + 4} & 1 + \frac{4}{b^{2}} \\ β_{0i} + β_{1}x & \frac{Ib^{2} + 2}{Ib^{2} + 4} & I + \frac{4}{b^{2}} \\ β_{0} + β_{1i}x & \frac{b^{2} + 2I}{b^{2} + 4I} & 1 + \frac{4}{b^{2}} I \\ β_{0i} + β_{1i}x & \frac{b^{2} + 2}{b^{2} + 4} & I + \frac{4}{b^{2}} I \end{matrix}

Table 1: One qualitative factor combined with linear regression,

optimum weights

matrix for the resulting design \overline{δ}_{1} ⊗ δ_{2}^{∗}, where δ_{2}^{∗} assigns weight w_{0}^{∗} to 0 and weight 1 − w_{0}^{∗} to b. The last entry of Table 1 shows complete product-type interactions and the optimum weight is obtained by Theorem 1.

Example 3. One-way layout combined with quadratic regression:

\begin{matrix}E(Y (i, x)) & w_{0}^{∗} & λ_{max} \\ β_{0} + β_{1}x + β_{2}x^{2} & \frac{3}{5} & 5 \\ β_{0i} + β_{1}x + β_{2}x^{2} & \frac{I + 2}{I + 4} & I + 4 \\ β_{0} + β_{1i}x + β_{2}x^{2} & \frac{2I − 1}{I^{2} + I − 1} & I + 2 + \frac{1}{I − 1} \\ β_{0} + β_{1}x + β_{2i}x^{2} & \frac{2I + 1}{4I + 1} & 4I + 1 \\ β_{0i} + β_{1i}x + β_{2}x^{2} & \frac{I}{2I − 1} & 2I + 1 + \frac{1}{I − 1} \\ β_{0i} + β_{1}x + β_{2i}x^{2} & \frac{3}{5} & 5I \\ β_{0} + β_{1i}x + β_{2i}x^{2} & \frac{2I + 1}{4I + 1} & 4I + 1 \\ β_{0i} + β_{1i}x + β_{2i}x^{2} & \frac{3}{5} & 5I \end{matrix}

Table 2: One qualitative factor combined with quadratic regression, optimum weights

All possible interaction structures are exhibited in Table 2. The underlying marginal design region is the unit intervall X = [−1, 1]. By a majorization argument we find that an E-optimum design \overline{δ}_{1}⊗δ_{2}^{∗} is given by a symmetric design δ_{2}^{∗} which assigns weight w_{0}^{∗} to the midpoint 0 and equal weights \frac{1}{2}(1 − w_{0}^{∗}) to both endpoints −1 and 1 of the interval. The optimum weights w_{0}^{∗} are given in Table 2, together with the corresponding maximum eigenvalues of the covariance matrix for the resulting design \overline{δ}_{1} ⊗ δ_{2}^{∗}. Different interaction structures give different optimum weights depending on the number I of levels of the qualitative factor as can be seen from the table. In particular, if I is large these differences are substantial.

A different structure of interactions is exhibited by the intra-class experiments

E(Y (i, x)) = f_{i}(x)′β^{(i)}, x ∈ X_{i}, i = 1, ..., I,  (5)

in which the regression functions f_{i} = (f_{i1}, ..., f_{ip_{i}})′ and the conditional design region X_{i} may also depend on the level i of the qualitative factor. With the notion of conditional designs (see Kurotschka, 1984) the E-optimum designs can readily be characterized (Schwabe, 1996, Theorem 7.1),

Theorem 4. Let, for every i, the design δ_{2|1}^{∗}(i, ·) be E-optimum in the conditional model E(Y_{i}(x)) = f_{i}(x)′β_{i}, x ∈ X_{i}, associated with the level i and let δ_{1}^{∗} assign weights λ_{i}/(∑_{j=1}^{I} λ_{j}) to level i, where λ_{i} is the largest eigenvalue of the conditional (generalized) covariance matrix for δ_{2|1}^{∗}. Then the mixture δ^{∗} = δ_{1}^{∗} ⊗δ_{2|1}^{∗} is E-optimum in the intra-class model.

6 Concluding remarks

The methods proposed throughout the paper permit a reduction of the problem to determine optimum designs for multi-factor models to components corresponding to single factors. Since a lot of optimum designs are already known for one-factor models and optimization is much easier in one dimension this enables us to construct optimum designs for complicated models which would be difficult to treat directly. For example, it would be interesting to transfer the known results on E-optimum designs in polynomial models (see e. g. Heiligers, 1994) to the present settings with an additional one-way layout.

Related results have been obtained by Kurotschka, Schwabe and Wierich (1992) and Schwabe (1996) on D- and A-optimum designs aiming at minimizing the determinant and the trace of the (generalized) covariance matrix, respectively.

Finally, we want to emphasize once more that the structure of the interactions has a strong influence on the performance of a design.

References

Bandemer, H. (ed.) (1977). Theorie and Anwendung der optimalen Versuchsplanung I. Hand-

buch zur Theorie. Akademie-Verlag, Berlin. Ermakov, S. M. and V. B. Melas (1995). Design and Analysis of Simulation Experiments.

Kluwer, Dordrecht. Fedorov, V. V. (1972). Theory of Optimal Experiments. Academic Press, New York. Heiligers, B. (1994). E-optimal design in weighted polynomial regression. Ann. Statist. 22,

917–929. Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory). Ann.

Statist. 2, 849–879. Kurotschka, V. G. (1984). A general approach to optimum design of experiments with qual-

itative and quantitative factors. In J. K. Ghosh and J. Roy (eds.), Statistics: Applications and New Directions: Proceedings of the Indian Statistical Institute Golden Jubilee International Conference Calcutta 1981, 353–368.

Kurotschka, V., R. Schwabe and W. Wierich (1992). Optimum designs for partly interacting qualitative and quantitative factors. In A. Pázman and J. Volaufová (eds.), Probastat ’91, Proc. Int. Conf. Probab. Math. Statist., Bratislava 1991. Slovak Academy of Sciences, Bratislava, 102–108.

Pukelsheim, F. (1993). Optimal Design of Experiments. Wiley, New York. Rafajłowicz, E. and W. Myszka (1988). Optimum experimental design for a regression on

a hypercube - generalization of Hoel’s result. Ann. Inst. Statist. Math. 40, 821–827. Schwabe, R. (1995). Optimal designs for additive linear models. Statistics (to appear). Schwabe, R. (1996). Optimum Designs for Multi-factor Models. Lecture Notes in Statistics.

Springer, New York (to appear).