optimality.criteria: Overview of optimality criteria in experimental design packages

Description

One function for calculating the S-optimality criterion is provided here. This help file documents this function and also describes optimality criteria from other related packages, referring to space filling optimality like the S criterion or to model-based optimality like the well-known D-criterion.

Usage

Scalc(design)
compare(...)

Value

Function Scalc returns a single number: the harmonic mean of all pairwise interpoint distances is calculated, based on the matrix design or the desnum attribute of the design design. (This value should be as large as possible for a space-filling design.)

Note that the resulting S value differs from the printed optimum value by function lhs.design for type optimum in two respects: the printed optimum value is the sum of inverse distances, i.e. the denominator of the harmonic mean. choose(nruns, 2) divided by the printed final optimal value is approximately equal to the calculated S; perfect equality cannot be achieved because the underlying the printed final optimum refers to an interim latin hypercube of integers that is subsequently rescaled to the unit cube and scrambled by random numbers.

Function compare returns a matrix, with rows representing the criteria and columns the different designs. Apart from many of the criteria mentioned above, the determinant of the correlation matrix is shown, which should ideally be close to one for a near-orthogonal design (at least in terms of linear effects).

Arguments

design: matrix, often normalized to unit cube,
OR
design (class design) of type lhs or Dopt.
For design objects, calculations are applied to the desnum attribute.
...: two or more designs, either all of type lhs or all of type Dopt, can be compared w.r.t. some optimality criteria that are stored in their design.info attribute (only works with designs created by DoE.wrapper version 0.7 or higher)

Author

Ulrike Groemping

Details

Function Scalc calculates the S criterion for optimality, which is employed in package lhs for most optimization purposes (exception: maximin designs). The criterion is the harmonic mean of all pairwise interpoint distances, and space-filling optimization tries to maximize it.

Function eval.design from package AlgDesign calculates various model-based optimality criteria:

confounding: a confounding matrix of effects, if requested
determinant: the k-th root of the determinant of Z'Z/N, where Z is the model matrix of the model under investigation, k is the number of columns of Z and N the number of rows; this is the quantity optimized for D-optimal designs
A: the arithmetic mean of coeffient variance multipliers, i.e. the average diagonal element of the inverse of Z'Z/N, intercept included
I: the average prediction variance over a space X; calculated only, if X is specified
Ge: the minimax normalized variance over X; calculated only, if X is specified
Dea: A lower bound on D efficiency for approximate theory designs. It is equal to exp(1-1/Ge), i.e. is also calculated only, if X is specified.
diagonality: the k-th root of the ratio of the determinant of M1 divided by the product of diagonal elements of M1, where M1 is Z'Z with the column and row referring to the intercept removed, and k the number of columns of M1; if this is 1, the coefficient estimates are uncorrelated.
gmean.variances: the geometric mean of normalized coeffient variance multipliers (intercept excluded), i.e. the geometric mean of the diagonal elements of the inverse of Z'Z/N, without the first element, if an intercept is in the model.

Several functions from package DiceDesign calculate optimality criteria regarding the space filling qualities of a design. These functions normalize the design to lie in the unit cube, if it does not yet do so. Application of these functions to designs with qualitative factors does not make sense and yields errors. The following functions are available:

mindist calculates the minimum distance between any pair of design points. This is the criterion which is maximized by maximin designs, i.e. should be large for a space-filling design.

For the next two distance metrics, it is helpful to define g_i as the minimal distance of design point i to any other design point.

meshRatio calculates the ratio of the maximum g_i to the minimum g_i (a small mesh ratio indicates a similar minimal distance for all design points).

coverage calculates the coefficient of variation of the g_i, however using the denominator n instead of n-1 for the standard deviation (a large coverage indicates that the average minimal distance for of design points is large relative to their standard deviation; large values are desirable).

Function discrepancyCriteria calculates several versions of L2 discrepancy.

Function link[skpr]{eval_design} calculates effect and parameter powers for a design for a specified model and a specified significance level alpha.

References

Beachkofski, B., Grandhi, R. (2002) Improved Distributed Hypercube Sampling. American Institute of Aeronautics and Astronautics Paper 1274.

Currin C., Mitchell T., Morris M. and Ylvisaker D. (1991) Bayesian Prediction of Deterministic Functions With Applications to the Design and Analysis of Computer Experiments, Journal of the American Statistical Association 86, 953--963.

Santner T.J., Williams B.J. and Notz W.I. (2003) The Design and Analysis of Computer Experiments, Springer, 121--161.

Shewry, M. C. and Wynn and H. P. (1987) Maximum entropy sampling. Journal of Applied Statistics 14, 165--170.

Fang K.-T., Li R. and Sudjianto A. (2006) Design and Modeling for Computer Experiments, Chapman & Hall.

Stein, M. (1987) Large Sample Properties of Simulations Using Latin Hypercube Sampling. Technometrics 29, 143--151.

Stocki, R. (2005) A method to improve design reliability using optimal Latin hypercube sampling. Computer Assisted Mechanics and Engineering Sciences 12, 87--105.

Examples

Run this code

   ## optimum design from package lhs (default)
   plan <- lhs.design(20,4,"optimum",
          factor.names=list(c(15,25), c(10,90), c(0,120), c(12,24)), digits=2)
   ## maximin design 
   plan2 <- lhs.design(20,4,"maximin",
          factor.names=list(c(15,25), c(10,90), c(0,120), c(12,24)), digits=2)
   ## purely random design (usually not ideal)
   plan3 <- lhs.design(20,4,"random",
          factor.names=list(c(15,25), c(10,90), c(0,120), c(12,24)), digits=2)
   ## genetic design 
   plan4 <- lhs.design(20,4,"genetic",
          factor.names=list(c(15,25), c(10,90), c(0,120), c(12,24)), digits=2)
   ## dmax design from package DiceDesign
   ## arguments range and niter_max are required
   ## ?dmaxDesign for more info
   plan5 <- lhs.design(20,4,"dmax",
        factor.names=list(torque=c(10,14),friction=c(25,35),
              temperature=c(-5,35),pressure=c(20,50)),digits=2,
              range=0.2, niter_max=500)
   ## Strauss design from package DiceDesign
   ## argument RND is required
   ## ?straussDesign for more info
   plan6 <- lhs.design(20,4,"strauss",
        factor.names=list(torque=c(10,14),friction=c(25,35),
              temperature=c(-5,35),pressure=c(20,50)),digits=2,
              RND = 0.2)
   ## compare all these designs
   compare(plan, plan2, plan3, plan4, plan5, plan6)

Run the code above in your browser using DataLab