caret (version 4.69)

xyplot.resamples: Lattice Functions for Visualizing Resampling Results

Description

Lattice functions for visualizing resampling results across models

Usage

## S3 method for class 'resamples':
xyplot(x, data = NULL, models = x$models[1:2], metric = x$metric[1], ...)
## S3 method for class 'resamples':
dotplot(x, data = NULL, models = x$models, metric = x$metric, conf.level = 0.95, ...)
## S3 method for class 'resamples':
densityplot(x, data = NULL, models = x$models, metric = x$metric, ...)
## S3 method for class 'resamples':
bwplot(x, data = NULL, models = x$models, metric = x$metric, ...)
## S3 method for class 'resamples':
splom(x, data = NULL, variables = "models", models = x$models, metric = NULL, panelRange = NULL, ...)
## S3 method for class 'resamples':
parallel(x, data = NULL, models = x$models, metric = x$metric[1], ...)

Arguments

x

an object generated by resamples

data

Not used

models

a character string for which models to plot. Note: xyplot requires exactly two models whereas the other methods can plot more than two.

metric

a character string for which metrics to use as conditioning variables in the plot. splom requires exactly one metric when variables = "models" and at least two when variables = "metrics".

variables

either "models" or "metrics"; which variable should be treated as the scatter plot variables?

panelRange

a common range for the panels. If NULL, the panel ranges are derived from the values across all the models

conf.level

the confidence level for intervals about the mean (obtained using t.test)

...

further arguments to pass to either histogram, densityplot, xyplot

Value

• a lattice object

Details

The ideas and methods here are based on Hothorn et al (2005) and Eugster et al (2008).

xyplot only uses two models in the plot. The plot uses difference of the models on the y-axis and the average of the models on the x-axis.

dotplot plots the average performance value (with two-sided confidence limits) for each model and metric.

densityplot and bwplot display univariate visualizations of the resampling distributions while splom shows the pair-wise relationships.

References

Hothorn et al. The design and analysis of benchmark experiments. Journal of Computational and Graphical Statistics (2005) vol. 14 (3) pp. 675-699

Eugster et al. Exploratory and inferential analysis of benchmark experiments. Ludwigs-Maximilians-Universitat Munchen, Department of Statistics, Tech. Rep (2008) vol. 30

See Also

resamples, dotplot, bwplot, densityplot, xyplot, splom

Examples

#load(url("http://caret.r-forge.r-project.org/Classification_and_Regression_Training_files/exampleModels.RData"))

resamps <- resamples(list(CART = rpartFit,
                          CondInfTree = ctreeFit,
                          MARS = earthFit))

dotplot(resamps, 
        scales =list(x = list(relation = "free")), 
        between = list(x = 2))

bwplot(resamps,
       metric = "RMSE")

densityplot(resamps,
            auto.key = list(columns = 3),
            pch = "|")

xyplot(resamps,
       models = c("CART", "MARS"),
       metric = "RMSE")

splom(resamps, metric = "RMSE")
splom(resamps, variables = "metrics")

parallel(resamps, metric = "RMSE")