microbenchmark (version 1.5.0)

microbenchmark: Sub-millisecond accurate timing of expression evaluation.

Description

microbenchmark serves as a more accurate replacement of the often seen system.time(replicate(1000, expr)) expression. It tries hard to accurately measure only the time it takes to evaluate expr. To achieved this, the sub-millisecond (supposedly nanosecond) accurate timing functions most modern operating systems provide are used. Additionally all evaluations of the expressions are done in C code to minimize any overhead.

Usage

microbenchmark(
  ...,
  list = NULL,
  times = 100L,
  unit = NULL,
  check = NULL,
  control = list(),
  setup = NULL
)

Value

Object of class ‘microbenchmark’, a data frame with columns expr and time. expr contains the deparsed expression as passed to microbenchmark or the name of the argument if the expression was passed as a named argument. time is the measured execution time of the expression in nanoseconds. The order of the observations in the data frame is the order in which they were executed.

Arguments

...

Expressions to benchmark.

list

List of unevaluated expressions to benchmark.

times

Number of times to evaluate each expression.

unit

Default unit used in summary and print.

check

A function to check if the expressions are equal. By default NULL which omits the check. In addition to a function, a string can be supplied. The string ‘equal’ will compare all values using all.equal, ‘equivalent’ will compare all values using all.equal and check.attributes = FALSE, and ‘identical’ will compare all values using identical.

control

List of control arguments. See Details.

setup

An unevaluated expression to be run (untimed) before each benchmark expression.

Author

Olaf Mersmann

Details

This function is only meant for micro-benchmarking small pieces of source code and to compare their relative performance characteristics. You should generally avoid benchmarking larger chunks of your code using this function. Instead, try using the R profiler to detect hot spots and consider rewriting them in C/C++ or FORTRAN.

The control list can contain the following entries:

order

the order in which the expressions are evaluated. “random” (the default) randomizes the execution order, “inorder” executes each expression in order and “block” executes all repetitions of each expression as one block.

warmup

the number of iterations to run the timing code before evaluating the expressions in .... These warm-up iterations are used to estimate the timing overhead as well as spinning up the processor from any sleep or idle states it might be in. The default value is 2.

See Also

print.microbenchmark to display and boxplot.microbenchmark or autoplot.microbenchmark to plot the results.

Examples

## Measure the time it takes to dispatch a simple function call
## compared to simply evaluating the constant \code{NULL}
f <- function() NULL
res <- microbenchmark(NULL, f(), times=1000L)

## Print results:
print(res)

## Plot results:
boxplot(res)

## Pretty plot:
if (requireNamespace("ggplot2")) {
  ggplot2::autoplot(res)
}

## Example check usage
my_check <- function(values) {
  all(sapply(values[-1], function(x) identical(values[[1]], x)))
}

f <- function(a, b)
  2 + 2

a <- 2
## Check passes
microbenchmark(2 + 2, 2 + a, f(2, a), f(2, 2), check=my_check)
if (FALSE) {
a <- 3
## Check fails
microbenchmark(2 + 2, 2 + a, f(2, a), f(2, 2), check=my_check)
}
## Example setup usage
set.seed(21)
x <- rnorm(10)
microbenchmark(x, rnorm(10), check=my_check, setup=set.seed(21))
## Will fail without setup
if (FALSE) {
microbenchmark(x, rnorm(10), check=my_check)
}
## using check
a <- 2
microbenchmark(2 + 2, 2 + a, sum(2, a), sum(2, 2), check='identical')
microbenchmark(2 + 2, 2 + a, sum(2, a), sum(2, 2), check='equal')
attr(a, 'abc') <- 123
microbenchmark(2 + 2, 2 + a, sum(2, a), sum(2, 2), check='equivalent')
## check='equal' will fail due to difference in attribute
if (FALSE) {
microbenchmark(2 + 2, 2 + a, sum(2, a), sum(2, 2), check='equal')
}