mlr (version 2.13)

friedmanPostHocTestBMR: Perform a posthoc Friedman-Nemenyi test.

Description

Performs a [PMCMR::posthoc.friedman.nemenyi.test] for a [BenchmarkResult] and a selected measure. This means *all pairwise comparisons* of `learners` are performed. The null hypothesis of the post hoc test is that each pair of learners is equal. If the null hypothesis of the included ad hoc [stats::friedman.test] can be rejected an object of class `pairwise.htest` is returned. If not, the function returns the corresponding friedman.test. Note that benchmark results for at least two learners on at least two tasks are required.

Usage

friedmanPostHocTestBMR(bmr, measure = NULL, p.value = 0.05,
  aggregation = "default")

Arguments

bmr

(BenchmarkResult) Benchmark result.

measure

(Measure) Performance measure. Default is the first measure used in the benchmark experiment.

p.value

(`numeric(1)`) p-value for the tests. Default: 0.05

aggregation

(character(1)) “mean” or “default”. See getBMRAggrPerformances for details on “default”.

Value

([pairwise.htest]): See [PMCMR::posthoc.friedman.nemenyi.test] for details. Additionally two components are added to the list:

f.rejnull (`logical(1)`)

Whether the according friedman.test rejects the Null hypothesis at the selected p.value

crit.difference (`list(2)`)

Minimal difference the mean ranks of two learners need to have in order to be significantly different

See Also

Other benchmark: BenchmarkResult, batchmark, benchmark, convertBMRToRankMatrix, friedmanTestBMR, generateCritDifferencesData, getBMRAggrPerformances, getBMRFeatSelResults, getBMRFilteredFeatures, getBMRLearnerIds, getBMRLearnerShortNames, getBMRLearners, getBMRMeasureIds, getBMRMeasures, getBMRModels, getBMRPerformances, getBMRPredictions, getBMRTaskDescs, getBMRTaskIds, getBMRTuneResults, plotBMRBoxplots, plotBMRRanksAsBarChart, plotBMRSummary, plotCritDifferences, reduceBatchmarkResults

Examples

# NOT RUN {
# see benchmark
# }