Return scenario object with default values.
defaultScenario(scenario = list(), params_def = .irace.params.def)
A list indexed by the irace parameter names, containing the default values for each parameter, except for those already present in the scenario passed as argument. The scenario list contains the following elements:
General options:
scenarioFile
Path of the file that describes the configuration scenario setup and other irace settings. (Default: "./scenario.txt"
)
execDir
Directory where the programs will be run. (Default: "./"
)
logFile
File to save tuning results as an R dataset, either absolute path or relative to execDir. (Default: "./irace.Rdata"
)
quiet
Reduce the output generated by irace to a minimum. (Default: 0
)
debugLevel
Debug level of the output of irace
. Set this to 0 to silence all debug messages. Higher values provide more verbose debug messages. (Default: 0
)
seed
Seed of the random number generator (by default, generate a random seed). (Default: NA
)
repairConfiguration
User-defined R function that takes a configuration generated by irace and repairs it. (Default: ""
)
postselection
Perform a postselection race after the execution of irace to consume all remaining budget. Value 0 disables the postselection race. (Default: 1
)
aclib
Enable/disable AClib mode. This option enables compatibility with GenericWrapper4AC as targetRunner script. (Default: 0
)
Elitist irace
:
elitist
Enable/disable elitist irace. (Default: 1
)
elitistNewInstances
Number of instances added to the execution list before previous instances in elitist irace. (Default: 1
)
elitistLimit
In elitist irace, maximum number per race of elimination tests that do not eliminate a configuration. Use 0 for no limit. (Default: 2
)
Internal irace
options:
sampleInstances
Randomly sample the training instances or use them in the order given. (Default: 1
)
softRestart
Enable/disable the soft restart strategy that avoids premature convergence of the probabilistic model. (Default: 1
)
softRestartThreshold
Soft restart threshold value for numerical parameters. (Default: 1e-04
)
nbIterations
Maximum number of iterations. (Default: 0
)
nbExperimentsPerIteration
Number of runs of the target algorithm per iteration. (Default: 0
)
minNbSurvival
Minimum number of configurations needed to continue the execution of each race (iteration). (Default: 0
)
nbConfigurations
Number of configurations to be sampled and evaluated at each iteration. (Default: 0
)
mu
Parameter used to define the number of configurations sampled and evaluated at each iteration. (Default: 5
)
Target algorithm parameters:
parameterFile
File that contains the description of the parameters of the target algorithm. (Default: "./parameters.txt"
)
parameters
Parameters space object (usually read from a file using readParameters
). (Default: ""
)
Target algorithm execution:
targetRunner
Executable called for each configuration that executes the target algorithm to be tuned. See the templates and examples provided. (Default: "./target-runner"
)
targetRunnerLauncher
Executable that will be used to launch the target runner, when targetRunner
cannot be executed directly (e.g., a Python script in Windows). (Default: ""
)
targetCmdline
Command-line arguments provided to targetRunner
(or targetRunnerLauncher
if defined). The substrings {configurationID}
, {instanceID}
, {seed}
, {instance}
, and {bound}
will be replaced by their corresponding values. The substring {targetRunnerArgs}
will be replaced by the concatenation of the switch and value of all active parameters of the particular configuration being evaluated. The substring {targetRunner}
, if present, will be replaced by the value of targetRunner
(useful when using targetRunnerLauncher
). (Default: "{configurationID} {instanceID} {seed} {instance} {bound} {targetRunnerArgs}"
)
targetRunnerRetries
Number of times to retry a call to targetRunner
if the call failed. (Default: 0
)
targetRunnerTimeout
Timeout in seconds of any targetRunner
call (only applies to target-runner
executables not to R functions), ignored if 0. (Default: 0
)
targetRunnerData
Optional data passed to targetRunner
. This is ignored by the default targetRunner
function, but it may be used by custom targetRunner
functions to pass persistent data around. (Default: ""
)
targetRunnerParallel
Optional R function to provide custom parallelization of targetRunner
. (Default: ""
)
targetEvaluator
Optional script or R function that provides a numeric value for each configuration. See templates/target-evaluator.tmpl (Default: ""
)
deterministic
If the target algorithm is deterministic, configurations will be evaluated only once per instance. (Default: 0
)
parallel
Number of calls to targetRunner
to execute in parallel. Values 0
or 1
mean no parallelization. (Default: 0
)
loadBalancing
Enable/disable load-balancing when executing experiments in parallel. Load-balancing makes better use of computing resources, but increases communication overhead. If this overhead is large, disabling load-balancing may be faster. (Default: 1
)
mpi
Enable/disable MPI. Use Rmpi
to execute targetRunner
in parallel (parameter parallel
is the number of slaves). (Default: 0
)
batchmode
Specify how irace waits for jobs to finish when targetRunner
submits jobs to a batch cluster: sge, pbs, torque, slurm or htcondor. targetRunner
must submit jobs to the cluster using, for example, qsub
. (Default: 0
)
Initial configurations:
initConfigurations
Data frame describing initial configurations (usually read from a file using readConfigurations
). (Default: ""
)
configurationsFile
File that contains a table of initial configurations. If empty or NULL
, all initial configurations are randomly generated. (Default: ""
)
Training instances:
instances
Character vector of the instances to be used in the targetRunner
. (Default: ""
)
trainInstancesDir
Directory where training instances are located; either absolute path or relative to current directory. If no trainInstancesFiles
is provided, all the files in trainInstancesDir
will be listed as instances. (Default: ""
)
trainInstancesFile
File that contains a list of training instances and optionally additional parameters for them. If trainInstancesDir
is provided, irace
will search for the files in this folder. (Default: ""
)
blockSize
Number of training instances, that make up a block' in \code{trainInstancesFile}. Elimination of configurations will only be performed after evaluating a complete block and never in the middle of a block. Each block typically contains one instance from each instance class (type or family) and the block size is the number of classes. (Default:
1)} } \item Tuning budget: \describe{ \item{
maxExperiments}{Maximum number of runs (invocations of \code{targetRunner}) that will be performed. It determines the maximum budget of experiments for the tuning. (Default:
0)} \item{
minExperiments}{Minimum number of runs (invocations of \code{targetRunner}) that will be performed. It determines the minimum budget of experiments for the tuning. The actual budget depends on the number of parameters and \code{minSurvival}. (Default:
NA)} \item{
maxTime}{Maximum total execution time for the executions of \code{targetRunner}. \code{targetRunner} must return two values: cost and time. This value and the one returned by \code{targetRunner} must use the same units (seconds, minutes, iterations, evaluations, ...). (Default:
0)} \item{
budgetEstimation}{Fraction (smaller than 1) of the budget used to estimate the mean computation time of a configuration. Only used when \code{maxTime} > 0 (Default:
0.05)} \item{
minMeasurableTime}{Minimum time unit that is still (significantly) measureable. (Default:
0.01)} } \item Statistical test: \describe{ \item{
testType}{Statistical test used for elimination. The default value selects \code{t-test} if \code{capping} is enabled or \code{F-test}, otherwise. Valid values are: F-test (Friedman test), t-test (pairwise t-tests with no correction), t-test-bonferroni (t-test with Bonferroni's correction for multiple comparisons), t-test-holm (t-test with Holm's correction for multiple comparisons). (Default:
"")} \item{
firstTest}{Number of instances evaluated before the first elimination test. It must be a multiple of \code{eachTest}. (Default:
5)} \item{
eachTest}{Number of instances evaluated between elimination tests. (Default:
1)} \item{
confidence}{Confidence level for the elimination test. (Default:
0.95)} } \item Adaptive capping: \describe{ \item{
capping}{Enable the use of adaptive capping, a technique designed for minimizing the computation time of configurations. Capping is enabled by default if \code{elitist} is active, \code{maxTime > 0} and \code{boundMax > 0}. (Default:
NA)} \item{
cappingType}{Measure used to obtain the execution bound from the performance of the elite configurations.\itemize{\item median: Median performance of the elite configurations.\item mean: Mean performance of the elite configurations.\item best: Best performance of the elite configurations.\item worst: Worst performance of the elite configurations.} (Default:
"median")} \item{
boundType}{Method to calculate the mean performance of elite configurations.\itemize{\item candidate: Mean execution times across the executed instances and the current one.\item instance: Execution time of the current instance.} (Default:
"candidate")} \item{
boundMax}{Maximum execution bound for \code{targetRunner}. It must be specified when capping is enabled. (Default:
0)} \item{
boundDigits}{Precision used for calculating the execution time. It must be specified when capping is enabled. (Default:
0)} \item{
boundPar}{Penalization constant for timed out executions (executions that reach \code{boundMax} execution time). (Default:
1)} \item{
boundAsTimeout}{Replace the configuration cost of bounded executions with \code{boundMax}. (Default:
1)} } \item Recovery: \describe{ \item{
recoveryFile}{Previously saved log file to recover the execution of \code{irace}, either absolute path or relative to the current directory. If empty or \code{NULL}, recovery is not performed. (Default:
"")} } \item Testing: \describe{ \item{
testInstancesDir}{Directory where testing instances are located, either absolute or relative to current directory. (Default:
"")} \item{
testInstancesFile}{File containing a list of test instances and optionally additional parameters for them. (Default:
"")} \item{
testInstances}{Character vector of the instances to be used in the \code{targetRunner} when executing the testing. (Default:
"")} \item{
testNbElites}{Number of elite configurations returned by irace that will be tested if test instances are provided. (Default:
1)} \item{
testIterationElites}{Enable/disable testing the elite configurations found at each iteration. (Default:
0`)
list()
Data structure containing irace
settings. The data structure has to be the one returned by the function
defaultScenario()
or readScenario()
.
data.frame()
Definition of the options accepted by
the scenario. This should only be modified by packages that wish to extend
irace.
Manuel López-Ibáñez and Jérémie Dubois-Lacoste
readScenario()
for reading a configuration scenario from a file.
printScenario()
prints the given scenario.
defaultScenario()
returns the default scenario settings of irace.
checkScenario()
to check that the scenario is valid.