rmcfs (version 1.3.6)

mcfs: MCFS-ID (Monte Carlo Feature Selection and Interdependency Discovery)

Description

Performs Monte Carlo Feature Selection (MCFS-ID) on a given data set. The data set should define a classification problem with discrete/nominal class labels. This function returns features sorted by RI as well as cutoff value, ID-Graph edges that denote interdependencies (ID), evaluation of top features and other statistics.

Usage

mcfs(formula, data,
    attrWeights = NULL,
    projections = 'auto',
    projectionSize = 'auto',
    featureFreq = 100,
    splits = 5,
    splitSetSize = 500,
    balance = 'auto',
    cutoffMethod = c("permutations", "criticalAngle", "kmeans", "mean", "contrast"),
    cutoffPermutations = 20,
    mode = 1,
    buildID = TRUE,
    finalRuleset = TRUE,
    finalCV = TRUE,
    finalCVSetSize = 1000,
    seed = NA,
    threadsNumber = 4)

Value

data

input data.frame limited to the top important features set.

target

decision attribute name.

RI

data.frame that contains all features with relevance scores sorted from the most relevant to the least relevant. This is the ranking of features.

ID

data.frame that contains features interdependencies as graph edges. It can be converted into a graph object by build.idgraph function.

distances

data.frame that contains convergence statistics of subsequent projections.

cmatrix

confusion matrix obtained from all \(s \cdot t\) decision trees.

cutoff

data.frame that contains cutoff values obtained by the following methods: mean, kmeans, criticalAngle, permutations (max RI).

cutoff_value

the number of features chosen as informative by the method defined by parameter cutoffMethod.

cv_accuracy

data.frame that contains classification results obtained by cross validation performed on cutoff_value features. This data.frame exists if finalCV = T.

permutations

this data.frame contains the following results of permutation experiments:

• perm_x all RI values obtained from all permutation experiments;

• RI RI obtained for reference MCFS experiment (i.e, the experiment on the original data); p-values from Anderson-Darling normality test applied separately for each feature to the cutoffPermutations RI set;

• t_test_p \(p\)-values from Student-t test applied separately for each feature to the cutoffPermutations RI vs. reference RI. This data.frame exists if parameter cutoffPermutations > 0.

jrip

classification rules produced by ripper algorithm and related cross validation result obtained for top features.

params

all settings used by MCFS-ID.

exec_time

execution time of MCFS-ID.

Arguments

formula

specifies decision attribute and relation between class and other attributes (e.g. class~.). The target attribute can be nominal (then MCFS-ID uses decision tree) or numerical (then MCFS-ID uses regression tree).

data

defines input data.frame containing all features with decision attribute included. This data.frame must contain proper types of columns. Columns character, factor, Date, POSIXct, POSIXt are treated as nominal/categorical and remaining columns as numerical/continuous. Decision attribute defined by formula can be nominal or numerical.

attrWeights

defines vector of length = ncol(data) of attributes weights - weight 10 denotes 10 times larger chance for the attribute to be selected to the random subset than if weight equals to 1.

projections

defines the number of subsets (projections) with randomly selected features. This parameter is usually set to a few thousands and is denoted in the paper as s. By default it is set to 'auto' and this value is based on size of input data set and featureFreq parameter.

projectionSize

defines the number of features in one subset. It can be defined by an absolute value (e.g. 100 denotes 100 randomly selected features) or by a fraction of input attributes (e.g. 0.05 denotes 5% of input features). This parameter is denoted in the paper as m. If is set to 'auto' then projectionSize equals to \(\sqrt{d}\), where d is the number of input features. Minimum number of features in one subset is 1.

featureFreq

determines how many times each input feature should be randomly selected when projections = 'auto'.

splits

defines the number of splits of each subset. This parameter is denoted in the paper as t. The size of the training set in the input subset is always set on 66%.

splitSetSize

determines whether to limit input dataset size. It helps to speedup computation for data sets with a large number of objects. If the parameter is larger than 1, it determines the number of objects that are drawn at random for each of the \(s \cdot t\) decision trees. If splitSetSize = 0 then the MCFS uses all objects in each iteration.

balance

determines the way to balance classes. It should be set to 2 or higher if input dataset contains heavily unbalanced classes. Each subset s will contain all the objects from the least frequent class and randomly selected set of objects from each of the remaining classes. This option helps to select features that are important for discovering a relatively rare class. The parameter defines the maximal imbalance ratio. If the ratio is set to 2, then subset s will contain the number of objects from each class (but the least frequent one) proportional to the square root of the class size \(size(c)^{1/2}\). If balance = 0 then balancing is turned off. If balance = 1 it is on but does not change the size of classes. Default value is 'auto'.

cutoffMethod

determines the final cutoff method. Default value is 'permutations'. The methods of finding cutoff value between important and unimportant attributes are the following:

• permutations - the method consists in permuting the decision attribute at least 20 times and running the MCFS-ID algorithm for each permutation. The set of the maximal RIs from all these experiments is assumed approximately normally distributed and a critical value based on the the one-sided (upper-tailed) Student's t-test (at 95% significance level) is provided. A feature is declared informative if its RI in the original ranking (without any permutation) exceeds the obtained critical value. A more detailed description of this method is included in the paper.

• criticalAngle - critical angle method is based on the plot of the features' RIs in decreasing order of size, with the corresponding features equally spaced along the abscissa. The plot can be seen as piecewise linear function, where each linear segment joins two neighboring RIs. Roughly speaking, the cutoff (placed on the abscissa) corresponds to this point on the plot where the slope of consecutive segments changes significantly.

• kmeans - the method is based on clustering the RI values into two clusters by the k-means algorithm. It sets the cutoff where the two clusters are separated. This method is quite valuable when data contains a subset of very informative features.

• mean - cutoff value is set on mean values obtained from all the implemented methods.

• contrast - This method adds 10% contrast (pure numerical random) atributes to the data then MCFS-ID is executed. Position of top 5% of them determines cutoff value. Usually it gives the largest cutoff beacause it select all attributes that are more informative than pure noise.

cutoffPermutations

determines the number of permutation runs. It needs at least 20 permutations (cutoffPermutations = 20) for a statistically significant result. Minimum value of this parameter is 3, however if it is 0 then permutations method is turned off.

mode

determines number of stages in MCFS filtering. If mode = 2 then MCFS is running new method that is based on two stage filtering. This method is much faster for BIG DATA - 1st stage filtering is performed based on contrast attributes (same as cutoffMethod = 'contrast') and 2nd stage is performed based on permutations experiments. If mode = 1 then it always runs one stage filtering the same as in rmcfs 1.2.x.

buildID

if = TRUE, Interdependencies Discovery is on and all ID-Graph edges are collected.

finalRuleset

if = TRUE, classification rules (by ripper algorithm) are created on the basis of the final set of features.

finalCV

if = TRUE, it runs 10 folds cross validation (cv) experiments on the final set of features. The following set of classifiers is used: C4.5, NB, SVM, kNN, logistic regression and Ripper.

finalCVSetSize

limits the number of objects used in the final cv experiment. For each out of 3 cv repetitions, the objects are selected randomly from the uniform distribution.

seed

seed for random number generator in Java. By default the seed is random. Replication of the result is possible only if threadsNumber = 1.

threadsNumber

number of threads to use in computation. More threads needs more CPU cores as well as memory usage is a bit higher. It is recommended to set this value equal to or less than CPU available cores.

References

M. Draminski, J. Koronacki (2018),"rmcfs: An R Package for Monte Carlo Feature Selection and Interdependency Discovery", Journal of Statistical Software, vol 85(12), 1-28, tools:::Rd_expr_doi("10.18637/jss.v085.i12")

Examples

  if (FALSE) ###dontrunbegin

  ####################################
  ######### Artificial data ##########
  ####################################
  # Set VM size for Java
  options(java.parameters = "-Xmx8g")
  library(rmcfs)
  
  # create input data and review it
  adata <- artificial.data(rnd_features = 10)
  showme(adata)
  
  # Parametrize and run MCFS-ID procedure
  result <- mcfs(class~., adata, cutoffPermutations = 3, featureFreq = 50,
                  buildID = TRUE, finalCV = FALSE, finalRuleset = FALSE, 
                 threadsNumber = 2)

  # Print basic information about mcfs result
  print(result)
  
  # Review cutoff values for all methods
  print(result$cutoff)
  
  # Review cutoff value used in plots
  print(result$cutoff_value)
  
  # Plot & print out distances between subsequent projections. 
  # These are convergence MCFS-ID statistics.
  plot(result, type = "distances")
  print(result$distances)
  
  # Plot & print out 50 most important features and show max RI values from 
  # permutation experiment.
  plot(result, type = "ri", size = 50)
  print(head(result$RI, 50))
  
  # Plot & print out 50 strongest feature interdependencies.
  plot(result, type = "id", size = 50)
  print(head(result$ID, 50))
  
  # Plot features ordered by RI. Parameter 'size' is the number of 
  # top features in the chart. By default it is set on cutoff_value + 10
  plot(result, type = "features", cex = 1)

  # Here we set 'size' at fixed value 10.
  plot(result, type = "features", size = 10)
  
  # Plot cv classification result obtained on top features.
  # In the middle of x axis red label denotes cutoff_value.
  # plot(result, type = "cv", cv_measure = "wacc", cex = 0.8)
  
  # Plot & print out confusion matrix. This matrix is the result of 
  # all classifications performed by all decision trees on all s*t datasets.
  plot(result, type = "cmatrix")
  
  # build interdependencies graph (all default parameters).
  gid <- build.idgraph(result)
  plot(gid, label_dist = 1)
  
  # build interdependencies graph for top 6 features 
  # and top 12 interdependencies and plot all nodes
  gid <- build.idgraph(result, size = 6, size_ID = 12, orphan_nodes = TRUE)
  plot(gid, label_dist = 1)

  # Export graph to graphML (XML structure)
  path <- tempdir()
  igraph::write_graph(gid, file = file.path(path, "artificial.graphml"), 
              format = "graphml", prefixAttr = FALSE)
  
  # Export and import results to/from csv files
  export.result(result, path = path, label = "artificial")
  result <- import.result(path = path, label = "artificial")

  # Find out how many trees with the given attribute has been built (and nodes based the 
  # attribute in total). Notice that result$RI$projections keeps the number of subsets where
  # the feature was randomly picked. The value: result$RI$projections*result$params$mcfs.splits
  # is the total number of trees for a given attribute that could be built based on the attribute.
  # This normalization takes into the consideration not the full number of st trees
  # but only the fraction that is trained on datasets with the attribute.

  
  result$RI$classifiers*(result$RI$projections*result$params$mcfs.splits)	
  result$RI$nodes*(result$RI$projections*result$params$mcfs.splits)


  ####################################
  ########## Alizadeh data ###########
  ####################################
  
  # Load Alizadeh dataset.
  # A 4026 x 62 gene expression data matrix of log-ratio values. The last column contains 
  # the annotations of the 62 samples with respect to the cancer types C, D, F.
  # The data are from the lymphoma/leukemia study of A. Alizadeh et al., Nature 403:503-511 (2000), 
  # http://llmpp.nih.gov/lymphoma/index.shtml
  
  alizadeh <- read.csv(file="http://home.ipipan.waw.pl/m.draminski/files/data/alizadeh.csv", 
                        stringsAsFactors = FALSE)
  showme(alizadeh)
  
  # Fix data types and data values - replace characters such as "," " " "/" etc. 
  # from values and column names and fix data types
  # This function may help if mcfs has any problems with input data
  alizadeh <- fix.data(alizadeh)
  
  # Run MCFS-ID procedure on default parameters. 
  # For larger real data (thousands of features) default 'auto' settings are the best.
  # This example may take 10-20 minutes but this one is a real dataset with 4026 features.
  # Set up more threads according to your CPU cores number.
  result <- mcfs(class~., alizadeh, featureFreq = 100, cutoffPermutations = 10, threadsNumber = 8)
  
  # Print basic information about mcfs result.
  print(result)
  
  # Plot & print out distances between subsequent projections. 
  plot(result, type="distances")
  
  # Show RI values for top 500 features and max RI values from permutation experiment.
  plot(result, type = "ri", size = 500)
  
  # Plot heatmap on top features, only numeric features are presented
  plot(result, type = "heatmap", size = 20, heatmap_norm = 'norm', heatmap_fun = 'median')
  
  # Plot cv classification result obtained on top features.
  # In the middle of x axis red label denotes cutoff_value.
  plot(result, type = "cv", cv_measure = "wacc", cex = 0.8)
  
  # build interdependencies graph.
  gid <- build.idgraph(result, size = 20)
  plot.idgraph(gid, label_dist = 0.3)
  
  ###dontrunend