pact (version 0.5.0)

eval.pact.cv: Evaluation functions for cross-validated predictions

Description

Methods for the evaluation of the cross-validated predictive scores obtained from pact.cv

Usage

eval.pact.cv(out.cv, method = c("discrete", "continuous"), g = log(1), plot.score = TRUE, plot.time = NULL, perm.test = FALSE, nperm = 100)

Arguments

out.cv

The object from pact.cv

method

The evaluation method. Currently two options, method='discrete' or method='continuous', are available. See 'Details'.

g

The cut-point for grouping scores into subsets 'benefit' and 'no benefit' from new treatment. Ignored for method='continuous'.

plot.score

Used only for plots if method='continuous' is chosen for survival response. Logical representing whether survival curves at specific quantiles of cross-validated scores are to be drawn. See 'Details'.

plot.time

Used only for plots if method='continuous' is chosen for survival response. Probability of survival greater than plot.time is plotted as a function of cross-validated score and Treatment. See 'Details'.

perm.test

Logical. If perm.test=TRUE, a permutation based test of significance is conducted for statistics computed from the cross-validated scores. See 'Value' and the package vignette and for more details on the permutation tests.

nperm

The number of permutations for the permutation test. Ignored if perm.test=FALSE

Value

The return object is of class eval.cv and is a list whose components depend on the family ('cox' or 'binomial') and the chosen evaluation method ('continuous' or 'discrete')

LR.Benefit

For family='cox' and method='discrete'. The log-rank statistic for the survival difference between E and C for the 'benefit' from E group.

LR.NoBenefit

For family='cox' and method='discrete'. The log-rank statistic for the survival difference between E and C for the 'do not benefit' from E group.

RR.T.Benefit

For family='binomial' and method='discrete'. The response rate for subjects getting E in the 'benefit' from E group.

RR.C.Benefit

For family='binomial' and method='discrete'. The response rate for subjects getting C in the 'benefit' from E group

RR.T.NoBenefit

For family='binomial' and method='discrete'. The response rate for subjects getting E in the 'do not benefit' from E group

RR.C.NoBenefit

For family='binomial' and method='discrete'. The response rate for subjects getting C in the 'do not benefit' from E group

.

pval.Benefit

If perm.test=TRUE, p-value from permutation test. For family='cox' and method='discrete', permutation based p-value for LR.Benefit. For family='binomial' and method='discrete', permutation based p-value for difference in response rates for E and C for the subset predicted 'benefit' from E.

pval.NoBenefit

If perm.test=TRUE, p-value from permutation test. For family='cox' and method='discrete', permutation based p-value for LR.NoBenefit. For family='binomial' and method='discrete', permutation based p-value for difference in response rates for E and C for the subset predicted 'no benefit' from E.

reg

For method='continuous', the regression model with treatment, predictive score and treatment x predictive score interaction

pval.twosided

For method='continuous'. Two-sided (non-directional) permutation based p-value for the treatment x predictive score interaction coefficient

pval.onesided

For method='continuous'. One-sided (directional, greater) permutation based p-value for the treatment x predictive score interaction coefficient

call

The function call

Additional plots for both method='discrete' as well as method='continuous'. print method is available for a nice display of objects of class eval.cv. See package vignette.

Details

Currently two methods are defined for the evaluation of the scores obtained from pact.cv. In method='discrete' a user specified cut-off score is used to classify the subjects into groups 'benefit' or 'do not benefit' from new treatment. In each of the 'benefit' and 'do not benefit' groups the actual responses in the control (C) and the experimental (E) groups are compared. For the 'cox' family, the 'score' for a subject represents the predicted change in the log hazard when the subject is treated with E as against C (with lower values denoting benefit with E). In the case of the 'binomial' family, the 'score' represents the predicted change in the log odds of a response when the subject is treated with E as against C (with higher values denoting benefit with E). For the 'cox' family, examples of the cut-point g could be g=log(1) with score < g meaning benefit with E. Or one could be more stringent and have g correspond to a 30% reduction in hazard (g=log(0.70)). For the 'binomial' family, g=log(1.20) with score > g meaning sensitive to E would mean that subjects predicted to receive at least 20% increase in odds of response with E are classified as benefitting from E. In method='continuous' no cut-off is applied to the cross-validated scores. A Cox proportional hazards (PH) regression or a logistic regression model (respectively for 'survival' and 'binary' response) is then developed that includes the main effect of treatment, main effect of cross-validated score, and treatment*score interaction. For survival response, this model is used to generate the Kaplan Meier survival curves for each treatment at the at 20th, 40th, 60th and 80th percentiles of predictive scores (plot.score = TRUE). The model is also used to compute the estimated probability of surviving beyond a landmark time specified in plot.time as a function of treatment and (cross-validated) score (if plot.time = NULL, this plot is not produced). For binary response, the output from evaluation is a plot of the probability of response as a functions of the predictive score and Treatment. If perm.test=TRUE, permutation based significance tests are performed on appropriate test statistics and p-values are computed. See 'Value' and the package vignette and for more details on the permutation tests.

Examples

### Survival response
data(prostateCancer)
Y <- prostateCancer[,3:4]
Xf <- prostateCancer[,7:8]
Xv <- prostateCancer[,c(5:6,9)]
Treatment <- as.factor(prostateCancer[,2])
p <- pact.fit(Y=Y, Xf=Xf, Xv=Xv, Treatment=Treatment, family="cox", varSelect="univar")
cv <- pact.cv(p, nfold=5)
## Not run: eval.pact.cv(cv, method="discrete", g=log(0.80), perm.test=TRUE, nperm=500)  ## At least 20% predicted reduction in HR classified as 'sensitive'
eval.pact.cv(cv, method="continuous", plot.score=TRUE, perm.test=FALSE)

### Binary response
data(EORTC10994)
Y <- as.factor(EORTC10994[,4])
Xv <- EORTC10994[,c(2,5:7)]
Treatment <- as.factor(EORTC10994[,3])
p <- pact.fit(Y=Y,Xv=Xv,Treatment=Treatment,family="binomial", varSelect="univar")
cv <- pact.cv(p, nfold=5)
## Not run: eval.pact.cv(cv, method="discrete", g=log(1), perm.test=TRUE, nperm=500)