Percentage of correct predictions (PCP) for models with binary outcome.
performance_pcp(model, ci = 0.95, method = "Herron")Model with binary outcome.
The level of the confidence interval.
Name of the method to calculate the PCP (see 'Details').
Default is "Herron". May be abbreviated.
A list with several elements: the percentage of correct predictions of the full and the null model, their confidence intervals, as well as the chi-squared and p-value from the Likelihood-Ratio-Test between the full and null model.
method = "Gelman-Hill" (or "gelman_hill") computes the
PCP based on the proposal from Gelman and Hill 2017, 99, which is
defined as the proportion of cases for which the deterministic prediction
is wrong, i.e. the proportion where the predicted probability is above 0.5,
although y=0 (and vice versa) (see also Herron 1999, 90).
method = "Herron" (or "herron") computes a modified version
of the PCP (Herron 1999, 90-92), which is the sum of predicted probabilities,
where y=1, plus the sum of 1 - predicted probabilities, where y=0, divided
by the number of observations. This approach is said to be more accurate.
The PCP ranges from 0 to 1, where values closer to 1 mean that the model predicts the outcome better than models with an PCP closer to 0. In general, the PCP should be above 0.5 (i.e. 50%), the closer to one, the better. Furthermore, the PCP of the full model should be considerably above the null model's PCP.
The likelihood-ratio test indicates whether the model has a significantly better fit than the null-model (in such cases, p < 0.05).
Herron, M. (1999). Postestimation Uncertainty in Limited Dependent Variable Models. Political Analysis, 8, 83<U+2013>98.
Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge; New York: Cambridge University Press, 99
# NOT RUN {
data(mtcars)
m <- glm(formula = vs ~ hp + wt, family = binomial, data = mtcars)
performance_pcp(m)
performance_pcp(m, method = "Gelman-Hill")
# }
Run the code above in your browser using DataLab