recommenderlab (version 0.2-6)

calcPredictionAccuracy: Calculate the Prediction Error for a Recommendation

Description

Calculate prediction accuracy. For predicted ratings MAE (mean average error), MSE (means squared error) and RMSE (root means squared error) are calculated. For topNLists various binary classification metrics are returned (e.g., precision, recall, TPR, FPR).

Usage

calcPredictionAccuracy(x, data, ...)
# S4 method for realRatingMatrix,realRatingMatrix
calcPredictionAccuracy(x, data, byUser=FALSE,...)
# S4 method for topNList,realRatingMatrix
calcPredictionAccuracy(x, data, byUser=FALSE, given=NULL, goodRating=NA,...)
# S4 method for topNList,binaryRatingMatrix
calcPredictionAccuracy(x, data, byUser=FALSE, given=NULL,...)

Arguments

x

Predicted items in a "topNList" or predicted ratings as a "realRatingMatrix"

data

Observed true ratings for the users as a "RatingMatrix". The users have to be in the same order as in x.

byUser

logical; Should the errors be averaged by user or over all recommendations?

given

how many items were given to create the predictions.

goodRating

If x is a "topNList" and data is a "realRatingMatrix" then goodRating is used as the threshold for determining what rating in data is considered a good rating.

...

further arguments.

Value

Returns a vector with the appropriate measures averaged over all users. For byUser=TRUE, a matrix with a row for each user is returned.

Details

The function calculates the accuracy of predictions compared to the observed true ratings (data).

If both, the predictions and the actual observed ratings are numeric ratings (i.e. a "realRatingMatrix"), then the error measures RMSE, MSE and MAE are returned.

If the predictions are a "topNList" and the observed data is a "binaryRatingMatrix", then binary classification measures like precision, recall, TPR and FPR are calculated.

If the ratings are a "topNList" and the observed data is a "realRatingMatrix" then goodRating is used to determine what rating in data is considered a good rating for calculating binary classification measures. This means that an item in the topNList is considered a true positive if it has a rating of goodRating or better in the observed data.

References

Asela Gunawardana and Guy Shani (2009). A Survey of Accuracy Evaluation Metrics of Recommendation Tasks, Journal of Machine Learning Research 10, 2935-2962.

See Also

'>topNList, '>binaryRatingMatrix, '>realRatingMatrix.

Examples

# NOT RUN {
### real valued recommender
data(Jester5k)

## create 90/10 split (known/unknown) for the first 500 users in Jester5k
e <- evaluationScheme(Jester5k[1:500,], method="split", train=0.9,
    k=1, given=15)
e

## create a user-based CF recommender using training data
r <- Recommender(getData(e, "train"), "UBCF")

## create predictions for the test data using known ratings (see given above)
p <- predict(r, getData(e, "known"), type="ratings")
p

## compute error metrics averaged per user and then averaged over all
## recommendations
calcPredictionAccuracy(p, getData(e, "unknown"))
head(calcPredictionAccuracy(p, getData(e, "unknown"), byUser=TRUE))

## evaluate topNLists instead (you need to specify given and goodRating!)
p <- predict(r, getData(e, "known"), type="topNList")
p
calcPredictionAccuracy(p, getData(e, "unknown"), given=15, goodRating=5)

## evaluate a binary recommender
data(MSWeb)
MSWeb10 <- sample(MSWeb[rowCounts(MSWeb) >10,], 50)

e <- evaluationScheme(MSWeb10, method="split", train=0.9,
    k=1, given=3)
e

## create a user-based CF recommender using training data
r <- Recommender(getData(e, "train"), "UBCF")

## create predictions for the test data using known ratings (see given above)
p <- predict(r, getData(e, "known"), type="topNList", n=10)
p

calcPredictionAccuracy(p, getData(e, "unknown"), given=3)
# }