Person fit consists of a set of techniques aimed at detecting unusual responses to tests or questionnaires. There are several person-fit statistics available in the literature, see Karabatsos (2003) and Meijer and Sijtsma (2001) for comprehensive reviews. Both dichotomous and polytomous types of items are considered. This R-package outputs the values of the chosen person-fit statistic, the IDs of the respondents that were flagged, and plots the sample distribution of the scores of the person-fit statistic. Nonparametric person response functions (Sijtsma and Meijer, 2001) may also be requested in order to help interpreting individual answering behaviors (dichotomous data only).
PerFit PerFit The PerFit package contains several person-fit functions. The goal is to detect response vectors that seem to be strange in terms of the sample of respondents or in terms to an item response theory (IRT) model.
There are many person-fit statistics available in the literature. Statistics are typically categorized according to the type of items (Dicho = dichotomous, Poly = polytomous) and the type of IRT model (NParam=nonparametric, Param=parametric) that they apply to. The current version of PerFit includes the following statistics:
| Person-fit statistic (R function) | Reference | Type item | Type IRT model |
r.pbis |
Donlon and Fisher (1968) | Dicho | NParam |
C.Sato |
Sato (1975) | Dicho | NParam |
G, Gnormed |
van der Flier (1977), Meijer (1994) | Dicho | NParam |
A.KB, D.KB, E.KB |
Kane and Brennan (1980) | Dicho | NParam |
U3, ZU3 |
van der Flier (1980, 1982) | Dicho | NParam |
Cstar |
Harnisch and Linn (1981) | Dicho | NParam |
NCI |
Tatsuoka and Tatsuoaka (1982, 1983) | Dicho | NParam |
lz |
Drasgow, Levine, and Williams (1985) | Dicho | Param |
lzpoly |
Drasgow, Levine, and Williams (1985) | Poly | Param |
Ht |
Sijtsma (1986) | Dicho | NParam |
Gpoly |
Molenaar (1991) | Poly | NParam |
Gnormed.poly |
Molenaar (1991), Emons (2008) | Poly | NParam |
lzstar |
Snijders (2001) | Dicho | Param |
U3poly |
Emons (2008) | Poly | NParam |
All functions above have an output of class PerFit.
The package provides other functions that help analyzing the data when conducting person-fit analyses:
| Function | Description |
cutoff |
Estimate cutoff values for the person-fit statistics, to be used as decision rules. |
flagged.resp |
Identify which respondents were flagged according to the chosen cutoff. |
plot (class PerFit) |
Plot the distribution of person-fit scores with the cutoff superimposed. |
More person-fit statistics will be added to the package in future updates.
Versions
Version 1.0 (April 2014)
Version 1.1 (May 2014)
Functions plot.PerFit and PRFplot now allow the user to edit the axes labels and the titles.
Version 1.2 (August 2014)
Some output values of some functions were renamed for the sake of consistency. The package documentation was adapted accordingly.
Version 1.3 (March 2015)
The package underwent a major revision:
Class PerFit now consists of a list with 12 objects.
New methods for objects of class PerFit were added (summary, print).
Routines accomodating for missing values were added.
Function cutoff was updated. Now, model-fitting item response patterns are generated in order to find the cutoff value.
Function plot.PerFit now allows displaying a bootstrap percentile confidence interval for the cutoff statistic, as well as ticks marking the flagged respondents.
Person response functions are now approximated by functional data objects computed by means of the fda package. The functional data objects are returned to the user.
Standard errors for the person-fit statistics are now available (see function PerFit.SE).
Many control checks were added throughout the entire code.
Version 1.4 (July 2015)
The default missing values approach is now pairwise elimination. The imputation methods introduced with version 1.3.1 are still available.
A bug was removed from function lzpoly() (many thanks to Marco Bressan for spotting it!).
The PerFit-package.Rd file was updated (version 1.4.2).
A bug in function U3poly was fixed (version 1.4.3, October 2018). All credit goes to Marek Muszynski for spotting the issue.
Recent changes to base plot() broke function PRFplot(). This was now fixed (version 1.4.4, January 2021). I thank Julie Webbs for pointing this out.
Version 1.4.5, February 2021: Updated affiliation.
Version 1.4.6, October 2021: Updated internal function estIP.poly() to properly manage cases where users supply item and/or person parameters.
Donlon, T. F., and Fischer, F. E. (1968) An index of an individual's agreement with group-defined item difficulties. Educational and Psychological Measurement, 28(1), 105--113.
Drasgow, F., Levine, M. V., and Williams, E. A. (1985) Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38(1), 67--86.
Emons, W. M. (2008) Nonparametric person-fit analysis of polytomous item scores. Applied Psychological Measurement, 32(3), 224--247.
Harnisch, D. L., and Linn, R. L. (1981) Analysis of item response patterns: Questionable test data and dissimilar curriculum practices. Journal of Educational Measurement, 18(3), 133--146.
Kane, M. T., and Brennan, R. L. (1980) Agreement coefficients as indices of dependability for domain-referenced tests. Applied Psychological Measurement, 4(1), 105--126.
Karabatsos, G. (2003) Comparing the Aberrant Response Detection Performance of Thirty-Six Person-Fit Statistics. Applied Measurement In Education, 16(4), 277--298.
Meijer, R. R. (1994) The number of Guttman errors as a simple and powerful person-fit statistic. Applied Psychological Measurement, 18(4), 311--314.
Meijer, R. R., and Sijtsma, K. (2001) Methodology review: Evaluating person fit. Applied Psychological Measurement, 25(2), 107--135.
Molenaar, I. W. (1991) A weighted Loevinger H-coefficient extending Mokken scaling to multicategory items. Kwantitatieve Methoden, 12(37), 97--117.
Sato, T. (1975) The construction and interpretation of S-P tables. Tokyo: Meiji Tosho.
Sijtsma, K. (1986) A coefficient of deviance of response patterns. Kwantitatieve Methoden, 7, 131--145.
Sijtsma, K., and Meijer, R. R. (2001) The person response function as a tool in person-fit research. Psychometrika, 66(2), 191--207.
Snijders, T. B. (2001) Asymptotic null distribution of person fit statistics with estimated person parameter. Psychometrika, 66(3), 331--342.
Tatsuoka, K. K., and Tatsuoka, M. M. (1982) Detection of aberrant response patterns and their effect on dimensionality. Journal of Educational Statistics, 7(3), 215--231.
Tatsuoka, K. K., and Tatsuoka, M. M. (1983) Spotting erroneous rules of operation by the individual consistency index. Journal of Educational Measurement, 20(3), 221--230.
Tendeiro, J. N., Meijer, R. R., and Niessen, A. S. M. (2016). PerFit: An R Package for Person-Fit Analysis in IRT. Journal of Statistical Software, 74(5), 1--27.
van der Flier, H. (1977) Environmental factors and deviant response patterns. In Y. H. Poortinga (Ed.), Basic problems in cross-cultural psychology. Amsterdam: The Netherlands.
van der Flier, H. (1980) Vergelijkbaarheid van individuele testprestaties [Comparability of individual test performance]. Lisse: The Netherlands.
van der Flier, H. (1982) Deviant response patterns and comparability of test scores. Journal of Cross-Cultural Psychology, 13(3), 267--298.
# NOT RUN {
# Load the inadequacy scale data (dichotomous item scores):
data(InadequacyData)
# As an example, compute the Ht person-fit scores:
Ht.out <- Ht(InadequacyData)
# Ht.out$PFscores
# Compute the cutoff value at 1% level:
set.seed(124) # To fix the random seed generator.
Ht.cut <- cutoff(Ht.out, Blvl=.01)
# Plot the sample distribution of the Ht scores with the above cutoff superimposed:
plot(Ht.out, cutoff.obj=Ht.cut)
# Determine which respondents were flagged by Ht at 1% level:
flagged.resp(Ht.out, cutoff.obj=Ht.cut, scores=FALSE)
# Flagged respondents: 30, 37, 46, 49,...
# Plot the person response function of respondent 30 (flagged as aberrant):
Resp30 <- PRFplot(InadequacyData, respID=30)
# Plot the person response function of respondent 35 (not flagged as aberrant):
Resp35 <- PRFplot(InadequacyData, respID=35)
# }
Run the code above in your browser using DataLab