Effect of Test Dimensionality and Strength of their Relationship on Statistical Properties and Standard Errors of Person Fit Indices
Keywords:
Wright index, drasgow index, almehrizi index, simulated data, nonfit patternsAbstract
The study aims to examine the effect of test dimensionality and the strength of their relationship on statistical properties and standard errors of three-person fit indices (Wright, Drasgow, Almehrizi) using seven simulated data sets of 1000 subjects. These data sets result from two factors: Number of dimensions (three levels: one, two, three) and relationship strength among dimensions (three levels: 0.0, 0.4,.8). Results revealed that Drasgow and Almehrizi indices showed the highest percentages of aberrant responses which were close to expected rates (5%) whereas Wright index showed the lowest. Descriptive statistics showed that the Almehrizi index was the closest to the descriptive statistics of standard normal distribution. Also, the result showed that test dimensionality and strength of interrelationship among dimensions affected the standard errors of both the mean and variance of the Wright index, whereas did not affect the standard errors of both the mean and variance of the Almehrizi index and Drasgow index. Almehrizi index was the most consistent index, especially with variance.
Downloads
References
Abu shindi, Y.(2008). The influence of test dimensionality and correlation between them on items parameters: A simulation study, Unpublished doctoral dissertation, Yarmouk University, Jordan.
Abu Shindi, Y., Al-mehrzi, R., & Omara, E.(2018). The Estimation Accuracy of True Scores at Different Degrees of Local Dependence among Test Items for Different Ability distributions. Journal of educational and psychological Science- Bahrain, 19(3), 466-491.
Ackerman, T.(1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied measurement in education, 7(4), 255-278.
Al-Mahrazi, R. (2004). Investigating a new modification of the residual-based person fit index and its relationship with other indices in dichotomous item response theory, Unpublished dissertation, University of Iowa, Iowa City.
Al-Mehrzi, R. (2010). Comparing among new residual-fit and wright's Indices for dichotomous three-parameter IRT model with standardized tests. Journal of Educational and Psychological Studies, Sultan Qaboos University, 4(2), 14-26
Al-Mqasqus, M.(2008). Comparing Methods for Detecting Local Independence of Multidimensional Items at Different Ability Levels: A Simulation Study, Unpublished doctoral dissertation, Yarmouk University, Jordan.
Alnuami, E.(2007). The effect of Local independent on item response theory estimation, Unpublished doctoral dissertation, Yarmouk University, Jordan.
Chen, C. & Wang, W. (2007). Effect of ignoring item interaction on item parameter estimation and detection of interacting items. Applied Psychological Measurement, (31), 388-411.
Deng, W., & Torre, J. (2008). Improving person fit assessment by correcting the ability estimate and its reference distribution. Journal of Educational Measurement, 45(2), 159-177.
Drasgow, F. (1982). Choice of the test models for appropriateness measurement. Applied Psychological Measurement, (6), 297-308.
Drasgow, F., Levine, M.V., & Williams, E.A. (1985). Appropriateness measurement with polytomous item response models and standardized indices. British Journal of Mathematical and statistical psychology, (38), 67-86.
Hambleton, R., & Swaminathan, H. (1985). Item Response Theory: Principles and applications. Boston: Kluwer-Nijhoff.
Hambleton, R., Swaminathan, H., Cook, L., Eignor, D., & Gifford, J.(1987). Developments in Latent trait theory: Models, technical issues, and applications. Review of Educational Research, (48), 467-510.
Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139-164.
Hulin, C., Drasgow, F., & Parsons, C. (1983). Item Response Theory: Application to Psychological Measurement. Homewood, II: Irwin.
Iasonas, L., Bill, B., & David, W.(2000). The consistency of examinee misfit across tests on the same subject and across subject: the case of the KS2 mathematics and science National Curriculum tests in England.
Jarrah, B.(2009). Comparative of Person fit indices in item response theory models by actual Data, Unpublished doctoral dissertation, Yarmouk University, Jordan.
Levine, M., & Rubin, D. (1979). Measuring the appropriateness of multiple - choice test scores. Journal of Educational Statistics, (4), 269-290.
Li, M. F., & Olijnike, s. (1997). The power of rasch person-fit statistics in detecting unusual person patterns. Applied Psychological Measurement, (21), 215-231.
Linn, R., & TatsukaK. (1983). Indications for detecting unusual patterns: Links between two general approaches and potential application. Applied psychological measurement, 7, 81-96.
Lopez, A., & Montesinos, H. (2005). Fitting rasch model using appropriateness Measure Statistics. The Spanish Journal of Psychology, (8), 100-110.
McKinley, R. L., & Way, W. D. (1992). The feasibility of modeling secondary TOEFL ability dimensions using multidimensional IRT models. ETS Research Report Series, (1), i-22.
Meijer, R. R., & Sijtsma, K. (1994). Detection of aberrant item score patterns: A review of recent developments. Applied Measurement in Education, 8(3), 261-272.
Meijer, R., & Van, K. (1999). The Null distribution of the person-fit statistics for conventional and adaptive tests. Applied psychological Measurement, 23, 327-345.
Muraki, E. (2000). RESGEN: Item response generator. Princeton, NJ: Educational Testing Service.
Odeh, A., Almehrzi, R., & Abu shindig, Y. (2019).An Improved method to interpret persons aberrant response pattern in tests and acomparative of five person fit indices. Alshariqa university for social and humanities Science.
Raise, S., & Due, A. (1991). The influence of test characteristic on the detection of aberrant response patterns. Applied psychological Measurement, (15), 217-226.
Reckase, M. (2000). Multidimensional Item Response Theory. New York: Springer.
Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9(4), 401-412.
Reese, L. M. (1999). A Classical Test Theory Perspective on LSAT Local Item Dependence. LSAC Research Report Series. Statistical Report.
Rogers, H., & Hattie, J.(1987). A Monte Carlo Investigation of several person and item fit statistics for item response models. Applied Psychological Measurement, (11), 47-57.
Smith, R.(1982). Detecting measurement distribution with the rash model, Unpublished doctoral dissertation, University of Chicago.
Thompson, T. D., & Pommerich, M. (1996). Examining the Sources and Effects of Local Dependence.
Waller, M. (1981). A procedure for comparing logistic latent trait models. Journal of Educational Measurement, (18), 119-125.
Way, W. D., Ansley, T. N., & Forsyth, R. A. (1988). The comparative effects of compensatory and non-compensatory two-dimensional data on unidimensional IRT estimation. Applied psychological Measurement, 12(3), 239-252.
Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Education Measurement, (114), 96-115.
Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch measurement. Chicago: Mesa Press.
Yen, W. (1993). Scaling performance assessments: strategies for managing local item dependence. Journal of Educational Measurement, 30(3), 187–213.
Zenisky, A. L., Hambleton, R. K., & Sireci, S. G. (2001). Effects of Local Item Dependence on the Validity of IRT Item, Test, and Ability Statistics. MCAT Monograph.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
