مقارنة فاعلية طرائق التعامل مع القيم المفقودة في دقة تقدير دالة معلومات الاختبار وتحقق افتراضي أُحادية البعد والاستقلال الموضعي الخاصة بنظرية الاستجابة للفقرة

Areej Mohammad Toamah  Al Ahmad; Muhammad Walid Musa Al-Batsh

doi:10.35516/edu.v51i3.5882

Authors

Areej Mohammad Toamah Al Ahmad Department of Educational Measurement and Evaluation, School of Educational Sciences, The University of Jordan, Amman, Jordan https://orcid.org/0009-0001-9146-645X
Muhammad Walid Musa Al-Batsh Department of Educational Measurement and Evaluation, School of Educational Sciences, The University of Jordan, Amman, Jordan https://orcid.org/0009-0008-1104-5577

DOI:

https://doi.org/10.35516/edu.v51i3.5882

Keywords:

Missing values, Estimation accuracy, Test information function, Individual ability, Unidimensionality, Local independence, Item Response Theory

Abstract

Objectives: The aim of this study was to compare the effectiveness of different methods for handling missing values in estimating test information accuracy and achieving unidimensionality and local independence within Item Response Theory. Five methods were evaluated: Listwise deletion (LW), treating missing responses as incorrect (NC), Multiple Imputation (MI), Expectation Maximization (EM), and response function (RF).

Methods: Real data with missing values were utilized from the International Mathematics Study (TIMSS) database of eighth-grade students in the academic year 2019. Data were fitted to One, Two, and Three Parameter logistic models, and the unidimensionality was assessed using Cronbach's alpha. Confirmatory factor analysis was conducted.

Results: The study found no significant difference in assumptions of unidimensionality and local independence across the methods for handling missing values. However, there were differences in the accuracy of estimating test information, favoring the Two Parameter logistic model and the RF method.

Conclusions: Any of the five methods (LW, NC, MI, EM, RF) can be adopted for handling missing values to achieve assumptions of unidimensionality and local independence. For accurate estimation of test information, the study recommends using the Two Parameter logistic model and the RF method.

Downloads

Download data is not yet available.

References

Al-zboon, H. & Alnasraween, M. (2021). The Effect of the Percentage of Missing Data on Estimating the Standard Error of the Items' Parameters and the Test Information Function According to the Three-Parameter Logistic Model in the Item Response Theory. Elementary Education Online, 20 (1), 887-898.

Baker, F. (2001). The basics of item response theory Clearinghouse on Assessment and Evaluation. Maryland, College Park.

Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.

Cokluk, O. & Kayri, M. (2011). The effect of Methods of Imputation for Missing Values on the Validity and Reliability of Scales. Educational Science: Theory and Practice, 11(1), 303-310.

De-Ayala, R., J. Plake, B., S. & Impara, J., C. (2006). The Impact of Omitted Responses on the Accuracy of Ability Estimation in Item Response Theory. Journal of Educational Measurement, 38(3), 213 – 234.

Embretson, S. & Reise, S. (2010). Item response theory for psychologists. (2nd rev.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Enders, C. K. (2010). Applied Missing Data Analysis. New York: Guilford Press.

Finch, H. (2008). Estimation of Item Response Theory Parameters in the Presence of Missing Data. Journal of Educational Measurement, 45(3), 225-245.

Gemici, S. Bednars, A. Lim, P. (2012). A Primer for Handling Missing Values in the Analysis of Education and Training Data. International Journal of Training Research, 10(3), 233-250

Goretzko, D. Heumann, C. & Bühner, M. (2020). Investigating Parallel Analysis in the Context of Missing Data: A Simulation Study Comparing Six Missing Data Methods. Educational psychological Measurement, 80(4), 756–774.

Hakstian, A. & Whalen, T. (1976). A k-samples significance test for independent alpha coefficients. Psychometrika,

(2), 219-231.

Hambelton, R. & Swaminthan, H. & Rogers, J. (1991). Fundamentals of Item Response Theory. Newbury Park California: Stage Publications.

Hambleton, R., & Swaminathan, H. (1985). Item Response Theory: principles and applications. Boston: Kluwer- Nijhoff Publishing.

Hattie, J. (1985). Methodology Review: Assessing Unidimensionality of Tests and Items. Applied psychological Measurement, 9(2), 139- 164.

Linn, R., L. (1989). Educational Measurement (third Ed.). New York: Macmillan

Kalkan, O. Kara, Y. & Kelecioglu. (2018). Evaluating Performance of Missing Data Imputation Methods in IRT Analyses. International Journal of Assessment Tools in Education, 5(3), 403 – 416.

Kim, S. Cohen, A. & Lin,Y. (2005). LDID: A Computer program for local dependence indices for dichotomous Items. Version 1.0.

Little, R., J. & Rubin, D., B. (2002). Statistical analysis with missing Data. Second edition. Hoboken, New Jersey: John Wiley.

Lord, F., M. (1980). Practical Applications of Item Characteristics Curve Theory. Journal of Educational Measurement. 14, 117- 138.

Reise, S. P. & Moore, T., M. (2023). Item response theory. APA handbook of research methods in psychology: Foundations, planning, measures, and psychometrics. American Psychological Association.

Shi, D. Lee, T. Fairchild, A. & Maydeu-Olivares, A. (2020). Fitting Ordinal Factor Analysis Models with Missing Data: A Comparison between Pairwise Deletion and Multiple Imputation. Educational and Psychological Measurement, 80(1), 41 – 66.

Xiao, J. Bulut, O. (2020). Evaluating the Performances of Missing Data Handling Methods in Ability Estimation from Sparse Data, Educational and Psychological Measurement, 80(5), 932-954.

Wu, T. Kim, Y. & Westine, C. (2023). Evaluating the Effects of Missing Data Handling Methods on Scale Linking Accuracy. Educational and Psychological Measurement, 83(6), 1202-1228.

Witta, E., L. (2000). Effectiveness of Four Methods of Handling Missing Data Using Samples from a National Database. American Educational Research Association, ERIC Document Reproduction Service No. ED442810, New Orleans, LA.

Yen, W. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145.