The Impact of Score Distribution Shape on National Test Results: Evaluating the Accuracy of Estimating Item Information Function and Test Reliability for Students' Scores

Abstract

Objectives: This study aimed to examine how the distribution of students' scores on national tests affects the accuracy of estimating the item information function and test reliability, utilizing the principles of Item Response Theory.

Methods: The study employed a descriptive comparative approach. A random sample of 12,000 male and female students from the eighth grade, who had taken the Arabic, English, Mathematics, and Science papers of The National Exam for Controlling the Quality of Education during the 2018/2019 academic year, was selected. Based on the skewness of their score distributions (positive, normal, negative), the students were divided into 12 groups, each containing 1,000 male and female students. The statistical program BILOG_MG3 was used to analyze the data, applying the Three-Parameter Model of Item Response Theory to extract the item information function and its standard error. Additionally, the theoretical and experimental reliability coefficients were determined based on the distribution of the data.

 Results: The findings revealed that scores with positive skewness had the highest mean standard error of the estimate for the item information function in national tests, and both the theoretical and empirical reliability of these tests were lower. Statistically significant differences were observed in the mean errors between data with a normal distribution and data with positive or negative skewness in the subjects of Arabic language, English language, and mathematics, while no significant differences were found in the Science subject.

Conclusions: Based on the study's results, it is recommended to consider the test information function as a criterion when designing national tests.