Examining the STEM-Science Achievement Test (SSAT) Using Rasch Dichotomous Measurement Model

Authors

  • Junainah Jamaludin1 , Lay Yoon Fah2,*, Khoo Chwee Hoon3 , Leong Suk Yee4

Abstract

The main purpose of this study is to develop a valid and reliable instrument for measuring the
STEM-Science Achievement Test (SSAT) of primary school students in Malaysia. The instrument was
develop by researcher with the integration of STEM elements and by referring to the Malaysia National
Curriculum Standard. The development focused on six topics in science Year 4 (Y4) primary syllabus;
Scientific Skills, Life Processes of Human, Properties of Materials, Measurement, Solar System, and
Importance of Technologies in Life, and six topics in science Year 5 (Y5) primary syllabus; Rules and
Regulation in Science Lab, Life Processes of Plants, Acid and Alkali, Electricity, Earth and Space Science,
and Technology and Sustainable Life. There are 226 of Year 4 and 226 of Year 5 primary school students in
Sabah responded to the instrument developed to test their STEM-Science knowledge. Rasch Dichotomous
Measurement Model approach was used to evaluate the validity and reliability of the SSAT. The validity
(item polarity (PTMEA-CORR), Principal Component Analysis of Residuals (PCAR), Mean Squared
(MNSQ) infit and outfit) and the reliability (Cronbach's alpha, item reliability and item separation) were
being assessed. The results Rasch Dichotomous Measurement Model analysis show that all scales were
unidimensional for objective and subjective items. For objective items, the Cronbach’s Alpha is .81 (Y4)
and .83 (Y5), Item Reliability is .95 (Y4) and .95 (Y5) whereas Item Separation is 4.21 (Y4) and 4.25 (Y5).
For item validity, PTMEA CORR were found positive varies between; .03 to .44 (Y4) and .05 to .47 (Y5)
after item deletion has been made. Principal Component Analysis of Residuals (PCAR) showed that raw
variance explained by measures in Eigenvalue unit is 9.8 (Y4) and 10.7 (Y5), variance unexplained is 60.0
(Y4) and 58.0 (Y5) and variance unexplained in Contrast 1 is 5.1 (Y4) and 3.7 (Y5) respectively.
Standardised Residual Correlation for Year 4 and Year 5 objective items showed satisfactory value for none
of item correlation exceed control level of .7. There are 5 items misfit in year 4 and 12 item misfit in year 5
objectives test and items need to be revised. For subjective items, the Cronbach’s Alpha is .78 (Y4) and .84
(Y5), item reliability is .91 (Y4) and .93 (Y5) whereas item separation is 3.27 (Y4) and 3.56 (Y5). For item
validity, PTMEA CORR were found positive varies between .00 to .50 (Y4) and .01 to .54 (Y5) with no
item deletion. Principal Component Analysis of Residuals (PCAR) showed that raw variance explained by
measures in Eigenvalue unit is 22.9 (Y4) and 14.1 (Y5), variance unexplained is 96 (Y4) and 70 (Y5) and
variance unexplained in Contrast 1 is 4.5 (Y4) and 3.0 (Y5) respectively. Standardised residual correlation
for Year 4 and Year 5 subjective items showed satisfactory value for none of item standardised residual
correlation exceed control level of .7. The assessment of SSAT has shown that the instrument is a valid and
reliable to measure Malaysian primary students’ knowledge in STEM-Science.

Published

2020-01-31

Issue

Section

Articles