The comparison between repeated measures ANOVA and multilevel modeling to investigate the effectiveness of language instruction

Abstract

The purpose of the current study is to compare two different statistical analyses—repeated measures analysis of variance (ANOVA) and multilevel modeling (MLM)—with regard to international second language (L2) students’ lexico-grammatical writing gains in the interventions. Fifty college students’ lexico-grammatical writing gains in the pre-test, immediate and delayed post-tests were collected from three groups—one control and two intervention groups. Research questions include: (1) Is there a change in the dependent variable over time through repeated measures ANOVA?; (2) How much on average do the individuals change over time through MLM?; and (3) What are the differences of the results between repeated measures ANOVA and MLM? Repeated measures ANOVA and MLM were conducted separately to answer the research questions. Both analyses revealed that there was no difference of the interventions over time. In terms of methodological considerations, however, the two analyses exhibited the results in a different manner. Repeated measures ANOVA requires several assumptions (e.g., homoschedasticity and sphericity) while MLM does not need the requirement of the data as long as it contains different periodic datasets. Furthermore, MLM enables researchers to investigate the rate of individual growth. This paper offers methodological insight into adequate quantitative analyses to investigate the effectiveness of language acquisition and learning.