Statistical Methods Used In Educational Technology Research 2012-2013
DOI:
https://doi.org/10.14738/assrj.711.9146Abstract
This article provides a content analysis of the research methodologies used in quantitative and mixed-methods articles in the top five educational technology journals between 2012 and 2013. These articles represented a total of 32,131 sampling procedures and statistical techniques recorded from 1,171 articles – the largest research synthesis of research methodologies in field of educational technology to date. Results indicate quantitative methods continue to dominate the field as a whole, yet specific journals appear to favor certain research methods over others. Most authors did not report the type of sampling procedure used in their investigations (617 articles). Fewer researchers reported score reliability estimates using their own data – with only 420 articles reporting reliability coefficients. Findings also suggest few authors reported informationally-adequate statistics. Recommendations for best statistical practices and implications for the field of educational technology are discussed.
References
Alper, A. & Gülbahar, Y. (2009). Trends and issues in educational technologies: A review of recent research in TOJET. The Turkish Online Journal of Educational Technology, 8(2), 124-135.
American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35, 33-40.
American Psychological Association (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
APA Publications & Communications Board Working Group on Journal Article Reporting Standards. (2008). Reporting standards for research in psychology Why do we need them? what might they be? American Psychologist, 63, 839-851.
Bulfin, S., Henderson, M., Johnson, N. F., Selwyn, N. (2014). Methodological capacity within the field of “education technology” research: an initial investigation. British Journal of Educational Technology, 45, 403-414.
Chen, W. & Hirschheim, R. (2004). A paradigmatic and methodological examination of information systems research from 1991 to 2001. Information Systems Journal, 14, 197-235.
Cheung, W. S. & Hew, K. F. (2009). A review of research methodologies used in studies on mobile handheld devices in K-12 and higher education settings. Australasian Journal of Educational Technology, 25, 153-183.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. 44
Ritter, N. L. (2020). Statistical Methods Used In Educational Technology Research 2012-2013. Advances in Social Sciences Research Journal, 7(11) 31-46.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159
Cooper, H. (2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Thousand Oaks, CA: Sage.
Farhoomand, A. & Drury, D.H. (1999). A historiographical examination of information systems. Communications of the Association for Information Systems, 1(5es), 1-27.
Gall, M. D., Gall, J. P., & Brog, W. R. (2007). Educational research an introduction (8th ed.). Boston, MA: Pearson Education.
Gronlund, N. E., & Linn, R. L. (1990). Measurement and evaluation in teaching (6th ed.). New York: Macmillan.
Hogan, T. P., Benjamin, A., & Brezinski, K. L. (2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60, 523-531.
Hrastinski, S. & Keller, C. (2007). An examination of research approaches that underlie research on educational technology: A review from 2000 to 2004. Journal of Educational Computing Research, 36, 175-190.
Johnson, R. B., & Onwuegbuzie. A. J. (2004). Mixed-methods research: A research paradigm whose time has come. Educational Researcher, 33, 14-26.
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.
Koble, M. A & Bunker, E. L. (1997). Trends in research and practice: An examination of The American Journal of Distance Education 1987 to 1995. American Journal of Distance Education, 11, 19-38.
Lee, Y., Driscoll, M. P., & Nelson, D. W. (2004). The past, present, and future of research in distance education: Results of a content analysis. American Journal of Distance Education, 18, 225-241.
Lee, Y., Driscoll, M. P., & Nelson, D. W. (2007). Trends in research: A content analysis of major journals. In M. G. Moore (Ed.), Handbook of Distance Education (2nd ed., pp. 31-41). Mahwah, NJ: Lawrence Erlbaum Associates.
Lipsey, M. W. & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
Mitchell, P. D. (1997). The impact of educational technology: A radical reappraisal of research methods. Association for Learning Technology Journal, 5(1), 48-54. doi:10.1080/0968776970050108
Peterson-Karlan, G. P. (2011). Technology to support writing by students with learning and academic disabilities: Recent research trends and findings. Assistive Technology Outcomes and Benefits, 7, 39-62.
Randolph, J. J. (2008). A methodological review of the program evaluations in K-12 computer science education. Informatics in Education, 7, 237-258.
Randolph, J., Julnes, G., Sutinen, E., & Lehman, S. (2008). A methodological review of computer science education research. Journal of Information Technology Education, 7, 135-162.
Reeves, T.C. (2000, April). Enhancing the worth of instructional technology research through “design experiments” and other development research strategies. In Session 41.29, International perspectives on instructional technology research for the 21st century. Symposium conducted at the annual meeting of the American Educational Research Association, New Orleans, LA.
Ritter, N. L. (2010, February). Understanding a widely misunderstood statistic: Cronbach's alpha. Paper presented at the meeting of Southwestern Educational Research Association Conference 2010, New Orleans, LA (ED526237).
Rourke, L., & Szabo, M. A. (2002). A content analysis of the Journal of Distance Education 1986–2001. Journal of Distance Education, 17(1), 63-74.
Shih, M., Feng, J. & Tsai, C. C. (2008). Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals. Computers & Education, 51, 955-967. Şİmşek, A., Özdamar, N., Uysal, O., Kobak, K. Berk, C., Kiliçer, T., & Çİğdem, H. (2009). Current trends in educational technology research in Turkey in the new millennium. Educational Sciences: Theory and Practice, 9, 961-966.
Skidmore, S. T. & Thompson, B. (2010). Statistical techniques used in published articles: A historical review of reviews. Educational and Psychological Measurement, 70, 777-795. 45
Thompson, B. (1992). Two and one-half decades of leadership in measurement and evaluation. Journal of Counseling and Development, 70, 434-438.
Thompson, B. (1994). Guidelines for authors. Educational and Psychological Measurement, 54, 837-847.
Thompson, B. (Ed.) (2003). Score reliability: Contemporary thinking on reliability issues. Newbury Park, CA: Sage.
Thompson, B. (2004). Exploratory and confirmatory factor analysis. Washington, DC: American Psychological Association
Thompson, B. (2006). Foundations of behavioral statistics. New York, NY: Guilford Press.
Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6-20.
Wang, F. & Lockee, B. B. (2010). Virtual worlds in distance education a content analysis study. Quarterly Review of Distance Education, 11, 183-186.
Warne, R.T. Lazo, M., Ramos, T., & Ritter, N. (2012). Statistical methods used in gifted education journals, 2006-2010. Gifted Child Quarterly, 56, 134-149.
Willson, V. L. (1980). Research techniques in AERJ articles: 1969 to 1978. Educational Researcher, 9(5), 5-10.
Wilkinson & the Task Force on Statistical Inference, (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Nicola L. Ritter
This work is licensed under a Creative Commons Attribution 4.0 International License.
