Development of the Online Course Overload Indicator and the Student Mental Fatigue Survey

Keywords: mental fatigue, cognitive overload, online learning, online course design, student support, online course development

Abstract

The purpose of this study is to develop and examine the psychometric properties of the Online Course Overload Indicator (OCOI) and the Student Mental Fatigue Survey (SMFS). The OCOI was designed to measure students’ perceptions of cognitive overload in online courses. The SMFS was used to assess students’ perceptions of mental fatigue while taking online courses. An exploratory factor analysis was conducted on a sample of 378 undergraduate students from various institutions offering online courses across the United States. Results of a factor and reliability analyses confirmed that the instruments are valid and reliable measures of students’ perceived mental fatigue and overload from online course elements. The analysis supported the model that students’ perceptions of overload in online courses consist of four constructs—information relevance, information overload, course design, and facilitation—in addition to the one-factor structure of the SMFS, which consists of the student mental fatigue construct.

Author Biographies

Gail Alleyne Bayne, Texas Tech University

Gail Alleyne Bayne received an Ed.D. degree from Texas Tech University in Instructional Technology. Dr. Alleyne Bayne is currently an Adjunct Instructor at Texas Tech University teaching undergraduate and graduate synchronous and asynchronous courses. Her research interests are online learning and mental fatigue, cooperative and project-based learning in online environments.

Fethi A. Inan, Texas Tech University

Fethi A. Inan is a President's Excellence in Teaching Professor of Instructional Technology. He has a diverse background and experience in learning intelligent tutoring systems, instructional multimedia, and data analytics. Dr. Inan's research interests are online learning, individual differences, adaptive systems, and technology integration.

References

Ackerman, P. L. (2011). Cognitive fatigue: Multidisciplinary perspectives on current research and future applications. American Psychological Association. https://doi.org/10.1037/12343-000

Ackerman, P. L., Kanfer, R., Shapiro, S. W., Newton, S., & Beier, M. E. (2010). Cognitive fatigue during testing: An examination of trait, time-on-task, and strategy influences. Human Performance, 23(5), 381–402. https://doi.org/10.1080/08959285.2010.517720

Al Ma’mari, Q., Sharour, L. A., Al Omari, O. (2020). Fatigue, burnout, work environment, workload and perceived patient safety culture among critical care nurses. British Journal of Nursing, 29(1), 28–34. https://doi.org/10.12968/BJON.2020.29.1.28

Alleyne Bayne, G. (2016, March 25). An exploratory study of university students’ mental fatigue in online courses [Poster presentation]. Texas Tech Univeristy 15th Annual Graduate Student Research Poster Competition, Lubbock TX.

Ayres, P. (2006). Using subjective measures to detect variations of intrinsic cognitive load within problems. Learning and Instruction, 16(5), 389–400. https://doi.org/10.1016/j.learninstruc.2006.09.001

Baker, C. (2010). The impact of instructor immediacy and presence for online student affective learning, cognition, and motivation. Journal of Educators Online, 7(1), 1–30. https://doi.org/doi:10.9743/JEO.2010.1.2

Bakker, R. J., van de Putte, E. M., Kuis, W., & Sinnema, G. (2009). Risk factors for persistent fatigue with significant school absence in children and adolescents. Pediatrics, 124(1), e89–e95. https://doi.org/10.1542/peds.2008-1260

Balkin, T. J., & Wesensten, N. J. (2011). Differentiation of sleepiness and mental fatigue effects. In P. L. Ackerman (Ed.), Cognitive fatigue: Multidisciplinary perspectives on current research and future applications (pp. 47–66). American Psychological Association. https://doi.org/10.1037/12343-002

Barnard, L., & Paton, V. O. (2007, November). Distance learning survey of Texas Tech University’s fall 2006 distance and off-campus students. https://www.depts.ttu.edu/opa/docs/Barnard_Paton_2007_Distance_Learning_Survey_TTU_F2006.pdf

Bartlett, M. S. (1951). The effect of standardization on a χ2 approximation in factor analysis. Biometrika, 38(3/4), 337–344. https://doi.org/10.2307/2332580

Beiske, A. G., & Svensson, E. (2010). Fatigue in Parkinson’s disease: A short update. Acta Neurologica Scandinavica, 122(s190), 78–81. https://doi.org/10.1111/j.1600-0404.2010.01381.x

Boksem, M. A. S., Meijman, T. F., & Lorist, M. M. (2005). Effects of mental fatigue on attention: An ERP study. Cognitive Brain Research, 25(1), 107–116. https://doi.org/10.1016/j.cogbrainres.2005.04.011

Boksem, M. A. S., & Tops, M. (2008). Mental fatigue: Costs and benefits. Brain Research Reviews, 59(1), 125–139. https://doi.org/10.1016/j.brainresrev.2008.07.001

Bolliger, D. U., & Inan, F. A. (2012). Development and validation of the Online Student Connectedness Survey (OSCI). The International Review of Research in Open and Distributed Learning, 13(3), 41-65. https://doi.org/10.19173/irrodl.v13i3.1171

Carpenter, S. (2018). Ten steps in scale development and reporting: A guide for researchers. Communication Methods and Measures, 12(1), 25–44. https://doi.org/10.1080/19312458.2017.1396583

Chalder, T., Berelowitz, G., Pawlikowska, T., Watts, L., Wessely, S., Wright, D., & Wallace, E. P. (1993). Development of a fatigue scale. Journal of Psychosomatic Research, 37(2), 147–153. https://doi.org/10.1016/0022-3999(93)90081-P

Chen, C. Y., Pedersen, S., & Murphy, K. L. (2011). Learners’ perceived information overload in online learning via computer-mediated communication. Research in Learning Technology, 19(2), 101–116. https://doi.org/10.1080/21567069.2011.586678

Clark, R., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (4th ed.). John Wiley & Sons. https://doi.org/10.1002/9781119239086

Clark, R., Nguyen, F., & Sweller, J. (2006). Efficiency in learning: Evidence-based guidelines to manage cognitive load. John Wiley & Sons.

Comrey, A. L., & Lee, H. (1992). A first course in factor analysis (2nd ed.). Lawrence Erlbaum Associates. https://doi.org/10.4324/9781315827506

Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation, 10(7), 1–9. https://doi.org/10.7275/jyj1-4868

Csathó, Á., van der Linden, D., Hernádi, I., Buzás, P., & Kalmár, G. (2012). Effects of mental fatigue on the capacity limits of visual attention. Journal of Cognitive Psychology, 24(5), 511–524. https://doi.org/10.1080/20445911.2012.658039

DeLuca, J. (Ed.). (2005). Fatigue as a window to the brain. MIT press. https://doi.org/10.7551/mitpress/2967.001.0001

DeVellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Sage Publications.

Edwards, J. R., & Cooper, C. L. (2013). The person-environment fit approach to stress: Recurring problems and some suggested solutions. In C. L. Cooper (Ed.), From stress to wellbeing: Vol. 1. Theory and research on occupational stress and wellbeing (pp. 91–108). Palgrave Macmillan. https://doi.org/10.1057/9781137310651_5

Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Sage Publications. https://uk.sagepub.com/en-gb/eur/discovering-statistics-using-ibm-spss-statistics/book257672

Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6

Guo, P. J., Kim, J., & Rubin, R. (2014, March). How video production affects student engagement: An empirical study of MOOC videos. L@S ’14: Proceedings of the First ACM Conference on Learning @ Scale Conference (pp. 41–50). Association for Computing Machinery. https://doi.org/10.1145/2556325.2566239

Guramatunhu, P. (2015). The gender shift in enrollment patterns in higher education: A case study of a school administration program. Advancing Women in Leadership, 35, 120–133. https://doi.org/10.21423/awlj-v35.a137

Hafezi, S., Zare, H., Najafi Mehri, S., & Mahmoodi, H. (2010). The Multidimensional Fatigue Inventory validation and fatigue assessment in Iranian distance education students. Proceedings of the 4th International Conference on Distance Learning and Education (ICDLE) (pp. 195–198) Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICDLE.2010.5606006

Harris, S. M., Larrier, Y. I., & Castano-Bishop, M. (2011). Development of the Student Expectations of Online Learning Survey (SEOLS): A pilot study. Online Journal of Distance Learning Administration, 14(4). https://eric.ed.gov/?id=EJ960578

Herlambang, M. B., Taatgen, N. A., & Cnossen, F. (2019). The role of motivation as a factor in mental fatigue. Human Factors, 61(7), 1171–1185. https://doi.org/10.1177/0018720819828569

Hinkin, T. R., Tracey, J. B., & Enz, C. A. (1997). Scale construction: Developing reliable and valid measurement instruments. Journal of Hospitality and Tourism Research, 21(1), 100–120. https://doi.org/10.1177/109634809702100108

Hockey, R. (2013). The psychology of fatigue: Work, effort and control. Cambridge University Press. https://doi.org/10.1017/CBO9781139015394

Holmes, C. M., & Reid, C. (2017). A comparison study of on-campus and online learning outcomes for a research methods course. The Journal of Counselor Preparation and Supervision, 9(2), 1–24. https://doi.org/10.7729/92.1182

Jensen, J. L., Berry, D. A., & Kummer, T. A. (2013). Investigating the effects of exam length on performance and cognitive fatigue. PLoS One, 8(8), e70270. https://doi.org/10.1371/journal.pone.0070270

Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20(1), 141–151. https://doi.org/10.1177/001316446002000116

Keller, J. (1983). Motivational design of instruction. In C. M. Reigeluth (Ed.), Instructional design theories and models: An overview of their current status (pp. 383–434). Lawrence Erlbaum. https://doi.org/10.4324/9780203824283

Lambert, M., Sattler, S., & Paton, V. O. (2009, June). Distance learning survey of Texas Tech University’s fall 2008 distance and off-campus students. https://www.depts.ttu.edu/opa/docs/Lambert_Sattler_Paton_2009_Distance_Learning_Survey_TTU_F2008.pdf

Lee, A., Son, S. M., & Kim, K. K. (2016). Information and communication technology overload and social networking service fatigue: A stress perspective. Computers in Human Behavior, 55, 51–61. https://doi.org/10.1016/j.chb.2015.08.011

Matsunaga, M. (2010). How to factor-analyze your data right: Do’s, don’ts, and how-to’s. International Journal of Psychological Research, 3(1), 97–110. https://doi.org/10.21500/20112084.854

Mayadas, F., Miller, G. E., & Sener, J. (2015, July). Updated e-learning definitions. Online Learning Consortium. http://onlinelearningconsortium.org/updated-e-learning-definitions/

Mertler, C. A., & Vannatta, R. A. (2016). Advanced and multivariate statistical methods: Practical application and interpretation (6th ed.). Routledge. https://doi.org/10.4324/9781315266978

Miller, G. (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101(2), 343–352. https://doi.org/10.1037/0033-295x.101.2.343

Mizuno, K., Tanaka, M., Fukuda, S., Yamano, E., Shigihara, Y., Imai-Matsumura, K., & Watanabe, Y. (2011). Low visual information-processing speed and attention are predictors of fatigue in elementary and junior high school students. Behavioral and Brain Functions, 7(1), 1–7. https://doi.org/10.1186/1744-9081-7-20

Mota, D. D. C. F., & Pimenta, C. A. M. (2006). Self-report instruments for fatigue assessment: A systematic review. Research and Theory for Nursing Practice: An International Journal, 20(1), 49–78. https://doi.org/10.1891/rtnp.20.1.49

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.

O’connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32(3), 396–402. https:/doi.org/10.3758/BF03200807

O’Sullivan, P. B., Hunt, S. K., & Lippert, L. R. (2004). Mediated immediacy: A language of affiliation in a technological age. Journal of Language and Social Psychology, 23(4), 464–490. https://doi.org/10.1177/0261927X04269588

Palmer, L. K. (2013). The relationship between stress, fatigue, and cognitive functioning. College Student Journal, 47(2), 312–325. https://www.ingentaconnect.com/content/prin/csj/2013/00000047/00000002/art00007

Plass, J. L., Moreno, R., & Brunken, R. (Eds.). (2010). Cognitive load theory. Cambridge University Press. https://doi.org/10.1017/CBO9780511844744

Plukaard, S., Huizinga, M., Krabbendam, L., & Jolles, J. (2015). Cognitive flexibility in healthy students is affected by fatigue: An experimental study. Learning and Individual Differences, 38, 18–25. https://doi.org/10.1016/j.lindif.2015.01.003

Roach, V., & Lemasters, L. (2006). Satisfaction with online learning: A comparative descriptive study. Journal of Interactive Online Learning, 5(3), 317–332. https://files.eric.ed.gov/fulltext/ED447310.pdf

Roberson, R. (2013, September). Helping students find relevance. Psychology Teacher Network, 23(2), 18–20.

Roelle, J., Lehmkuhl, N., Beyer, M., & Berthold, K. (2015). The role of specificity, targeted learning activities, and prior knowledge for the effects of relevance instructions. Journal of Educational Psychology, 107(3), 705–723. https://doi.org/10.1037/edu0000010

Sarkar, S., & Parnin, C. (2017). Characterizing and predicting mental fatigue during programming tasks. Proceedings: 2017 IEEE/ACM 2nd International Workshop on Emotion Awareness in Software Engineering (pp. 32–37). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/SEmotion.2017.2

Slavin, R. E. (2007). Educational research in an age of accountability. Allyn & Bacon.

Sweller, J. (2008). Human cognitive architecture. In J. M. Spector, M. D. Merrill, J. Van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 369–381). Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203880869

Sweller, J., Van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296. https://doi.org/10.1023/A:1022193728205

Tugtekin, U. (2022). Development and validation of an instrument for online learning fatigue in higher education. In G. Durak & S. Çankaya (Eds.), Handbook of research on managing and designing online courses in synchronous and asychronous environments (pp. 566–586). IGI Global. https://doi.org/10.4018/978-1-7998-8701-0.ch028

van der Linden, D. (2011). The urge to stop: The cognitive and biological nature of acute mental fatigue. In P. L. Ackerman (Ed.), Cognitive fatigue: Multidisciplinary perspectives on current research and future applications. (pp. 149–164). American Psychological Association. https://doi.org/10.1037/12343-007

Vercoulen, J. H. M. M., Swanink, C. M., Fennis, J. F., Galama, J. M., van der Meer, J. W., & Bleijenberg, G. (1994). Dimensional assessment of chronic fatigue syndrome. Journal of Psychosomatic Research, 38(5), 383–392. https://doi.org/10.1016/0022-3999(94)90099-x

Wanstreet, C. E. (2006). Interaction in online learning environments: A review of the literature. Quarterly Review of Distance Education, 7(4), 399–411. https://www.learntechlib.org/p/106711/

Wilcox, R. R. (2013). Introduction to robust estimation and hypothesis testing (3rd ed.). Elsevier. https://doi.org/10.1016/C2010-0-67044-1

Worthington, R., & Whittaker, T. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127

Published
2022-11-01
How to Cite
Alleyne Bayne, G., & Inan, F. A. (2022). Development of the Online Course Overload Indicator and the Student Mental Fatigue Survey. The International Review of Research in Open and Distributed Learning, 23(4), 75-92. https://doi.org/10.19173/irrodl.v23i4.6223
Section
Research Articles