Abstract

Sentiments expressed on massive open online course (MOOC) discussion forums significantly influence learning effectiveness and academic performance. The evolution of learning sentiments on MOOC discussion forums is a dynamic process; however, a gap exists in the current understanding of the interplay between evolving sentiments and their impact on MOOC efficacy. Consequently, to enhance MOOC effectiveness further empirical research is needed to uncover the underlying patterns and temporal dynamics of learning sentiments. This study collected online discussions from 158 MOOC participants and examined the discussions using epistemic network analysis to identify how learning sentiment patterns differed according to performance level and learning topics. The results showed that learning sentiment patterns were affected by both performance level and learning topics, with participants in the high-score group exhibiting stronger associations between engagement-neutral and neutral-frustration, and fewer connections between frustration-delight and frustration-boredom when compared to those in the low-score group. In addition, this study found that engagement was strongly linked to all learning topics in the high-score group, whereas for the low-score group, only engagement and experience showed strong connections. Based on these findings, we discuss the implications for learners and instructors in paving the way for the development of targeted interventions and instructional strategies tailored to optimize MOOC effectiveness.

Keywords: learning sentiments, epistemic network analysis, MOOC discussion forum, learning topics, MOOC effectiveness

Introduction

In recent years, Massive Open Online Courses (MOOCs) have gained popularity as an effective online learning model due to their customized services, real-time feedback and flexible learning. Notably, discussion forums in MOOCs can provide an interactive environment for students (Wei et al., 2024). Several studies have shown that MOOC forums generate a significant volume of discussion data containing comprehensive records of behaviors during the MOOC learning process, which provides an opportunity to explore more deeply the sentiment patterns on MOOC forums (Ye & Zhou, 2022).

There has been increased emphasis on exploring learning sentiments in online learning. Positive sentiments (e.g., flow, delight) may be modulators of high-quality interactions and knowledge construction, whereas negative sentiments (e.g., boredom, frustration) could detrimentally impact interactions during the learning process (Shao et al., 2023). Furthermore, current research has shown that learning sentiments differ based on the occasion (Harley et al., 2015) and topic (Tan & Jung, 2024). Several studies have also emphasized that sentiments undergo dynamic changes during the learning process, particularly when learners encounter cognitive disequilibrium and equilibrium (D’Mello & Graesser, 2012). In the online learning context, learning sentiments become more complex because the non-face-to-face learning environment affects the emotional atmosphere (Huang et al., 2021; Shao et al., 2023). Therefore, it is important to further understand the developmental trajectory of learning sentiments in MOOC discussion forums.

Previous research has demonstrated that sentiments significantly influence learners’ academic performance in MOOC discussion forums and are therefore critical for assessing the effectiveness of MOOCs (Ye & Zhou, 2022). Some researchers have investigated the relationship between academic performance and sentiments (Parker et al., 2021; Shao et al., 2023). Specifically, King et al. (2015) focused on the sentiments of boredom, anxiety, and enjoyment, as these can significantly influence learners’ academic performance. High-performing learners who experienced higher levels of positive sentiments were more engaged and exhibited lower levels of disaffection. In contrast, low-performing learners typically experienced more negative sentiments (King et al., 2015). Overall, it is widely recognized that different academic performance groups exhibit varying learning sentiments. However, specific differences in learning sentiment patterns between these groups within the context of MOOC discussion forums remain largely unexplored. Therefore, this study applied epistemic network analysis (ENA; Lund et al., 2017) to identify how learning sentiment patterns differed between high- and low-score groups in MOOC discussion forums.

Literature Review

Sentiment Analysis in Online Discussions

Recently, research interest in exploring learning sentiments within online learning discussions has displayed a notable surge (Ye & Zhou, 2022; Huang et al., 2019). The terms sentiment and emotion are frequently employed interchangeably. This primarily stems from the fact that both emotions and sentiments are influenced by a range of components that encompass cognitive, motivational, affective, physiological, and expressive elements (Pekrun & Marsh, 2022). However, in the learning context, sentiments diverge from emotions in terms of the duration that learners experience them. Sentiments arise and persist, whereas emotions typically endure for relatively short periods, generally ranging from a few seconds to a few minutes (Huang et al., 2019). In online learning context, learning sentiments reflect the attitude or perspectives on things when interacting with others (Ye & Zhou, 2022). This study specifically focused on feelings directly related to learning tasks in MOOCs, encompassing learners’ feelings about comprehending course content, perception of challenges in exercises, and feelings of collaboration with peers. Flow theory was chosen to support the analysis of learning sentiments in this study (Kiuru et al., 2022).

Specific studies have examined the sentiment categories prevalent in online learning contexts, leading to the observation that various sentiments may manifest during online learning discussions (Tan & Jung, 2024). In the field of positive sentiment research, D’Mello and Graesser (2012) have described engagement as a cognitive state characterized by intense focus, concentration, and full immersion in a given task. Avry et al. (2020) determined that the attainment of one’s goals and the successful resolution of problems can lead to feelings of delight. A substantial literature has consistently indicated a positive correlation between feelings of delight and academic performance (Liu et al., 2022). Surprise might be elicited by cognitive incongruity caused by the disconnect between incoming information and prior knowledge (Yang et al., 2024). Lehman et al. (2012) found that confusion arose when learners encountered information that conflicted with their existing knowledge, leaving them uncertain about the way forward. Similarly, D’Mello et al. (2014) posited that confusion was likely to manifest when newly acquired information resisted integration into pre-existing mental models, and revealed a positive correlation between confusion and academic performance. Conversely, another study reported a negative association between confusion and academic performance (Richey et al., 2019). Frustration was a common sentiment among learners participating in collaborative online learning environments (Yang et al., 2021). Peterson and Zengilowski (2024) found that when learners failed to resolve the uncertainty associated with confusion, they may feel frustrated. Boredom ensues when an individual perceives an inherent lack of meaning within an activity (Beymer & Schmidt, 2023). Previous research has provided evidence of associations between suboptimal learning outcomes and both boredom and frustration (Baker et al., 2010). Gasper and Danube (2016) posited that neutral states often manifested in the absence or minimal presence of both positive and negative sentiments. Arguel et al. (2019) highlighted the possibility of neutral sentiment during the knowledge construction process. Hence, this study focused on seven distinct sentiment states: engagement, delight, surprise, boredom, confusion, frustration, and neutral. These sentiments have been recognized as among the most common learning-centered sentiments and have the potential to predict academic performance (Zheng & Huang, 2016).

A growing research stream has recently explored the evolving nature of learning sentiments ( Huang et al., 2021; Rebolledo-Mendez et al., 2021). Lehman et al. (2012) introduced a conceptual framework to elucidate the dynamics of sentiment states, including engagement, frustration, boredom, and confusion. This model affirmed that learners in a state of engagement may encounter cognitive disequilibrium and confusion when faced with obstacles to their goals or other challenges. Rebolledo-Mendez et al. (2021) investigated the temporal dynamics of sentiments, including confusion, boredom, neutral, frustration, and engagement, and found that learners with limited sentiment regulation abilities frequently transitioned from boredom to frustration and from engagement to neutral. Overall, these findings indicated that sentiments undergo temporal changes concurrent with the evolution of cognitive processes during learning. Hence, this study aimed to uncover the underlying patterns and temporal dynamics of learning sentiments in MOOC discussion forums.

Group Differences in Learning Sentiments

Numerous previous studies have objectively noted differences in sentiments among various groups (Ye & Zhou, 2022). For example, Ye and Zhou (2022) indicated that the most significant difference between higher- and lower- performing groups was positive sentiments, suggesting that positive sentiments can promote learners’ behavioral interactions. The higher- performing group had stronger associations around positive and confused sentiments; lower-performing groups had stronger associations around off-topic discussion. Han et al. (2021) proposed patterns of sentiments for four categories of learners, divided by social interactions type (i.e., posts, views, replies, votes) in MOOC forums. Learners with persistent interactions with various sentiments showed significantly higher frequencies of sentiments than did other learners. Huang et al. (2024) investigated emotion sequence patterns in the posts of MOOC discussion forums and revealed that learners in the low-level interaction group experienced more emotion transition from boredom to frustration than did the average- and high- level interaction group. Huang et al. (2021) examined the evolution of sentiments across three interaction levels in blended learning, namely surface, deep, and social-emotional. Their results indicated that during deep interactions, learning sentiments could evolve from negative to insightful. In contrast, the sentiment network derived from social-emotional interactions showed stronger connections in joking-positive and joking-negative sentiments compared to the other two interaction levels. Overall, previous studies have provided evidence of group differences in learning sentiment. However, existing research has typically focused on one or only a few aspects. The type and inner structure of learning sentiment differences in regard to performance remain unclear and warrant further investigation.

Epistemic Network Analysis

ENA combines traditional qualitative and quantitative methodologies with contemporary computational techniques and data analytics. This integration enables researchers to extract deep insights from their datasets (Andrist et al., 2018). ENA is a data analysis approach that focuses on reducing dimensionality and modeling connections among concepts within coded data. ENA leverages cognition, communication, action, and other pertinent aspects of group interaction and systematically characterizes them using suitable coding schemes, aligning with established practices in content analysis (Alonso-Nuez et al., 2020). An epistemic network is constructed by assigning codes to different elements present in online discussions, in which each node in the network represents these codes. The connections between nodes are determined by the occurrence of the codes within a pertinent unit of analysis, such as an individual discussion message or message sequence. Thus, each concept or significant feature within a dataset is depicted as a distinct node within the network. When a specific feature is identified within a data segment, the corresponding data are coded accordingly. ENA uses coded data to construct ENA networks by examining the co-occurrence of codes within a dataset. This process involves quantifying the co-occurrences to formulate weighted network models. In these models, the thickness of the edges (representing connections between nodes), size of nodes corresponding to specific codes, and spatial arrangement of nodes relative to one another collectively provide valuable insights into the dataset. This is understood through a systematic examination of codes within defined time windows, followed by the assignment of weights to their co-occurrences (Lund et al., 2017). Weight-based structuring, accomplished through dimensionality reduction, leads to a visualization focused on the selected variables of interest, enabling insightful comparisons.

Research Questions

Our aim was to expand the coding frameworks for learning sentiments established by previous researchers and provide additional evidence of the effectiveness of ENA in the analysis of MOOC discussion forums. In this study, learning sentiments expressed on MOOC discussion forums were conceptualized as a network comprising seven distinct dimensions. ENA was used to investigate the interrelationships among these dimensions and compare the salient properties of the epistemic networks generated by different participant groups. The primary research questions addressed in this study were as follows:

RQ1: What is the frequency distribution of participants’ sentiments on MOOC discussion forums?

RQ2: What are the time-series characteristics of participants’ sentiments on MOOC discussion forums?

RQ3: What distinctions exist in the epistemic network characteristics of sentiments between participants in high- and low-score groups?

RQ4: What distinctions exist in the epistemic network characteristics of sentiments between high- and low-score groups concerning different learning topics?

Methodology

Research Design and Participants

The study sample consisted of 158 learners who had registered for an online course titled Applications of Mind Maps in Teaching, offered on the Chinese MOOC University platform, one of the largest online learning communities in China. The course’s primary objective was to enhance learners’ proficiency in using mind maps as a pedagogical strategy. The course was accessible to the public and was not affiliated with any college or university curriculum.

During each week of the course, participants typically spent approximately three to six hours engaging with course materials. These activities included reading course materials, viewing instructional videos, participating in discussion forums, and completing unit quizzes. Notably, the MOOC did not operate entirely on a self-paced basis; each unit was made available at the start of the scheduled week. The course covered six learning topics related to mind maps, as shown in Table 1.

Table 1

Learning Topics in MOOC Online Discussions

Data Collection

We collected data from three distinct categories: demographic characteristics, performance data, and online discussions. The demographic data encompassed information such as the participants’ IDs, names, genders, majors, and regions. Participant performance was based on scores on tests, assignments, online discussions, and the final examination (worth 35%, 20%, 15% and 30% of the final grade, respectively). Subsequently, based on their final performance scores, learners who scored above the average were placed in the high-score group, while those who scored below the average were placed in the low-score group.

Participants’ online discussions served as reflections of their learning sentiments. The online discussion messages were organized chronologically and stored in Excel to facilitate subsequent coding and analysis processes and to reveal the underlying learning sentiment patterns.

Learning Sentiment Coding Scheme

Before conducting ENA, it was necessary to convert participants’ qualitative discussion message data from the MOOC forums into quantitative data. Drawing on an extensive literature review, we aimed to investigate variations in learning sentiments in online discussions. To accomplish this, seven sentiment categories closely linked to the learning process were identified: engagement, delight, confusion, frustration, boredom, surprise, and neutral. The coding scheme is presented in Table 2. Previous research has indicated that these categories are sufficient for distinguishing learning sentiments in online discussions (Zheng & Huang, 2016). In general, each message was assigned a single label. In cases where a message exhibited multiple sentiments, multiple labels were appended accordingly.

Table 2

Coding Scheme for Learning Sentiments in Online Discussions

By employing these coding schemes for learning sentiments, 1,316 online discussion messages encompassing six learning topics were systematically coded by two proficient raters. The coders possessed expertise in both sentiment framework and content analysis for encoding data from online discussions. First, a test set comprising 400 messages, representing approximately 30% of the complete dataset, was drawn from the MOOC discussion forum. This test set was employed to assess coding consistency between the two raters. The inter-coder reliability coefficient was computed, yielding a value of 0.85 (Cohen’s kappa), signifying a robust level of reliability. The two raters subsequently negotiated to reconcile discrepancies and enhance their understanding of the coding scheme. The remaining messages were then evenly distributed and independently coded by both raters. Ultimately, this process yielded 1,349 codes representing learning sentiments.

Data Analysis Methods

We conducted a comprehensive three-stage analysis of the three types of data, namely demographics, performance, and online discussion. The initial stage aimed to investigate the categories and frequency distributions of the learning sentiments by statistical analysis. In the second stage, time series analysis was carried out to examine the time-series characteristics of learning sentiments by using the ggplot data visualization package in R software. Based on the results of qualitative content analysis, the third and fourth stages compared learning sentiments across varying performance levels and topics using the ENA Web Tool (https://www.epistemicnetwork.org/).

To address the third and fourth research questions, seven learning sentiments from the coding scheme were designated as codes and the stanza size was fixed at four. To compare the differences between two groups, we analyzed the locations of projected points in the ENA. Normality checks suggested that the distributions of projected points were nonnormal. Then, a two-sample Mann-Whitney U test was used to compare differences in X- and Y-axis values between groups. We interpreted the meaning of any variance by computing the mean networks, which involved averaging the connection weights across the networks within each group. Additionally, we compared the means and individual networks using network difference graphs. These graphs were derived by subtracting the weight of each connection in one network from the corresponding connections in another network.

Results

Frequency Distribution of Participants’ Sentiments on MOOC Discussion Forums

Table 3 shows the frequency distribution of the participants’ learning sentiments. All seven learning sentiments appeared in the participants’ online discussions, albeit with different proportions. Engagement occurred most frequently (EN: 508, 37.7%), followed by frustration (FR: 284, 21.1%). Boredom (BO: 11, 0.8%) and surprise (SU: 5, 0.4%) had the lowest values.

Table 3

Descriptive Statistics for Learning Sentiments

Time-Series Characteristics of Participants’ Sentiments on MOOC Discussion Forums

We employed time as the horizontal axis and the number of learning sentiments as the vertical axis to construct a time-series diagram, as illustrated in Figure 1. Figure 1 shows the evolving dynamics of sentiments throughout the learning process over time. Notably, the density of learning sentiments varied at different time points. The seven types of learning sentiments alternated in appearance; however, their distributions exhibited variations. Engagement and delight consistently emerged as the most prevalent sentiments, which persisted throughout the learning process. In contrast, boredom and surprise were infrequently observed throughout the duration of the course.

Figure 1

Time-Series Characteristics of Learning Sentiments

Distinctions in the Epistemic Network Characteristics of Sentiments Between Participants in the High- and Low-Score Groups

We then performed ENA to identify the learning sentiment patterns within the high- and low-score groups. Figure 2 shows the epistemic networks of participants in the high-score group (left, marked in red) and the low-score group (right, marked in blue) in an online discussion; a difference network graph (center) indicates how the learning sentiments of each group differed. In this comparison plot, the connection weights were compared between the two groups. The thicker lines indicate stronger connections. Regarding the differences in epistemic connections, participants in the high-score group exhibited stronger connections between EN and NE, as well as between EN and FR. Conversely, the low-score group displayed stronger connections between FR and DE.

Figure 2

Mean Discourse Networks of Learning Sentiments on MOOC Discussion Forums

The mean value for the high-score group was positioned towards the left side of the ENA space, as illustrated in Figure 3. Conversely, the low-score group was located on the right side of the ENA space. Based on the distribution of learning sentiments within the ENA space, the high-score group expressed EN and NE more frequently, while the low-score group expressed DE and FR more frequently.

Figure 3

Comparison of High-Score (Red) and Low-Score (Blue) Groups

We also conducted two-sample Mann-Whitney U test to assess the potential differences in ENA characteristics between the high- and low-score groups. Table 4 presents the results. In terms of the first dimension (X-axis), the test assuming unequal variances revealed a statistically significant difference between the high- and low-score groups at alpha = 0.05 level. However, along the second dimension (Y-axis), the test assuming unequal variances indicated that the high-score group was not significantly different from the low-score group at an alpha level of 0.05.

Table 4

Results of the T-test for ENA Characteristics Between High- and Low-Score Groups

Note. ^⁎ p < 0.05.

Distinctions in the Epistemic Network Characteristics of Sentiments Between High- and Low-Score Groups Concerning Different Learning Topics

Figure 4(a) displays the group averages for both the high-score (red) and low-score (blue) groups, illustrating the relationship between learning sentiments and learning topics. The visualization employed X and Y, which collectively accounted for 8.6% and 10.6% of the variability in the epistemic networks established by the participants, respectively. The circles represent the high-score group in red and the low-score group in blue. Rectangles denote group-averaged networks, with each encircled by lines indicating 95% confidence intervals.

Figure 4(b) reveals a significant difference between the groups along the X-axis (t = -8.32; p = 0.00; r = 0.95), where the effect size (r = 0.95) is notably high. The results indicated that the learning topics were predominantly situated at the center of the network, except for the learning topics of experience and strategy, which appeared to hold singular importance in the course and were primarily captured along the Y-axis. Furthermore, Figure 4(b) indicates that the high-score group displayed more connections with EN, whereas the low-score group tended to contribute more online discussions linked to DE.

Figure 4

Comparison of High-Score (Red) and Low-Score (Blue) Groups Regarding Learning Topics

Table 5 shows the connection coefficients of the epistemic networks within both the high- and low-score groups. The values in Table 5 represent the frequency-weighted occurrence of each connection in the online discussion. The results revealed several significant connections between learning sentiments, such as the link between FR and EN. In addition, the conditions and strategies of the learning topics had significant connections. However, focusing on the relationship between the learning topic and sentiments, EN showed strong links to all learning topics, in which the values were greater than 0.15 in the high-score group, whereas for the low-score group, only EN and experience showed strong connections, with values greater than 0.15. In addition, FR and experience showed significant connections for the high- and low-score groups, with values reaching 0.17.

Table 5

ENA Network Weights for Learning Sentiments

Discussion

On MOOC discussion forums, EN (508 discourses, 37.7%) and DE (230 discourses, 17.0%) were the most common positive sentiments, while BO (11 discourses, 0.8%) was the least common negative sentiment. Numerous studies have shown that positive sentimental states such as engagement and delight manifest when a learner’s existing knowledge aligns with new information being acquired or when learning tasks have been accomplished (Tan & Jung, 2024). In contrast, negative sentiments such as boredom and frustration were likely to arise when individuals encountered impediments to their learning objectives (Camacho-Morles et al., 2021). In this study, frustration significantly outweighed boredom. This underscored the importance of instructors meticulously tracing the underlying causes of emerging frustration. Lehman et al. (2012) demonstrated that as frustration escalates, learners may encounter increasing challenges in devising new strategies to achieve their goals. Frustration could happen if a subject one studies is highly complex even though one has studied or prepared for it. These findings are consistent with flow theory, which suggests that highly challenging situations that surpass an individual’s current skill level can evoke feelings of anxiety and frustration (Wei et al., 2024). Moreover, confusion emerged as the most prevalent sentiment, which aligned with the findings of D’Mello et al. (2014), who suggested that confusion may be conducive to complex learning and deep cognitive processing. Although positive sentiments were common in this study, it is crucial to address it promptly once it is detected. Failure to do so may lead to negative learning outcomes, potentially culminating in frustration for learners.

In addition, we found that the seven types of learning sentiments appeared alternately in the online discourse, and the density of learning sentiments varied at different times. This finding was consistent with prior research that argued learning sentiments displayed periodic and dynamic features during the online learning process (Huang et al., 2019). Such research offered evidence that, when engaging in intricate MOOC learning tasks or assignments, learners’ sentiments were subject to dynamic fluctuations influenced by their goals and knowledge levels (Ye & Zhou, 2022). This highlighted the importance of instructors tracing the underlying causes of learning sentiment dynamics. For example, D’Mello and Graesser (2012) developed a model to elucidate the dynamics of affective states that arose during deep learning activities to address pedagogical and motivational strategies.

Moreover, the results indicated that the high-score group engaged in more discourse characterized by engagement and neutral sentiment while displaying fewer instances of delight and frustration compared to the low-score group. These findings aligned with Yang et al. (2024), who highlighted the notable correlation between learners’ academic performance and the occurrence of positive sentiments. Learning sentiment has also been identified as a significant predictor of academic performance (Xing et al., 2019). Regarding the learning sentiment patterns, we found that the high-score group exhibited stronger connections between engagement and neutral, as well as between engagement and frustration. However, the low-score group experienced more frustration, followed by delight, than did the high-score group. If a learner experiences chronic frustration, they may be operating at the limits of their current abilities, often associated with learning within the zone of proximal development. Consequently, endeavoring to identify and address such frustrations, with a special focus on offering additional guidance to learners, is imperative. This intervention aimed to break the cycle in which learners became bored and remained bored for prolonged periods. However, frustrations are not always negative; they can direct learners to be grittier when faced with frustrations in learning.

As an additional significant contribution, our results offered insights into the association between learning topics and sentiments for the high- and low-score groups. These results indicated a difference between the learning topics and learning sentiments for distinct groups. Our research confirmed that participants in the high-score group had more links between all other learning topics and evaluations than did their peers in the low-score group. This finding aligned with the conclusion of Alonso-Nuez et al. (2020) who showed that learners who engaged in more evaluation activities exhibited stronger academic performance. Additionally, our findings suggested that engagement was strongly linked to all learning topics (i.e., theory, condition, experience, strategy, application, and evaluation). This was consistent with the conclusions of Huang et al. (2019), who discovered that a learning task or topic could evoke several diverse positive sentiments (engagement) at the beginning of the learning process. In addition, frustration and experience had significant connections in the high- and low-score groups. We posit this phenomenon resulted from the features of the learning topic of experience. Most participants always experienced problems that could not be solved in time when operating the mind-mapping software, which often led to frustration.

This study has significantly advanced our understanding of learning sentiments in educational contexts, clarifying the dynamic nature of sentimental changes across various groups and tasks during MOOC learning. By incorporating both learning sentiments and topics into our analysis, we enhanced our ability to decipher the evolving connections among shifting sentiments. This research provided valuable insights for instructional designers and educators, providing them with effective pedagogical strategies to facilitate positive sentimental transitions and effective sentimental regulation in MOOCs. For example, instructional designers and educators can design teaching strategies and evaluate the complexity of learning tasks carefully to motivate learners to invest effort and achieve success. This finding aligned with the conclusion of (Peterson & Zengilowski, 2024); pedagogical strategies such as providing learners with challenges can support optimal positive sentimental when the challenge is appropriate or may result in sentiments like frustration when the challenge is too great or lacks support.

Conclusion and Limitations

Contemporary education researchers have increasingly highlighted the essential role of learning sentiments in online learning communities. However, further research is needed in the educational field regarding learning sentiments. Compared with traditional conceptualizations, learning analytics, particularly ENA, have the potential to explore learning sentiment dimensions through unconventional approaches. This study gathered online discussions from 158 participants and analyzed them using ENA to determine how individual learning sentiment patterns varied based on performance level and learning topics. Our research has marked a transition from the static paradigm of conceptualizing sentiments to analyzing dynamic sentimental processes. This novel approach can reveal underlying patterns and temporal dynamics of learning sentiments, thereby enhancing MOOC effectiveness.

This study had several limitations that merit acknowledgment. First, the assessment of online discussions should be automated to enable differentiation between distinct categories within learning sentiment dimensions. Guidelines for automated analysis of online discourse offer valuable insights. Second, it is conceivable that employing different coding schemes or analyzing a separate dataset to code learning sentiments may yield contrasting results. Future research should investigate learning sentiment coding patterns across various contexts and participant groups. Third, dichotomization of the final scores may have introduced statistical errors that could affect the rigor of the conclusions. One avenue for future research would involve exploring more effective methods for group differentiation. Finally, the study relied on forum discussions data only to detect the sentimental states, which could have potentially led to inaccuracies in the results. Future research should employ mixed methods to collect data and explore sentimental patterns.

Funding

This work was supported by National Natural Science Foundation of China (Grant Number: 62407041), Scientific Research Fund of Zhejiang Provincial Education Department (Grant Number: Y202250445) and General Topic of Zhejiang Provincial Education Science Planning/Research on Value-added Evaluation of Classroom Learning Effectiveness Empowered by Intelligent Technology (Grant Number: 2023SCG392).

References

Alonso-Nuez, M. J., Gil-Lacruz, A. I., & Rosell-Martínez, J. (2020). Assessing evaluation: Why student engages or resists to active learning? International Journal of Technology and Design Education, 31(5), 1001-1017. https://doi.org/10.1007/s10798-020-09582-1

Andrist, S., Ruis, A. R., & Shaffer, D. W. (2018). A network analytic approach to gaze coordination during a collaborative task. Computers in Human Behavior, 89, 339-348. https://doi.org/10.1016/j.chb.2018.07.017

Arguel, A., Lockyer, L., Chai, K., Pachman, M., & Lipp, O. V. (2019). Puzzle-solving activity as an indicator of epistemic confusion. Frontiers in Psychology, 10, 163. https://doi.org/10.3389/fpsyg.2019.00163

Avry, S., Chanel, G., Bétrancourt, M., & Molinari, G. (2020). Achievement appraisals, emotions and socio-cognitive processes: How they interplay in collaborative problem-solving? Computers in Human Behavior, 107, 106267. https://doi.org/10.1016/j.chb.2020.106267

Baker, R. S. J. D., D’Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4), 223-241. https://doi.org/10.1016/j.ijhcs.2009.12.003

Beymer, P. N., & Schmidt, J. A. (2023). Can you hear it? Toward conceptual clarity of emotional cost and negative emotions. Contemporary Educational Psychology, 74, 102198. https://doi.org/10.1016/j.cedpsych.2023.102198

Camacho-Morles, J., Slemp, G. R., Pekrun, R., Loderer, K., Hou, H., & Oades, L. G. (2021). Activity achievement emotions and academic performance: A meta-analysis. Educational Psychology Review, 33(3), 1051-1095. https://doi.org/10.1007/s10648-020-09585-3

D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145-157. https://doi.org/10.1016/j.learninstruc.2011.10.001

D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153-170. https://doi.org/10.1016/j.learninstruc.2012.05.003

Gasper, K., & Danube, C. L. (2016). The scope of our affective influences: When and how naturally occurring positive, negative, and neutral affects alter judgment. Personality and Social Psychology Bulletin, 42(3), 385-399. https://doi.org/10.1177/0146167216629131

Han, Z. M., Huang, C. Q., Yu, J. H., & Tsai, C. C.. (2021). Identifying patterns of epistemic emotions with respect to interactions in massive online open courses using deep learning and social network analysis. Computers in Human Behavior, 122(2), 106843. https://doi.org/10.1016/j.chb.2021.106843

Harley, J. M., Bouchet, F., Hussain, M. S., Azevedo, R., & Calvo, R. (2015). A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system. Computers in Human Behavior, 48, 615-625. https://doi.org/10.1016/j.chb.2015.02.013

Huang, C. Q., Han, Z. M., Li, M. X., Jong, M. S.-y., & Tsai, C. C. (2019). Investigating students’ interaction patterns and dynamic learning sentiments in online discussions. Computers & Education, 140, 103589. https://doi.org/10.1016/j.compedu.2019.05.015

Huang, C., Han, Z., Li, M., Wang, X., & Zhao, W. (2021). Sentiment evolution with interaction levels in blended learning environments: Using learning analytics and epistemic network analysis. Australasian Journal of Educational Technology, 37(2), 81-95. https://doi.org/10.14742/ajet.6749

Huang, C., Yu, J., Wu, F., Wang, Y., & Nian‐Shing Chen. (2024). Uncovering emotion sequence patterns in different interaction groups using deep learning and sequential pattern mining. Journal of Computer Assisted Learning, 40(4), 1777-1790. https://doi.org/10.1111/jcal.12977

King, R. B., McInerney, D. M., Ganotice, F. A., & Villarosa, J. B. (2015). Positive affect catalyzes academic engagement: Cross-sectional, longitudinal, and experimental evidence. Learning and Individual Differences, 39, 64-72. https://doi.org/10.1016/j.lindif.2015.03.005

Kiuru, N., Malmberg, L.-E., Eklund, K., Penttonen, M., Ahonen, T., & Hirvonen, R. (2022). How are learning experiences and task properties associated with adolescents’ emotions and psychophysiological states? Contemporary Educational Psychology, 71, 102095. https://doi.org/10.1016/j.cedpsych.2022.102095

Lehman, B., D’Mello, S., & Graesser, A. (2012). Confusion and complex learning during interactions with computer learning environments. The Internet and Higher Education, 15(3), 184-194. https://doi.org/10.1016/j.iheduc.2012.01.002

Liu, S., Liu, S., Liu, Z., Peng, X., & Yang, Z. (2022). Automated detection of emotional and cognitive engagement in MOOC discussions to predict learning achievement. Computers & Education, 181, 104461. https://doi.org/10.1016/j.compedu.2022.104461

Lund, K., Quignard, M., & Williamson Shaffer, D. (2017). Gaining insight by transforming between temporal representations of human interaction. Journal of Learning Analytics, 4(3). https://doi.org/10.18608/jla.2017.43.6

Parker, P. C., Perry, R. P., Hamm, J. M., Chipperfield, J. G., Pekrun, R., Dryden, R. P., Daniels, L. M., & Tze, V. M. C. (2021). A motivation perspective on achievement appraisals, emotions, and performance in an online learning environment. International Journal of Educational Research, 108, 101772. https://doi.org/10.1016/j.ijer.2021.101772

Pekrun, R., & Marsh, H. W. (2022). Research on situated motivation and emotion: Progress and open problems. Learning and Instruction, 81, 101664. https://doi.org/10.1016/j.learninstruc.2022.101664

Peterson, E. G., & Zengilowski, A. (2024). Educators’ perceptions of expectancy, value, and cost for supporting student emotions. Contemporary Educational Psychology, 78, 102294. https://doi.org/10.1016/j.cedpsych.2024.102294

Rebolledo-Mendez, G., Huerta-Pacheco, N. S., Baker, R. S., & du Boulay, B. (2021). Meta-affective behaviour within an intelligent tutoring system for mathematics. International Journal of Artificial Intelligence in Education, 32(1), 174-195. https://doi.org/10.1007/s40593-021-00247-1

Richey, J. E., Andres-Bray, J. M. L., Mogessie, M., Scruggs, R., Andres, J. M. A. L., Star, J. R., Baker, R. S., & McLaren, B. M. (2019). More confusion and frustration, better learning: The impact of erroneous examples. Computers & Education, 139, 173-190. https://doi.org/10.1016/j.compedu.2019.05.012

Shao, K., Kutuk, G., Fryer, L. K., Nicholson, L. J., & Guo, J. (2023). Factors influencing Chinese undergraduate students’ emotions in an online EFL learning context during the COVID pandemic. Journal of Computer Assisted Learning, 39(5), 1465-1478. https://doi.org/10.1111/jcal.12791

Tan, S. E., & Jung, I. (2024). Unveiling the dynamics and impact of emotional presence in collaborative learning. International Journal of Educational Technology in Higher Education, 21(1). https://doi.org/10.1186/s41239-024-00477-y

Wei, X., Chen, Y., Shen, J., & Zhou, L. (2024). Fail or pass? Investigating learning experiences and interactive roles in MOOC discussion board. Computers & Education, 217, 105073. https://doi.org/10.1016/j.compedu.2024.105073

Xing, W., Tang, H., & Pei, B. (2019). Beyond positive and negative emotions: Looking into the role of achievement emotions in discussion forums of MOOCs. The Internet and Higher Education, 43, 100690. https://doi.org/10.1016/j.iheduc.2019.100690

Yang, S., Shu, D., & Yin, H. (2021). ‘Frustration drives me to grow’: Unraveling EFL teachers’ emotional trajectory interacting with identity development. Teaching and Teacher Education, 105, 103420. https://doi.org/10.1016/j.tate.2021.103420

Yang, Y., Yuan, K., Zhu, G., & Jiao, L. (2024). Collaborative analytics-enhanced reflective assessment to foster conducive epistemic emotions in knowledge building. Computers & Education, 209, 104950. https://doi.org/10.1016/j.compedu.2023.104950

Ye, J.-m., & Zhou, J. (2022). Exploring the relationship between learning sentiments and cognitive processing in online collaborative learning: A network analytic approach. The Internet and Higher Education, 55, 100875. https://doi.org/10.1016/j.iheduc.2022.100875

Zheng, L., & Huang, R. (2016). The effects of sentiments and co-regulation on group performance in computer supported collaborative learning. The Internet and Higher Education, 28, 59-67. https://doi.org/10.1016/j.iheduc.2015.10.001

Analyzing Learning Sentiments on a MOOC Discussion Forum Through Epistemic Network Analysis by Jianhui Yu is licensed under a Creative Commons Attribution 4.0 International License.