Volume 17, Number 3
Omid Noroozi1, Simon McAlister2, Martin Mulder3
1,3Wageningen University, The Netherlands, 2University of East London, UK
The goal of this study was to explore how students debate with their peers within a designed context using a digital dialogue game, and whether their epistemic beliefs are significant to the outcomes. Epistemic beliefs are known to colour student interactions within argumentative discourse, leading some students to hold back from interactions. By designing an online small group activity based around an issue both important and controversial to the students, with multiple viewpoints in each group and with the scaffolding provided by a dialogue game, it was examined whether these epistemic effects were still evident within their argumentative discourse. Furthermore, the study examined whether the activity design improves students' willingness to argue with each other, and their openness to attitudinal change. A pretest, posttest design was used with students who were assigned to groups of four or five and asked to argue on a controversial topic. Their aim was to explore various perspectives and to debate the pros and cons of the use of Genetically Modified Organisms (GMOs). While previous research has shown that some epistemic beliefs lead to less critical engagement with peers, the results presented here demonstrate that activity design is also an important factor in successful engagement within argumentative discourse.
Keywords: argumentation, attitudinal change, epistemic beliefs, dialogue, digital game
The advancements of the educational and communication technologies have acted as a catalyst for educational transformation in the past two decades. With the growth of the Internet and related technologies and wide accessibility to the World Wide Web, there is an inevitable need in modern society to confront and analyse complex issues. To prepare society for tackling these complexities, professionals and experts from diverse disciplines need to collaborate in new learning and working contexts. It is the case in education particularly, that students need to gain ample experience working in learning groups to become capable and qualified professionals who can actively participate in the knowledge society, analyse, synthesize, and cope with complex and societal issues. To do so, students need to be able to engage in dialogical argumentation, build arguments and support a position, to consider and weigh arguments and counter-arguments, to test, enlighten, and clarify uncertainties, and thus review misconceptions and correct false viewpoints (Aleixandre-Jimenez, 2007).
In line with the innovation and latest developments in the field of educational technology, many international universities have started to develop open and distance learning programmes in addition to their on-campus programmes. International professionals and students are keenly interested in such programmes due to the possibility to combine work, family responsibilities, and study; the lower annual costs; and the assumed flexibility. Like any other programme, open and distance learning programmes may have their own specific risks and disadvantages, especially with respect to lack of engagement and high student dropout rates. Despite the benefits of open and distance learning programmes, the physical separation of learners from their peers and instructors may cause a weaker sense of belonging to a classroom community and a lack of engagement, communication, and interactivity (Chaiprasurt & Esichaikul, 2013). Many scholars have concluded that this lack of belonging to a classroom community may reduce learners’ motivation which could also result in dissatisfaction, poor performance, and dropout (Balaban-Sali, 2008; Rau, Gao, & Wu, 2008). Therefore, additional motivational factors are needed to compensate for the physical separation of students from their peers and also their instructors in open and distance learning programmes.
One approach for stimulating learners’ motivation and satisfaction with open and distance learning programmes is to provide them with the opportunity to work together as a learning group to encouraging a spirit of social learning, praising and recognition of one another’s positive efforts, and fostering knowledge about the learning process (Sharan & Shaulov, 1990). The collaborative learning can thus be seen as a motivational factor for students to work on their assignments in a challenging yet pleasant learning group rather than individual work which in turn may positively influence students’ engagement and academic, social, and attitudinal outcomes (Miyake, 2007).
Argumentation is a vehicle for collaborative learning process not only for traditional forms of classrooms but also for blended and online learning settings such as open and distance learning programmes in higher education (see Noroozi, Busstra, Mulder, Biemans, Tobi, Geelen, et al., 2012). Argumentation is considered to be significant to education due to the importance of discourse in the acquisition of scientific knowledge (Osborne, 2010). Due to the importance of argumentation, scientific evidence suggests to explicitly design for critical discussion, argumentation, and reasoning rather than expecting it to occur naturally in open and distance learning programmes (see Noroozi, Weinberger, Biemans, Mulder, & Chizari, 2012; Ravenscroft, 2011).
Engaging in dialogical argumentation with learning partners for carrying out complex learning tasks can be more difficult in traditional settings rather than blended and online learning environments. There could be several reasons for the difficulties for argumentation in traditional (face-to-face) settings as compared with blended and online learning environments. Traditional class settings may create some emotional and social barriers for students to engage in face-to-face argumentation with learning partners. For example, there are some students who might be holding epistemic emotions (being curious and anxious when receiving counter-arguments in a face-to-face situation), while others might be emotional based on their achievements (being proud of their success or shameful on their failure) during argumentation-based collaborative learning. This might just hold back some students to engage in dialogical argumentations with their learning peers in the face-to-face setting because they want to avoid getting into a verbal fight with their learning partner(s) (Andriessen, 2006). These difficulties for engaging in dialogical argumentation can to some extent be tackled in online or blended learning environments when students’ identities are not revealed to their learning partners. Being unknown to learning partners through nicknames in online and blended learning environments may provide a safer and more respectful learning opportunity for learning groups who engage in argumentative discourse activities. Even when the unanimity of the learning partners is kept secret, students may still face difficulties during collaborative argumentation-based learning. The ill-defined nature of argumentation makes it almost impossible for learners to follow a set of strict rules and unbending laws on constructing arguments, responding to counter-arguments, and engaging in transactive argumentation in order to gain and construct knowledge, reject false viewpoints, refine and modify claims, and eliminate misunderstandings and misconceptions about the issue at stake (see Andriessen, 2006; Noroozi, Weinberger, et al., 2012).
Various instructional approaches have been proposed to help students learn the skills of argumentation and critical discussion for engaging in effective dialogical argumentation. The most recent approach is the use of computer-support, such as online argument awareness representations, computer-supported collaboration scripts, and digital dialogue games to scaffold collaborative argumentation and support the building, representing, and sharing of arguments with the aim of learning. These computer-support systems are fully described in studies by Kirschner, Buckingham-Shum, and Carr (2003), McLaren, Scheuer, and Mikšátko (2010), Scheuer, Loll, Pinkwart, and McLaren (2010), as well as Noroozi, Weinberger, et al. (2012). Most of these computer-support systems lack motivational aspects of learning leading to less student satisfaction and motivation in these instructional environments. Specifically, they focused mostly on challenging intellectual tasks, prompting and supporting students in arguing and debating issues in science (Suthers, 2003), the law (Pinkwart, Ashley, Lynch, & Aleven, 2009), and ethics (McLaren et al., 2010), without regard for the students’ motivation. This is a striking omission since in real educational settings, motivational factors and willingness to argue play a key role in the extent to which students approach or avoid arguments (Infante & Rancer, 1982). Willingness to argue can be influenced by the learner's level of assertiveness, which may in turn determine whether they engage in, or avoid, critical reasoning and arguments (Nussbaum & Bendixen, 2003). Due to the fear of getting into verbal fight, some learners may avoid disagreements with their learning partners, while other students may dislike to be questioned by their peers (Nussbaum, Hartley, Sinatra, Reynolds, & Bendixen, 2004). Also, less assertive students avoid arguments due to the challenging and competitive nature of argumentation and debating (Nussbaum, Sinatra, & Poliquin, 2008). For all these students, the structuring of discussion within a game context may make a significant difference to their motivations and level of interactions with peers.
Another factor that is crucial for the extent to which students engage in, or avoid, critical reasoning and arguments is their epistemic beliefs (Noroozi, Weinberger, et al., 2012; Nussbaum et al. 2008). Epistemic beliefs can be defined as one’s own opinion on the nature, structure, and certainty of knowledge and justification for knowing with regard to knowledge acquisition (see Hofer, 2000; Hofer & Pintrich, 1997; King & Kitchener, 1994). From this perspective, students’ epistemic beliefs can be labelled as: (a) absolutism, (b) multiplism, and (c) evaluativism. Absolutists view knowledge as objective, simple, certain, and fixed that cannot be changed meaning that there is only one right or wrong answer and only authority figures have those answers. Multiplists perceive knowledge as subjective and contextual where viewpoints are seen as mere opinions. In this case, students are exposed to various perspectives of the issue at hand with the aim of concluding that one point of view is as good as another. In the most developed and sophisticated scenario, evaluativists perceive knowledge as verified true belief meaning that there are multiple possibilities in which knowledge claims must always be evaluated for their quality of arguments in different contexts (see Muis, 2007). Scientific empirical evidence has shown that students engage in argumentation differently with respect to their epistemic beliefs. For example, multiplists are less critical regarding inconsistencies and misconceptions and less interactive with their partners than other belief groups (Nussbaum et al. 2008). It is also shown that evaluativists are more critical and active in eliciting information from their partners (Nussbaum et al. 2008), compared with absolutists who are less inclined to explore alternative solutions (Oh & Jonassen, 2006).
Epistemic beliefs can therefore be seen as a factor that influences the way students engage in argumentative discourse and critical discussion and reasoning. With regard to willingness to argue, since argumentation and debating involve social learning processes (O’Keefe, 1982) and guide student attention towards exploring various sides of issue at stake (Noroozi, Biemans, Busstra, Mulder, & Chizari, 2011; Nussbuam et al., 2008), we hypothesise that students’ attitudes towards GMOs would be modified after the discourse. The goal of this study is to explore how undergraduate students with various epistemic beliefs engage in argumentative discourse by exposing them to a controversial topic and conflicting views within a dialogue game which is fun to play, but encourages challenges. Prior research has not investigated the effects of epistemic beliefs on student argumentative discourse when student willingness to argue is enhanced by such activity design. The picture is also unclear when it comes to relationship between argumentative discourse activities, epistemic beliefs, and change in attitude. Therefore, the main question for this study is the following: what is the relationship between students’ argumentative discourse activities, epistemic beliefs, and change in attitude when exposing them to an activity design with enhanced motivational characteristics?
The study took place at Wageningen University in the Netherlands, with a focus on the life sciences, especially food and health, sustainability, and the healthy living environment. Students at this university are encouraged to combine natural and social sciences: from plant sciences to economics and from food technology to sociology. The participants were 25 BSc students who enrolled for the 168-h course "Life-Science Communication and Learning in the Digital Age." This new course is part of the Minor program on Environmental Education, which consists of Applied Environmental Education and Communication; Sustainable Development: Integrating Worldviews, Disciplines and Practices; Life-Science Communication and Learning in the Digital Age; and Didactic Skills courses with a total of 24 credits. In this course, students acquire insights into the possibilities and limitations of communication and learning in the digital age particularly in a life-sciences and sustainability-oriented context. Students explore how their own engagement in screen-based interaction affects their own ability to communicate and the way they learn. The mean age of the participants was 22.00 ( SD = 1.82), and the majority (80%) were female. This almost mirrors the proportion of female and male students at Wageningen University.
Participants were divided into five groups of four students, and one group of five students, taking into account their perspective on the controversial issue of the discussion. This was done to stimulate the discussion in the group by making sure that each group includes students who are neutral, in favour of, and against the controversial issue. Although all participants were distributed around a single classroom and even though they knew one another in advance, nicknames were assigned to them so that they did not know who other members of their online group were.
The topic for discussion was the application of Genetically Modified Organisms (GMOs) in agriculture. Students were provided with the description of the controversial issue and a summary of the theoretical text regarding GMOs. They were also provided with some additional links to websites to further study the concept of the GMOs. For example, in the description of the controversial issue, it was explained that:
GMOs have been touted as both nightmare and salvation—a nightmare to human health and biodiversity, and salvation for human food production, reducing pesticide emissions to the environment, and, by extension, biodiversity and human health. In short—the application of GMOs in our global food chains is rife with controversy. And despite the seeming ease with which one can paint the controversial picture of GMOs, the matter itself is rather complex. The financial and corporate stakes are sky high and of importance to our economies. Even issues with intellectual property rights on certain genes, as well as issues with plant breeders’ rights have come to the fore. In sum, the application of GMOs should be something that we can argue about—at length!
The students’ task was to read materials, discuss, and argue the GMO issue with three other members in the group while taking into account the various perspectives on the need—or lack thereof—of implementing GMOs in agriculture.
The four or five learning partners in each group were distributed over different locations of a classroom with anonymous online names. A synchronous text-based discussion board called InterLoc was used in the study for the collaboration phase. InterLoc is a Web-enabled platform for structured dialogue and discussion that is used to stimulate discussions between members of a group in an active learning environment by getting students thinking together about topics, media, or material that is relevant to them. It guides the interaction style for synchronous dialogue—promoting reasoning, critical discussion, and justified arguments—and allows the players to produce reusable content from their group experiences using a saved transcript. This guiding is done by requiring a choice of a sentence opener to begin each contribution to the discussion. A variety of sentence openers1 are embedded in the design for provoking and promoting students’ reasoning and the argumentative dialogue processes and practices of the players (see Figure 1 for a screenshot of the game environment). For instance, "I agree because..." encourages a player to provide a fully reasoned agreement. Other sentence openers deal with statements, evidence, support or criticism, and conclusions. In all cases these are formulated to cue the players to provide fuller, constructive contributions, which require more forethought.
Equally important for guiding, and a key feature of the game, is the list of suggested openers for players' responses to others, dynamically based on what has gone before. The suggested list of openers for replies is derived from a conception of how a well-reasoned discussion should proceed, (i.e., from statements to fuller explanations) and from evidence to deriving justified conclusions. Each opener in the suggested list therefore explores more deeply the current stage or takes the discussion to the next stage2.
The epithet of a dialogue game accurately describes the interactions within the discussion, as in a game there are rules about what (dialogue) moves can be made at different times, and so it is with InterLoc3. The presentation of an InterLoc discussion as a game to students is important psychologically because of the motivational aspects of participating in games; being involved in a game allows participants full reign to be critical, because such moves are sanctioned within a game environment.
The full description of the game and the roles of the players and tutors can be found in in the InterLoc literature which examines the differences in group discussions through using InterLoc versus using a free discussion board (for example McAlister, Ravenscroft, & Scanlon, 2004; Ravenscroft, 2011; Ravenscroft & McAlister, 2006).
Figure 1. A screenshot of the digital dialogue game with associated sentence opener.
Overall, the session took about 2.5 hours and consisted of three main phases:
A pretest posttest questionnaire was used to measure students’ attitudinal change on the GMO issue. This questionnaire consisted of two questions on a 5-point Likert scale ranging from strongly disagree, disagree, neutral, agree, through to strongly agree. Specifically, both in the pretest and posttest, each student was asked to indicate the extent to which they agreed with the following two statements: (a) GMOs should be further developed for the market to improve sustainability; and (b) GMOs are a danger to biodiversity. The data from the pretest served as an indicator of students’ preliminary opinions on the GMO issue in order to assign learning groups with the full range of opinions. Then, the data from posttest were compared with the pretest data in order to detect any shift of the student opinion on the GMO from pretest to posttest.
The students were measured on their epistemic beliefs using a 15-item instrument developed by Kuhn Cheney, and Weinstock (2000). This instrument was used to measure students’ beliefs about knowledge according to the judgement domains. Each item of this questionnaire consisted of a pair of contrasting statements, attributed to two individuals—Robin and Chris—in five domains including (a) judgements of taste, (b) aesthetic judgements, (c) value judgements, (d) judgements of truth about the physical world, and (e) judgements of truth about the social world. Following each pair of statements, this question was posed:
Can only one of their views be right, or could both have some rightness?
Response options were as follows:
only one right (coded as absolutist)
both could have some rightness
The following question, which was contingent on the response to the first, was as follows:
Could one view be better or more right than the other (If both could have some rightness was selected)?
one could not be more right (coded as multiplist)
one could be more right than the other (coded as evaluativist)
Based on the data from this questionnaire, each student was classified into three epistemic orientations: absolutists, multiplists, and evaluativists (Kuhn et al., 2000; Nussbaum et al., 2008). An absolutist believes that only one answer could be right. A multiplist believes that all opinions can be equally valid, so one opinion could not be more right than another in their view. An evaluativist believes that criteria exist whereby opinions or judgements can be evaluated and one can be shown to be better than another (so one opinion or judgement could be more right than another). Therefore, each student was classified as an absolutist when the first response option (i.e., only one right ) was dominant in his or her answers to the questions in various judgement domains. Accordingly, each student was classified as a multiplist when the second response option (i.e., both could have some rightness but one could not be more right ) was dominant in his/her answers to the questions in various judgement domains. And finally, each student was classified as an Evaluativist when the third response option (i.e. both could have some rightness but one could be more right than the other ) was dominant in his or her answers to the questions in various judgement domains. The reliability of the epistemic belief measure was satisfactory (Cronbach α =.74).
A content analysis coding scheme was adapted to measure quality of argumentative discourse activities. This was done to assess how well the discussion notes of each student and also his or her group reflected the criteria of a scientific argument (Noroozi, Weinberger, Biemans, Mulder, & Chizari, 2013; Noroozi, Teasley, Biemans, Weinberger, & Mulder, 2013; Noroozi, Biemans, Weinberger, Mulder, & Chizari, 2013; Weinberger & Fischer, 2006). Every message posted during the online discussion on the GMO issue was coded as one of the following: externalization, elicitation (asking a question), agreement, integration, or disagreement.
When students displayed a claim, the message was coded as externalization. Typically, this was the first message in a thread, though sometimes students might juxtapose externalizations (i.e., reply to earlier externalizations by a further externalization). It was also distinguished if the externalization was supported with evidence or not. When students asked for, or invited a reaction from their learning partners, we coded the message as elicitation and the answering response as externalisation. When students agreed with what had been said before, without any modification, or just by repeating what was said, the message was coded as agreement. Students might have taken over perspectives from their peers and built syntheses of (various) arguments and counter-arguments that learning partners had uttered before. In such a case, the message was coded as integration. Also, small additions to a peer’s utterances were coded as integration. Any rejection, denial, or negative answer or evaluation was coded as disagreement. Beyond saying "No" or "I disagree", any kind of modification or replacement of what had been said before was also coded as conflict. Thus, small repairs to a peer’s utterances were coded as conflict. This included taking note of the phenomenon of alleviating critiques by initializing responses with phrases such as "I totally agree, but...."
Table 1 presents descriptive information on students’ epistemic orientation (based on Kuhn et al., 2000), including attitudinal change, number of discussion messages per individuals and groups, and type of contribution based on argumentative discourse activities. This shows that 16 (64%) of the participants were classified as multiplist, 9 (36%) as evaluativists and none as absolutists. There were a total of 403 discussion messages generated during the discourse, with an average of 16.12 per student ( SD = 5.227). Out of 403 discussion messages, 129 messages were categorized as externalization, 83 messages as elicitation, 68 messages as agreement, 66 messages as integration, and 57 messages as disagreement. In total, Group 1 produced 61 messages, Groups 2 and 3 each produced 69 messages, Group 4 produced 90 messages; Group 5 produced 49 messages and Group 6 produced 67 messages.
Table 1
Students’ Epistemic Orientation, Attitude Shift, Number of Messages, and Types of Contribution
| Learning group | Nickname | Epistemic orientation | Attitude shift | Number of messages | Message proportion in group (%) | Types of contribution | |||||||||
| Externalization | Elicitation | Agreement | Integration | Disagreement | |||||||||||
| N | % | N | % | N | % | N | % | N | % | ||||||
| Group 1 Total messages: 61 | Simon | Multiplist | Yes | 13 | 21.31 | 3 | 23.1 | 2 | 15.4 | 2 | 15.4 | 2 | 15.4 | 4 | 30.8 |
| Robert | Multiplist | No | 21 | 34.43 | 4 | 19.0 | 2 | 9.5 | 3 | 14.3 | 8 | 38.1 | 4 | 19.0 | |
| Jane | Evaluativist | Yes | 15 | 24.59 | 3 | 20.0 | 4 | 26.7 | 1 | 6.7 | 5 | 33.3 | 2 | 13.3 | |
| Tom | Multiplist | No | 12 | 19.67 | 4 | 33.3 | 3 | 25.0 | 2 | 16.7 | 1 | 8.3 | 2 | 16.7 | |
| Group 2 Total messages: 68 | Carla | Multiplist | Yes | 17 | 25.00 | 7 | 41.2 | 4 | 23.5 | 2 | 11.8 | 3 | 17.6 | 1 | 5.9 |
| Anke | Evaluativist | No | 21 | 30.88 | 8 | 38.1 | 4 | 19.0 | 6 | 28.6 | 2 | 9.5 | 1 | 4.8 | |
| Eline | Multiplist | No | 12 | 17.65 | 5 | 41.7 | 1 | 8.3 | 3 | 25.0 | 1 | 8.3 | 2 | 16.7 | |
| Femke | Multiplist | No | 18 | 26.47 | 4 | 22.2 | 5 | 27.8 | 2 | 11.1 | 4 | 22.2 | 3 | 16.7 | |
| Group 3 Total messages: 68 | Anna | Evaluativist | Yes | 26 | 38.24 | 9 | 34.6 | 2 | 7.7 | 1 | 3.8 | 6 | 23.1 | 8 | 30.8 |
| Martin | Multiplist | Yes | 10 | 14.70 | 5 | 50.0 | 1 | 10.0 | 0 | 0.0 | 0 | 0.0 | 4 | 40.0 | |
| Sara | Evaluativist | Yes | 15 | 22.06 | 8 | 53.3 | 3 | 20.0 | 2 | 13.3 | 1 | 6.7 | 1 | 6.7 | |
| Ester | Evaluativist | No | 17 | 25.00 | 8 | 47.1 | 1 | 5.9 | 0 | 0.0 | 0 | 0.0 | 8 | 47.1 | |
| Group 4 Total messages: 90 | Patricia | Multiplist | Yes | 21 | 23.33 | 5 | 23.8 | 11 | 52.4 | 4 | 19.0 | 0 | 0.0 | 1 | 4.8 |
| Bianca | Multiplist | Yes | 9 | 10.00 | 1 | 11.1 | 1 | 11.1 | 4 | 44.4 | 3 | 33.3 | 0 | 0.0 | |
| Ilona | Multiplist | No | 20 | 22.22 | 6 | 30.0 | 2 | 10.0 | 8 | 40.0 | 2 | 10.0 | 2 | 10.0 | |
| Eelco | Multiplist | Yes | 17 | 18.89 | 6 | 35.3 | 4 | 23.5 | 1 | 5.9 | 4 | 23.5 | 2 | 11.8 | |
| Jeroen | Evaluativist | No | 23 | 25.56 | 2 | 8.7 | 9 | 39.1 | 6 | 26.1 | 5 | 21.7 | 1 | 4.3 | |
| Group 5 Total messages: 49 | Hanneke | Multiplist | No | 10 | 20.41 | 6 | 60.0 | 1 | 10.0 | 0 | 0.0 | 2 | 20.0 | 1 | 10.0 |
| Amanda | Multiplist | No | 7 | 14.29 | 4 | 57.1 | 0 | 0.0 | 3 | 42.9 | 0 | 0.0 | 0 | 0.0 | |
| Elizabeth | Multiplist | Yes | 17 | 34.69 | 5 | 29.4 | 2 | 11.8 | 3 | 17.6 | 4 | 23.5 | 3 | 17.6 | |
| Sophie | Multiplist | Yes | 15 | 30.61 | 2 | 13.3 | 6 | 40.0 | 3 | 20.0 | 1 | 6.7 | 3 | 20.0 | |
| Group 6 Total messages: 67 | Nienke | Evaluativist | No | 23 | 34.33 | 9 | 39.1 | 4 | 17.4 | 5 | 21.7 | 4 | 17.4 | 1 | 4.3 |
| Steven | Evaluativist | Yes | 9 | 13.43 | 2 | 22.2 | 6 | 66.7 | 0 | 0.0 | 1 | 11.1 | 0 | 0.0 | |
| Anouk | Evaluativist | Yes | 11 | 16.42 | 7 | 63.6 | 0 | 0.0 | 0 | 0.0 | 2 | 18.2 | 2 | 18.2 | |
| Jessy | Multiplist | Yes | 24 | 35.82 | 6 | 25.0 | 5 | 20.8 | 7 | 29.2 | 5 | 20.8 | 1 | 4.2 | |
A check was performed on students’ attitudinal change on the GMO issue from pretest to posttest. Figure 2 depicts these results, showing for each student the Likert value they chose, pretest and posttest, on the question of development of GMOs for the market to improve sustainability. It shows that 14 of 25 (56%) students shifted their opinion on the GMO issue, from pretest to posttest. In other words, for these students, there was an indication that the dialogue and argumentation affected their attitude to GMOs.
Of the fourteen students who shifted their opinions on the controversial issue from pretest to posttest, nine of these students were multiplists and five were evaluativists, giving the same split (56%) within both these subgroups as the whole group.
The frequency of types of discourse messages was examined between these subgroups, looking at the number of messages in each category (externalisation, elicitation etc.), to see if there was any systematic variation or if epistemic beliefs were independent of discourse activities (null hypothesis). No significant differences in types of discourse messages were found between those students categorised as multiplist and those categorised as evaluativist, x 2(4, N = 25) = 0.14, p = 0.54. A further test splitting multiplists into those that shifted attitude on GMOs and those that did not, and evaluativists in the same way, also did not reveal significant variations in discourse activity by type of message, x 2(12, N = 25) = 1.08, p = 0.24. The largest variation in discourse activity between the two categories was that multiplists posted more agreement messages and evaluativists posted fewer agreement messages. Multiplists who did not shift their attitude asked fewer questions (elicitations) and posted more agreement compared with multiplists who did shift their attitude. Evaluativists who shifted their attitude posted much fewer agreements than those evaluativists who did not shift their attitude.
Figure 2. Attitude shift of student opinions from pretest to posttest in terms of development of GMOs for the market to improve sustainability. The letter S represents student names.
A positive response for students’ satisfaction with the learning experience and its outcomes was obtained during the plenary discussion sessions at the end of the experiment. During the plenary discussion sessions, students appreciated the game with regard to its dynamic nature, user-friendliness, and variation of the sentence openers. Furthermore, they said that the game was useful with respect to practicing, provoking, and promoting their critical reasoning and argumentation skills.
Qualitative analysis of the data for each learning group is presented below in order to have a clear picture of the relationship between argumentative discourse activities and students’ epistemic beliefs. Overall, the proportions of the type of contributions to the game were highest for externalization (32%), and then agreement (21%), elicitation (17%) and integration (16%), with the least belonging to disagreement (14%).
Group 1 consisted of three multiplists and one evaluativist. Students in this group produced 61 messages, of which, 14 messages (23%) were categorized as externalization, 11 messages (18%) as elicitation, eight messages (13%) as agreement, 16 messages (26%) as integration, and 12 messages (20%) as disagreement. In this group, the opinion of two students on the controversial issue shifted from pretest to posttest. Characteristics of each student in this group in terms of epistemic orientation, attitudinal change, number of messages per individuals and groups, and type of contribution based on argumentative discourse activities are presented below.
Group 2 consisted of three multiplists, and one evaluativist. Students in this group produced 68 messages, of which, 24 messages (35%) were categorized as externalization, 14 messages (21%) as elicitation, 13 messages (19%) as agreement, 10 messages (15%) as integration, and seven messages (10%) as disagreement. In this group, the opinion of only one student on the controversial issue shifted from pretest to posttest.
Group 3 is rather different from most other groups in having three evaluativists and only one multiplist. Students in this group produced 68 messages, of which 30 were categorized as externalization (44%). Only seven messages (10%) were categorized as elicitation, and another three messages (4%) were categorized as agreement. Of the remaining 28 messages, seven were classified as integration (10%) and 21 as disagreement (31%). In this group, the opinion of three students on the controversial issue shifted from pretest to posttest.
Group 4 consisted of four multiplists and one evaluativist. Students in this group produced 90 messages, of which 20 messages (22%) were categorized as externalization, 27 messages (30%) as elicitation, 23 messages (26%) as agreement, 14 messages (16%) as integration, and only six messages (7%) as disagreement. In this group, the opinion of three students on the controversial issue shifted from pretest to posttest.
Group 5 is rather different from other groups in terms of the distribution of students on their epistemic orientation, since it consists of only multiplists. Students in this group produced 49 messages, of which, 17 messages (35%) were categorized as externalization, nine messages (18%) as elicitation, nine messages (18%) as agreement, seven messages (14%) as integration, and seven messages (14%) as disagreement. In this group, the opinion of two students on the controversial issue shifted from pretest to posttest.
Group 6 consists of three evaluativists and one multiplist. Students in this group produced 67 messages, of which 24 were categorized as externalization (36%), and 15 messages (22%) as elicitation. The same number of messages have been classified as agreement and integration: 12 messages (18%). A mere four messages (6%) were classified as disagreement. In this group, the opinion of three students on the controversial issue shifted from pretest to posttest.
Figure 3 shows a typical fragment of the discussion from Group 1 demonstrating the nature of their interactions. This particular fragment was chosen because it includes disagreement. The first part starts with Tom eliciting group views on the topic, to which Simon outlines two opposing positions on the nature of GMOs (externalisation). Robert requests clarification (elicit.) by challenging the position for GMOs and Tom defends by elaborating on the pro-GMO position (ext.). Robert makes a tentative disagreement and Simon elaborates (externalisation) on Tom's position. In the second part, Robert agrees with, and elaborates (integration) on Simon's first statement on the unnaturalness of GMOs. Tom disagrees with Robert and restates both their respective positions. In turn, Simon disagrees with Tom and criticises his position.
Figure 3. Fragment of Group 1 GMO discussion demonstrating disagreements.
The discussion pattern reveals the style of discussion throughout, in that students used good reasoning and elaborated well, and were not afraid to use disagreement, if needed, to challenge their peers. The dialogue game provided guidance and shape to the students' choice of responses through the selection of openers and encouraged explicit and elaborated statements (e.g. "Let me say more about that..."). It also encouraged them to commit to disagreement ("I disagree because...") or provide evidence ("I read that...") through the use of the suggested openers, whereas most students in other contexts, are normally circumspect with regard to these dialogue moves. Previous work on InterLoc has shown that the dialogue game significantly changes the style of discussion and the frequency of the lesser used dialogue moves, compared with a non-guiding environment (chat).
The results show little effect of epistemic beliefs on the style and frequency of particular types of contribution by students. It was expected that multiplists would interact less and be less critical of their peers than evaluativists. Also that multiplists would be less susceptible to attitude shifts as a consequence. However, this did not appear to be the case in the present study and so a number of factors were considered to see why this might have been so. The study of Kuhn et al. (2000) tested epistemic beliefs in several domains, including social truths and physical truths, and across age and experience ranges, including undergraduates. Assuming the GMO issue in this study falls into both social and physical truths domains, the average across these two domains for the Kuhn study for undergraduates was as follows: evaluativist (42.5%), multiplist (47.5%), and absolutist (5%). The proportions for this study (all undergraduates) were not dissimilar, and were as follows: evaluativist (36%), multiplist (64%), and absolutist (0%). Fourteen (56%) students shifted their opinion (as measured by the Likert scale) on the GMO issue, between the pretest and posttest on the two attitude statements ("GMOs should be further developed for the market to improve sustainability" and "GMOs are a danger to biodiversity"). The first issue was the subject for the main debate between the students, and this showed the most shifts in opinion with seven students responding more positively after the debate and four students more negatively. Each opinion shift recorded represents a clear revision by the student. For instance, two students moved from a neutral position to being against further development, and three students from being neutral to supporting development. However, we believe another factor that may be involved in change of opinion is the strength or weakness of prior knowledge on the controversial issue. If prior knowledge is weak and the pretest opinion is not firmly held, an opinion may be changed with relatively little argument.
The proportion of multiplists and evaluativists shifting opinion was the same at 56%. There were six students who strongly favoured further development of GMOs at the outset and who did not change their opinion. Mainly, they did not regard GMOs as a threat to biodiversity. Looking for differences between evaluativists and multiplists in the discussion, it appears that there were few differences, and none statistically significant. Evaluativists produced more messages, with fewer agreement and more disagreement messages, while multiplists produced fewer integration messages. This study did not replicate the findings of Nussbaum et al. (2008), that multiplists are less critical regarding inconsistencies and misconceptions and less interactive with their partners than other belief groups. The percentage of multiplist messages categorised as disagreement (12.9%) was slightly less than the average (14.1%), and the percentage of externalizations (32.3%) slightly less than the average (34.5%). Since externalizations are viewed as the least interactive category, multiplists in this study appeared to interact about as much as the others.
Previous studies in this field had found differences in the style and strength of interactions within the discussion emerging from the differences in epistemic beliefs, but this study found none. Having followed the methodology from the Kuhn study (Kuhn et al., 2000), there are only a few differences in context that could have caused these results. One background difference (which seems to the authors to be unlikely) is culture, since this study was carried out in The Netherlands. There are three likely remaining differences, and each affects the willingness to argue with peers: the use of a controversial issue for discussion; the context of the discussion within a university course; and, the use of a dialogue game. The controversial issue of GMOs caused quite passionate views on both sides of the argument, increasing the willingness to argue. We can see the strong views expressed by many of the students on the controversial issue, indicated by their positions towards the extremes on the Likert scales in Figure 2 and, for example, the number of disagreements in the discussion fragment in Figure 3. The context of the discussion, within a university course the students had chosen, confirmed their personal interest in the topic and would be expected to raise their willingness to argue. Finally, the use of a dialogue game which promotes and scaffolds critical reasoning and argument and enabled the students to engage with their peers without recourse to, or fear of, personal (ad hominum) statements, increasing willingness to argue.
While we can make no strong claims from a small number study such as this, the indications are that the activity design is responsible for the enhanced willingness to argue and engage with others for all students, eliminating the expected interaction differences due to multiplist or evaluativist beliefs. In addition, the results showed the same percentage of students making attitude shifts between the two groups, indicating an equal openness to persuasion. The change between being neutral to supporting a proposal about developing GMOs is in fact a relatively large change for a student studying the topic, so none of the attitude shifts recorded were trivial or unconsidered to the students involved. This study did not test conceptual change, but the equal frequency of attitude shifts, which indicate openness to persuasion, are at least likely to have implications for frequency of conceptual change also between the groups.
Implementation of the digital dialogue game for debating a controversial issue was evaluated positively by undergraduate students as can be seen in their satisfaction with the learning experience and its outcomes. This was documented through the plenary discussion session at the end of the game. This was apparently a challenging, yet pleasurable, learning opportunity for undergraduate students to debate with fellow students over GMOs—one of the most controversial issues of the social and physical sciences. The use of sentence openers guided the students to elicit information, ask clarifying questions, express agreements and disagreements, and integrate various points of view. The dialogue game provided a safe and respectful environment for students to practice their argumentation and exercise critical discussion and reasoning skills. It was also easy for students to quickly learn various functionalities of this game-based environment. Taking all together, the user-friendliness and the design of the sentence openers of the game offered through InterLoc were positively reflected in the learners’ scores for satisfaction with the learning experience.
Within this study a learning activity was designed to positively engage students by (a) choosing a controversial topic, (b) creating groups that contain conflicting opinions, and (c) using a dialogue game to direct the conflict towards deeper reasoning and engagement. The results showed that more than half of the students shifted their opinions in the questionnaire after the debate, and the transcripts demonstrated that they had engaged deeply with their peers. The measurement of different epistemic beliefs among the students led to expected differences in style of argumentation and level of interaction between students with multiplist beliefs and those with evaluativist beliefs. No significant differences between the two groups were found in this study, leading to a re-evaluation of the relationship between students’ argumentative discourse activities, epistemic beliefs, and change in attitude when exposing them to an activity design with enhanced motivational characteristics. Based on this study, the indications are that use of an appropriate learning design, such as a digital dialogue game and an appropriate choice of controversial issue, enhanced the willingness to argue of all students, whatever a student's epistemic beliefs. It is also extremely likely that selection of group members to include opposite views played a part in increasing willingness to argue. Many variables are involved in such engagements, not all under the learning designer's control, but the activity design and the authenticity of the engagement are undoubtedly the two major factors contributing to willingness to argue. This is good news for learning designers who may be able to arrange suitably engaging and thoughtful argumentative discourse, despite students' epistemic beliefs and reluctance to engage. The argumentative discourse in this study caused most students to change their position and shift their opinion, an outward sign that the activity initiated thinking, and rethinking, among the students. Of course, further and more detailed research is needed to confirm these indications, and determine if epistemic beliefs have more subtle effects on students' participation in such debates as were undertaken here.
One of the strengths of the present study is that the activity took place in an educational setting with high ecological validity, and not in an artificial setting which might reduce student motivation. Our criticism of previous studies in this area would be that lack of authentic motivation might dramatically affect the results obtained (with regard to the lack of engagement of multiplists for instance), yet in real-life situations where the issues matter to the students, the multiplists may argue equally as hard as evaluativists. This study provided the opportunity to shed light on the effects of such a designed game-based activity on argumentative discourse and attitudinal change as they occur in authentic learning situations. In educational settings, not only are students’ self-reported opinions on the learning important, but also their actual learning and achievement. Therefore, we might aim to incorporate any learning metrics, from course exams or assignments, to measure actual learners’ achievement in educational settings with such interventions.
This study did not count for students’ prior knowledge on GMOs. As discussed previously, prior knowledge might play a role in shifting perceptions on the controversial issues. When prior knowledge is strong then often opinion is firmly held and would not be easily changed or shifted compared with a weak prior knowledge that can be changed with relatively little argument. Thus, future research should focus on taking prior knowledge of players into consideration. This study did not experiment with group size, and so future work should be scaled up in order to test alternative configurations of group size and conflicting opinions within a group. Finally, with further research we will be able to draw more certain conclusions about the likely impact of epistemic beliefs on willingness to argue and the likelihood of rethinking and opinion shift in students, and whether, as this study indicates, these are in fact variables in the control of the learning designer.
Such debates as were undertaken here, are part of a learning designer's goal to facilitate thoughtful and deeper argumentation which in turn facilitate students' grasp of what it means to be a 'critical thinker'. We conclude that even this small scale study reminds us of the many variables at work within a learning design affecting willingness to argue and engagement in argumentative discourse. They include the ecological validity of the setting, knowledge and pertinence of the issue at hand, whether there is confrontation of conflicting views, students' epistemic beliefs, and, by no means least, the style of engagement (in this case an online dialogue game). Outcomes are not determined by one variable alone, so learning designers will need to keep in mind the full range of factors that will facilitate thoughtful and deeper argumentation.
1 There are 53 sentence openers in the default game which have been carefully chosen and refined in hundreds of trials to ensure both usability for players and guidance towards the desired style of dialogue.
2 However the suggested list can be overridden by the player if they require to make a specific contribution and they can choose from the full set of openers.
3 In fact, within InterLoc there are alternative games the administrator can select–the most commonly used is the (default) Critical Reasoning game.
Aleixandre-Jimenez, M. (2007). Designing argumentation learning environments. In S. Erduran., & M. Aleixandre-Jimenez (Eds.), Argumentation in science education: Perspectives from classroom-based research (pp. 91–115). New York, NY: Springer. http://dx.doi.org/10.1007/978-1-4020-6670-2_5
Andriessen, J. (2006). Arguing to learn. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 443–460). New York, NY: Cambridge University Press.
Balaban-Sali, J. (2008). Designing motivational learning systems in distance education. Turkish Online Journal of Distance Education, 9(3), 149–161.
Chaiprasurt, C., & Esichaikul, V. (2013). Enhancing motivation in online courses with mobile communication tool support: A comparative study. The International Review of Research in Open and Distributed Learning, 14(3), 377–401.
Hofer, B. K. (2000). Dimensionality and disciplinary differences in personal epistemology. Contemporary Educational Psychology, 25(4), 378–405. Retrieved from http://dx.doi.org/10.1006/ceps.1999.1026
Hofer, B. K. (2001). Personal epistemology research: Implications for learning and teaching. Educational Psychology Review, 13( 4), 353–383. Retrieved from http://dx.doi.org/10.1023/A:1011965830686
Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67(1), 88–140. Retrieved from http://dx.doi.org/10.3102/00346543067001088
Infante, D. A., & Rancer, A. S. (1982). A conceptualization and measure of argumentativeness. Journal of Personality Assessment, 46(1), 72–80. Retrieved from http://dx.doi.org/ 10.1207/s15327752jpa4601_13
King, P., & Kitchener, K. S. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco, CA: Jossey-Bass.
Kirschner, P. A., Buckingham-Shum, S. J., & Carr, C. S. (Eds.). (2003). Visualizing argumentation: Software tools for collaborative and educational sense-making. London, UK: Springer.
Kuhn, D., Cheney, R., & Weinstock, M. (2000). The development of epistemological understanding. Cognitive Development, 15(3), 309–328. Retrieved from http://dx.doi.org/10.1016/S0885-2014(00)00030-7
McAlister, S., Ravenscroft, A., & Scanlon, E. (2004). Combining interaction and context design to support collaborative argumentation using a tool for synchronous CMC. Journal of Computer Assisted Learning, 20(3), 194–204. Retrieved from http://dx.doi.org/10.1111/j.1365-2729.2004.00086.x
McLaren, B. M., Scheuer, O., & Mikšátko, J. (2010). Supporting collaborative learning and e-discussions using artificial intelligence techniques. International Journal of Artificial Intelligence in Education, 20(1), 1–46.
Miyake, N. (2007). Computer supported collaborative learning. In R. Andrews & C. Haythornthwaite (Eds.), Handbook of e-learning research (pp. 263–280). London, UK: Sage
Muis, K. R. (2007). The role of epistemic beliefs in self-regulated learning. Educational Psychologist, 42(3), 173–190. Retrieved from http://dx.doi.org/10.1080/00461520701416306
Noroozi, O., Biemans, H. J. A., Busstra, M. C., Mulder, M., & Chizari, M. (2011). Differences in learning processes between successful and less successful students in computer-supported collaborative learning in the field of human nutrition and health. Computers in Human Behaviour, 27(1), 309–318. Retrieved from http://dx.doi.org/10.1016/j.chb.2010.08.009
Noroozi, O., Biemans, H. J. A., Weinberger, A., Mulder, M., & Chizari, M. (2013). Scripting for construction of a transactive memory system in a multidisciplinary CSCL environment. Learning and Instruction, 25(1), 1–12. Retrieved from http://dx.doi.org/10.1016/j.learninstruc.2012.10.002
Noroozi, O., Busstra, M.C., Mulder, M., Biemans, H.J.A., Tobi, H., Geelen, M.M.E.E., … Chizari, M. (2012). Online discussion compensates for suboptimal timing of supportive information presentation in a digitally supported learning environment. Educational Technology Research and Development, 60(2), 193-221. Retrieved from http://dx.doi.org/10.1007/s11423-011-9217-2
Noroozi, O., Teasley, S. D., Biemans, H. J. A., Weinberger, A., & Mulder, M. (2013). Facilitating learning in multidisciplinary groups with transactive CSCL scripts. International Journal of Computer-Supported Collaborative Learning, 8(2), 189–223. Retrieved from http://dx.doi.org/10.1007/s11412-012-9162-z
Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M., & Chizari, M. (2012). Argumentation-based computer supported collaborative learning (ABCSCL): A systematic review and synthesis of fifteen years of research. Educational Research Review, 7(2), 79–106. Retrieved from http://dx.doi.org/10.1016/j.edurev.2011.11.006
Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M., & Chizari, M. (2013). Facilitating argumentative knowledge construction through a transactive discussion script in CSCL. Computers and Education, 61(2), 59–76. Retrieved from http://dx.doi.org/10.1016/j.compedu.2012.08.013
Nussbaum, E. M., & Bendixen, L. D. (2003). Approaching and avoiding arguments: The role of epistemological beliefs, need for cognition, and extraverted personality traits. Contemporary Educational Psychology, 28(4), 573–595. Retrieved from http://dx.doi.org/10.1016/S0361-476X(02)00062-0
Nussbaum, E. M., Hartley, K., Sinatra, G. M., Reynolds, R. E., & Bendixen, L. D. (2004). Personality interactions and scaffolding in on-line discussions. Educational Computing Research, 30(1-2), 113–137. Retrieved from http://dx.doi.org/10.2190/H8P4-QJUF-JXME-6JD8
Nussbaum, E. M., Sinatra, M. G., & Poliquin, A. (2008). Role of epistemic beliefs and scientific argumentation in science learning. International Journal of Science Education, 30(15), 1977–1999. Retrieved from http://dx.doi.org/10.1080/09500690701545919
Oh, S., & Jonassen, D. H. (2006). Scaffolding online argumentation during problem solving. Journal of Computer Assisted Learning, 23(2), 95–110. Retrieved from http://dx.doi.org/10.1111/j.1365-2729.2006.00206.x
O’Keefe, D. J. (1982). The concept of argument and arguing. In J. R. Cox & C. A. Willard (Eds.), Advances in argumentation theory and research (pp. 3–23). Carbondale, IL: Southern Illinois University Press.
Osborne, J. F. (2010). Arguing to learn in science: The role of collaborative, critical discourse. Science, 328(5977), 463–466. Retrieved from http://dx.doi.org/10.1126/science.1183944
Pinkwart, N., Ashley, K. D., Lynch, C., & Aleven, V. (2009). Evaluating an intelligent tutoring system for making legal arguments with hypotheticals. International Journal of Artificial Intelligence in Education, 19(4), 401–424.
Rau, P.L. P., Gao, Q., & Wu, L. M. (2008). Using mobile communication technology in high school education: Motivation, pressure, and learning performance. Computers & Education, 50(1), 1–22. Retrieved from http://dx.doi.org/10.1016/j.compedu.2006.03.008
Ravenscroft, A. (2011). Dialogue and connectivism: A new approach to understanding and promoting dialogue-rich networked learning. International Review of Research in Open and Distributed Learning, 12(3), 139–160.
Ravenscroft, A., & McAlister, S. (2006). Digital games and learning in cyberspace: A dialogical approach. E-Learning and Digital Media, 3(1), 37–50. Retrieved from http://dx.doi.org/10.2304/elea.2006.3.1.37
Scheuer, O., Loll, F., Pinkwart, N., & McLaren, B. M. (2010). Computer-supported argumentation: A review of the state of the art. International Journal of Computer-Supported Collaborative Learning, 5(1), 43–102. Retrieved from http://dx.doi.org/10.1007/s11412-009-9080-x
Sharan, S., & Shaulov, A. (1990). Cooperative learning, motivation to learn, and academic achievement. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 173–202). New York, NY: Praeger.
Suthers, D. (2003). Representational guidance for collaborative inquiry. In J. Andriessen, M. Baker, & D. Suthers (Eds.), Arguing to learn: Confronting cognitions in computer-supported collaborative learning environments (pp. 27–46). Dordrecht, the Netherlands: Kluwer.
Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers and Education, 46(1), 71–95. Retrieved from http://dx.doi.org/10.1016/j.compedu.2005.04.003
Impacts of a Digital Dialogue Game and Epistemic Beliefs on Argumentative Discourse and Willingness to Argue by Omid Noroozi, Simon McAlister, and Martin Mulder is licensed under a Creative Commons Attribution 4.0 International License.