Teacher- Versus AI-Generated (Poe Application) Corrective Feedback and Language Learners’ Writing Anxiety, Complexity, Fluency, and Accuracy

Authors

  • Dan Wang School of Foreign Studies, East China University of Political Science and Law, ShangHai, China

DOI:

https://doi.org/10.19173/irrodl.v25i3.7646

Keywords:

artificial intelligence, AI, corrective feedback, writing anxiety, fluency, accuracy

Abstract

This study examines the effects of corrective feedback (CF) on language learners’ writing anxiety, writing complexity, fluency, and accuracy, and compares the effectiveness of feedback from human teachers with an AI-driven application called Poe. The study included three intact classes, each with 25 language learners. Using a quasi-experimental design with pretest and posttest measures, one class received feedback from the teacher, one from the Poe application, and the third received no response to their writing. Data were generated though tests and a writing anxiety scale developed for the study. Data analysis, conducted using one-way ANOVA tests, revealed significant effects of teacher and AI-generated feedback on learners’ writing anxiety, accuracy, and fluency. Interestingly, the group that received AI-generated feedback performed better than the group that received teacher feedback or no AI support. Additionally, learners in the AI-generated feedback group experienced a more significant reduction in writing anxiety than their peers. These results highlight the remarkable impact of AI-generated CF on improving writing outcomes and alleviating anxiety in undergraduate language learners at East China University of Political Science and Law . The study demonstrates the benefits of integrating AI applications into language learning contexts, particularly by promoting a supportive environment for students to develop writing skills. Educators, researchers, and developers can use these findings to inform pedagogical practices and technological interventions to optimize the language learning experience in primary school settings. This research highlights the effectiveness of AI-driven applications in language teaching. It highlights the importance of considering learners’ psychological well-being, particularly anxiety levels, when developing effective language learning interventions.

References

Benavides, L. M. C., Tamayo Arias, J. A., Arango Serna, M. D., Branch Bedoya, J. W., & Burgos, D. (2020). Digital transformation in higher education institutions: A systematic literature review. Sensors, 20(11), Article 3291. https://doi.org/10.3390/s20113291

Benson, S., & DeKeyser, R. (2018). Effects of written corrective feedback and language aptitude on verb tense accuracy. Language Teaching Research, 23(6), 702–726. https://doi.org/10.1177/1362168818770921

Biber, D., Nekrasova, T., & Horn, B. (2011). The effectiveness of feedback for L1-English and L2-writing development: A meta-analysis. ETS Research Report Series, 2011(1), i–99.https://doi.org/10.1002/j.2333-8504.2011.tb02241.x

Bonilla López, M., Van Steendam, E., Speelman, D., & Buyse, K. (2018). The differential effects of comprehensive feedback forms in the second language writing class. Language Learning, 68(3), 813–850. https://doi.org/10.1111/lang.12295

Bozkurt, A., Karadeniz, K., Baneres, D., Guerrero-Roldán, A. E., & Rodríguez, M. E. (2021). Artificial intelligence and reflections from educational landscape: A review of AI studies in half a century. Sustainability, 13(2), Article 800. https://doi.org/10.3390/su13020800

Bozkurt, A., Xiao, J., Lambert, S., Pazurek, A., Crompton, H., Koseoglu, S., Farrow, R., Bond, M., Nerantzi, C., Honeychurch, S., Bali, M., Dron, J., Mir, K., Stewart, B., Costello, E., Mason, J., Stracke, C. M., Romero-Hall, E., Koutropoulos, A., … Jandrić, P. (2023). Speculative futures on ChatGPT and generative artificial intelligence (AI): A collective reflection from the educational landscape. Asian Journal of Distance Education, 18(1), 53–130. https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/709

Cao, L. (2023). Trans-AI/DS: Transformative, transdisciplinary and translational artificial intelligence and data science. International Journal of Data Science and Analytics,15(1), 119–132. https://doi.org/10.1007/s41060-023-00383-y

Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020). Application and theory gaps during the rise of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1(2020), Article 100002. https://doi.org/10.1016/j.caeai.2020.100002

Chu, H.-C., Hwang, G.-H., Tu, Y.-F., & Yang, K.-H. (2022). Roles and research trends of artificial intelligence in higher education: A systematic review of the top 50 most-cited articles. Australasian Journal of Educational Technology, 38(3), 22–42. https://ajet.org.au/index.php/AJET/article/view/7526

Cox, A. M. (2021). Exploring the impact of artificial intelligence and robots on higher education through literature-based design fictions. International Journal of Educational Technology in Higher Education, 18(2021), Article 3. https://doi.org/10.1186/s41239-020-00237-8

DeKeyser, R. (2007). Skill acquisition theory. In B. VanPatten & J. Williams (Eds.), Theories in second language acquisition: An introduction (2nd ed., pp. 94–112). Routledge.

Diebold, G. (2023, January 17). Higher education will have to adapt to generative AI—And that’s a good thing. Center for Data Innovation. https://datainnovation.org/2023/01/higher-education-will-have-to-adapt-to-generative-ai-and-thats-a-good-thing/

Dogan, M. E., Goru Dogan, T., & Bozkurt, A. (2023). The use of artificial intelligence (AI) in online learning and distance education processes: A systematic review of empirical studies. Applied Sciences, 13(5), Article 3056. https://doi.org/10.3390/app13053056

Ellis, R. (2009a). A typology of written corrective feedback types. ELT Journal, 63(2), 97–107. https://doi.org/10.1093/elt/ccn023

Ellis, R. (2009b). Corrective feedback and teacher development. L2 Journal: An electronic refereed journal for foreign and second language educators, 1(1), 2–18. https://doi.org/10.5070/l2.v1i1.9054

Ellis, R. (2009c). Task-based language teaching: Sorting out the misunderstandings. International Journal of Applied Linguistics, 19(3), 221–246. https://doi.org/10.1111/j.1473-4192.2009.00231.x

Evans, N. W., Hartshorn, K. J., Cox, T. L., & Martin de Jel, T. (2014). Measuring written linguistic accuracy with weighted clause ratios: A question of validity. Journal of Second Language Writing, 24(1), 33–50. https://doi.org/10.1016/j.jslw.2014.02.005

Goksel, N., & Bozkurt, A. (2019). Artificial intelligence in education: Current insights and future perspectives. In S. Sisman-Ugur & G. Kurubacak (Eds.), Handbook of research on learning in the age of transhumanism (pp. 224–236). IGI Global. https://doi.org/10.4018/978-1-5225-8431-5.ch014

Han, Y., & Hyland, F. (2015). Exploring learner engagement with written corrective feedback in a Chinese tertiary EFL classroom. Journal of Second Language Writing, 30(1), 31–44. https://www.doi.org/10.1016/j.jslw.2015.08.002

Hartshorn, K. J., & Evans, N. W. (2015). The effects of dynamic written corrective feedback: A 30-week study. Journal of Response to Writing, 1(2), 6–34.

Humble, N., & Mozelius, P. (2022). The threat, hype, and promise of artificial intelligence in education. Discover Artificial Intelligence, 2(2022), Article 22. https://doi.org/10.1007/s44163-022-00039-z

Hyland, K., & Hyland, F. (2006). Feedback on second language students’ writing. Language Teaching, 39(2), 83–101. https://doi.org/10.1017/S0261444806003399

Kang, E., & Han, Z. (2015). The efficacy of written corrective feedback in improving L2 written accuracy: A meta-analysis. The Modern Language Journal, 99(1), 1–18. https://doi.org/10.1111/modl.12189

Karim, K., & Nassaji, H. (2018). The revision and transfer effects of direct and indirect comprehensive corrective feedback on ESL students’ writing. Language Teaching Research, 24(4), 519–539. https://doi.org/10.1177/1362168818802469

Karim, K., & Nassaji, H. (2019). The effects of written corrective feedback: A critical synthesis of past and present research. Instructed Second Language Acquisition, 3(1), 28–52. https://doi.org/10.1558/isla.37949

Kurzer, K. (2018). Dynamic written corrective feedback in developmental multilingual writing classes. TESOL Quarterly, 52(1), 5–33. https://doi.org/10.1002/tesq.366

Kurzweil, R. (2014). The singularity is near. In R. L. Sandler (Ed.), Ethics and emerging technologies (pp. 393–406). Palgrave Macmillan UK. https://doi.org/10.1057/9781137349088_26

Leeman, J. (2010). Feedback in L2 learning: Responding to errors during practice. In R. DeKeyser (Ed.), Practice in a second language: Perspectives from applied linguistics and cognitive psychology (pp. 111–138). Cambridge University Press. https://doi.org/10.1017/cbo9780511667275.007

Li, S., & Vuono, A. (2019). Twenty-five years of research on oral and written corrective feedback in System. System, 84(1), 93–109. https://doi.org/10.1016/j.system.2019.05.006

Lim, S. C., & Renandya, W. A. (2020). Efficacy of written corrective feedback in writing instruction: A meta-analysis. TESL-EJ, 24(3), 1–26. https://tesl-ej.org/wordpress/issues/volume24/ej95/ej95a3/

Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 30(1), 66–81. https://doi.org/10.1016/j.jslw.2015.08.011

Liu, Y., & Huang, J. (2020). The quality assurance of a national English writing assessment: Policy implications for quality improvement. Studies in Educational Evaluation, 67(2), Article 100941. https://doi.org/10.1016/j.stueduc.2020.100941

Long, M. H. (1980). Input, interaction, and second language acquisition. University of California.

Luo, Y., & Liu, Y. (2017). Comparison between peer feedback and automated feedback in college English writing: A case study. Open Journal of Modern Linguistics, 7(04), 197–215. https://doi.org/10.4236/ojml.2017.74015

Luan, H., Geczy, P., Lai, H., Gobert, J., Yang, S. J. H., Ogata, H., Baltes, J., Guerra, R., Li, P., & Tsai, C.-C. (2020). Challenges and future directions of big data and artificial intelligence in education. Frontiers in Psychology, 11, Article 580820. https://doi.org/10.3389/fpsyg.2020.580820

Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2(2021), Article 100041. https://doi.org/10.1016/j.caeai.2021.100041

Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27(6), 7893–7925. https://doi.org/10.1007/s10639-022-10925-9

Polio, C. (2012). The relevance of second language acquisition theory to the written error correction debate. Journal of Second Language Writing, 21(4), 375–389. https://doi.org/10.1016/j.jslw.2012.09.004

Polio, C., & Yoon, H. J. (2018). The reliability and validity of automated tools for examining variation in syntactic complexity across genres. International Journal of Applied Linguistics (United Kingdom), 28(1), 165–188. https://doi.org/10.1111/ijal.12200

Riazantseva, A. (2012). Outcome measure of L2 writing as a mediator of the effects of corrective feedback on students’ ability to write accurately. System, 40(3), 421–430. https://doi.org/10.1016/j.system.2012.07.005

Robinson, P. (2011). Second language task complexity, the cognition hypothesis, language learning, and performance. In P. Robinson (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 3–38). John Benjamins.

Russell, J., & Spada, N. (2006). The effectiveness of corrective feedback for the acquisition of L2 grammar: A meta-analysis of the research. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 133–164). John Benjamins.

Schmidt, R. (2012). Attention, awareness, and individual differences in language learning. Perspectives on Individual Characteristics and Foreign Language Education, 6(27), 27-49.

Selwyn, N., Hillman, T., Bergviken-Rensfeldt, A., & Perrotta, C. (2023). Making sense of the digital automation of education. Postdigital Science and Education, 5(1), 1–14. https://doi.org/10.1007/s42438-022-00362-9

Sharma, R. C., Kawachi, P., & Bozkurt, A. (2019). The landscape of artificial intelligence in open, online and distance education: Promises and concerns. Asian Journal of Distance Education, 14(2), 1–2. https://doi.org/10.5281/zenodo.3730631

Sia, P. F. D., & Cheung, Y. L. (2017). Written corrective feedback in writing instruction: A qualitative synthesis of recent research. Issues in Language Studies, 6(1), 61–80. https://www.ils.unimas.my/images/pdf/v6n1/ILS_Vol6No1_Sia.pdf

Stefanou, C., & Révész, A. (2015). Direct written corrective feedback, learner differences, and the acquisition of second language article use for generic and specific plural reference. The Modern Language Journal, 99(2), 263–282. https://doi.org/10.1111/modl.12212

Storch, N. (2009). The impact of studying in a second language (L2) medium university on the development of L2 writing. Journal of Second Language Writing, 18(2), 103–118. https://api.semanticscholar.org/CorpusID:62520708

Tang, K.-Y., Chang, C.-Y., & Hwang, G.-J. (2021). Trends in artificial intelligence-supported e-learning: A systematic review and co-citation network analysis (1998–2019). Interactive Learning Environments, 31(4), 2134–2152. https://doi.org/10.1080/10494820.2021.1875001 Thi, K, N., & Nikolov, M. (2021). Feedback treatments, writing tasks, and accuracy measures: A critical review of research on written corrective feedback. TESL-E, 25(3), 1-25. https://www.tesl-ej.org/pdf/ej99/a16.pdf

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), Article 15. https://doi.org/10.1186/s40561-023-00237-x

Truscott, J. (2010). Further thoughts on Anthony Bruton’s critique of the correction debate. System, 38(4), 626–633. https://doi.org/10.1016/j.system.2010.10.003

Truscott, J., & Hsu, A. Y.-P. (2008). Error correction, revision, and learning. Journal of Second Language Writing, 17(4), 292–305. https://doi.org/10.1016/j.jslw.2008.05.003

Winterson, J. (2022). 12 Bytes: How artificial intelligence will change the way we live and love. Penguin Random House.

Wolfe-Quintero, K. (1998). The connection between verbs and argument structures: Native speaker production of the double object dative. Applied Psycholinguistics, 19(2), 225–257. https://doi.org/10.1017/S0142716400010055

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–Where are the educators? International Journal of Educational Technology in Higher Education, 16(1), Article 39. https://doi.org/10.1186/s41239-019-0171-0

Zhang, T. (2021). The effect of highly focused versus mid-focused written corrective feedback on EFL learners’ explicit and implicit knowledge development. System, 99(2), Article 102493. https://doi.org/10.1016/j.system.2021.102493

Published

2024-08-26

How to Cite

Wang, D. (2024). Teacher- Versus AI-Generated (Poe Application) Corrective Feedback and Language Learners’ Writing Anxiety, Complexity, Fluency, and Accuracy. The International Review of Research in Open and Distributed Learning, 25(3), 37–56. https://doi.org/10.19173/irrodl.v25i3.7646

Publication Facts

Metric
This article
Other articles
Peer reviewers 
2
2.4

Reviewer profiles  N/A

Author statements

Author statements
This article
Other articles
Data availability 
N/A
16%
External funding 
N/A
32%
Competing interests 
N/A
11%
Metric
This journal
Other journals
Articles accepted 
81%
33%
Days to publication 
278
145

Indexed in

Editor & editorial board
profiles
Academic society 
N/A
Publisher 
Athabasca University Press