A System of Indicators for the Quality Assessment of Didactic Materials in Online Education

Authors

DOI:

https://doi.org/10.19173/irrodl.v22i1.5069

Keywords:

online education, didactic materials, quality, indicators, system, assessment

Abstract

The quality of didactic materials is a source of concern for teachers, users, and educational institutions that offer online education. There is a lack of indicators to help assess the quality of three key types of didactic materials commonly used in online education: didactic units (i.e., materials that contain program contents), didactic guides (i.e., materials that provide information), and additional didactic materials (materials to deepen knowledge). The objective of this article is to present a system of indicators designed to assess the quality of these types of didactic materials and guide their creation process. The system was developed based on a critical analysis of existing models designed to assess the quality of digital didactic materials. The system was validated by 16 international experts in online education, and a trial application of the system assessed five didactic guides and didactic units used by online universities in three different countries. Results of the validation process were triangulated with relevant literature, allowing the authors to make decisions regarding changes to the system in terms of maintaining, reformulating, or removing indicators. The resulting system comprises 43 assessment indicators and serves as a guide for designers, teachers, and users in the creation and selection of didactic materials for use in online education and in the assessment of their quality.

Author Biographies

Renata Marciniak, University of Vic - Central University of Catalonia

Renata Marciniak holds a PhD in Education with the International Doctoral Research Component, a Special Award for Doctoral Studies from the Autonomus University of Barcelona (UAB), and a Master’s degree in Research in Education from the Autonomus University of Barcelona (UAB). She also holds a PhD in Economics and Management from the University of Economics in Kraków (Poland). She is a professor accredited by the National Agency for Quality Assessment and Accreditation of Spain, ANECA and the Catalan University Quality Assurance Agency, AQU. She is a associate professor at the Faculty of Social Sciences at Manresa, University of Vic - Central University of Catalonia (UVic-UCC), and EUNCET Business School (Spain). Her research interests include management of the quality in online education, e-learning technologies, knowledge management in small and medium companies, and virtual organizations.

Cristina Cáliz Rivera, EUNCET Business School

Director of the Administration and Business Management degree and Marketing and Digital Communication degree.

References

Asinsten, J. C. (2011). Producción de material didáctico en ambientes virtuales de aprendizaje [Production of didactic material in virtual learning environments]. Virtual Educa. https://virtualeduca.org/documentos/manual_del_contenidista.pdf

Bautista Liébana, J. R., Martínez Romero, R., & Sainz Ibáñez, M. (2001). La evaluación de materiales didácticos para la educación a distancia [The evaluation of didactic materials for distance education]. Revista Iberoamericana de Educación a Distancia, 4(1), 73–95. http://revistas.uned.es/index.php/ried/article/view/1192/1095

Blanco Gil, R., Casado García, L., Martín Ruiz, F. J., Mediano Villanueva, F. J., Ramos Ruiz, J., & Utiel Ortega, C. (2010). Didáctica de la tecnología [Teaching technology]. Grao.

Bravo Ramos, J. L. (2005). Elaboración de materiales educativos para la formación a distancia [Development of educational materials for distance learning] Instituto de Ciencias de la Educación (ICE) UPM. http://www.ice.upm.es/wps/jlbr/Documentacion/Libros/Elabora_mat_img.pdf

Cepeda Romero, O., ; Gallardo, I. M., & Rodríguez Rodríguez, J. (2017), La evaluación de los materiales didácticos Digitales [The evaluation of digital didactic materials]. Revista Latinoamericana de Tecnología Educativa, 16(2). https://doi.org/10.17398/1695-288X.16.2.79

Espinoza, O., & González, L. E. (2006). Procesos universitarios dinámicos. El modelo de gestión de la calidad total [Dynamic university processes. The total quality management model]. Calidad de la Educación, 1(24), 15–34. http://dx.doi.org/10.31619/caledu.n24.265

Fernández-Pampillón Cesteros, A. M., Domínguez Romero, E., & Armas Ranero, I. (2012a). Guía para la producción y evaluación de materiales didácticos digitales [Guide for the production and evaluation of digital didactic materials]. Universidad Complutense de Madrid. https://eprints.ucm.es/id/eprint/12533/1/COdAv1_1_07jul2012.pdf

Fernández-Pampillón Cesteros, A. M., Domínguez Romero, E., & Armas Ranero, I. (2012b). Diez criterios para mejorar la calidad de los materiales didácticos digitales [Ten criteria to improve the quality of digital didactic materials]. In A. Sanz Cabrerizo, J. A. López Orozco, & A. Baratas Díaz (Eds.). VII jornadas de campus virtual UCM: valorar, validar y difundir campus virtual (pp. 25–34). Universidad Complutense de Madrid. https://eprints.ucm.es/id/eprint/20297/

García Aretio, L. (2014). Bases, mediaciones y futuro de la educación a distancia en la sociedad digital [Bases, mediations, and future of distance education in the digital society]. Sintesis.

García Martín, A., Briones Peñalver, A. J., Busquier Sáez, S., García Cascales, M. S., De Miguel-Gómez, M. D., Mulas Pérez, J., Pérez García, J., Vicente Chicote, C., García Sanchez, A. J., Martínez Conesa, E. J., Serrano Martínez, J. L., Pérez Pérez, J., & Sánchez Pérez, L. Á. (2010). Manual de elaboración de guías docentes adaptadas al EEES. [Manual for preparing teaching guides adapted to the EEES]. Universidad de Cartagena. https://repositorio.upct.es/xmlui/bitstream/handle/10317/1747/isbn9788469350317.pdf

Hyla, M. (2016). Przewodnik po e-learningu. [A guide for e-learning]. Wolters Kluwer.

Illum Hansen, T., & Toke Gissel, S. (2017). Quality of didactic materials. IARTEM e-Journal, 9(1), 122–141. https://doi.org/10.21344/iartem.v9i1.601

Marciniak, R. (2016). Autoevaluación de programas de educación universitaria virtual [Self-assessment of higher virtual education programmes] (Doctoral dissertation). Autonomous University of Barcelona, Barcelona, Spain.

Marciniak, R. (2017). A methodological proposal for the Project design of the online course: Pilot application. APERTURA. Revista de Innovación Educativa, 9(2), 74-95. http://www.udgvirtual.udg.mx/apertura/index.php/apertura/article/view/991/797

Marciniak, R. (2018). Quality assurance for online higher education programmes: Design and validation of an integrative assessment model applicable to Spanish universities. The International Review of Research in Open and Distributed Learning 19(2), 126-154. http://www.irrodl.org/index.php/irrodl/article/view/3443

Marzal, M. A., Calzada-Prado, J., & Vianello, M. (2008). Criterios para la evaluación de la usabilidad de los recursos educativos virtuales: un análisis desde la alfabetización en información. [Criteria for evaluating the usability of virtual educational resources]. Information Research, 13(4), 387. http://informationr.net/ir/13-4/paper387.html

Morales Morgado, E. M., García Peñalvo, F. J., & Olmos Migueláñez, S. (2010). Diseño de objetos de aprendizaje para potenciar el desarrollo de competencias y su evaluación con

HEODAR [Design of learning objects to enhance the development of skills]. J. Sánchez (Ed.). IE 2010, Segundo Congreso Iberoamericano de Informática Educativa (pp. 683–690). Universidad de Chile. http://www.tise.cl/volumen6/TISE2010/Documento97.pdf

National Agency for Quality Assessment and Accreditation (ANECA). (2019). Guide to self-evaluation: Ex post accreditation for official Bachelor's Degrees, Master's Degrees and Doctoral Degrees ACREDITA Procedure. ANECA. http://www.aneca.es/eng/content/view/full/12612

National Distance Education University of Spain. (n.d.). El protocolo de evaluación de materiales didácticos a distancia [The evaluation protocol for distance learning materials]. http://www.uned.es/iued/subsitio/html/documentos/Protocolo_MADI.pdf

Nesbit, J. C., Belfer, K., & Vargo, J. (2002). A convergent participation model for evaluation of learning objects. Canadian Journal of Learning and Technology, 28(3), 105–120. https://doi.org/10.21432/T25C8C

Nielsen, J., & Morkes, J. (1998). Applying writing guidelines to Web pages. Nielsen Norman Group. https://www.nngroup.com/articles/applying-writing-guidelines-web-pages/

Opdenacker, L., Stassen, I., Vaes, S., Waes, L., & Jacobs, G. (Eds.). (2009). QuADEM:Manual for the quality assessment of digital educational material. University of Antwerp.

Padrón, C., Díaz, P., & Aedo, I. (2006). MD2 method: The didactic materials creation from a model based perspective. In: W. Nejdl & K. Tochtermann (Eds.), Innovative Approaches for Learning and Knowledge Sharing. Lecture Notes in Computer Science (vol. 4227, pp. 366–382). Springer. https://doi.org/10.1007/11876663_29

Rodríguez Rodríguez, J., & Martínez Bonafé, J. (2016). Libros de texto y control del currículum en el contexto de la sociedad digital. Revista Latinoamericana de Tecnología Educativa, 16(2), 79–95. https://doi.org/10.17398/1695-288X.16.2.79

Rushby, N., & Surry, D. (2016). Wiley handbook of learning technology. John Wiley and Sons. https://doi.org/10.1002/9781118736494

Shattuck, K. (2014). Assuring quality in online education: Practices and processes at theteaching, resource, and program levels. Sterling: Stylus Publishing.

Spanish Association for Standardization and Certification (AENOR). (2017). UNE 71362. Calidad de los materiales educativos digitales [UNE 71362. Quality of digital educational materials]. Madrid: AENOR.

Vizoso, C. M. (2018, November 29). Calidad de los MED: Materiales educativos digitales[Quality of the Digital Educational Materials]. Global Campus Nebrija. https://www.nebrija.com/medios/nebrijaglobalcampus/2018/11/29/calidad-de-los-med-materiales-educativos-digitales/

Vlachopoulos, D. (2016). Assuring quality in online course design: The roadmap. The International Review of Research in Open and Distance Learning, 17(6), 183–205. http://dx.doi.org/10.19173/irrodl.v17i6.2784

Downloads

Published

2021-03-11

How to Cite

Marciniak, R., & Cáliz Rivera, C. (2021). A System of Indicators for the Quality Assessment of Didactic Materials in Online Education. The International Review of Research in Open and Distributed Learning, 22(1), 180–198. https://doi.org/10.19173/irrodl.v22i1.5069

Issue

Vol. 22 No. 1 (2021)

Section

Research Articles

License

This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.

This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.

You are free to

  • Share — copy and redistribute the material in any medium or format
  • Adapt — remix, transform, and build upon the material for any purpose, even commercially.

The licensor cannot revoke these freedoms as long as you follow the license terms below:

  • No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.