Development and Evaluation of an Open-Source, Online Training for the Measurement of Adult-Child Responsivity at Home and in Early Childhood Education and Care Settings

  • Michelle Rodrigues University of Toronto
  • Alessandra Schneider University of Toronto
  • Nina Sokolovic University of Toronto
  • Ashley Brunsek University of Toronto
  • Beatriz Oré NGO Alto Perú and Independent Consultant
  • Michal Perlman University of Toronto
  • Jennifer Jenkins University of Toronto
Keywords: responsive caregiving, parental sensitivity, online learning, observational measurement, low- and middle-income countries

Abstract

Efforts to monitor and improve responsive caregiving for young children, because of its importance for child development, are part of the United Nations Sustainable Development Goals. Two brief observational measures of responsive caregiving have been developed and validated (Responsive Interactions for Learning—parent [RIFL-P] and educator [RIFL-Ed] versions), with the RIFL-P available in English, Portuguese, and Spanish. The aim of the current study was to present and evaluate two online training programs for the RIFL measures. These distance learning courses were designed as open-source and asynchronous to enable their use in low- and middle-income countries and remote areas. The following course components are used: readings, lectures, observation of interactions on video, coding practice with automated feedback on item coding, and quizzes. Of the 76 trainees who registered for one of the online courses, 58 (76%) completed all theoretical module components. Student performance was generally high. Marks on quizzes ranged between 83%–100%. Ninety percent of those who took the reliability tests passed (40/44). Student satisfaction during and after the course was high. The effective online training programs are available free of charge and the RIFL suite of measures is efficient to implement. Implications for research and practice are discussed.

References

Bailey, H. N., Bernier, A., Bouvette‐Turcot, A. A., Tarabulsy, G. M., Pederson, D. R., & Becker‐Stoll, F. (2017). Deconstructing maternal sensitivity: Predictive relations to mother‐child attachment in home and laboratory settings. Social Development, 26(4), 679–693. https://doi.org/10.1111/sode.12220

Barteit, S., Guzek, D., Jahn, A., Bärnighausen, T., Jorge, M. M., & Neuhann, F. (2020). Evaluation of e-learning for medical education in low-and middle-income countries: A systematic review. Computers & Education, 145, 103726. https://doi.org/10.1016/j.compedu.2019.103726

Black, M. M., Walker, S. P., Fernald, L. C. H., Andersen, C. T., DiGirolamo, A. M., Lu, C., McCoy, D. C., Fink, G., Shawar, Y. R., Shiffman, J., Devercelli, A. E., Wodon, Q. T., Vargas-Barón, E., & Grantham-McGregor, S. (2017). Early childhood development coming of age: Science through the life course. The Lancet, 389(10064), 77–90. https://doi.org/10.1016/S0140-6736(16)31389-7

Britto, P. R., Lye, S. J., Proulx, K., Yousafzai, A. K., Matthews, S. G., Vaivada, T., Perez-Escamilla, R., Rao, N., Ip, P., Fernald, L. C., MacMillan, H., Hanson, M., Wachs, T. D., Yao, H., Yoshikawa, H., Cerezo, A., Leckman, J. F., & Bhutta, Z. A. (2017). Nurturing care: Promoting early childhood development. The Lancet, 389(10064), 91–102. https://doi.org/10.1016/S0140-6736(16)31390-3

Burgos, C., Campanario, M. L., de la Peña, D., Lara, J. A., Lizcano, D., & Martínez, M. A. (2018). Data mining for modeling students’ performance: A tutoring action plan to prevent academic dropout. Computers & Electrical Engineering, 66, 541–556. https://doi.org/10.1016/j.compeleceng.2017.03.005

Chang, T. P., Pham, P. K., Sobolewski, B., Doughty, C. B., Jamal, N., Kwan, K. Y, Little, K., Brenkert, T. E., & Mathison, D. J. (2014). Pediatric emergency medicine asynchronous e‐learning: A multicenter randomized controlled Solomon four‐group study. Academic Emergency Medicine, 21(8), 912–919. https://doi.org/10.1111/acem.12434

Davis, D., Chen, G., Hauff, C., & Houben, G. J. (2018). Activating learning at scale: A review of innovations in online learning strategies. Computers & Education, 125, 327–344. https://doi.org/10.1016/j.compedu.2018.05.019

Gladwell, M. (2005). Blink: The power of thinking without thinking. Little, Brown and Company.

González, A., Fernández, M. V. C., & Paoloni, P. V. (2017). Hope and anxiety in physics class: Exploring their motivational antecedents and influence on metacognition and performance. Journal of Research in Science Teaching, 54(5), 558–585. https://doi.org/10.1002/tea.21377

Hallal, P. C., Bertoldi, A. D., Domingues, M. R., da Silveira, M. F., Demarco, F. F., da Silva, I. C., Barros, F. C., Victora, C. G., & Bassani, D. G. (2018). Cohort profile: The 2015 Pelotas (Brazil) birth cohort study. International Journal of Epidemiology, 47(4), 1048–1048h. https://doi.org/10.1093/ije/dyx219

Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.

Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educause Quarterly, 31(4), 51–55. https://er.educause.edu/articles/2008/11/asynchronous-and-synchronous-elearning

Jeong, J., Franchett, E. E., Ramos de Oliveira, C. V., Rehmani, K., & Yousafzai, A. K. (2021). Parenting interventions to promote early child development in the first three years of life: A global systematic review and meta-analysis. PLoS Medicine, 18(5), e1003602–e1003602. https://doi.org/10.1371/journal.pmed.1003602

Jeong, J., Pitchik, H., & Yousafzai, A. (2018). Stimulation interventions and parenting in low- and middle-income countries: A meta-analysis. Pediatrics, 141(4), e20173510. https://doi.org/10.1542/peds.2017-3510

Khalil, H., & Ebner, M. (2014). MOOCs completion rates and possible methods to improve retention: A literature review. In J. Viteli & M. Leikomaa (Eds.), Proceedings of EdMedia 2014--World Conference on Educational Media and Technology (pp. 1305–1313). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/147656/

La Paro, K. M., Hamre, B. C., & Pianta, R. C. (2009). The classroom assessment scoring system, Toddler version. Teachstone.

Lotzin, A., Lu, X., Kriston, L., Schiborr, J., Musal, T., Romer, G., & Ramsauer, B. (2015). Observational tools for measuring parent–infant interaction: A systematic review. Clinical Child and Family Psychology Review, 18(2), 99–132. https://doi.org/10.1007/s10567-015-0180-z

Madigan, S., Prime, H., Graham, S., Rodrigues, M., Anderson, N., Khoury, J., & Jenkins, J. M. (2019). Parenting behavior and child language: A meta-analysis. Pediatrics, 144(4), e20183556. https://doi.org/10.1542/peds.2018-3556

Matias, C., O’Connor, T. G., Futh, A., & Scott, S. (2014). Observational attachment theory-based parenting measures predict children’s attachment narratives independently from social learning theory-based measures. Attachment & Human Development, 16(1), 77–92. https://doi.org/10.1080/14616734.2013.851333

Means, B., Toyama, Y., Murphy, R, Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. U.S. Department of Education, Office of Planning, Evaluation, and Policy Development. https://repository.alt.ac.uk/629/

Molin, F., Haelermans, C., Cabus, S., & Groot, W. (2020). The effect of feedback on metacognition: A randomized experiment using polling technology. Computers & Education, 152, 103885. https://doi.org/10.1016/j.compedu.2020.103885

Paton, R. M., Fluck, A. E., & Scanlan, J. D. (2018). Engagement and retention in VET MOOCs and online courses: A systematic review of literature from 2013 to 2017. Computers & Education, 125, 191–201. https://doi.org/10.1016/j.compedu.2018.06.013

Pauker, S., Perlman, M., Prime, H., & Jenkins, J. (2018). Caregiver cognitive sensitivity: Measure development and validation in Early Childhood Education and Care (ECEC) settings. Early Childhood Research Quarterly, 45, 45–57. https://doi.org/10.1016/j.ecresq.2018.05.001

Pederson, D. R., Moran, G., Sitko, C., Campbell, K., Ghesquire, K., & Acton, H. (1990). Maternal sensitivity and the security of infant‐mother attachment: AQ‐sort study. Child Development, 61(6), 1974–1983. https://doi.org/10.2307/1130851

Pierce, H. (2021). Nurturing care for early childhood development: Path to improving child outcomes in Africa. Population Research and Policy Review, 40(2), 285–307. https://doi.org/10.1007/s11113-020-09581-z

Prime, H., Browne, D. T., Akbari, E., Wade, M., Madigan, S., & Jenkins, J. M. (2015). The development of a measure of maternal cognitive sensitivity appropriate for use in primary care health settings. Journal of Child Psychology and Psychiatry, 56(4), 488–495. https://doi.org/10.1111/jcpp.12322

Prime, H., Pauker, S., Plamondon, A., Perlman, M., & Jenkins, J. M. (2014b). Sibship size, sibling cognitive sensitivity, and children’s receptive vocabulary. Pediatrics, 133(2), e394–e401. https://doi.org/10.1542/peds.2012-2874

Prime, H., Perlman, M., Tackett, J. L., & Jenkins, J. M. (2014a). Cognitive sensitivity in sibling interactions: Development of the construct and comparison of two coding methodologies. Early Education and Development, 25(2), 240–258. https://doi.org/10.1080/10409289.2013.821313

Prusak, K., Dye, B., Graham, C., & Graser, S. (2010). Reliability of pre-service physical education teachers’ coding of teaching videos using studiocode analysis software. Journal of Technology and Teacher Education, 18(1), 131–159. https://www.learntechlib.org/primary/p/29396/

Roggman, L. A., Cook, G. A., Innocenti, M. S., Jump Norman, V., & Christiansen, K. (2013). Parenting interactions with children: Checklist of observations linked to outcomes (PICCOLO) in diverse ethnic groups. Infant Mental Health Journal, 34(4), 290–306. https://doi.org/10.1002/imhj.21389

Santos, I., Munhoz, T., Barcelos, R., Blumenberg, C., Bortolotto, C., Matijasevich, A., Santos, H. G. dos, Jr., Marques, L., Correia, L., Souza, M., Lira, P., Pereira, V., Botelho, V., Silva, R., Chicaro, M., & Victora, C. (2020). Estudo de linha de base da avaliação de impacto do Programa Criança Feliz [Baseline study of the impact evaluation of the Criança Feliz program]. Ministério da Cidadania, Secretaria de Avaliação e Gestão da Informação. http://www.epidemio-ufpel.org.br/uploads/downloads/82359dcf63d2c7b882037e499a417bfe.pdf

Scherer, E., Hagaman, A., Chung, E., Rahman, A., O’Donnell, K., & Maselko, J. (2019). The relationship between responsive caregiving and child outcomes: Evidence from direct observations of mother-child dyads in Pakistan. BMC Public Health, 19(1), 252. https://doi.org/10.1186/s12889-019-6571-1

Schneider, A., Rodrigues, M., Falenchuk, O., Munhoz, T. N., Barros, A. J., Murray, J., Domingues, M. R., & Jenkins, J. M. (2021). Cross-cultural adaptation and validation of the Brazilian Portuguese version of an observational measure for parent–child responsive caregiving. International Journal of Environmental Research and Public Health, 18(3), 1246. https://doi.org/10.3390/ijerph18031246

Sokolovic, N., Borairi, S., Rodrigues, M., Perlman, M., & Jenkins, J. M. (2021a). Validating an efficient measure of responsivity in father–child interactions. Canadian Journal of Behavioral Science/Revue canadienne des sciences du comportement, 53(1), 84–89. https://doi.org/10.1037/cbs0000185

Sokolovic, N., Brunsek, A., Rodrigues, M., Borairi, S., Jenkins, J. M., & Perlman, M. (2021b). Assessing quality quickly: Validation of the Responsive Interactions for Learning - Educator (RIFL-Ed.) measure. Early Education and Development, 1–16. https://doi.org/10.1080/10409289.2021.1922851

Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research, and Evaluation, 9(1), 4. https://doi.org/10.7275/96jp-xz07

United Nations Children’s Fund [UNICEF] & Countdown to 2030. (2020, November 21). Thrive. Nurturing care for early childhood development: Country profiles for early childhood development. https://nurturing-care.org/wp-content/uploads/2020/11/English.pdf

Vermeer, H. J., van IJzendoorn, M. H., Cárcamo, R. A., & Harrison, L. J. (2016). Quality of child care using the environment rating scales: A meta-analysis of international studies. International Journal of Early Childhood, 48(1), 33–60. https://doi.org/10.1007/s13158-015-0154-9

Watts, L. (2016). Synchronous and asynchronous communication in distance learning: A review of the literature. Quarterly Review of Distance Education, 17(1), 23–32.

Whipple, N., Bernier, A., & Mageau, G. A. (2011). Broadening the study of infant security of attachment: Maternal autonomy‐support in the context of infant exploration. Social Development, 20(1), 17–32. https://doi.org/10.1111/j.1467-9507.2010.00574.x

World Health Organization [WHO], United Nations Children’s Fund [UNICEF], & World Bank Group. (2018). Nurturing care for early childhood development: A framework for helping children survive and thrive to transform health and human potential. https://apps.who.int/iris/bitstream/handle/10665/272603/9789241514064-eng.pdf

Published
2021-04-27
How to Cite
Rodrigues, M., Schneider, A., Sokolovic, N., Brunsek, A., Oré , B., Perlman, M., & Jenkins, J. (2021). Development and Evaluation of an Open-Source, Online Training for the Measurement of Adult-Child Responsivity at Home and in Early Childhood Education and Care Settings. The International Review of Research in Open and Distributed Learning, 22(3), 1-18. https://doi.org/10.19173/irrodl.v22i3.5472
Section
Research Articles