Development and Evaluation of an Open-Source, Online Training for the Measurement of Adult-Child Responsivity at Home and in Early Childhood Education and Care Settings

  • Michelle Rodrigues University of Toronto
  • Alessandra Schneider University of Toronto
  • Nina Sokolovic University of Toronto
  • Ashley Brunsek University of Toronto
  • Beatriz Oré NGO Alto Perú and Independent Consultant
  • Michal Perlman University of Toronto
  • Jennifer Jenkins University of Toronto
Keywords: responsive caregiving, parental sensitivity, online learning, observational measurement, low- and middle-income countries


Efforts to monitor and improve responsive caregiving for young children, because of its importance for child development, are part of the United Nations Sustainable Development Goals. Two brief observational measures of responsive caregiving have been developed and validated (Responsive Interactions for Learning—parent [RIFL-P] and educator [RIFL-Ed] versions), with the RIFL-P available in English, Portuguese, and Spanish. The aim of the current study was to present and evaluate two online training programs for the RIFL measures. These distance learning courses were designed as open-source and asynchronous to enable their use in low- and middle-income countries and remote areas. The following course components are used: readings, lectures, observation of interactions on video, coding practice with automated feedback on item coding, and quizzes. Of the 76 trainees who registered for one of the online courses, 58 (76%) completed all theoretical module components. Student performance was generally high. Marks on quizzes ranged between 83%–100%. Ninety percent of those who took the reliability tests passed (40/44). Student satisfaction during and after the course was high. The effective online training programs are available free of charge and the RIFL suite of measures is efficient to implement. Implications for research and practice are discussed.


Bailey, H. N., Bernier, A., Bouvette‐Turcot, A. A., Tarabulsy, G. M., Pederson, D. R., & Becker‐Stoll, F. (2017). Deconstructing maternal sensitivity: Predictive relations to mother‐child attachment in home and laboratory settings. Social Development, 26(4), 679–693.

Barteit, S., Guzek, D., Jahn, A., Bärnighausen, T., Jorge, M. M., & Neuhann, F. (2020). Evaluation of e-learning for medical education in low-and middle-income countries: A systematic review. Computers & Education, 145, 103726.

Black, M. M., Walker, S. P., Fernald, L. C. H., Andersen, C. T., DiGirolamo, A. M., Lu, C., McCoy, D. C., Fink, G., Shawar, Y. R., Shiffman, J., Devercelli, A. E., Wodon, Q. T., Vargas-Barón, E., & Grantham-McGregor, S. (2017). Early childhood development coming of age: Science through the life course. The Lancet, 389(10064), 77–90.

Britto, P. R., Lye, S. J., Proulx, K., Yousafzai, A. K., Matthews, S. G., Vaivada, T., Perez-Escamilla, R., Rao, N., Ip, P., Fernald, L. C., MacMillan, H., Hanson, M., Wachs, T. D., Yao, H., Yoshikawa, H., Cerezo, A., Leckman, J. F., & Bhutta, Z. A. (2017). Nurturing care: Promoting early childhood development. The Lancet, 389(10064), 91–102.

Burgos, C., Campanario, M. L., de la Peña, D., Lara, J. A., Lizcano, D., & Martínez, M. A. (2018). Data mining for modeling students’ performance: A tutoring action plan to prevent academic dropout. Computers & Electrical Engineering, 66, 541–556.

Chang, T. P., Pham, P. K., Sobolewski, B., Doughty, C. B., Jamal, N., Kwan, K. Y, Little, K., Brenkert, T. E., & Mathison, D. J. (2014). Pediatric emergency medicine asynchronous e‐learning: A multicenter randomized controlled Solomon four‐group study. Academic Emergency Medicine, 21(8), 912–919.

Davis, D., Chen, G., Hauff, C., & Houben, G. J. (2018). Activating learning at scale: A review of innovations in online learning strategies. Computers & Education, 125, 327–344.

Gladwell, M. (2005). Blink: The power of thinking without thinking. Little, Brown and Company.

González, A., Fernández, M. V. C., & Paoloni, P. V. (2017). Hope and anxiety in physics class: Exploring their motivational antecedents and influence on metacognition and performance. Journal of Research in Science Teaching, 54(5), 558–585.

Hallal, P. C., Bertoldi, A. D., Domingues, M. R., da Silveira, M. F., Demarco, F. F., da Silva, I. C., Barros, F. C., Victora, C. G., & Bassani, D. G. (2018). Cohort profile: The 2015 Pelotas (Brazil) birth cohort study. International Journal of Epidemiology, 47(4), 1048–1048h.

Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.

Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educause Quarterly, 31(4), 51–55.

Jeong, J., Franchett, E. E., Ramos de Oliveira, C. V., Rehmani, K., & Yousafzai, A. K. (2021). Parenting interventions to promote early child development in the first three years of life: A global systematic review and meta-analysis. PLoS Medicine, 18(5), e1003602–e1003602.

Jeong, J., Pitchik, H., & Yousafzai, A. (2018). Stimulation interventions and parenting in low- and middle-income countries: A meta-analysis. Pediatrics, 141(4), e20173510.

Khalil, H., & Ebner, M. (2014). MOOCs completion rates and possible methods to improve retention: A literature review. In J. Viteli & M. Leikomaa (Eds.), Proceedings of EdMedia 2014--World Conference on Educational Media and Technology (pp. 1305–1313). Association for the Advancement of Computing in Education (AACE).

La Paro, K. M., Hamre, B. C., & Pianta, R. C. (2009). The classroom assessment scoring system, Toddler version. Teachstone.

Lotzin, A., Lu, X., Kriston, L., Schiborr, J., Musal, T., Romer, G., & Ramsauer, B. (2015). Observational tools for measuring parent–infant interaction: A systematic review. Clinical Child and Family Psychology Review, 18(2), 99–132.

Madigan, S., Prime, H., Graham, S., Rodrigues, M., Anderson, N., Khoury, J., & Jenkins, J. M. (2019). Parenting behavior and child language: A meta-analysis. Pediatrics, 144(4), e20183556.

Matias, C., O’Connor, T. G., Futh, A., & Scott, S. (2014). Observational attachment theory-based parenting measures predict children’s attachment narratives independently from social learning theory-based measures. Attachment & Human Development, 16(1), 77–92.

Means, B., Toyama, Y., Murphy, R, Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. U.S. Department of Education, Office of Planning, Evaluation, and Policy Development.

Molin, F., Haelermans, C., Cabus, S., & Groot, W. (2020). The effect of feedback on metacognition: A randomized experiment using polling technology. Computers & Education, 152, 103885.

Paton, R. M., Fluck, A. E., & Scanlan, J. D. (2018). Engagement and retention in VET MOOCs and online courses: A systematic review of literature from 2013 to 2017. Computers & Education, 125, 191–201.

Pauker, S., Perlman, M., Prime, H., & Jenkins, J. (2018). Caregiver cognitive sensitivity: Measure development and validation in Early Childhood Education and Care (ECEC) settings. Early Childhood Research Quarterly, 45, 45–57.

Pederson, D. R., Moran, G., Sitko, C., Campbell, K., Ghesquire, K., & Acton, H. (1990). Maternal sensitivity and the security of infant‐mother attachment: AQ‐sort study. Child Development, 61(6), 1974–1983.

Pierce, H. (2021). Nurturing care for early childhood development: Path to improving child outcomes in Africa. Population Research and Policy Review, 40(2), 285–307.

Prime, H., Browne, D. T., Akbari, E., Wade, M., Madigan, S., & Jenkins, J. M. (2015). The development of a measure of maternal cognitive sensitivity appropriate for use in primary care health settings. Journal of Child Psychology and Psychiatry, 56(4), 488–495.

Prime, H., Pauker, S., Plamondon, A., Perlman, M., & Jenkins, J. M. (2014b). Sibship size, sibling cognitive sensitivity, and children’s receptive vocabulary. Pediatrics, 133(2), e394–e401.

Prime, H., Perlman, M., Tackett, J. L., & Jenkins, J. M. (2014a). Cognitive sensitivity in sibling interactions: Development of the construct and comparison of two coding methodologies. Early Education and Development, 25(2), 240–258.

Prusak, K., Dye, B., Graham, C., & Graser, S. (2010). Reliability of pre-service physical education teachers’ coding of teaching videos using studiocode analysis software. Journal of Technology and Teacher Education, 18(1), 131–159.

Roggman, L. A., Cook, G. A., Innocenti, M. S., Jump Norman, V., & Christiansen, K. (2013). Parenting interactions with children: Checklist of observations linked to outcomes (PICCOLO) in diverse ethnic groups. Infant Mental Health Journal, 34(4), 290–306.

Santos, I., Munhoz, T., Barcelos, R., Blumenberg, C., Bortolotto, C., Matijasevich, A., Santos, H. G. dos, Jr., Marques, L., Correia, L., Souza, M., Lira, P., Pereira, V., Botelho, V., Silva, R., Chicaro, M., & Victora, C. (2020). Estudo de linha de base da avaliação de impacto do Programa Criança Feliz [Baseline study of the impact evaluation of the Criança Feliz program]. Ministério da Cidadania, Secretaria de Avaliação e Gestão da Informação.

Scherer, E., Hagaman, A., Chung, E., Rahman, A., O’Donnell, K., & Maselko, J. (2019). The relationship between responsive caregiving and child outcomes: Evidence from direct observations of mother-child dyads in Pakistan. BMC Public Health, 19(1), 252.

Schneider, A., Rodrigues, M., Falenchuk, O., Munhoz, T. N., Barros, A. J., Murray, J., Domingues, M. R., & Jenkins, J. M. (2021). Cross-cultural adaptation and validation of the Brazilian Portuguese version of an observational measure for parent–child responsive caregiving. International Journal of Environmental Research and Public Health, 18(3), 1246.

Sokolovic, N., Borairi, S., Rodrigues, M., Perlman, M., & Jenkins, J. M. (2021a). Validating an efficient measure of responsivity in father–child interactions. Canadian Journal of Behavioral Science/Revue canadienne des sciences du comportement, 53(1), 84–89.

Sokolovic, N., Brunsek, A., Rodrigues, M., Borairi, S., Jenkins, J. M., & Perlman, M. (2021b). Assessing quality quickly: Validation of the Responsive Interactions for Learning - Educator (RIFL-Ed.) measure. Early Education and Development, 1–16.

Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research, and Evaluation, 9(1), 4.

United Nations Children’s Fund [UNICEF] & Countdown to 2030. (2020, November 21). Thrive. Nurturing care for early childhood development: Country profiles for early childhood development.

Vermeer, H. J., van IJzendoorn, M. H., Cárcamo, R. A., & Harrison, L. J. (2016). Quality of child care using the environment rating scales: A meta-analysis of international studies. International Journal of Early Childhood, 48(1), 33–60.

Watts, L. (2016). Synchronous and asynchronous communication in distance learning: A review of the literature. Quarterly Review of Distance Education, 17(1), 23–32.

Whipple, N., Bernier, A., & Mageau, G. A. (2011). Broadening the study of infant security of attachment: Maternal autonomy‐support in the context of infant exploration. Social Development, 20(1), 17–32.

World Health Organization [WHO], United Nations Children’s Fund [UNICEF], & World Bank Group. (2018). Nurturing care for early childhood development: A framework for helping children survive and thrive to transform health and human potential.

How to Cite
Rodrigues, M., Schneider, A., Sokolovic, N., Brunsek, A., Oré , B., Perlman, M., & Jenkins, J. (2021). Development and Evaluation of an Open-Source, Online Training for the Measurement of Adult-Child Responsivity at Home and in Early Childhood Education and Care Settings. The International Review of Research in Open and Distributed Learning, 22(3), 1-18.
Research Articles