Volume 18, Number 5
Dr Grace Clifton
The Open University, United Kingdom
This paper evaluates the implementation of Learning Design on the production of a core FHEQ level 6 (QAA, 2008)1 unit of study at a UK distance learning institution. By comparing student (n=656) and tutor (n=42) survey data with questionnaire responses (n=9) from the unit of study's core production team, this paper assesses the impact of incorporating the Open University Learning Design Initiative (OULDI) methodology into curriculum production by looking specifically at barriers and facilitators in the application of Learning Design and its impact on module development, delivery, and the resulting student and tutor learning experience. With a focus on developing and embedding Learning Design into the curriculum planning and production process, the paper explores how lessons learned from this experience have helped to guide and inform the future implementation of Learning Design into module and qualification level frameworks.
Keywords: learning design, curriculum production, student learning experience, distance learning
Learning Design (LD) is the practice of planning, sequencing, and managing learning activities, usually using ICT-based tools to support both design and delivery. The aim of Learning Design is to describe pedagogic intentions in a particular learning context; providing "a broad picture of a series of planned pedagogical actions rather than detailed accounts of a particular instructional event" (Lockyer, Heathcote, & Dawson, 2013, p. 1442).
Literature offers a number of different perspectives on what is meant by the term "Learning Design." From one perspective, it refers to a research methodology, which "seeks to develop a descriptive framework for teaching and learning activities and to explore how this framework can assist educators to share and adopt great teaching ideas" (Dalziel et al., 2013, p.5). A second perspective is that of an educational practice methodology - a systematic approach to the development of a unit of learning that supports the more effective integration of technologies for learning. Here the focus on "design" is seen as significant because it re-frames the scholarship of teaching in response to a context of new and rapidly changing pedagogies (Beetham & Sharpe, 2007). The third perspective sees LD as a change process mechanism. That is, an enabler for restructuring and informing institutional curriculum planning practices (Anderson, Bullen, Alltree, & Thornton, 2008; Salmon, Jones, & Armellini, 2008).
In contrast to the field of Instructional Design, which draws primarily on cognitivist learning theories (Dick & Carey, 1996; Gagné & Driscoll, 1988) the field of LD is firmly positioned within a tradition of socio-cultural educational research. For example, LD references concepts such as Activity Theory (Vygotsky, 1978; Engeström, 2015) in the notion of "mediating artefacts" for learning and design (Conole, Dyke, Oliver, & Seale, 2004). LD research also emphasises the importance of the social and contextual aspects of a student's learning experience (Laurillard, 2002; Oliver, Harper, Hedberg, Wills, & Agostinho, 2002; Conole, 2013). This emphasis is in line with a movement over the last 20 years in UK and Australian educational discourse towards more social, contextual and distributed models of learning - away from an "acquisition" metaphor towards a "participation" metaphor - where knowledge or "knowing about" is considered to be an inherently situated activity (Sfard, 1998).
The OU is the largest provider of distance-learning, part-time undergraduate and postgraduate study in the United Kingdom, for which students may register without formal entry qualifications. The OU's pedagogical approach is somewhat different to face to face institutions. We describe this teaching method as "blended learning" (Bonk & Graham, 2006) in that it can take place in a face-to-face environment (through regular tutorials) but also at a distance (through on-line tutorials, forums, and one-to-one telephone calls). Associate Lecturers (ALs) are employed by the OU to deliver and teach at a distance the materials written by central academic teams (module teams) for students. There are increased expectations for central academics to play extended roles within the modules (or "chunks" of study) they have created - for example, delivering podcasts, moderating student conferences, and monitoring AL marking and feedback. Although we have a physical headquarters in Milton Keynes, the reality is that our students and ALs can be located anywhere in the world.
LD is particularly pertinent to the distance learning context for two reasons. First, it is important that a broad framework for the structure and pedagogy of curriculum can be clearly transmitted to ALs and students, learning, and working at a distance. Second, it is more difficult for a distance learning provider to identify and fix problems in curriculum materials when a course is underway as print materials will already be in use by students and ALs. LD provides the module team with a way of identifying potential issues prior to materials being released, by using module maps and activity planners to visualise the likely teaching and learning experience (Galley, 2012).
This paper evaluates a project trialling a new module design process which " frontloaded" the LD activity in advance of the start of module production on a popular and well regarded 60 credit module EK310 Research with Children and Young People into a new module EK313 Issues in Research with Children and Young People. This is a compulsory level 6 module (equivalent to the last year of undergraduate study) across four different qualification pathways and draws on students from a range of different discipline backgrounds, including Childhood and Youth Studies, Early Years, Youth Justice, and Primary Studies. In particular, this pilot sought to develop an approach which provided greater emphasis on pedagogy and support for LD, and that would lead to the design and development of a module which met the aims of the Faculty of Education and Language Studies' own Learning and Teaching Strategy by:
Two, three hour LD workshops were developed and delivered using the OULDI LD methodology (Cross, Galley, Brasher & Weller, 2012; Galley, 2012). The OULDI methodology makes use of five distinct tools to enable module production: the use of Cloudworks to provide a space for users to discuss production activity, an activity/pedagogy profiling tool, a module mapping tool, a set of Course Features and Information Literacies cards to facilitate and promote discussions about design features and, finally, the development of software CompendiumLD which produced visual representations and conceptual designs of the emerging module design. These LD workshops were designed to sit alongside regular module team meetings where administration, marketing, content and, eventually, the critical appraisal of emerging curriculum material were considered. EK313 was first made available to students in February 2014.
Unlike other evaluations of LD (Britain, 2004; Conole et al. 2004), this paper is written from the perspective of an academic team member of the module being produced. This is significant in that it is the academic team, not the LD team, at the OU which have onward responsibility for the quality assurance (QA) and student experience of the module. This paper draws on data from the QA process in order to provide a deeper and richer understanding of the impact of LD on the teaching and learning experience.
In order to evaluate the impact of the LD process, three separate data sets were employed. First, open ended questionnaires (n=9) were circulated to the academic and academic-related team as well as the Faculty's Associate Dean (Learning & Teaching). Second, to consider the impact of LD on the final module "product," data resulting from the module's post launch (quality assurance) review, were reviewed. This included student (n=656) and AL (n=42) survey data (a mixture of closed and open questions) to identify how key LD design principles had an impact on the learning and teaching experience. Finally, data were compared with the student survey data taken from the module's predecessor, EK310, which had not been through the LD process.
The data were analysed around four emerging broad "impact" themes which closely aligned with the Faculty's own Learning and Teaching Strategy as outlined in section 3: emphasis and support in the design phase for pedagogy and learning design, increased proportion of student activity on the VLE, increased proportion of active student learning, and integration of student digital literacy skills.
LD helped to promote a coherent structural curriculum design for EK313 and led to the production of both the Module Map (see Figure 1) and Activity Planner (see Figure 2) to support and guide production activities. These provided a visual representation of the planned learning and teaching experience for the new module, setting out the tools and resources for use by students and tutors, and an indication of student workload. Second, it was felt that the two design workshops brought about greater collaboration among members of the module team compared to module production experiences on other modules where LD had not been used. It was felt that the design representations mediated design discussion and promoted collaboration and sharing:
It's helpful to have people from different backgrounds, different strengths getting together to work something through [...] the [learning design] process helped them work together because they were all looking at the same things. (Associate Dean, Learning and Teaching)
There was some indication that frontloading these design workshops at the start of the process only might create difficulties in maintaining a strong pedagogical focus on student engagement throughout later development, as the module team chair clearly recognised:
I think there's a danger that you do the thinking really early on and that is productive and then that thinking is not threaded through the future decisions and you go through [...] as the pace increases, how can we keep that pedagogical focus there? (Academic lead, EK313)
Data suggested that the student demographic information could have been considered more thoroughly during the LD process and that this may have helped the academic team to produce material more relevant to all students taking the module: "This course is mandatory for the Youth degree and I felt it did not fit in with my aims as per the work based courses" (Youth Justice degree student).
Figure 1. Module Map for new planned module.
Figure 2. Activity Planner for the new planned module.
The initial LD workshops encouraged the module team members to map out curriculum content using the Module Map and Activity Planner which show more integrated use of VLE tools such as the block review forums and glossary, and other synchronous and asynchronous tools, for example OU Live (our on-line tutorial software). The extent to which this level of online study was recognised by students varied. For example, there was a clear recognition for the way in which the reader chapters and online materials allowed students to consolidate their knowledge and understanding of core themes in EK313: "All aspects seemed interconnected so were relevant to gaining more understanding," and "I found working my way through all the online learning activities was interesting and really helped to consolidate my learning." (Student responses). Similar benefits to supporting, and engaging in, online learning were recognised by the ALs: "The online material is very well written and clear for students," and "Given that EK313 has mostly online materials, I have certainly found this second presentation more user friendly - probably more to do with continuing to share the study materials and discussions re access and content with the students". (AL responses). Despite a recognition for the overall quality of materials and the value of online delivery, the focus on online study was often unexpected for many students, as, up to that point, students had completed modules within previous qualification frameworks which relied heavily on a traditional printed format: "The fact that everything was on-line except one reader signalled a significant change for me, and took some getting used to." (Student response)
Through the design workshops, the module team developed a very clear focus for improving active engagement through the development of the Activity Planner that emphasised students' active involvement in online discussion/review forums, analysing information, and dealing with real-life examples. However, this focus on active learning was not entirely translated into the module produced. For example, the original discussions and mapping of the module at the early LD workshops focussed on creativity and experiential learning, whereas the revised planner submitted towards the end of the production process suggested there were fewer contact hours and there had been a move back towards more assimilative types of activities.
One way to support productive activities in the Activity Planner was to promote student collaboration and communication through tutor group forum activities and "end of block" review fora. In essence, this was where students created something concrete online which could then receive feedback from the AL in order to feed into the preparation of their next assignment. However, the importance of jointly constructing forum threads were not realised by students and ALs. This aspect of the LD did not work as originally intended, despite the review fora having a strong pedagogic value for teaching and learning:
On line forums - lack of use by other students made participating in activities suggested in the modules frustrating. In the end I stopped using both the module and tutor group forums which was disappointing. (AL)
During the two initial LD workshops, the module team focussed heavily on designing online activities that supported students in acquiring a range of Level 6 academic skills including critical reviews, carrying out online literature searches, and synthesising research evidence. The module team members perceived that the LD activities enabled the module team to embed digital and information literacy into the existing module: "I think things like setting out the Module Map [...] has made us more aware of what the existing material was trying to do in terms of the different skills including Information Literacy" (Academic module team member).
The LD activities, however, also revealed the module teams concerns about their own lack of skills or experience in designing online activities and embedding these into the curriculum. For students, it was not the nature of online study specifically, but the development of study skills that caused some tensions. For example, feedback from ALs clearly recognised that acquiring these study skills represented a key challenge for many students, particularly those who struggled with basic reading and writing skills:
If students have not the time to study they won't do it and if they have not the basic skills required at level [6] (reading, writing, extracting the main points from a text) they will have difficulties with everything. (AL)
The findings clearly demonstrate a disconnect between the successful process of incorporating LD into the design of the module (as seen in the interview data with module staff) and the actual learning and teaching experience of the module (as seen in the data from students and ALs). It appears, therefore, that although the module was built on strong pedagogical principles, including shifting from a transmission to a participatory model, the final product did not fully reflect the original design plan. Some of these innovative design features were not recognised or experienced by the ALs or students. This might be because the original design plan was not referred back sufficiently during production so the module delivery did not fully reflect the design plan as initially intended. In this sense, it is clear that the actual experience of LD on EK313 is somewhat at odds with the perspective that LD should be seen as a change process mechanism (Salmon et al., 2008; Anderson et al., 2008) as, for EK313, LD was not a continual process but a series of one-off events.
The LD process did not include a consideration of broader student information (e.g., Socio-economic status, educational background, prior study) which would have helped to focus the LD workshops onto who were the students who were going to study the module and, therefore, what their needs might be. As such, the process did not take into account some of the key social and contextual aspects of the student learning experience (Conole, 2013; Laurillard, 2002; Oliver et al., 2002). Similarly, the LD workshop did not consider the module within the context of all the different qualifications it contributed to. As a new module within relatively "old" qualification pathways, some of the more innovative aspects of teaching promoted by the LD process were at odds with some of the more "traditional" teaching methods that went on in the modules that students would have studied prior to EK313. Potentially, these early learning design workshops could start at a qualification rather than module level to ensure consistency in student learning journey across the entire qualification pathway.
ALs were involved in the critical reading of materials and a group of students were employed to test two units of materials. Crucially this input was accessed after the module had been designed and written, so these important stakeholders were not involved in the actual design of the module. The rationale for decisions made in relation to pedagogy and design were explored within the LD workshops were not sufficiently communicated to ALs and, therefore, an opportunity to benefit from some of the rich and professionally relevant feedback from ALs was missed. In this way, the relationship between the module's design, pedagogy, and the student and tutor experience were not fully explored or appreciated.
The LD process mapped learning design only. It did not sufficiently focus on the FHEQ level of the module and how and in what way the pedagogical design could have enabled students working at level 6. In this way, it could be argued that the LD practice methodology was not systematic enough in terms of module development (Beetham & Sharpe, 2007) and the related requirements of supporting the needs of students registered on different qualification pathways.
This LD pilot has influenced the way that LD continues to be refined, developed, and implemented across the whole of the OU. As a result of recommendations made from the EK313 evaluation, the University has now moved to a process whereby the OULDI LD is compulsory for all new curriculum production. In addition, the new Technology Enhanced Learning (TEL) team work alongside the academic team throughout production, not just at the start, in order to help the team to reflect on the nature and level of communicative activities adopted. This insures that the final curriculum product can be compared to its original intention, whilst maintaining flexibility to make changes to the product where it is felt that there is a strong pedagogical need to do so. In addition, the University is moving towards incorporating the student view within LD at a qualification level as part of its on-going student consultative work.
1. Frameworks for Higher Education Qualifications in England, Wales and Northern Ireland, produced by the Quality Assurance Agency for Higher Education (2008) - this framework provides a reference point for all Higher Education providers for the setting and assessing of academic standards.
The author gratefully acknowledges the contribution of Dr Lee Sheriston and Rebecca Galley in carrying out this project and writing this paper.
Anderson, I. Bullen, P. Alltree, J., & Thornton, H. (2008). CABLE: An approach to embedding blended learning in the curricula and across the institution. Reflecting Education, 4(1), pp 30-41.
Beetham, H., & Sharpe, R. (2007). Rethinking pedagogy for a digital age. London: Routledge Falmer.
Bonk, G., & Graham, C. (Eds.) (2006). The handbook of blended learning: Global perspectives, local designs. San Fransisco: Wiley.
Britain, S. (2004). A review of learning design: Concept, specifications and tools. A report for the JISC E-learning Pedagogy Programme [Electronic document]. Retrieved from https://staff.blog.ui.ac.id/harrybs/files/2008/10/learningdesigntoolsfinalreport.pdf
Conole, G. (2013). Designing for learning in an open world. New York: Springer.
Conole, G., Dyke, M., Oliver, M., & Seale, J. (2004). Mapping pedagogy and tools for effective learning design. Computers & Education, 43, p. 17-33.
Cross, S. Galley, R. Brasher, A., & Weller, M. (2012). OULDI-JISC project evaluation report: The impact of new curriculum design tools and approaches on institutional process and design cultures [Electronic document]. Retrieved from http://www.open.ac.uk/blogs/OULDI/?page_id=113
Dalziel, J., Conole, G., Wills, S., Walker, S., Bennett, S. Dobozy, E., ... Bower, M. (2013). The larnaca declaration on learning design - 2013. Retrieved from http://www.larnacadeclaration.org
Dick, W., & Carey, L. (1996). The systematic design of instruction (4th ed.). New York, NY: Harper Collins.
Engeström, Y. (2015). Learning by expanding: An activity-theoretical approach to developmental research (2nd ed). Cambridge, MA: Cambridge University Press.
Galley, R. (2012). Using a learning design approach to supporting evolving design practices at the OU [Electronic document].Retrieved from http://www.open.ac.uk/blogs/OULDI/wp-content/uploads/2012/07/Support_pilot_Final.pdf
Gagné, R., & Driscoll, M. (1988). Essentials of learning for instruction. Englewood Cliffs, NJ: Prentice-Hall.
Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies (2nd Ed.). London: Routledge Falmer.
Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action aligning learning analytics with learning design. American Behavioral Scientist, 57(10), pp1439-1459.
Oliver, R., Harper, B., Hedberg, J., Wills, S., & Agostinho, S. (2002, July 7-10). Formalising the description of learning designs. Proceedings of the 25th HERDSA Annual Conference, Quality Conversations (pp 496-504), Perth, Western Australia. Retrieved from http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=4903&context=ecuworks
Quality Assurance Agency for Higher Education. (2008). Frameworks for higher education qualifications in England, Wales and Northern Ireland. Retrieved from http://www.qaa.ac.uk
Salmon, G., Jones, S., & Armellini, A. (2008). Building institutional capability in e-learning design. Research in Learning Technology, 16(2), pp 95-109.
Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27(2).
Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
An Evaluation of the Impact of "Learning Design" on the Distance Learning and Teaching Experience by Dr Grace Clifton is licensed under a Creative Commons Attribution 4.0 International License.