Using educational design research to inform teaching and learning in the health professions

Abstract

Teaching has always been at the core of what it means to practice in the health professions. Health professionals generally accept that as part of their role they will be involved in educating future generations in their discipline. However, whilst health professional educators typically have extensive knowledge and skills in their discipline areas, the art and science of teaching, particularly in the clinical setting, is often overlooked in the context of competing priorities. Currently, there is no formal requirement in Australia that health professional educators demonstrate teaching competency. Indeed, teaching pre and post qualification students often goes unrewarded and unsupported, thus suggesting that it is an undervalued element of the health professional educator's role. Furthermore, time-poor health professionals have limited opportunities to attend educational courses and workshops away from their workplaces. This paper discusses how educational design research influenced a hybrid mobile-web learning solution for providing educational professional development to health professional educators. It is argued that educational design research provides a rigorous framework in which to ground educational design and development processes, particularly when dealing with embryonic and unproven learning technologies.

Background

One of the strategic goals of the University of Notre Dame Australia (UNDA) is to provide educational professional development (EPD) and support to academic staff to enable them to enhance their teaching skills. As such, UNDA, like many universities, offers a Graduate Certificate in University Teaching (GCUT). This course is well attended by staff across multiple disciplines, although most participants tend to be those who teach on campus as opposed to those who teach in workplace-based settings. Whilst staff from the medical school, and other health professional faculties, have participated in the GCUT, no hospital-based medical teachers have enrolled to date.

In an effort to gain more insight into this issue, Bate and Steketee (2010) examined the EPD needs of medical educators. They found that the traditional delivery mode of courses such as the GCUT, were inadequate in that they did not take into account the complex and busy nature of working in health settings. It was also found that medical doctors were open to enrolling in higher degree programs in order to attain qualifications in medical education, particularly if the objectives of the course were aligned with their specific needs of teaching in a clinical, workplace-based setting. An environmental scan of other health professional educators' needs across the University revealed similar sentiments.

These findings were the impetus behind the development of two new flexible postgraduate courses in health professional education (HPE). The Graduate Diploma and Masters of HPE, which together comprise six units plus a dissertation, were conceived to build upon the generic educational objectives of the GCUT and to focus educational theory more specifically on the needs of teaching and learning in health contexts. However, the course developers recognised that, while providing flexible access might encourage a greater up-take of the courses, flexible access in itself would not ensure a rich, relevant and meaningful learning experience for participants. The courses would need to be designed to accommodate the multifarious disciplines that comprise the health professions such that all participants could apply concepts to their varied educational needs. This was crucial if EPD was to be embraced and valued as integral to being a health professional educator. Additionally, the course developers wanted to take full advantage of new learning technologies as a means of augmenting flexible access to the courses and providing a richer flexible learning experience.

Educational design research offered a promising framework by which to address the complexity of this problem in a systematic and rigorous way. This exploratory methodology acknowledges that when employing technological solutions to teaching and learning problems, the technology can become the primary focus, often to the detriment of the quality of the teaching and learning outcome. Instead, educational design research proposes that the focus must be on the meaningful impact any solution might have on the quality of the teaching and learning outcome (Reeves, McKenney & Herrington, 2011). While new learning technologies would play an important role in providing flexible EPD to health professional educators, these technologies were seen as being more than just a vehicle through which participants could access quality learning experiences. Just what 'quality learning experiences' entailed, however, was initially unclear. Educational design research provided a means through which design, research and practice could evolve concurrently (Wang & Hannafin, 2005) without losing sight of the overall goal of providing practical and constructive EPD to health professional educators.

Educational design research

Educational design research, also known as 'design-based research' (Reeves et al, 2011) is grounded in the belief that useful educational research cannot be disconnected from the challenges and complexities of everyday practice. It must emanate from and "speak directly to problems of practice" (National Research Council, cited in The Design-Based Research Collective, 2003, p.5) if it is to generate useful, useable knowledge and to make any valuable impact on educational outcomes. Contextual influences inform research and therefore cannot be separated from it. This means that for research such as this that aims to 'test' designs, it must do so in context.

Often, instructional design processes and research are two separate activities. Research can be conducted after instructional design processes are completed in order to test the design's effectiveness, rather than to confront issues of educational practice (Wang & Hannafin, 2005). This distinguishes educational design research from action research whereby an intervention is designed, implemented then tested. In educational design research, the design emerges through the research process. "In effect, design is embodied in research, and research is embodied in design" (Wang & Hannafin, 2005, p.13).

In this regard an emphasis is placed on an iterative and cyclical process, whereby data emerging from each iteration not only serves to refine the design but also informs the principles upon which it is based. These emergent design principles can subsequently "guide similar research and development endeavours" (Amiel & Reeves, 2008, p.35). Similarly, Barab and Squire (2004, p. 2) note the advantages of educational design research as a tool to both guide and impact upon teaching and learning:

[Educational design] research is not so much an approach as it is a series of approaches, with the intent of producing new theories, artifacts, and practices that account for and potentially impact learning and teaching in naturalistic settings.

Reeves (2006) suggests four phases in the conduct of educational design research:

Analysis of practical problems.

Development of solutions informed by existing design principles and technological innovations.

Iterative cycles of testing and refinement.

Reflection to produce new design principles and enhance future implementation.

In an attempt to configure, understand and apply design principles that would engage health professional educators in EPD via flexible postgraduate courses, these four phases were used to guide the research in practice. The following sections describe these phases and the way in which they have facilitated the research to date.

Phase 1 - conceptualisation of the problem

In educational design research, the thorough conceptualisation of the problem is fundamental to identifying a plausible solution and to the ultimate success of the overall project. Articulation of the problem creates a purpose for the research. The creation of potential solutions forms the focus of the entire study (Herrington, McKenney, Reeves & Oliver, 2007).

Conceptualising the problem was an important step in the current research. Whilst it was evident that the problem centred on an inability to engage health professional educators in EPD, it was not fully understood why this was the case, nor how it could be resolved effectively. This vacuum in understanding crystallised after a series of attempts of trialling different methods and models of EPD, all of which met with limited success. It also emerged as a result of research into the EPD needs of medical educators at UNDA (Bate & Steketee, 2010). Consultations with medical graduates and teaching staff in the medicine, nursing, physiotherapy and counselling faculties gave further insight, and were instrumental in recalibrating the issue of EPD as one that extended beyond just medical educators to other health professional educators. As potential future 'students' of the courses, conversations with these individuals provided the researchers with a better appreciation of the multiple educational contexts that comprise health professional education and that a design that proposed a 'one-size-fits-all' approach would not work. Finally, an extensive literature review into the professional development of health professionals as educators gave clarity to the scope of the problem, as well as the challenges to be faced by the course developers in their attempts to address it.

As a result of this process, the following problem was posed:

Although health professionals have a professional duty to teach, very few are prepared for this aspect of their role. This is problematic given that from a very early point in their careers, they are required to train pre and / or post qualification health professional students. Many revert to how they were taught and adopt teaching strategies and styles that they experienced as trainees. Often, these methods are suboptimal and result in passive and sometimes stressful learning (MacDougall & Drummond, 2005). Despite this, health professional educators do not fully engage in EPD due to the fact that their complex and busy schedules leave little time for attendance at campus-based workshops and courses. Additionally, the generic educational focus of many of these courses does not appeal to the unique clinical workplace-based teaching challenges faced by health professional educators. Historically and culturally, the educational aspect of their roles is not highly valued.

Clarifying the problem gave the project overall focus and direction. It highlighted the problem as both a flexible access and flexible learning issue. Herrington et al (2007) suggest that many researchers begin by thinking of a solution - such as a technology-based intervention, an educational game, or a technology tool - before they consider the educational problem it could solve. In doing so, researchers run the risk of only partially addressing the problem or missing it altogether.

Distinguishing between flexible access and flexible learning was an important step in the evolution of the current research. Whilst flexible access posed the problem of making learning available to participants at a time and place which is convenient to them, the broader construct of flexible learning necessitates that students engage in a cognitively rich and authentic environment that lends itself to knowledge construction. This environment should be "fit for purpose" meaning that it is developed for, and with, its target audience.

Formulating the problem helped to shape the research questions that would guide the study:

Were the Graduate Diploma and Masters of HPE courses an effective means of providing EPD to health professional educators?

To what extent did the design of the courses address issues of flexibility in terms of access as well as meeting diverse learning needs?

Were the activities relevant and reflective of contemporary challenges faced by health professional educators?

Did the courses succeed in changing health professional educators' perceptions of EPD?

What aspects of the learning environment were most and least effective for the target audiences?

What level of impact has the course design had for health professionals taking part?

Phase 2 - development of the solution

With a clear understanding of the nature and scope of the problem firmly at hand, the next phase of educational design research is the development of a feasible solution. This typically involves wide consultation of literature that will provide a theoretical platform upon which the problem can be better understood and that will inform a solution that is anchored in scholarly principles. The Design-Based Research Collective (2003, p. 6) wrote, "[Solutions] embody specific theoretical claims about teaching and learning, and reflect a commitment to understanding the relationships among theory, designed artifacts and practice".

The course developers therefore conceived the solution as entailing the development and delivery of flexible, authentic and credentialed postgraduate courses thereby encouraging health professional educators to not only engage in EPD, but come to value it as an integral aspect of their roles. Through a review of the literature on adult learning theory and design principles for flexible learning environments, it was apparent that for the courses to be authentic, they needed to embody certain features that would support the construction of meaningful learning outcomes. Knowles' (1990) principles of andragogy suggest that, as adults, the participants in these courses would be more inclined to engage in the content if activities were purposeful and relevant to their specific learning needs and contexts. They would be motivated to learn if they could link new and existing knowledge and experiences, and if activities were problem-centred rather than subject-centred. Additionally, their learning would be enriched if they had opportunities to collaborate with other participants in the course such that they could share perspectives and co-construct understandings.

Knowles' (1990) assumptions about how adults learn best are closely aligned with contemporary design principles of e-learning environments as proposed by Herrington (2006). She argues that to ensure e-learning courses result in meaningful learning, they should engage learners in complex, authentic activities that involve problem solving and reflection. These authentic activities essentially are the vehicle through which learners engage with content and attempt to make sense of it by relating it to their own contexts and learning needs. For health professionals, e-learning environments could act as a mirror to their own professional practices helping them to see their practices differently and/or with more clarity. Assessment should also be authentic and integrate seamlessly with learning activities. Importantly, learning should be scaffolded, primarily by a teacher but also via collaboration with the other students and learning resources.

These findings from the literature influenced the development of a set of nine design principles that would inform and drive the instructional design of the Graduate Diploma and Masters of HPE. These design principles required a range of technical and logistical solutions which are detailed in Table 1. In keeping with the tenets of educational design research, it was acknowledged, however, that these nine principles could and should evolve over the period of the study.

Table 1: Design principles underpinning the Graduate Diploma and Masters of HPE, and proposed technical/logistical solutions.

Design principle	Description	Proposed technical/ logistical solution
Flexible access	Courses will be developed to enable participants to access activities anytime, anywhere.	Activities provided through web and mobile interfaces.
Flexible design	With the exception of the dissertation unit, all learning activities can be accessed at a module level. These modules can be consumed in any order, and participants may choose whether or not to be assessed.	Activities to be discrete and modularised and available on mobile devices.
Prior knowledge	Participants are encouraged to share their prior knowledge with their peers and/or the unit facilitator through the process of engaging with learning activities and contributing to the learning community. The application of existing knowledge and expertise to learning activities (e.g. by participants using artefacts such as lesson plans, marking guides, etc.) is equally encouraged.	Blog and e-portfolio functions activated in the UNDA learning management system (Blackboard).
Authentic activities	The central theme for each module is that learning will be contextualised and relevant to participants. Learning activities will have explicit meaning in at least one of the health professions, but will have application to other health professions. Learning activities will mirror real world practices, problems and dilemmas in health professional education.	Where possible activities to be context-free. Where this is not possible, activities to be developed in such a way as to optimise generalisability.
Community	Participants will be encouraged to share their perspectives through a learning community They will also be encouraged to 'collaborate' such that participants come to view each other as integral resources for pushing their thinking and learning to higher levels. Where numbers permit, participants will be offered opportunities to work on learning activities in small groups.	Blog function activated in Blackboard.
Facilitation	Educational expertise will be provided at the unit level to support, and provide feedback to participants as they work through activities that comprise each module. The unit facilitator will also be responsible for assessment.	Support provided via email, telephone, social networking tools and face to face.
Scaffolding	Participants will be provided with resources and tools that will help them to complete designated tasks. Where possible, these will be functional resources and tools used in health professional education.	Electronic resources provided through an online bibliographic management tool (RefWorks).
Purposeful assessment	Assessments emanate from learning activities. Each assessment will be designed so that the product of the assessment will be of use to the participant in her/his teaching role.	E-portfolio function activated in Blackboard.
Reflection	Participants will be encouraged to consider what and how they learnt and also share the products of their learning with others, being open to feedback and opportunities to improve their practice.	E-portfolio function activated in Blackboard.

As mentioned previously, it was acknowledged that building courses electronically for ubiquitous access would only partially address the requirement that they had to be flexible. Flexibility was needed also to address and cater for the context within which health professional educators find themselves. A hybrid mobile-web learning environment comprising three dimensions was conceived in response to this problem. Each of the three dimensions - mobile learning, web (through RefWorks) and a learning management system (Blackboard) - has a clearly defined function and purpose. The conceptual design is represented diagrammatically as Figure 1.

Figure 1: Conceptual design of the hybrid mobile-web learning environment

In a recent study of nursing students' use of mobile devices (Jamieson-Proctor, Albion, Redmond, Harris, Yuginovich, Maxwell, Fasso, Sander & Larkin, 2012), it was found that the lack of a defined role for the use of the device (in this case an iPod Touch) was a major impediment to the success of the initiative. For participants to gain optimum advantage from the technological affordances offered in the conceptual design of the hybrid mobile-web environment, each component needs to have an identified function and work seamlessly with other components. Each is an important cog in the overall educational design.

The rationale for using mobile learning technologies is integrally connected with the characteristics of the target audience for the Graduate Diploma and Masters of HPE. Many health professionals work across multiple sites and are familiar with using devices such as tablet computers and smart phones. These devices help them to (a) be contactable and (b) access large amounts of complex information. Mobile devices also present significant opportunities to facilitate learning in a way which previously has not been possible (Jamieson-Proctor et al, 2012). The use of e-books as a mechanism to mediate learning tasks was seen an attractive option for a number of reasons. Firstly, the EPUB (electronic publication) version 3.0 standard supports media formats such as video, audio, interactive elements (e.g. instantaneous feedback) and avatars (IDPF, 2013). EPUBs can then be opened using an e-reader on Android and Apple devices thus providing access to a rich multimedia environment. Another advantage of developing for an e-reader is that design products (e.g. Apple's i-Author) do not require a high level of technical competence, alleviating the need for expensive development. Finally, EPUBs can also be read on non-mobile devices meaning that tasks can be mediated in a truly flexible way.

The affordances offered through the EPUB 3.0 format can help to consolidate prior knowledge, promote learning through engagement with avatars that are introduced in simulated learning environments, and help to guide research through links to web-based resources and templates. In summary, using the EPUB standard provided a student-centred mobile learning environment that potentially could mediate tasks to busy health professionals in an authentic manner. Tasks are presented as guided research and are supported by web-based resources. An online bibliographic management tool, RefWorks, provided an interface for all supporting resources whether these are located in refereed journals, newspaper articles or YouTube video clips. This one point of reference provides a consistent interface for all guided research that is embedded in the mobile learning environment. It is acknowledged that mobile devices may have limitations for the creation and production of knowledge. Therefore, participants are invited to use tools of their choice before uploading the products of their work to their e-portfolio which is developed in Blackboard. This work is assessed formatively and summatively. Synchronous and asynchronous communication tools are also activated in Blackboard to provide facilitated support for course participants.

In summary, the proposed solution, therefore, was to develop two postgraduate courses in EPD for health professional educators. The design of these courses would be in keeping with contemporary theories of adult learning and influenced by principles of authentic e-learning environments. These courses would maximise the potential afforded by mobile learning devices but also adapt to regular web-based applications. A hybrid mobile-web learning environment would essentially be the platform upon which participants would engage in flexible, authentic and credentialed professional development opportunities.

Phase 3 - iterative cycles of testing and refinement

The third phase of educational design research involves the implementation of the proposed solution. In order to gain an in-depth insight into the effectiveness of the solution, implementation is iterative and occurs over at least two cycles where data is gathered and evaluated. The researchers, in collaboration with the participants, learn whether the solution is effective in practice or if it needs to be improved in some way. In support of this approach to educational research, Reeves (1999, p. 18) wrote "... research and evaluation efforts should be primarily developmental in nature ... the purpose of such enquiry should be to improve, not to prove".

In educational design research, the researchers and participants in the study work closely together to interpret and make sense of the data to emerge during the implementations of the proposed solution. As participants interact with the solution in practice, its limitations and design constraints are identified and used by the researcher to refine and redesign subsequent solutions. Consequently both parties are instrumental in advancing the "pragmatic and theoretical aims" (Wang & Hannafin, 2005, p. 6) of the study. This often leads to a blurring of the lines between designers, practitioners, researchers and participants in educational design research (Wang & Hannafin, 2005). In the context of this study, the researchers were also the practitioners (co-facilitators for the courses) which meant that they were at the same time designers and participants. The students taking the course constituted the other participants.

There will be two primary iterations for this project, both of which will coincide with the implementation of the first two units of the Graduate Diploma of HPE; one in Semester 1, 2013 which is presently underway and the second in Semester 2, 2013. Prior to these iterations, a pilot was conducted which involved two peers testing the semester one unit for usability and educational coherency. Feedback resulted in minor modifications to the design of the e-portfolio and RefWorks environments.

Data will be gathered from the students participating in the courses in the form of a post-unit questionnaire and a mid-semester focus group session (via the video-conferencing tools in the Blackboard learning management system). Students will also be invited to maintain a blog which they can share at the end of each unit. The types of questions they will be asked during these data collection sessions will focus on whether the design of the courses allowed for transferability of content across multiple health professional disciplines; whether it encouraged students to integrate new content with prior knowledge and experiences; if the activities and assessments were relevant, authentic and purposeful; if the resources mediated and supported higher order thinking; if the tools embedded in the design fostered collaboration and reflection, and the extent to which the hybrid mobile-web environment encouraged flexible access. The two co-facilitators for the courses, who as discussed are also the researchers, will also provide data in the form of self-reflective journals which will be kept for the duration of the two iterations of delivery.

The purpose of this data will be to test the design principles which constitute the proposed solution. Data analysis will be primarily an inductive process using a combination of Miles and Huberman's (1994) three step method of data reduction, data display and conclusion drawing, and Glaser and Strauss' (cited in Lincoln & Guba, 1985) constant comparative method. These methods will allow for thick description of the themes to emerge and, ultimately, conclusions to be drawn based on patterns that are evident in the themes.

Phase 4 - Reflection

Whilst the third phase of educational design research is focused on incremental refinement and improvement, Phase 4 involves explicitly confirming, and where necessary re-drafting, design principles to inform future implementation. Boud, Keogh and Walker (1995, p.19) described reflection as "a generic term for those intellectual and affective activities in which individuals engage to explore their experiences in order to lead to new understandings and appreciations". The researchers will draw upon Kolb and Fry's (1975) learning cycle, which provides a schema for integrating concrete experiences with observations and reflections to form and test new abstract concepts and generalisations.

In future educational design research iterations, re-calibrated design principles will be developed by embedding Kolb and Fry's (1975) learning cycle into the educational design research process. These re-calibrated design principles will also be disseminated and discussed with peers to help generate deeper insights and further improvement opportunities.

Discussion

The Design-Based Research Collective argue that the success of educational design research should ultimately be determined by the extent to which it can improve educational practice and outcomes (2003). In the context of this study, this will be measured in terms of the extent to which the design of the two postgraduate courses promoted effective EPD for health professional educators. As the study is currently in progress, the jury is still out on this front.

However, the benefits of using this methodology, without having yet completed the study, are already evident to the researchers. The process of engaging in educational design research and, immersion in its literature base, was instrumental in focusing the aims of the study on teaching and learning outcomes. Early conversations between the researchers tended to focus on how mobile technologies might address the issue of flexible access to the courses, such that more health professionals could participate, but through the process of clarifying the problem via the educational design research process, it became abundantly apparent that flexibility as it pertained to access was only a small part of the problem; a problem of utility and convenience only. Had the researchers not engaged in this process, it is possible that the focus would have remained on the affordances associated with mobile devices as tools to simply broaden access to the courses. Whilst useful, this outcome would not have addressed the real issue of how to engage health professional educators in meaningful EPD.

The benefits of using educational design research as a process of design and exploration can have scientific, practical and societal outputs (Herrington et al., 2007). Scientific outputs relate to the design principles that are expected to emerge as a result of the process. Because these principles are generally couched in thick, rich description of the context, the procedures of implementation and the outcomes, they can be applied as solutions to other similar settings. Herrington et al. (2007, p.4095) believe that design principles "contain substantive and procedural knowledge with comprehensive and accurate portrayal of the procedures, results and context, such that readers may determine which insights may be relevant to their own specific settings". Although the present study has not yet run its course, the foundational principles of design in the hybrid mobile-web environment have already been applied to other postgraduate courses currently under development. For example, given similarities in the target audience and requirements of flexible learning, developers of a Graduate Certificate in Tele-health have applied the principles from the present study to their own course which will also be delivered via a hybrid mobile-web platform. The Graduate Certificate in Tele-health aims to provide a skills-based training course targeting practice managers and administration staff working in medical practices to learn how to provide support to general practitioners through tele-health services. It is hoped that educational design research will also guide the ongoing improvement of this course.

Another type of output is the actual tangible product which has been developed as a result of the research. In this instance, the product of design will be two postgraduate courses in health professional education. Due to the systematic and rigorous process followed in their development, it is anticipated that the benefits will be far reaching. They will provide hundreds, if not thousands of health professional educators with authentic and engaging opportunities to become effective teachers of future health professionals. The indirect outcome of these courses is better patient care which, in effect, is a type of societal output of educational research design.

Whilst the nine foundational design principles have been integral to the initial design solution, it is accepted that they may be added to or changed as a result of the iterations of implementation. Although they have evolved as a result of consultation and immersion in the literature it is not until they are tested in practice that they will be fully understood. The actual features of the setting will no doubt influence the outcome, allowing insight into whether a design will be successful and why (or why not, as the case may be). In this regard, educational design research has been an invaluable mechanism by which to explore a complex problem but at the same time contribute to theories of teaching and learning.

Conclusions

Educating health professionals about best practices in teaching and learning is an important element in their overall development as professionals. Given that these individuals will be responsible for educating future generations of health professionals, it is necessary that they do so according to current thinking on how adults learn best. However, current approaches to EPD have been less than adequate, and there tends to be an assumption (particularly in clinical settings) that practitioners' experiences as learners translate readily into adequate teaching practices. This is not the case and a review of the literature has revealed that, unsupported, health professional educators can provide substandard learning experiences. Providing EPD to health professionals, however, is problematic given the complex and frequent demands on their expertise.

In trying to address this issue, two courses have been developed that take into consideration the cultural and historical nuances of training health professionals who have a teaching role. Using educational design research methodology, a hybrid mobile-web learning environment has been developed according to sound principles of instructional design. In following stages of educational design research methodology, as proposed by Reeves (2006), cycles and products have been reviewed regularly. Educational design research has, therefore, provided a systematic framework in which to conceptualise, re-think and re-engineer the design of the hybrid mobile-web learning environment. The approach encouraged course designers to develop an integrated system of learning, one which acknowledged the educational affordances of new learning technologies and articulated a clear role and purpose for each of the technologies adopted. It is evident that the educational design research methodology has ensured that designers either stay true to identified design principles or rigorously defend modifications.

References

Amiel, T. & Reeves, T. C. (2008). Design-based research and educational technology: Rethinking technology and the research agenda. Educational Technology & Society, 11(4), 29-40. http://www.ifets.info/journals/11_4/3.pdf

Barab, S. & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1-14. http://dx.doi.org/10.1207/s15327809jls1301_1

Bate, F. & Steketee, C. (2010). Understanding the pedagogical world-view of clinical teachers in Australia: An evidence-based approach. Paper presented at the European Association for Practitioner Research on Improving Learning (EAPRIL) Conference, Lisbon, Portugal. http://researchonline.nd.edu.au/cgi/viewcontent.cgi?article=1046&context=med_conference

Boud, D., Keogh, R. & Walker, D. (1995). Promoting reflection in learning: a model. In D. Boud, R. Keogh & D. Walker (Eds.), Reflection: Turning experience into learning (pp. 18-40). London: Kogan Page Ltd.

Herrington, J. (2006). Authentic e-learning in higher education: Design principles for authentic learning environments and tasks. World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2006, Chesapeake, VA, AACE. http://researchrepository.murdoch.edu.au/5247/1/Authentic_e%2Dlearning(authors).pdf

Herrington, J., McKenney, S., Reeves, T. & Oliver, R. (2007). Design-based research and doctoral students: Guidelines for preparing a dissertation proposal. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 4089-4097). Chesapeake, VA: AACE. http://researchrepository.murdoch.edu.au/6762/1/design_based_doctoral.pdf

IDPF (International Digital Publishing Forum) (2013). EPUB. http://idpf.org/epub

Jamieson-Proctor, R., Albion, P., Larkin, K., Redmond, P., Harris, J., Fasso, W., Sander , T. Yuginovich, T. & Maxwell, A. (2012). Learning at times and places chosen by the learner: Adapting to study with mobile digital devices. Paper presented at the Australian Computers in Education (ACEC) Conference. Perth, Western Australia. http://acec2012.acce.edu.au/sites/acec2012.acce.edu.au/files/proposal/83/DEHub_mLearning.pdf

Knowles, M. (1990). The adult learner: A neglected species (4th ed.). Houston: Gulf Publishing.

Kolb, D. A. & Fry, R. (1975). Towards an applied theory of experiential learning. In C. L. Cooper (Ed.), Theories of group processes (pp 33-58). London: John Wiley.

Lincoln, Y. & Guba, E. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.

MacDougall, J. & Drummond, M. J. (2005). The development of medical teachers: An enquiry into the learning histories of 10 experienced medical teachers. Medical Education, 39(12), 1213-1220. http://dx.doi.org/10.1111/j.1365-2929.2005.02335.x

Miles, M. B. & Huberman, A. M. (1984). Qualitative data analysis: A sourcebook of new methods. Thousand Oaks CA: Sage Publications.

RefWorks (undated). http://www.refworks.com/

Reeves, T. C. (1999). A research agenda for interactive learning in the new millennium. In P. Kommers & G. Richards (Eds.), World Conference on Educational Multimedia, Hypermedia and Telecommunications 1999 (pp. 93-99). London: Routledge. http://www.aace.org/conf/edmedia/99/reeves.htm

Reeves, T. C. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research (pp. 52-66). London: Routledge.

Reeves, T. C., McKenney, S. & Herrington, J. (2011). Publishing and perishing: The critical importance of educational design research. Australasian Journal of Educational Technology, 27(1), 55-65. http://www.ascilite.org.au/ajet/ajet27/reeves.html

The Design-Based Research Collective (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. http://www.designbasedresearch.org/reppubs/DBRC2003.pdf

Wang, F. & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23. http://dx.doi.org/10.1007/BF02504682