A Design Research Methodology for Online Learning Course Design (#edumooc) Posted on July 5, 2011 by Rebecca

For all the #edumooc'ers out there, I apologize that most of the references in this piece are of academic nature, and therefore are not in the public domain. Below are key excerpts from a synthesis paper I wrote about design research. I see now that the methodology that I propose would also easily apply to the development of online learning in fact, I actually use this methodology somewhat informally as I design and re-design various iterations of one of the online courses that I teach. If you find this useful, or plan to use this methodology, please leave me a comment. Thanks. A Design Research Methodology for Online Learning Course Design Wang and Hannafin define design-base research as a systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories [underline added] (2005, p. 6). Design principles are "aims and goals guiding design decisions that occur during the development life cycle" (Mariage, Vamderdonckt, & Pribeau, 2005, p.689). Design principles are generic and are not generally places in context. When context is added to a design principle, it becomes a design guideline. Design research is pragmatic: it provides a solution to a real-world educational problem (such as teaching terminology) and adds to educational theory in the form of design principles that can be used by educational practitioners (Brown, 1992, p. 143; Collins, 1992, p. 15). Depending on the context in which it is being used, design research may also be known as design studies, design experiments, development/developmental research, formative research, formative evaluation, or engineering research (van den Akker, Gravemeijer, McKenney, & Nieveen, 2006, p. 4). Since design research is intended to both solve a real world problem and to develop reusable design principles, it is well suited to help develop design theories for mobile learning. There are many approaches to design research all of which begin with a theory gathering or literature review phase and end with a reflection on both the design solution and the design research process. The middle part of the process the iterative cycles of analysis, design, development, and implementation very among the different approaches. Most online learning projects involve both instructional designers and software developers; therefore, a design research approach to online learning should take into account the process used by these practitioners. Specifically, an ADDIE-based instructional design approach and an Agile-based software development approach. For online learning (and mobile learning), I recommend a simplified four-phased design research approach that draws on the characteristics of the design research approaches presented in research (Ma and Harmon, 2009;Middleton, Gorard, Taylor, and Bannan-Ritland, 2008; Gravemeijer and Cobb, 2006) with instructional design methodologies and the agile software development method. I call this approach ASER (analyze, strategize, experiment, reflect). As illustrated below, the ASER process has four phases: 1) analyze real-world problem, 2) strategize design experiment, 3) conduct context specific design experiment, and 4) reflect on entire project.

In the analyze phase the researcher performs a detailed analysis of a real-world problem. When analyzing the problem, the researcher ensures that scope of the problem is general enough to warrant design research, and that the problem is indeed a design problem. This phase requires crafting a research question. The strategize phase requires the development of strategies to support the design research project. The researcher performs detailed literature reviews and chooses or develops frameworks based upon current theories. The researcher defines the strategies for the experiment phase, including identifying research methods and data collection methods, and identifying specific goals for each experiment cycle. In the strategy, the researcher defines the specific boundaries for the beginning and ending of cycles and the project as a whole. These boundaries provide the necessary scope for the additional team members (instructional designers and software developers). For example, the researcher may define the means of delivering the content to the devices not within the scope of the project, but the coding of the content itself within scope. In addition, the researcher defines the scope of each iteration small enough to permit multiple iterations of the experiment phase. During the strategize phase, additional team members perform their analysis. Instructional designers perform the analysis phase of ADDIE. Software developers perform the initial planning and architecture design portion of their process. The frameworks defined by the researcher are used as input to the instructional designers and the software developers and for their analysis. The experiment phase is the iterative phase in the ASER approach, clearly illustrated by the circular arrows in the diagram. The research team uses iterative cycles of design, develop, instructional experiment, and evaluate, with the evaluate step providing input to the next design stage and the first iteration acting as the feasibility test for the project. The four cycles align directly with the design, develop, implement, and evaluate phases in ADDIE. The iterative development and testing phases of the agile software development method also align. Each iteration has a specific goal, defined in the research design strategy, and there is specific software and instructional material developed to support that goal. Reflect is the final phase of the project. In this phase, all data collected throughout the iterative cycles is analyzed as a whole. The researcher seeks to extract design principles based upon the results of the design experiments. The software development team completes the final phases of their process, seeking to commercialize the results of their development. In addition to the analysis of the research results, an evaluation of the overall research process is performed and learnings are shared. References Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141178. doi:10.1207/s15327809jls0202_2 Collins, A. (1992). Towards a design science of education. In E. Scalon & T. O'Shea (Eds.), New Directions in Educational Technology (pp. 15-22). Berlin, Germany: Springer-Verlag.

Gravemeijer, K., & Cobb, P. (2006). Design research from a learning design perspective. In J. van den Akker, K. gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational Design Research (pp. 17-51). Abingdon, UK: Routledge. doi:http://www.routledge.com/ Ma, Y., & Harmon, S. W. (2009). A case study of design-based research for creating a vision prototype of a technology-based innovative learning environment. Journal of Interactive Learning Research, 20(1), 75-93. doi:http://www.aace.org/ Middleton, J., Gorard, S., Taylor, C., & Bannan-Ritland, B. (2008). The "Compleat" Design Experiment. In A. E. Kelly, R. A. Lesh, & J. Y. Baek (Eds.), Handbook of design research methods in education (pp. 21 46). New York: Routledge, Taylor & Francis. Mariage, C., Vamderdonckt, J., & Pribeanu, C. (2005). State of the art of web usability guidelines. In R. W. Proctor & K.-P. L. Vu (Eds.), Handbook of human factors in web design (pp. 688 700). Mahwah, NJ: Lawrence Erlbaum Associates. van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (2006). Introducing educational design research. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational Design Research (pp. 3-13). Abingdon, UK: Routledge. doi:http://www.routledge.com/ Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23. doi:10.1007/BF02504682