Professional Documents
Culture Documents
Abstract Adults over age 50 are increasingly pursuing online instructional modalities of higher education coursework. While research shows that late-career adults typically prefer traditional, face-to-face instruction, some are embracing the flexibility and convenience inherent in online instruction. This mixed-method study examined both the academic performance and instructional support needs of late-career adults (aged 50-65) in an online course as compared to earlycareer (aged 21-35) and mid-career (aged 36-49) adults. Surprisingly, results of the study indicate that not only are late-career adults satisfied with the online delivery, but they actually find the experience to be more rewarding than their early- and mid-career peers despite the differences in technical abilities. Additionally, results reveal that for late-career adults to be successful in online classes, they initially require higher levels of technology support and digital interaction. However, after receiving the technical assistance, they perform as good as or better than their younger peers. Keywords: postsecondary education, older adults, continuing education, online instruction, distance education
Introduction It is an easy task to reflect back upon an experience that was as positive as this was. It began with apprehension, the first class I had taken in many years, uncertainty about the qualities of the materials and some indecision as to wanting to dedicate the amount of time necessary to make the process meaningful. It is ending with a desire to continue. This has been a rewarding experience and has been a benefit to my role as teacher. This was written by a 58-year old male student on an end-of-course evaluation of an online graduate course. Literature Review Information from the U.S. Census Bureau (2008) reveals that the overall population in the United States is aging, and their projections show that in the next few decades the fastest growing segment of the population will be older adults. This holds true for the workforce as well, with the number of workers over the age of 55 increasing at a higher rate than any other age group (Alley & Crimmins, 2007). Additionally, we know that our economy and workforce demand life-long learners who continually update and upgrade skills (Shen, Pitt-Catsouphes, & Smyer, 2007), and that late-career workers value workplace lifelong learning (Fredericksen, 2006). Despite the emergence of the late-career student, there is scant research on the educational needs and performance of students ages 50-65 in higher education (Paulson & Boeke, 2006). Interestingly, the American Council on Education recently published research results entitled Reinvesting in the Third Age that identified the need for higher education to focus more on individuals aged 50 and older (Lakin, Mullane, & Robinson, 2007 & 2008). Recommendations from this focus group research suggested that older adults prefer education skill-ettes (i.e., short, specialized instruction focused on a particular need) and colleges should learn more about the interests and needs of this age group. This is consistent with the finding that late-career workers possessed positive attitudes toward learning, but only if it was relevant and helped them do their jobs better (Fuller & Unwin, 2006). To compound matters, higher education is experiencing a shift from traditional face-to-face instruction to fully online courses (Grant & Thornton, 2007; Rose, 2009). Online enrollment continues to rise rapidly with over 20 percent of students taking online courses (Allen & Seaman, 2008). It is expected that this shift in learning modalities will become even more prevalent in the next decade, so to maintain credentials and engage in life-long learning, late-career adults will have little choice but to attempt online coursework. This online interaction is second nature for digital natives (i.e., those who grew up with computer technology), but it requires new learning for digital immigrants (Prensky, 2001). Additionally, this shift in instructional formats implies that instructors must follow the process of interaction design to create an effective environment for diverse learners in their online coursework (Preece, Rogers & Sharp, 2002; Tallent-Runnels, Thomas, Lan, Cooper, Ahern, T.C., Shaw, et al., 2006). To identify these learning needs, instructors will need to conduct an analysis of necessary technology support and assignment options that match the learning styles of their course participants. While late-career adults are becoming more technologically savvy, these digital immigrants are still reluctant to take online coursework. In 2007, AARP reported that most adults (69%) aged 50 to 64 used the Internet; however, they rarely participated in formalized online learning. When asked why they did not participate in online coursework, older adults most often cited poor computer skills and loss of face-to-face connections as the primary reasons (Lakin et al., 2008). Contrary to the research, this study found that more than a third of all students in the online graduate course Introduction to Transition Education and Services were late-career adults. The characteristics of the students aged 50 to 65 and their learning outcomes from the course are the focus of this study. Methods This study examined the learner characteristics, academic performance, and satisfaction of latecareer teachers (aged 50-65) in an online graduate course. Research questions included:
1. 2. 3. 4. 5.
Why did late-career adults choose to take this online course? What level of content and technology knowledge did participants have prior to the online course? How did late-career adults perform in this online course? What level of technology support was necessary to facilitate the learning of late-career adults in the online environment? Were late-career adults satisfied with the online course content and instructional methods?
Setting and Content The asynchronous online graduate course, Introduction to Transition Education & Services was designed for secondary special educators who support students with disabilities in high school. The course was the first in a series of five online graduate courses, each worth 1 graduate credit hour (totaling 5 graduate credits) at a Midwestern research university. Employing a cohort model, course participants advanced through the series together, with research-based interaction design and instructional support components embedded into each course. These included: (a) a syllabus that outlined all assignments, expectations, and due dates; (b) detailed technical assistance instructions with screen shots; (c) structured discussions with a rubric posted on the course website; (d) a forum to post course questions; (e) content and media options that addressed a variety of learning styles; (f) student choice in application activities that related the content to their teaching; and (g) a reflection and evaluation of the instruction and learning experience that was used to continually enhance the instruction and learning environment. This standardized format enabled learners to master the learning format during the first course and then continue to use these newly-acquired skills in the subsequent courses. As the first course in the series, Introduction to Transition Education & Services was offered during the fall and summer semesters using the open-source course management platform Moodle (http://moodle.org/). One week prior to the start date, students received access to the course and login instructions so they could explore the website freely. The course website provided students the syllabus, grading rubric, information about technical formats, and all necessary resources needed for successful completion. Students submitted all assignments on the course website and e-mails could be sent to the instructor through the website or via students personal e-mail accounts. Participants In 2007-2009, 136 graduate students completed Introduction to Transition Education & Services. Two state Departments of Education (one Midwestern and one Eastern State) offered limited stipends to high school special education teachers in their state who chose to take the course. The course was customized with state-specific content for the cohorts in these states. Other participants enrolled online through the universitys continuing education division and paid full tuition. The results of this study reflect the data from the seven cohorts of students who participated in Introduction to Transition Education & Services between 2007 and 2009 (see Table 1). Table 1: Cohorts Percentage of Students aged 50 and above 53% 43% 26% 46% 38%
Cohort State A, Cohort 1 State A, Cohort 2 State B, Cohort 1 National, Cohort 1 State A, Cohort 3
Number of Students 17 28 19 13 24
2009
23 12 136
While the program did not intentionally recruit late-career adults, 51 individuals aged 50-65 chose to enroll. These students were primarily female (82%) and the majority held a masters degree or higher (67%). Job titles of these older adults included: special education teacher (30), transition specialist (7), related-services provider (6), administrator (3), college faculty (2), community agency consultant (2), and parent of a child with a disability (1). Most of these individuals had a long-term career in the field of education (i.e., 74% for 10+ years, 16% for 7-9 years, 6% for 4-6 years, and 4% for 1-3 years). Measures Several quantitative and qualitative measures were implemented throughout the online graduate course to collect background information on the participants and assess their change in knowledge, attitude, and skill. Furthermore, data were archived throughout the course to continually improve the course content and instructional strategies. These measures are described next. Demographic Survey. Prior to starting the course, participants completed a survey that gathered demographic information as well as their use of and comfort with technology. Descriptive and comparative analyses were used to develop a detailed picture of the course participants. Competency Survey. The competency survey was based on the transition specialist indicators identified by the Council for Exceptional Childrens Division on Career Development and Transition (2000). Participants were asked to rate their current aptitude on 40 indicators using a 4-point Likert scale. This pre-assessment survey enabled course content to be tailored to meet the needs of the participants. Case-based Learning Pre/Post Assessment. During the second week of the course, participants completed a case-based learning experience on transition education compliance and best practice (Morningstar, Gaumer Erickson, Lattin & Wade, 2008). This learning experience utilized performance-based assessments that required participants to apply their learning to case study examples and their own students. The pre/post assessment consisted of a 20-item multiplechoice test on key points of the Individuals with Disabilities Education Act (2004). Satisfaction Survey. After completing the case-based learning component, participants were asked to rate their satisfaction with the interactive content and online learning in general on a 20item survey using a 5-point Likert scale. Questions on this survey evaluated the time required to complete the learning experience, comfort with technology, components of the case-based learning experience that were most beneficial, and future uses for the information gained through the learning experience. Discussion Forums. Asynchronous discussions were utilized during two of the four weeks of the course. A topic in the first weeks discussion asked participants to introduce themselves and share their hopes and concerns related to the course. In addition to the week-long discussion, the instructor asked participants to post their questions about course content in a forum titled, General Class Questions. This enabled the instructor to post responses that could be accessed by all course participants. For this study, discussions from both Week 1 and General Class Questions were analyzed. These qualitative data were collected, printed, and coded to reveal themes related to the comfort with technology and reasons for pursuing the course. It was then quantified revealing the number of posts for each course participant related to the themes. E-mail Communication. All e-mail communication with the instructor was archived. While many students posted their questions to the General Class Questions discussion forum, others felt more comfortable sending an e-mail directly to the instructor. These e-mails were coded through the same procedure as described above for the discussion forums. Course Reflection. During the last week of the course, participants were asked to reflect on the
course content. Specifically they were asked to: Write a 1-2 page single-spaced reflection on this online learning experience. Be sure to identify: (a) information, resources, & activities you found most useful, (b) how you will use the information to improve transition services in your school or community, and (c) suggestions for improving this online learning experience. A random sample of twenty-five reflections from participants aged 50-65 were coded and themed to identify the course content that they found to be most beneficial and the application activities they planned to undertake based on their learning. Additionally their suggestions for improving the online experience were analyzed to identify overarching support needs of this age group. Quantitative data analyses consisted of descriptive statistics (i.e., mean and standard deviation), analysis of variance (ANOVA), and paired-samples t tests. For all analyses, the course participants were divided into three groups (early-career participants aged 21-35; mid-career participants aged 36-49; and late-career participants aged 50-65). ANOVA procedures evaluated the relationship between factors and the dependent variable (e.g., the relationship between technology skills and the age of participants). Because each ANOVA included variables with more than two levels, they were followed with pairwise comparisons (i.e., Dunetts C if variances were unequal or the least significant difference (LSD) procedure if variances were not statistically different). A paired-sample t test evaluated the performance across time with two data points (i.e., case-based instruction pre/post test performance). The a priori level of 0.05 was set for all statistical tests (Green & Salkind, 2003). Results Throughout the results section, course participants are compared using three groups. Those aged 50-65 are termed late-career; aged 36-49 termed mid-career; and aged 21-35 termed early-career. Because the vast majority of individuals who participated in the course were practicing teachers, these employment terms accurately represent the participants. Why did late-career adults choose to take this online course? When asked why they chose to take the course, late-career participants cited two main reasons: 1) their interest in the topic and 2) the ability to earn recertification credits. As one student noted, I see the courses as a great opportunity to learn knowledge and skills that will better equip my students to meet their post-secondary goals. Others described the appealing layout, This seemed to be a good way to learn more about the field in an efficient and timely manner and I like the opportunity to gain new information in a short period of time. I also like the intensive focus on one topic at a time. What level of content and technology knowledge did participants have prior to the course? The subject-area competency was similar across all age groups. When asked to rate competency on 40 transition-related skills, the mean scores of late-career participants ranged from 1.90 (not prepared) to 3.70 (very prepared), with an average rating of 2.5 (somewhat prepared). This was similar to their mid- and early-career counterparts. These means reveal that on average the course participants, regardless of their age, felt that they had a moderate level of competency related to the course content prior to enrolling in the course (see Table 2). Thirteen of the 51 late-career adults (25%) had previously completed an online course. A oneway analysis of variance was conducted to evaluate the relationship between the number of online courses taken and age of the student. Error rates on follow-up analyses were controlled for using the LSD approach. These analyses found that previous online course-taking of latecareer participants was significantly lower than the online course-taking of early-career participants (see Table 2). The online course-taking for mid-career participants was between that of early- and late-career participants and thus not statistically different from either group. Late-career adults identified having a moderate level of technology skills and used technology moderately in their daily work. While their technology usage rated at the same level as earlyand mid-career participants, the mid- and late-career participants felt less experienced
technologically than the early-career participants (see Figure 1). A one-way analysis of variance was conducted to evaluate the relationship between technology skills and age of the student. Because Levenes test found that equal variance could not be assumed, error rates on follow-up analyses were controlled for using the Dunnett C approach. There was no statistical difference between the technology skills of mid- and late-career participants, but both groups rated their technology skills statistically lower than the early-career participants (see Table 2).
Figure 1: Technology Use and Skill How did late-career adults perform in this online course? Late-career adults had high levels of success in this online course. All (100%) late-career participants successfully completed the course requirements. Students receiving graduate credit were graded on an A-F system with 22 earning an A (90-100%) and 2 earning a B (80-89%). Other participants chose to earn Continuing Education Units (CEUs) with a pass/fail system through which the remaining 27 participants earned a passing grade. The case-base learning pre/post assessment reinforced the data from the competency survey that identified similar levels of proficiency in the subject area for all age groups. On the pre-test, late-career participants averaged 62% and increased their scores to 80% on the post-test. No significant differences were found in either the pre- or post-test scores when compared to their younger counterparts. A paired-samples t-test was conducted to evaluate the increase in knowledge from the pre-test to the post-test. The results for late-career participants indicated that the mean score on the post-test (M=80.42, SD=11.56) was significantly higher than the mean score on the pre-test (M=61.59, SD=15.57). Results for mid- and early-career participants also showed significant increases in knowledge. What level of technology support was necessary to facilitate the learning of late-career adults in the online environment? The discussion forum and e-mail analyses revealed that students aged 50-65 ask more technology-related questions than their younger counterparts. These questions included asking for directions regarding posting comments, submitting assignments, and accessing online resources. Approximately 40% of the late-career adults asked a technology-related question. An ANOVA followed by a Dunetts C test revealed that late-career participants asked significantly more technology-related questions than early-career participants. Results for mid-career participants were not significantly different from either of the other age groups (see Table 2). Other discussion forum and e-mail analyses did not reveal significant differences among the age
groups. The themes included asking course content questions, expanding learning by discussing other transition-related topics, and providing technology-related support to peers on the discussion forums. Were late-career adults satisfied with the online course content and instructional methods? Some variance was identified among the age groups on the satisfaction survey. Late-career participants spent more time completing the case-based learning experience, but they also gave higher ratings to the following statements: (a) the case-based learning experience kept my attention and interest; and (b) the case-based learning experience could be an important resource to me in the future. These items on the satisfaction survey were analyzed using a oneway analysis of variance. Post-hoc procedures included the LSD approach when variance was assumed (i.e., the case-based learning experience kept my attention and interest) and the Dunnets C approach when variance could not be assumed (i.e., the case-based learning experience could be an important resource to me in the future). The analyses revealed that the ratings of mid- and late-career participants were significantly higher than those of early-career participants when asked if the case-based learning experience kept their attention. On the item that asked if the case-based learning experience could be an important resource in the future, the ratings of late-career participants were significantly higher than those of early-career participants. The ratings of mid-career participants fell between the early- and late-career participants, and thus were not statistically different from either group (see Figure 2). Table 2 provides mean scores, standard deviations, and p-values.
Figure 2: Satisfaction with Case-Based Learning Experience Upon completion of the course, participants were asked to reflect on the course content. These reflections revealed that participants aged 50-65 highly valued the applicability of the course content to their jobs and the array of resources provided throughout the course. All late-career participants sampled identified the resources (articles, videos, and website) as contributing to their learning. Most also felt that the case-based learning experience (76%) and discussions (68%) were beneficial. These participants expanded on the information by identifying ways they would use their newly-acquired knowledge. Responses included disseminating information to colleagues and parents, improving the transition education processes for students, and advocating for increased collaboration and additional services in schools. I have been utilizing what I have learned as I have assisted students in their transition planning. It has been invaluable, enriching experience. I have allowed the students to take more control so they feel more confident.
I have printed out the articles and some of the information from the websites, and have incorporated them into a note book with information that can be used in the transition planning process. I have also notified my colleagues that I have this information, which will be located in our special education office/library. I have also e-mailed a list of websites to them. I would also like to begin sending out brochures to parents or guardians before the IEP [Individualized Education Program] meetings so that they come to the meetings better informed. I actually called my Special Education Director to tell her, This is the first time transition goals for a student felt individualized and real! My e-mail has been busy sending new and seasoned special educators in the school bits and pieces of this class. I am already using the information garnered in this course in my IEP meetings. When asked how they would improve the online learning experience, half of the late-career adults (50%) reiterated their satisfaction with the course. Others identified technology, time commitment, and discussion forum strategies. As to areas of improvement, my more traditional habits of learning cause me to seek out more topic specific discussion forums. Time commitment to do the activities continues to be a concern to me, but the information is invaluable. The lack of programs to view some of the videos is a bummer but being able to read the text is some consolation. Overall late-career adults came into the course with moderate levels of prior knowledge and showed stellar performance on all assignments. They required higher levels of technology support, but once they became proficient in the technology requirements, they found the course content and format to be highly beneficial and applicable to their work. This satisfaction was evident in the enrollment rate of the next course with 94% of the late-career participants in the Introduction to Transition course enrolling in the second online course in the series. Discussion & Implications The course was not designed for or specifically marketed to late-career adults, but many individuals chose it as their first online learning experience. While one study found that 25% of teachers in the United States are over the age of 50, 38% of participants in this online course were in that demographic (Miller, Sen, Malley, & Burns, 2009). This reaffirms research that older adults prefer highly specific, short-term learning opportunities based on their interests and job requirements (Fuller & Unwin, 2006; Lakin et al., 2008; Shen et al., 2007; Tallant-Runnels, Thomas, et. al., 2006). Teacher re-certification requirements and employer expectations also encouraged participation in this professional development opportunity. Teachers must participate in professional development throughout their teaching career, but they typically have extensive flexibility in the professional development options they choose, including school and district in-services, workshops, conferences, and coursework (National Center for Education Statistics, 1999). All course participants could have met re-certification requirements without participating in online training, but it is unlikely that face-to-face training options would have been as specialized as that provided in this course. While the teacher re-certification requirements should be considered when generalizing the results of this study, its important to note that this online course was sought out specifically by late-career teachers to meet these requirements. These late-career adults used technology for their work (primarily teaching) at similar levels as their younger colleagues, but they reported that they did not feel as skilled in technology use. This was substantiated by both their self-ratings of technology skills and use, as well as the number of technology-related questions posed during the course. While many of the late-career
adults entered the course concerned about their technology skills, they were willing to work through the barriers with the instructor because they valued the information. This interaction required higher levels of invisible labor by the instructor (Blair & Hoy, 2006), but it also produced an online learning community that extended the learning opportunities within the course (Grant & Thornton, 2007). Table 2: Performance and Perception by Age Group Participants Aged 21-35 M SD Number of online courses 2.12 Technology use in daily work Technology skills 3.48 2.52 Subject-area competency 2.53 Satisfaction Case-based learning experience kept my attention Satisfaction Case-based learning experience could be an important resource in the future Number of technologyrelated questions 0.39 2.55 0.42 2.50 0.30 0.83 2.01 0.51 0.51 1.28 3.50 2.20 1.85 0.51 0.69 1.12 3.36 2.20 1.79 0.53 0.49 0.05 0.37 0.03 Participants Aged 36-49 M SD Participants Aged 50-64 M SD
3.97
0.73
4.30
0.61
4.28
0.57
0.05
4.45 0.17
0.83 0.45
4.80 0.49
0.41 0.84
4.84 0.88
0.37 1.51
0.01 0.01
It is interesting to note that late-career adults gave higher satisfaction scores to some components of the course, specifically the case-based learning experience that required participants to read research-based content and then apply the information to case study examples. Hypotheses for these higher ratings could be that digital immigrants have a high appreciation for content that is directly applicable to their jobs or that digital natives expect more active interfaces (i.e., game-like atmospheres) in online environments (Zemke, Raines & Filipczak, 2001). In addition, the case-base learning followed a standardized format with a balance of content and application. This learning-while-applying approach has been found to be effective for late-career learners (Charness, Czaja, & Sharit, 2007, pp. 233). Additional research is needed on the perceptions and performance of late-career adults in online learning environments. Because these individuals prefer highly specialized courses, additional data need to be collected by institutes of higher education on their continuing education course participants. Older adults are continuing to grow as a market niche in education, so to maintain a competitive edge, institutes must identify the needs, interests, and necessary online supports of this age group. Institutions of higher education should reflect on their online course content and delivery systems. The course in this study was unique in that it was completed over a four-week duration. Additionally, it was content-specific with direct application to the job requirements of the participants. This level of specificity and application was found to be highly valued by late-career adults. As institutions of higher education expand their online course offerings, they should undergo a rigorous evaluation process addressing the context, interactions, and desired
outcomes for students (Preece et al., 2002; Ruhe & Zumbo, 2009; Scanlon, Jones, Barnard, Thompson, & Calder, 2000). Foresight in course design can then lead to higher levels of learning for individuals of all ages. Finally, the invisible labor of faculty teaching online courses must be understood and valued. When the instructor of the course in this study was asked about instructional time, she responded: Even after teaching this course seven times, it still requires more of my time than any face-to-face course I teach. In addition to updating assignments and grading, I access the course at least five days per week to respond in weekly discussions and to answer questions from students. Once students understand the course layout and technology requirements, my instructional time decreases substantially in the four courses that follow Introduction to Transition Education & Services. This is aligned with research that identifies student-staff contact and prompt feedback as core principles in effective online teaching (Grant & Thornton, 2007; Stein & Glazer, 2003). Many students, especially late-career students, could benefit from an introductory course that exposes them to the online learning environment and subject-area content prior to participating in advanced online courses.
References AARP. (2007). The state of 50+ America. Washington, DC : Author. Retrieved January 22, 2010 from http://assets.aarp.org/rgcenter/econ/fifty_plus_2007.pdf. Allen, E. & Seaman, J. (2008). Staying the course: Online education in the United States, 2008. The Sloan Consorium. Retrieved January 19, 2010, from http://www.sloanconsortium.org/publications/survey/pdf/staying_the_course.pdf. Alley, D. & Crimmins, E. (2007). The demography of aging and work. In K.S. Shultz & G.A. st Adams (Eds.), Aging and Work in the 21 Century (pp. 7-23). Mahwah, NJ: Lawrence Erlbaum Associates. Blair, K. & Hoy, C. (2006). Paying attention to adult learners online: The pedagogy and politics of community. Computers and Composition 23, 32-48. Charness, N., Czaja, S., & Sharit, J. (2007). Age and technology for work. In K.S. Shultz & G.A. st Adams (Eds.), Aging and Work in the 21 Century (pp. 225-249). Mahwah, NJ: Lawrence Erlbaum Associates. Council for Exceptional Children Division on Career Development and Transition. (2000). Transition Specialist Competencies. Retrieved June 3, 2006 from http://www.dcdt.org/factsheets/DCDT_Fact_Sheet_Compentencies_3.pdf. Fredericksen, R.S. (2006). Work attitudes and values of older US public service employees. In T. Tikkanen & B. Nyhan (Eds.), Promoting lifelong learning for older workers: An international overview (pp. 123-139). Luxembourg: Office for Official Publication of the European Communities. Fuller, A. & Unwin, L. (2006). Older workers learning in changing workplace contexts: Barriers and opportunities. In T. Tikkanen & B. Nyhan (Eds.), Promoting lifelong learning for older workers: An international overview (pp. 257-270). Luxembourg: Office for Official Publication of the European Communities. Grant, M.R. & Thornton, H.R. (2007). Best practices in undergraduate adult-centered online learning: Mechanisms for course design and delivery. MERLOT Journal of Online Learning and Teaching, 3(4), 346-362. Green, S. B., & Salkind, N. J. (2003). Using SPSS for Windows and Macintosh: Analyzing and understanding data (3rd ed.). Upper Saddle River, NJ: Pearson Education. Grant, M.R. & Thornton, H.R. (2007). Best practices in undergraduate adult-centered online
learning: Mechanisms for course design and delivery. MERLOT Journal of Online Learning and Teaching, 3(4), 346-362. Lakin, M.B., Mullane, L. & Robinson, S.P. (2008). Mapping new directions: Higher education for older adults. Washington DC: American Council on Education. Lakin, M.B., Mullane, L. & Robinson, S.P. (2007). Framing new terrain: Older adults and higher education. Washington DC: American Council on Education. Miller, D.C., Sen, A., Malley, L.B., and Burns, S.D. (2009). Comparative Indicators of Education in the United States and Other G-8 Countries: 2009 (NCES 2009-039). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC. Morningstar, M.E., Gaumer Erickson, A.., Lattin, D.L. & Wade, D.K. (2008). Best Practices in Planning for Transition. University of Kansas, Department of Special Education, Transition Coalition. National Center for Education Statistics. (1999). Teacher quality: A report on the preparation and qualifications of public school teachers. Washington, DC: U.S. Department of Education. Paulson, K. & Boeke, M. (2006). Adult learners in the United States: A national profile. Washington, DC: American Council on Education. Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction design: Beyond human-computer interaction. New York, NY: John Wiley & Sons, Inc. Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5), 1-6. Rose, K.K. (2009). Student perceptions of the use of instructor-made videos in online and faceto-face classes. MERLOT Journal of Online Learning and Teaching, 5(3), 487-495. Ruhe, V. & Zumbo, B.D. (2009). Evaluation in distance education and e-learning. New York: The Guilford Press. Scanlon, E., Jones, A., Barnard, J., Thompson, J., & Calder, J. (2000). Evaluating information and communication technologies for learning. Educational Technology and Society, 3(4), 110. Shen, C., Pitt-Catsouphes, M., & Smyer, M.A. (2007). Todays multi-generational workforce: A proposition of value. Workplace Flexibility Issue Brief, 10, 1-11. Stein, D. & Glazer, H. R. (2003). Mentoring the adult learner in academic midlife at a distance education university. The American Journal of Distance Education, 17(1), 7-23. Tallent-Runnels, M.K., Thomas, J.A., Lan, W.Y., Cooper, S., Ahern, T.C., Shaw, S.M., et al. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76(1), 93-135. U.S. Census Bureau. (2009). National population projections. Washington DC: Author. Retrieved January 19, 2010 from http://www.census.gov/population/www/projections/2009projections.html. U.S. Department of Education. (2010). Fall Enrollment Survey 2002, 2007. Washington DC: National Center for Education Statistics, Integrated Postsecondary Education Data System. Retrieved January 19, 2010 from http://nces.ed.gov/ipeds/datacenter/Statistics.aspx. Zemke, R., Raines, C., & Filipczak, B. (2001) Generation markers. Across the Board, 39(11), 4854. Manuscript received 15 Feb 2010; revision received 18 May 2010.
This work is published under a Creative Commons Attribution-Non-Commercial-Share-Alike License For details please go to: http://creativecommons.org/licenses/by-nc-sa/3.0/us/
QUALITY SCORECARE FOR THE ADMINISTRATION OF ONLINE EDUCATION PROGRAMS: A Work in Progress
SLOAN-C:
A
Consortium
of
Individual,
Institution
and
Organization
Committed
to
Quality
Online
Educations
Accessed
online
at
http://sloanconsortium.org/quality_scoreboard_online_program
This scorecard is for measuring and quantifying elements of quality within online education programs in higher education. It is an easy-to-use tool for online administrators for program evaluation. By evaluating each of the respective quality indicators within the established categories, an online administrator can determine strengths and weaknesses of their program. The identification of the weaknesses can be used to support program improvement and strategic planning initiatives. The scorecard could also be used to demonstrate to accrediting bodies, elements of quality within the program as well as an overall level of quality. The scorecard contains 70 quality indicators--each indicator is worth up to three points. The administrator will determine at what level their program meets the intent of the quality indicator after examining all procedures and processes. 0 points = Not Observed. The administrator does not observe any indications of the quality standard in place. 1 point = Insufficiently Observed. The administrator has found a slight existence of the quality standard in place. Much improvement is still needed in this area. 2 points = Moderate Use. The administrator has found there to be moderate use of the quality standard. Some improvement is still needed in this area. 3 points = Meets Criteria Completely. The administrator has found that the quality standard is being fully implemented and there is no need for improvement in this area.
The quality scorecard is versatile enough to be used to demonstrate the overall quality of online education programs, no matter what size or type of institution. The following steps for use and implementation are suggested that will yield a measurable result: 1. The online education administrator examines the online program for evidence of each of the 70 quality indicators. Based upon the level of evidence observed, the administrator chooses one of the following values: 0 points - not observed, 1 point - insufficient, 2 points - moderate use, 3 points completely meets criteria. For each indicator, the online education administrator should provide examples of the observed evidence. For example, the first indicator listed in the Institutional Support category is: The institution has put in place a governance structure to enable effective and comprehensive decision making related to distance learning. To substantiate the score for this indicator, evidence should be documents such as digital copies of organizational charts, reporting structures, and advisory committee minutes demonstrating how a decision is processed. The online education administrator totals the score for each indicator and then determines the level of quality observed: A perfect score = 210 points. 90-99% = 189-209 - Exemplary (little improvement is needed) 80-89% = 168-188 - Acceptable (some improvement is recommended) 70-79% = 147-167 - Marginal (significant improvement is needed in multiple areas) 60-69% = 126-146 - Inadequate (many areas of improvement are needed throughout the program) 59% and below = 125 points and below - Unacceptable.
2.
3.
This scorecard was developed through a Delphi study involving 43 experts in online education administration (83% had more than nine years of experience). The quality scorecard contains adaptations of the 24 quality standards identified by the Institute for Higher Education Policy report, Quality on the Line: Benchmarks for Success in Internet-based Distance Education (2000). QUALITY SCORECARD FOR THE ADMINISTRATION OF ONLINE EDUCATION PROGRAMS Not Moderate Insufficient Observed Use INSTITUTIONAL SUPPORT The institution has put in place a governance structure to enable effective and comprehensive decision making related to distance learning. Policies are in place to authenticate that students enrolled in online courses and receiving college credit are indeed those completing the course work. Policy for copyright ownerships of course materials exists. The institution has defined the strategic value of distance learning to its enterprise and to its relevant parts. Points Meets Possible Criteria Score Per Completely Category 12
1.
2.
3.
4.
TECHNOLOGY SUPPORT A documented technology plan that includes electronic security measures (e.g., password protection, encryption, secure online or 1. proctored exams, etc.) is in place and operational to ensure quality standards, adherence to FERPA, and the integrity and validity of information. *** The technology delivery systems are highly reliable and operable with measurable 2. standards being utilized such as system downtime tracking or task benchmarking. ***
18
3.
4.
5.
6.
A centralized system provides support for building and maintaining the distance education infrastructure. *** The course delivery technology is considered a mission critical enterprise system and supported as such. The institution maintains system backup for data availability. Faculty, staff, and students are supported in the development and use of new technologies and skills.
COURSE DEVELOPMENT AND INSTRUCTIONAL DESIGN Guidelines regarding minimum standards are used for course development, design, and delivery of online instruction. *** Technology is used as a tool to achieve learning outcomes in delivering course content. *** Instructional materials, course syllabus, and learning outcomes are reviewed periodically to ensure they meet program standards. *** Courses are designed so that students develop the necessary knowledge and skills to meet learning objectives at the course and program level. These may include engagement via analysis, synthesis and evaluation. *** Learning objectives describe outcomes that are measurable. Selected assessments measure the course learning objectives and are appropriate for an online learning environment. Student-centered instruction is considered during the coursedevelopment process. There is consistency in course development for student retention and quality. Course design promotes both
36
1.
2.
3.
4.
5.
6.
7.
8. 9.
0 0
1 1
2 2
3 3
faculty and student engagement. Current and emerging technologies are evaluated and 10. recommended for online teaching and learning. Instructional design is provided for creation of 11. effective pedagogy for both synchronous and asynchronous class sessions. Curriculum development is a 12. core responsibility for faculty. COURSE STRUCTURE The online course site includes a syllabus outlining course objectives, learning outcomes, evaluation methods, textbook information, and other related course information, making course requirements transparent at time of registration. *** The institution ensures that all distance education students, regardless of where they are located, have access to library/learning resources adequate to support the courses they are taking (SACS statement). *** Expectations for student assignment completion, grade policy, and faculty response are clearly provided in the course syllabus. *** Links or explanations of technical support are available in the course. Instructional materials are easily accessible and usable for the student. The course adequately addresses the special needs of disabled students via alternative instructional strategies and/or referral to special institutional resources. Opportunities/tools provided to encourage student-student
24
1.
2.
3.
4.
5.
6.
7.
collaboration (i.e, web conferencing, instant messaging, etc). Documents attached to modules are in a format that is easily accessed with multiple 8. operating systems and productivity software (PDF, for example). TEACHING AND LEARNING Student-to-Student interaction and Faculty-to-Student interaction are essential characteristics and are facilitated through a variety of ways. *** Feedback on student assignments and questions is constructive and provided in a timely manner. *** Students learn appropriate methods for effective research, including assessment of the validity of resources and the ability to master resources in an online environment. *** Students are provided access to library professionals and resources that help them to deal with the overwhelming amount of online resources. Instructors use specific strategies to create a presence in the course.
15
1.
2.
3.
4.
5.
SOCIAL AND STUDENT ENGAGEMENT Students should be provided a way to interact with other 1. students in an online community. FACULTY SUPPORT Technical assistance in course development and assistance 1. with the transition to teaching online is provided [for faculty]. *** Instructors are prepared to 2. teach distance education
18
3.
4.
5.
6.
courses and the institution ensures faculty receive training, assistance, and support at all times during the development and delivery of courses. *** Faculty receive training and materials related to Fair Use, plagiarism, and other relevant legal and ethical concepts. *** Faculty are provided on-going professional development related to online teaching and learning. Clear standards are established for faculty engagement and expectations around online teaching. Faculty workshops are provided to make them aware of emerging technologies and the selection and use of these tools.
STUDENT SUPPORT Before starting an online program, students are advised about the program to determine if they possess the self-motivation and commitment to learn at a distance. *** Before starting an online program, students are advised about the program to determine if they have access to the minimal technology required by the course design. *** Students receive (or have access to) information about programs, including admission requirements, tuition and fees, books and supplies, technical and proctoring requirements, and student support services prior to admission and course registration. *** Students are provided with access to training and information they will need to secure required materials through electronic databases,
51
1.
2.
3.
4.
interlibrary loans, government archives, new services and other sources. *** Throughout the duration of the course/program, students have 5. access to appropriate technical assistance and technical support staff. *** Student support personnel are available to address student 6. questions, problems, bug reporting, and complaints. *** Students have access to 7. effective academic, personal, and career counseling. Minimum technology 8. standards are established and made available to students. Student support services are provided for outside the 9. classroom such as academic advising, financial assistance, peer support, etc. Policy and process is in place 10. to support ADA requirements. Students are provided relevant information: ISBN numbers, suppliers, etc. and delivery 11. modes for all required; instructional materials: digital format, e-packs, print format, etc. to ensure easy access. Program demonstrates a student-centered focus rather 12. than trying to fit service to the distance education student in on-campus student services. Efforts are made to engage 13. students with the program and institution. Students are instructed in the appropriate ways of 14. communicating with faculty and students. The institution provides guidance to both students and 15. faculty in the use of all forms of technologies used for course delivery. Tutoring is available as a 16. learning resource. Students are instructed in the 17. appropriate ways of enlisting help from the program.
0 0
1 1
2 2
3 3
EVALUATION AND ASSESSMENT The program is assessed through an evaluation process 1. that applies specific established standards. *** A variety of data (academic and administrative information) are used to 2. regularly and frequently evaluate program effectiveness and to guide changes toward continual improvement. *** Intended learning outcomes at the course and program level 3. are reviewed regularly to ensure clarity, utility, and appropriateness. *** A process is in place for the 4. assessment of faculty and student support services. Course and program retention 5. is assessed. Recruitment and retention are 6. examined and reviewed. Program demonstrates compliance and review of 7. accessibility standards (Section 508, etc.) Course evaluations are 8. examined in relation to faculty performance evaluations. Faculty performance is 9. regularly assessed. Alignment of learning 10. outcomes from course to course exists. Course evaluations collect student feedback on quality of 11. content and effectiveness of instruction.
33
0 0 0
1 1 1
2 2 2
3 3 3
0 0 0
1 1 1
2 2 2
3 3 3
Perfect Score = 210 *** Adapted from the Institute for Higher Education Policy's report Quality on the Line: Benchmarks for Success in Internet-based Distance Education (2000).
Time Management As part of an initial assignment, ask students to describe how they will manage the course work and to detail any activities they engage in that may cause them to have difficulty completing work on time. Set expectations with students with respect to instructor response times. Indicate when you will be grading assignments, responding to email and posting to discussion boards. Also tell students if late assignments will be graded on the same or extended timeline.
Working with an Instructional Designer Instructional designers bring a focus on learning theory to the course development process. One key contribution is tying all course content and activities back to learning objectives and outcomes. Faculty are deep in the course content and have valuable classroom experience to inform the online course development. IDs bring the experience in learning theory and online methodologies. Faculty and instructional designer should develop a shared vision for the content/course -- this includes providing faculty with examples of how the content will be presented online. For subject areas in which the instructional designer has limited exposure or expertise, faculty may want to spend some time at the outset educating them on the course content. Prepare learning objectives and content overview before engaging with the instructional designer. This will guide the overall course development.
Challenges Faced by Online Faculty Students who dont connect within the course are disheartening. In a face-to-face course, there are numerous ways to engage the student -- even taking the student out for a cup of coffee. In online learning, those opportunities feel limited. Without seeing students in person, it is more challenging to hold them accountable for on time submission of assignments. Students tend to submit assignments in batches rather than on a well-paced schedule. Written feedback is not always the most effective delivery method. It is sometimes more straightforward to discuss an issue with a student in person rather than via email or discussion forums.
ONLINE PEDAGOGY: LINKS, RESOURCES, RESEARCH The Sloan Consortium Sloan-C, a consortium of individuals, institutions, and organizations committed to quality online education, provides workshops, webinars, publications, and resources relevant to online teaching and learning. Sloan Consortium Annual Conferences include the International Conference on Online Learning, Emerging Technologies for Online Learning, and the Blended Learning Conference and Workshop. Sloan-Cs Effective Practices are built around Five Pillars of Quality: Access, Learning Effectiveness, Faculty Satisfaction, Student Satisfaction, and Scale. The International Association for K-12 Online Learning (iNACOL) The International Association for K-12 Online Learning (iNACOL) focuses on advocacy, research, professional development, and networking for K-12 online teachers. The Guide to K-12 Online Learning offers advice to those starting an online program. iNACOL hosts the Virtual School Symposium each year and publishes National Standards for Quality Online Teaching. Educause Educause is a non-profit organization whose mission is to advance higher education by promoting the intelligent use of information technology. The Blended and Online Learning Group provides a number of resources to support the field. Educause holds a number of conferences, seminars, and institute programs. Research Journals American Journal of Distance Education Educause Quarterly International Review of Research in Open and Distance Learning (IRRODL) Journal of Asynchronous Learning Networks (JALN) Journal of Distance Education (JDE) Journal of Educators Online (JEO) Journal of Interactive Online Learning (JIOL) Journal of Online Learning and Teaching (JOLT) Online Journal of Distance Learning Administration (OJDLA) Open Learning: The Journal of Open, Distance and e-Learning
FACULTY
RESOURCES
MESA
COMMUNITY
COLLEGE
accessed
online
at
http://www.mesacc.edu/eLearning/faculty.html
Practices in eLearning:
These features are recommended as best practices in online course design and instruction. This document is designed as an informational resource only. For more detailed information, review the Distance Education Handbook at www.mc.maricopa.edu/distance/faculty.html and Distance Learning Resources at www.mc.maricipa.edu/~johnson/DLMGRSRC.HTM
Learning Module/Assignment Objectives Defined, measurable and consistent with course competencies.
Assessment
Assessing student participation and progress regularly is essential to good course design; student feedback can also provide valuable information when considering course modifications and updating. Objectives Alignment Align course competencies/learning objectives, learning activities and assessment/evaluation activities. Assessment Process Identify and explain assessment policy and procedures. Evaluation Provide detailed description or criteria for evaluating student work. For example, rubrics or grading explanations/criteria. For rubric examples go to http://ctl.mc.maricopa.edu/wiki/index.php/Grading_Forms. Instruments Describe activities that monitor student progress throughout the course and measure learning objectives/outcomes. For example, tests, projects, essays, etc. Assessment Safeguards State cheating, student-only work and plagiarism policy. See http://ctl.mc.maricopa.edu/_resources/helpdocs/turnitin.html for further information. Also, see the Course Integrity statement under Legal Matters. Grading Policy Provide a detailed explanation of grading policy. For example, point/percentage requirements for assignments and for the overall letter grade. Grade Book Provide students online access to their progress in the course. Course Evaluation Provide a tool that captures student feedback. For example, suggestion box or anonymous survey. Examples of such tools can be found at here.
Communication
It is important to engage students in your course and make them feel part of a community. Frequent communication helps to promote retention and increases student participation and interaction. Contact Information Clearly stated contact information and procedures, including instructor availability. Response Criteria Clearly stated timeframes for responding to student questions (usually 24 to 48 hours during the week), weekend/holiday responses, providing feedback on assignments, and posting grades. Student Communication Plan Outlined process for communicating with students. For example, class announcements and emails. Class Interaction Methods to allow student/student, student/instructor, student/content and student/technology interaction. Create and Nurture a Learning Community Process for class introductions and ongoing activities that create a sense of community and comfort. For examples, visit http://onlinelearninglab.wikispaces.com/icebreakers.
tutorials, MCC Learning Enhancement Services at www.mc.maricopa.edu/library/LE/, MCC Library services at www.mc.maricopa.edu/library/ and other resources. Technology Support State how students can obtain technological support. For example, TSS at www.mc.maricopa.edu/its/tss or outside support options if using a publisher-provided instructional management system. Learning Support Technology tools should support learning objectives and promote active learning. For examples, see "Teaching and Learning on the WEB." www.mcli.dist.maricopa.edu/tl/index.html.
Legal Matters
It is important to address security and legal issues. Copyright Have proper permissions and citations for any copyrighted materials or creative commons, http://creativecommons.org/ used in your class. See www.maricopa.edu/legal/ip/ for copyright information. Course Integrity State cheating, student-only work and plagiarism policy. See MCCCD policies at www.mc.maricopa.edu/students/publications.html.
Accessibility
It is important that the course be accessible to all students. Disability Include an ADA statement and willingness to make accommodations based on disabilities. www.mc.maricopa.edu/students/disability/links.html. Some thought should be given to alternative modalities based on student needs. Readability Consideration should be given to the readability of course information. For example, type of font and colors used. www.section508.gov/. *For more information on best practices, please visit http://www.qualitymatters.org/ and link on FIPSE Grant Project. This document was develop by the Mesa Community College Distance Education CommitteeApril 2008
Class Roster and Verifying Attendance Classroom and Location Info Course Evaluations (by Students) Course Registration Requests (for Dis.Ed. students wanting to take a class offered to a different cohort) Course Registration Requests (for main campus students wanting to take a Dis.Ed. course) Course Scheduling Disruptive or Troubled Students FERPA Grades Hiring, Contracts, and Compensation Inclement Weather Cancellations & Closures Incompletes Interactive TV Course Delivery Library Assistance for Extension Faculty SmarThinking Student Support Support & Services Teaching for Extension vs Summer Technology Information Textbooks Travel & Vehicle Reservations
COPYRIGHT AND INTELLECTUAL PROPERTY The TEACH Toolkit: An Online Resource for Understanding Copyright and Distance Education
Description: This site explains the Technology, Education, and Copyright Harmonization (TEACH) Act as it pertains to higher education. Fair use and the TEACH Act is also covered.