ARE YOU EXPERIENCED?

DETERMINING EXISTING ONLINE TEACHING COMPETENCE

IN NEWLY HIRED FACULTY

Thomas Cavanagh, Ph.D. -- Assistant Vice President, Distributed Learning

Linda Futch, Ed.D. -- Associate Department Head
Nancy Swenson -- Instructional Designer
University of Central Florida
Audience Survey
 Is your institution experiencing growth in online
courses?
 How many of you provide faculty development for
online courses?
 Is your faculty development required?
 How many of you receive questions about faculty with
previous online experience?
 How do you deal with these questions?
 Why Faculty Development?
(Internal Pressures)

 Rapid growth in online courses

 12.9% growth in 2008 (national: Sloan)
 31% growth Fall 08-09 (UCF)
 Minnesota State system proposes 25% of all MnSCU
credits be earned online by 2015.
 Faculty who have never taught online are being
assigned (or have expressed an interest)
 Why Faculty Development?
(Internal Pressures)

 Enable faculty success (and student success)

 As opposed to onerous administrative requirement
 True online course is not simply a “remediation” of
classroom presentation
 Video/Theater analogy
 Accreditation Requirements
(External Pressures)

 Regional accreditation bodies often require training

and support unique to DL
 Southern Association of Colleges and Schools (SACS)
 Western Association of Schools and Colleges (WASC)
 Accreditation Requirements
 Southern Association of Colleges and Schools
 Faculty support services are appropriate and
specifically related to distance education.
 Faculty who teach in distance education programs and
courses receive appropriate training.
 Accreditation Requirements
 Western Association of Schools and Colleges
 The institution provides an ongoing program of appropriate
technical, design, and production support for participating
faculty members.
 The institution provides to those responsible for program
development the orientation and training to help them become
proficient in the uses of the program’s technologies, including
potential changes in course design and management.
 The institution provides to those responsible for working
directly with students the orientation and training to help them
become proficient in the uses of the technologies for these
purposes, including strategies for effective interaction.
 University of Central Florida

•Public
•Large
•Metropolitan
•Multi-disciplinary
Programs
University of Central Florida
 Combined (W & M) = almost 19% of SCH
Dynamic Environment
 High growth
 High demand for online courses
 Required faculty development
 1990’s: online education was new and no one had
experience
 Existing models designed for faculty new to online
instruction
Current UCF Faculty Development
Webcourses IDV Essentials ADL5000 IDL6543
Essentials
Required to teach Required to teach Required to teach Required to design
“web-enhanced” lecture-capture/ existing and teach original
Face-to-face video streaming online/blended online/blended
course course course

Technology Focus Design and Pedagogy, Deeper Design and

Delivery Focus Logistics, Teaching Focus
Technology

5 hrs 8 hrs 35 hrs 80 hrs

Ad hoc training, open labs, and advanced topics sessions

The Challenges
 New faculty being hired with prior online teaching
experience
 Deans’ comments
 Various models for online teaching
 Templates, web-enhanced, document posting, synchronous,
asynchronous, SL, etc.
 Limited use of online adjuncts who are not
geographically proximate
 UCF-specific technologies, processes, and expectations
IDL6543 Overview
 Flagship faculty development program for online
instruction
 Award winning (Sloan-C Excellence in Faculty
Development 2003)

Online
 Three key strategies

Consultations F2F
IDL6543 Objectives
 Verbalize an informed opinion regarding the importance and
contribution to higher education made by "distributed learning."
 Demonstrate understanding of the UCF course development process
for online courses (including timelines, preferred content formats,
and materials submission protocols).
 Reflect on your existing teaching strategies and consider their
suitability for inclusion in your online course.
 Collaborate with peers and facilitators (using online learning
technologies) to design an online course consistent with best
practices and protocols demonstrated in IDL6543.
 Formulate student-centered learning objectives / course outcomes
for your online course.
 Create assignments / activities for your online course based on your
learner outcomes.
IDL6543 Objectives
 Develop an assessment strategy for your online course based on your
learner outcomes.
 Assemble grading criteria for your online course based on your learner

outcomes and your assessment strategy.

 Compose Web-based content for your online course based on your learner

outcomes.
 Choose instructional graphics / media for your online course which

demonstrate visual literacy (e.g., constitutes an attractive, motivational,

instructional Web interface).
 Develop a set of student interaction protocols (for e-mail, discussions, etc.)

to be followed in your online course.

 Draft specific tactics for developing a "learning community" in your online

course based on the group work guidelines presented.

 Devise a strategy for implementing "learner support" in your online course.
Proposed Solution
 Competency-based instrument
 Artifacts showing evidence of experience
 Plan:
 Determine evaluation criteria
 Develop online submission form prototype
 Create assessment rubric
 Run a pilot test to validate instrument and establish
inter-rater reliability
 Revise as necessary and implement
Instrument Development
 Drafted by Tom based on Bb Greenhouse and QM
 Reviewed and revised by ID team
 IDs experienced as exemplary course reviewers
Criteria
 Online Teaching Record
 Previously taught online for a college or university?
 For which institutions?
 How many different courses?
 How many sections of those courses have you taught
online and how many students were in each section?
Criteria
 Online Teaching Record
 What was the delivery mode of the majority of the
courses?
 Have you won any awards for your online teaching?
 How did you address accessibility issues for students
with disabilities in your online course(s)?
 Have you ever been through an external assessment of
your online course(s), such as the Quality Matters
peer-review process?
Criteria
 Learning Management System
 Have you developed original online courses from scratch
or have you taught primarily from existing course
templates or content from a publisher?
 What course management programs have you used to
teach online?
 Which of the following technologies have you used in
your online courses?
 Animation/Simulation, Audio/Podcasts, Blogs, Course
Statistics, Calendars, Chat, Discussions, E-mail, External Media
(e.g., DVD, CD-ROM), Glossary, Gradebook, Graphics/images,
Performance Dashboard, Quizzes, RSS Feeds, Self Test, Student
Presentations, Survey, Video, Whiteboard, Wikis, Other
Criteria
 Professional Development
 Have you completed any training or professional
development related to online design and/or teaching?
Please describe the professional development and
provide certificates/transcripts, if available.
 Have you ever worked with an instructional designer
to develop an online course? Please describe how that
relationship worked.
Criteria
 Strategies
 Please provide detailed descriptions related to what you feel is
(are) your best online course(s). Areas you should consider
include:
 Organization and navigation
 Learner support strategies
 Course learning objectives
 Types of interactions and associated activities
 Assessments
 How you integrated technology
 Important or creative design features
 Course Design:
 Please describe in detail your instructional design strategy, including
structural and media decisions.
Criteria
 Interaction and Collaboration:
 Describe course interaction and collaboration strategies.
Include use of synchronous tools (such as whiteboards,
chat, webcasting) and asynchronous tools (such as email
and discussions).
 Also include how you addressed the different types of
interaction: student-to-student, student-to-instructor, and
student-to-content/technology.
 Did you use online group projects?
 Did you have netiquette expectations? How did all of these
strategies enhance student learning?
Criteria
 Assessment:
 Describe course assessment strategies. How did you ensure
that assignments aligned with course objectives? Did you
use rubrics? Did you use authentic assessment strategies
such as projects, portfolios or papers?
 Did you use scaffolding strategies?
 How did you try to ensure academic integrity in multiple
choice or other auto-graded tests? Did you use testing
centers or proctored exams?
 How did you provide performance feedback to students?
Provide example assessments, if possible.
Criteria
 Wrap-Up
 Please provide a syllabus from one or more of your
online courses.
 Is there any other information that you would like to
share that you feel would be helpful in evaluating your
readiness to teach online at UCF?
Demo
 If available
 Encouraged to
compose offlin
e
Screen Capture
Screen Capture
Interactive Activity
 Go to the handout titled – Sample Output
 Take a few minutes to go over the output
 How would you evaluate this individual’s online
experience?
 Why do you feel this way?
 How would you justify your evaluation?
Document Output
 Creates raw file of data
 Example
 Difficult to read
 Reformatted Example
Sample Output
Rubric Development
 Draft rubric has been developed
 MS Excel File
 Problematic in some areas: viewing entries
Demo
Challenges…PDF Version
Usability Testing
 ID review team
 Half testing with Excel
 Half testing with PDF
 Preliminary results
 Goal of inter-rater reliability
Instructional Design Review
 Considerations
 Anonymity
 Usability
 Readability
 Efficacy of point system
 Weighted criteria?
 Objective criterion referenced vs. subjective holistic
evaluation
EIS
 Demo of EIS data
Outcomes
 IDL6543 equivalency
 Requirement to complete the Essentials course for
IDL6543 equivalency
 Requirement to complete ADL5000 to deliver an
already-developed course
 Requirement to complete specified elements/modules
 E.g., Accessibility
 Requirement to complete IDL6543 in its entirety
 Provide additional materials (e.g., syllabus)
 *Meet to learn “UCF specific” policies and procedures
Remediation Modules
 Webcourses@UCF Essentials

 ADL5000
 https://obojobo.ucf.edu/

 Individual consultation
 Evaluation plan
 Coordinated by CDL’s Research Initiative for Teaching
Effectiveness
 Recruit small group of Faculty volunteers
 Complete the form to the best of their abilities
 Likely candidates new to UCF
 Recruit Instructional Designer Volunteers
 Review submissions and provide ratings
 Establish inter-rater reliability
 Usability testing of different methods
 Validate the instrument as an effective tool
Current Status
 Initial testing ongoing
 Refinements underway
 1 faculty member exempted from IDL
 1 faculty member required to take IDL6543
 Several more under review
Challenges
 Recruiting faculty participants
 System of tracking performance of “exempted”
faculty over time to see if quality results and
support needs are comparable to “trained” faculty
 Compensation issues
Special Thanks
Contributors to Project:
Baiyun Chen, Ph.D., Instructional Designer
George Bradford, Ph.D., Instructional Designer
Questions / Comments / Feedback
Thomas Cavanagh, Ph.D., Asst. VP, Distributed
Learning
cavanagh@mail.ucf.edu
Linda Futch, Ed.D., Assoc. Dept. Head
lsfutch@mail.ucf.edu
Nancy Swenson, Instructional Designer
nswenson@mail.ucf.edu