Designing and Evaluating Online Instruction
Tigers Project 2003 Summary
http://www.csuchico.edu/~lseder/eoi

Laura Sederberg, ITC,
Technology & Learning Program

 

Project Objective

The overriding objective was for CSU-wide implementation of a method for evaluating online instruction. We realized that some campuses used WebCT and others Blackboard. We wanted to see if the "Rubric for Online Instruction" was adaptable across the LMS and if we felt that we could initiate a system-wide implementation.

As we began to discuss this goal, it became obvious that beyond the evaluation of online instruction, demonstration of online instruction by examples was as critical, if not more important to point people in the right direction for development of online courses.

Project Summary

Members of our team included: Gayle Burns, CSU-LA; Judith Norton, CVC; Laura Sederberg, CSU-Chico; Joan Van Duzer, CSU-Humboldt.

Each member of this TIGERS committee reviewed the Rubric for Online Instruction and Exemplary Online Instruction Program as set forth by CSU, Chico. Each decided how to talk about it with her community of faculty/IT staff and agreed to test it on her campus.

Humboldt State University applied the Rubric for Online Instruction (ROI) to two Blackboard courses, allowing the committee access to review the courses following Chico's program. Although both campuses use different Learning Management Systems (Chico uses WebCT and Humboldt uses Blackboard), it didn't matter for applying the ROI.

Cal State LA is working on a centralized approach to implementing the EOI Rubric across the university by introducing it to the campus WebCT Users Group.

California Virtual Campus (CVC) applied the ROI to training provided to community colleges. They already had a means of showcasing exemplary instruction, but not with a rubric, and so now they have introduced how to apply that to courses via a training course available to anyone who joins their Professional Development Center web site, http://pdc.cvc.edu/common/. (Note: it is not necessary to be a member of CVC in order to join.) Judith Norton presented at the Annual ASTD Conference, "You've Gone Online...Now It's Time to Evaluate Your Course!"

The committee gave feedback to Chico State on its process and the Web site showing examples of courses with exemplary online instruction. Feeback was incorporated into the Web site. It is recommended that the Web site be expanded to show examples of not only exemplary rated online instruction, but for comparison purposes to also show effective or baseline rated examples. This will take some time to add.

This project will be presented at the Annual CATS Conference in March 2004. A similar presentation (hands-on workshop will be held at the WebCT Annual Users Conference in July 2004. The presentation will include hands-on involvement with the Rubric for Online Instruction, as well as collaboration with others on determining rubric categories and criteria for measure exemplary online instruction. A discussion, as part of this, is most important.

The outcome of this project was multiple.

  • We all got better acquainted with how different our individual universities (or organizations) handles this process.
  • We each applied the Rubric for Online Instruction in a different way and compared notes.
  • We all agreed that there was not one way to implement this program to all CSUs.
  • We all agreed to share our resources, and continue to add to them as we can.
  • We decided to create a Web site of resources for the public, linked from CATS Web site.
  • We agreed that the Rubric for Online Instruction could be applied across LMS, in a variety of ways from institutional Exemplary Online Instruction Awards (like from Chico), to individual department application, to individual instructor self-assessments.
  • We know that including a module on this into training for new faculty developing online courses is a good idea, and as important in the design phase as in the evaluation phase.
  • CVC sharing this training module with the CSU is a step in the right direction for sharing resources in Higher Education.
  • There are many other resources available out there, that we can all share with each other.
  • This is a work in progress.

Lessons Learned

  • CSU campuses are very diverse in their approach to providing faculty training, IT-to-administration relationships, handling faculty development issues, and recognition of faculty successes. There was not ONE WAY or EASY WAY to implement a program across the system. However, we share lots of common problems and ideas for solutions.
  • Our TIGERS team was committed and attended our teleconference meetings.
  • Rubric for Online Instruction works well across both LMS's, WebCT and Blackboard!
  • All team members worked beyond the hours covered in the proposed budget (isn't that true of all TIGERS teams?).
  • We were very fortunate to have Judith Norton, volunteer member from CVC, participate with us. Community colleges and CSUs have a lot in common to share/benefit.
  • Instead of a deliverable of a HorizonLive session, we decided a Web site would be more beneficial, as a living an growing document of resources, pointing to examples of exemplary online instruction, changing dynamically as it evolves.

Recommended Links

Exemplary Online Instruction program web site from CSU, Chico
Including examples of exemplary online instruction based on the categories of the Rubric for Online Instruction and the process on annual EOI Awards at the CELT Conference each fall.
http://www.csuchico.edu/celt/roi

Rubric for Online Instruction
(Six categories with criteria spelled out for three rankings).
This rubric is currently being used by approximately 25 institutions, across the USA.
It is featured online at these web sites: e-Sloan Consortium, Educause, and MERLOT.
http://www.csuchico.edu/tlp/webct/rubric/rubric_final.pdf

Instructional Design Tips for Online Instruction
For use with the Rubric for Online Instruction, one page per category and resources.
Compiled and edited by Joan Van Duzer, Humboldt State, University.
http://www.humboldt.edu/~jdv1/InstructionalDesignTips.pdf

ASTD Web Site (American Society for Training & Development)
http://www.astd.org/astd

ASTD Conference Presentation, "You've Gone Online...Now It's Time to Evaluate Your Course!"
Feb 2004 by Judith Norton (Word document). Resources are particularly relevant.

Educause Web Site (for conferences, effective practices, collaborations)
http://www.educause.edu

Distance Education Report and other higher ed. publications recommended reading
http://www.magnapubs.com

CIT Infobits online is an electronic service of the University of North Carolina at Chapel Hill ITS Center for Instructional Technology.
http://www.unc.edu/cit/infobits/

Virtual Instructional Designer web site
http://thevid.org

Web Style Guide (detailed document of all phases of web page design)
http://info.med.yale.edu/caim/manual/contents.html


Recommended Articles

Rubric Clearly Describes Exemplary Online Instruction
by Laura Sederberg as published in
Distance Education Report, Vol. 7, No. 23, December 1, 2003 (in PDF format)

Learner-Centered Theory and Practice in Distance Education: 
Cases from Higher Education (Word doc converted to html)
In the article, the authors examine theoretically-based distance learning environments in higher education and discuss issues of theory, pedagogy, design, assessment, communities of practice, collaboration, and faculty development. The descriptions, strategies, and principles discussed potentially inform the design of continuing education, as well as degree-based education and corporate education and training, and distance education programs for adults.
Edited by Thomas M. Duffy and Jamie R. Kirkley, Indiana University. All rights reserved. ISBN: 0-8058-4576-3. Lawrence Erlbaum and Associates. 10 Industrial Avenue, Mahwah, NJ 07430-2262 http://erlbaum.com/ Reprinted with permission. Suggested by Rick Reis, reis@stanford.edu

Criteria for Evaluating the Quality of Online Courses
Clayton R. Wright, wrightc@macewan.ca
Instructional Media and Design
Grant MacEwan College, Edmonton, Alberta T5J 4S2
The criteria presented in this guide are based on the national and international experiences of staff in the Instructional Media and Design department at Grant MacEwan College. Although they were developed to assist educators in evaluating the effectiveness of online courses, they may also be used as guidelines for course developers.

Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses
by L. Dee Fink (Shared by University of Oklahoma Significant Learning Website)
Complimentary web site to the book, talking about "Significant Learning for Significant Living".

http://www.csuchico.edu/~lseder/eoi/
Submitted and maintained by Laura Sederberg
April 2, 2004