Problem #6 - Difficulties in Assessing Outcomes Initially, assessment was viewed rather narrowly. At the beginning, the project team addressed assessment by creating the following two instruments: Attitudes Survey -This survey allows students to self-report their feelings about various aspects of the course and their impressions of the impact that the course had on their writing, interpersonal, and computer skills. Knowledge Exam - This multiple choice exam was designed to test a student's acquisition of knowledge in the areas covered by specific course subobjectives. This exam was also conducted on a pre- and post-test basis. After relying exclusively on these two assessment instruments for the first three semesters of the grant period (Fall, 1992; Spring, 1993; and Fall, 1993), the project team began to realize that, for several reasons, the assessment results were not particularly revealing as to what features of the new curriculum was working and what was not working: Time period - One semester - or even two semesters - is a very short period over which to expect to see significant changes in a student's writing skills or interpersonal skills. This is particularly true in a course, such as accounting, that is focused primarily on knowledge acquisition and only secondarily on development of writing and interpersonal skills. Lack of effort - To be diagnostic, knowledge exams--both pre- and post-test-- must be taken seriously by the student. After reviewing test results for the first three semesters and finding no clear patterns, the project team began to question whether students were in fact working hard on the exams. To address this question further, focus group sessions were conducted at the end of the Fall 1993. Among other things, students were asked about their performance on the skills exam. Students reported that, since the multiple choice test results were not used as a factor in a student's course grade, the exams were not being taken seriously by many students. Solution to Problem #6 Midway through the three-year grant period, the project team decided to expand its assessment activities considerably, adopting a much more comprehensive multiple methods/multiple measures approach. The following four new assessment methods were introduced at the end of the Fall 1993: Common Finals - The project team began to control carefully the semester-to-semester changes in the final exam for each course so that a fixed number of questions would continue to address the same issues at essentially the same level of difficulty. Exit Interviews - An exit interview questionnaire was designed for administration by our senior mentors in one-on-one interviews with randomly selected students in each course. Introduction to Accounting I mentors interviewed Introduction to Accounting I and vice versa. Exit Focus Groups - One non-randomly selected student group from each course met with an instructor from outside the accounting department who facilitated a discussion of their experience in their respective courses. Although the focus group sessions addressed the same issues covered in the exit interviews, students had the opportunity to explore other issues as well. Exit Questionnaires - These questionnaires, provided to students who were not involved in exit interviews or focus groups, addressed the same types of questions addressed in those other forums. The results of these and other assessment activities are discussed in the Outcomes Measures section, presented later. [ Previous | Next ] Home | Overview | Introduction | Motivation for Change Structure of the New Courses | Obstacles | Resources | Outcome Measures Reaction of Interested Parties | Recommendations for Interested Parties | Links | Index