EVALUATION RESEARCH METHODS
FALL 2006 HCSV 425
Professor: Dr.
Lyndall Ellingson Office:
E-Mail: lellingson@csuchico.edu Telephone:
898-6310
Office Hours: MWF 9-11 am
*About
e-mail and telephone messages: If you
have an emergency that requires you to miss a class without prior notification,
it is a good idea to leave me a message before you leave town. However, leaving a message does not negate
your responsibilities for attendance and timely submission of papers.
COURSE SPECIFICS
Times & Location: Tuesday/Thursday
8 – 9:15 am
Required text: Babbie, E. (1999). The Basics of Social Research
(3rd ed.).
Recommended:
·
Hacker, Diana
(1999). A pocket style manual. (3rd
ed.).
·
American
Psychological Association (1994). Publication manual (4th
ed.)
·
Wagenaar, T.,
& Babbie, E. (2001). Practicing
social research (10th ed.).
COURSE DESCRIPTION
This course is an introduction to the language and methodology of evaluation research. It is also a writing proficiency (WP) course - a grade of C- or better certifies writing proficiency for majors. Thus, the course is designed to a) introduce, examine, and understand the basic concepts of research methodology in the context of evaluation and b) to evaluate and improve student writing skills. To facilitate improvement in writing skills written assignments may be re-submitted for additional evaluation. The course will comprise lecture, discussion, and group work.
SPECIFIC COURSE OBJECTIVES
At the conclusion of the
course, students will:
1) Identify and explain program evaluation in the context
of program planning;
2) Identify and explain evaluation types and theories;
3) Apply the basic concepts of research methodology to
program evaluation;
4) Develop scientifically sound measurement items;
5) Gain experience in data collection;
6)
Exhibit
proficient writing ability.
STUDENT RESPONSIBILITIES
1) Attendance
will be taken in all class sessions.
For each absence over 2, your final grade will be reduced 1/3
letter grade. Class sessions that have an asterisk are required to participate
in field work. Absence from these sessions will result in missing field work
and associated class credit.
2) Professionalism:
a) Arrive on time, prepared and ready to discuss readings
and issues.
b) Submit high quality assignments on time. Assignments
must be submitted at the beginning of the class session (within 15 minutes of
start time) in which they are due. Any
papers submitted after this time will be deducted 1/2 letter grade. A letter grade will be deducted for each additional
day late, including weekends. No
papers will be accepted after one week from the due date for any reason.
c) Professional preparation and demeanor in presentations
and interactions.
3) Requirements for
Written Papers:
a) Title page with name, class, date and title of
assignment (APA format)
b) 8.5” x 11” paper, one-inch margins on all sides, including
first page.
c) Stapled, typed, high quality (high contrast) print,
double-spaced, no larger than 12 point font
d) Free of typographical, grammatical, and spelling
errors
e) Proper and complete citation for all materials and
sources using APA style.
Ideas/information taken from others MUST be properly cited. Any direct
quote must be enclosed in quotation marks and must be cited.
f)
All sources must
be from 2000 or later.
Purpose: To encourage
timely reading and comprehension of lecture material and to prepare for exams.
Quizzes will be multiple choice and short answer questions covering the text
reading and lecture material covered since the previous quiz, but will NOT
include readings assigned the day of the quiz (see list on schedule). See
sample quiz questions on WebCT. Quiz questions may also appear on exams!
Students must purchase and
bring to class a Scantron form No. 886-E for each quiz.
No scantron = no quiz = no points.
Exams
Examinations will cover
lecture and readings and will include multiple choice, true/false, short
answer, and essay. Exams are conceptually cumulative. I.e., you may be asked to apply knowledge
covered in a previous exam on a later exam. No make-up exams without prior
approval of the instructor. Make-up exams are offered during finals week by
appointment.
Purpose: To assist in your
learning, encourage application of theoretical concepts, and to provide an
opportunity to improve your grade. You may earn up to (but no more than) 15
additional points towards your final grade. Each opportunity is worth 5
points. Quality of work will be
evaluated, thus you may earn anywhere from 0-5 points depending on the accuracy
of your work. Extra credit work is due on the days noted on the schedule. No
late work will be accepted.
Analyze assigned article
using McDermott & Sarvela’s Guiding Questions (see below). Keep in mind many of the concepts will be new
to you and will be covered in great detail throughout this semester. The purpose of this assignment is to
introduce these concepts and establish a framework within which to place the
content of this course. Your analysis should be in essay form, NOT
question/answer. Your analysis should cover the following questions:
1) Why?: What
were the objectives of the evaluation?
2) Whom?: What
kind(s) of people were the subjects in this study? (for example: gender? age?
ethnicity? health status? school children?)
3) What?: What
was studied &/or measured? (e.g., Knowledge? Attitudes? Behaviors?)
4) In very basic terms, describe the “intervention” or
“program” or (in research vocabulary) the “stimulus”?
5) Where/When?:
Where and when did the evaluation take place? (e.g., in a classroom? at home
via mail or telephone?)
6) How?: How
was the information (data) collected? (e.g., Was it a test? A survey?
Interviews?)
7) Who collected the data? (e.g., was independent
researchers? Program staff? Classroom Teachers?)
You
will be presenting your analysis in a small group in class on the day it is
due.
Analyze assigned article OR find one peer-reviewed research article
associated with your study question/issue/problem (Your instructor will tell
you which one). Spend some time looking, hopefully you will use this for your
literature review. Write a 200 word
abstract for each. Attach a complete photocopy/print out of each article
reviewed. You will be presenting
your analysis in a small group in class on the day it is due.
Address these questions in
your abstract:
1) What was the purpose of the study?
2) What kind of research study was it? (Experimental?
Descriptive? Exploratory? Etc)
3) What was the population sample? How large was the sample? Any other defining characteristics (gender,
age, etc.)? How were they selected? Was
there an experimental & control group?
4) What variables were measured?
5) How were the variables measured (survey? test?
Interviews? etc.)
6) What were the findings/results? Were they statistically significant? (be VERY
brief!)
7) What were the limitations of the study?
8) What were the implications of the study?
Research Proposal (225 pts)
This is the primary writing
assignment for this course. It should
represent your cumulative knowledge of research methodology. This assignment will be submitted in stages
as noted in the schedule. As noted on the schedule, your complete Proposal is
due on the Friday of the last week of finals week. This final submission will
be a formal complete submission of the stages (I – III). Grades earned on each stage will only
be awarded upon submission of the final and complete Report.
A. Significance of Problem
1. The extent and nature of the problem
2. Literature review of factors associated with the
subject of your study
B. Significance of Evaluation
1. The importance of program evaluation
2. Literature review of evaluation similar to your
study
A.
Organizational Overview
1.
Philosophy of Organization
2.
Organizational Context of program
B.
Program Overview
1.
Objectives of program
2.
Content of program
3.
Program procedures
A. Purpose of Study
B. Research questions
C. Study Design & Data Collection
D. Sampling & Subject Recruitment
E. Constraints & Limitations
F. Instrument*
G. Projected Data Analysis
Group Participation Points (15 pts)
You are required to work in
at least a 2 person group with a common research purpose. Groups will meet
continuously throughout the semester, sharing their ER stage assignments. Working groups may attend various research
activities associated with your project. Each group will also present their
report orally at the end of the semester.
You will be graded on your presence, active participation, contribution,
and preparation for group work, as well as the quality of your final
presentation. Non-participation in the final presentations will result in a
forfeiture of all participation points.
Course Evaluation:
Two Exams (100 pts/each) 200 pts
9 quizzes (15 pts each) 135 pts
Article Abstracts 50 pts
EP I: Program Overview
75 pts
EP II: Literature Review
75 pts
EP III: Methods 75 pts
Group participation 15 pts
635 pts
·
Attendance is taken in daily. For each
absence over 2, your final grade will be reduced 1/3 letter grade.
·
All writing
assignments may be re-submitted for a better grade. Re-submission can
earn up a 10% improvement on initial grade. Re-submission must be
received within one week of instructor’s return.
·
Overall Report
grade will not be awarded until Final Report is submitted.
·
To check up on
your progress/current grade/attendance records, log on to your WebCT account
for this course and go to “My Grades”.
These will be updated at least monthly.
If you note ANY discrepancies come to my office hours immediately to
rectify! Any discrepancies must be
rectified by December 1, 2006.
Grades in this course
follow CSU-Chico Academic Policy:
A:
B: Very Good Work - A high level of achievement clearly better than
adequate competence in the subject matter, but not as good as the unusual,
superior achievement of students earning an A.
C: Adequate Work - A level of achievement indicating adequate competence
in the subject matter. This level will
usually be met by a majority of students in the course.
D: Minimally Acceptable
Work - A level of achievement that
meets the minimum requirements of the course.
F: Unacceptable Work - A level of achievement that fails to meet the
minimum requirements of the course.
HEALTH EDUCATION STUDENT OUTCOME ASSESSMENTS
HCSV 425 meets, in part, the
following Program Learning Objectives in the Department of Health &
Community Services. Learning objectives
are based on the Competency-Based Framework for the Professional Development
of Certified Health Education Specialists, The National Commission for
Health Education Certification, Inc.
1. Assessing Individual and Community Needs for Health
Education (Resp. I)
a. Obtain health-related data about social and cultural
environments, growth, and development factors, needs, and interests.
2. Evaluate the effectiveness of health education
programs (Resp. IV).
a. Interpret results of program evaluation
3. Act as a Resource Person in Health
Education (Resp VI)
a. Utilize computerized health information retrieval
systems effectively.
b. Access principal on-line and other database health
information resources.
4. Communicating Health and Health Education
Needs, Concerns, and Resources (Resp. VII)
a. Demonstrate proficiency in communicating health
information and health education needs.
b. Evaluate the state-of-the-art health education.
c. Analyze the foundations of the discipline of health
education.
|
Date |
Subject |
Readings/Assignments |
Quiz |
Extra Credit |
|
8/22 |
Introductions |
|
|
|
|
8/24 |
Intro to Evaluation
Research |
R1 (pgs 6-15) & R2 (on
WebCT) |
|
|
|
8/29 |
Theories of Evaluation
Research |
Evaluation
Article Analysis
|
|
|
|
8/31 |
Research Projects & EP II |
|
Q1 |
EC1
|
|
9/5 |
Research Fundamentals |
B1 & B2 (pgs.42-56)
|
|
|
|
9/7 |
Research Fundamentals |
|
Q2 |
1.3 |
|
9/12 |
Research Design |
B4; Research Article Abstract |
|
|
|
9/14 |
Measurement Principles
(NOIR) |
B5 |
Q3 |
4.3 |
|
9/19 |
Measurement Principles
(rel) |
|
|
|
|
9/21 |
Measurement Principles
(val) |
EP II due |
|
|
|
9/26 |
Exam Review |
|
|
|
|
9/28 |
Exam 1 |
|
Q4 |
5.2; 5.5 |
|
10/3 |
Writing the Literature
Review |
|
|
|
|
10/5 |
Sampling Methods |
B7 |
|
|
|
10/10 |
Sampling Methods |
|
|
|
|
10/12 |
Experimental Research |
B8 |
Q5 |
7.1,7.2, 7.3 |
|
10/17 |
Experimental Research |
|
|
|
|
10/19 |
Survey Research |
B9 |
Q6 |
8.2, 8.3 |
|
10/24 |
Survey Research |
EP I due |
|
|
|
10/26 |
Instrument Development |
|
|
|
|
10/31 |
Qualitative Research |
B10 |
Q7 |
9.2 |
|
11/2 |
Qualitative Research |