|
|
|
Department of Political Science
Prepared by: Dr. Elizabeth A. Bennion
Part I: Introduction
The Political Science Department seeks to prepare students to assume
the duties and obligations of citizenship, to provide special knowledge
and skills useful in public service, and to lay foundations for scholarly
study of government and politics. In order to achieve these goals, we
seek to create courses that enhance student knowledge, comprehension,
application, analysis, synthesis, and evaluation of political concepts.
This report outlines our recent attempts to improve our ability to assess
the degree to which we achieve these goals and objectives.
Part II: Assessment Methods Review
First, we sought a detailed and comprehensive list of course evaluation
questions currently being used on campus. We collected copies of these
materials and created a single list of questions used on these forms.
A copy of this summary is available upon request.
As the university considers standardized on-line assessment, perhaps allowing
for campus, divisional, department, and instructor designed sections,
we wanted to consider what types of questions we might use as a department
and as instructors. Reviewing these questions allowed us to reflect upon
our pedagogical goals in our individual courses and in the department
as a whole.
In the immediate future, we have decided to retain our qualitative course
assessment (student evaluation) instrument. We value students' thoughtful
and detailed reflections about our courses, and find this type of rich,
qualitative data to be most useful in revising and improving our courses.
Second, we sought a better understanding of the different program assessment
methods employed on campus. Informal interviews with department chairs
and program directors indicate that the following methods of assessment
are currently being used on campus:
1. Capstone Course
2. Alumni Survey
3. Graduating Student Survey
4. Course Portfolios
5. Program Portfolios
6. Focus Group Sessions
7. Comprehensive Post-Tests
8. Graduate Student Advisory Boards
9. Juried Performances and Exhibits
10. Computerized Testing of Particular Skills
Our program has decided to adopt the first three of these approaches,
while also considering the use of focus groups in the near future.
It would be helpful for the assessment committee to make a list of which
departments use each of these techniques and to post sample guidelines
and questions designed by each department on the assessment website. This
would increase intercampus communication and education about assessment.
We were unable to gather such extensive and systematic data through our
informal contacts with department/program heads.
Part III: Program Assessment Literature Review
Dr. Bennion worked with a political science major to conduct a review
of the program assessment literature. A copy of this essay is available
upon request. The review stresses the importance of clearly defined learning
objectives, and highlights six universal objectives of higher education
(Miller, et al. 1998), fourteen characteristics of an effective assessment
program (Center for Assessment and Research Studies, James Madison University,
1998), and six common approaches to program assessment (Kelly, et al.,
2003). The report distinguishes between indirect measures (exit interviews,
graduate school acceptance rates, job placement rates, and employer surveys)
and direct measures (comprehensive exams, national exams, local pretest-posttests,
performance assessments, portfolios, and capstone courses). Some departments
also use the senior thesis and internships as measures of knowledge retention,
skills acquisition and knowledge application. Additional direct assessment
techniques include the capstone oral exam and comparing entry versus exit
writing samples (see University of Georgia, 1988).
The Carnegie Foundation recommends three capstone requirements: senior
thesis, oral presentation, and portfolio preparation (Moore, 2003). The
Foundation also stresses that capstone courses should be tailored to measure
outcomes in a particular field while encouraging students to perform at
higher level learning by requiring them to engage in analysis, synthesis
and evaluation of past learning and apply it to new experiences.
We find that the most popular assessment techniques among political science
departments in the United States are the senior capstone course, faculty
observations, and exit interviews (Kelly et al, 2003). Truman State professor
John Ishiyama has recently argued for discipline-wide assessment guidelines.
In a recent article in PS: Political Science and Politics, Ishiyama provides
evidence that a structured program promotes student knowledge to a greater
degree than an unstructured program. In "the Case for Assessment"
(The Political Science Educator, April 2004) Ishiyama argues that political
science, as a whole, has done very little to develop systematic or uniform
assessment goals and techniques.
Part IV: Alumni Survey
We designed and administered an open-ended survey to graduating seniors
and past alumni. Because our department (as now constituted) was not in
place until five years ago, we focused on alumni graduating over the past
five years. We decided to use an open-ended qualitative approach due to
the small number of alumni as well as our desire to allow maximum opportunity
for original reflections and suggestions.
Students enrolled in the Senior Seminar completed a self-administered
survey, while alumni participated in a telephone survey. A copy of this
survey is available upon request.
We decided to use an alumni telephone interview due to the low response
rate obtained by most written surveys, particularly those with numerous
open-ended questions. The telephone interview questions were written by
the faculty, but conducted by a political science student. The student
was advised to: (1) introduce herself as a student, (2) use a chatty,
informal tone, (3) explain that we were seeking information about how
to improve our program, rather than information about a specific professor
or course, (4) stress that we value alumni feedback about their experiences
in the program, (5) gather information about what they are doing now,
and if/how the political science major helped them in their life after
graduation.
The surveys and interviews have been completed. We are currently examining
the results of these surveys. Overall, it appears that students were happy
with the program, though a few indicated that they wished that the program
had given them a greater understanding of research methodologies and polling
techniques that they could use in their lives beyond college. We will
consider alumni comments as we consider further changes to our program.
Part V: Changes Resulting from Grant Activities
The assessment grant allowed us to conduct an informal survey of current
assessment practices at IU South Bend, to compile a comprehensive list of student
evaluation questions employed on campus, to conduct a review of the literature
on program assessment in higher education (with an emphasis on political
science), and to conduct a survey of graduating seniors and alumni.
After reviewing the various assessment methods employed on our campus,
the department has decided to use faculty observation, the capstone course,
the exit interview, and alumni surveys as our basic tools of assessment.
The department relies heavily upon faculty observations in the classroom
and during independent study and internship supervision. We recently supplemented
these efforts with an exit interview and alumni survey to gather student
perceptions of program strengths and weaknesses. In accordance with the
Carnegie Foundation suggestions, the department requires a capstone "Senior
Seminar" which includes oral presentations and a major research paper.
These assignments are decided to encourage students to analyze, synthesize,
and evaluate new information based upon what they have learned throughout
the program. The seminar was recently designated as a "Writing Level
II" course, indicating that special attention is paid to honing and
developing the research and writing skills that students have acquired
as undergraduates.
Based upon Ishiyama's call for greater structure in political science
programs, and on alumni comments about the importance of research skills
development, we have decided to create (and require) a new 200-level course
covering the scope and methods of the field. While our department values
the flexibility that our major gives students and faculty, we have decided
that our students need some common understanding of the basic questions
addressed by political scientists and the diverse methodological approaches
political scientists use to study these questions. This course will include
a discussion of the various subfields of our discipline (as studied in
the United States). These fields include political theory, American politics,
comparative politics, and international relations. Students will also
learn about a variety of qualitative and quantitative techniques of investigation,
including interviews, focus groups, participant observation, content analysis,
close reading, surveys, and experiments. We believe that this course may
also be a place for faculty to present their own research to students
(who seldom have any idea what faculty do outside the classroom).
We believe that this course will give students the tools they need to
complete the final research paper required in the senior seminar. In addition,
by equipping students with a better understanding of various types and
modes of knowledge acquisition, we believe this course will better prepare
students for graduate school and for life beyond college.
Part VI: Conclusion
The data generated through our grant-funded activities have allowed our
department to re-evaluate our assessment plan. Ultimately, we have opted
to retain some of our current practices while changing others. In addition,
we are committed to a continued discussion of these ideas as we seek to
effectively maximize and measure the effectiveness of our program in meeting
our stated goals and objectives.
|
|
