Executive Summary
I. Educational Goals and Assessment Techniques
II.Instrumentation
III.Analysis of Assessment Data
IV Actions Taken
V. Proposed Next Steps

Executive Summary

Over the past three years, the School of Business and Economics has used the techniques of a capstone experience and a comprehensive post-test to assess student learning in four broad areas. Analysis of data from these assessment techniques indicates that undergraduate students are mastering discipline specific knowledge and higher order thinking skills, with some consistent differences in level of learning among disciplines. Improvement is occurring in the areas of basic academic success skills and of personal development and career preparation, although substantial variation remains among seniors. Few program changes have occurred over the past three years, because of other duties of the School's Planning and Assessment Committee and the Associate Dean. In the coming years, the School should improve implementation of its existing assessment plans by conducting interviews of graduating students and surveying employers on a more regular schedule. The School should address differences in student knowledge of specific disciplines. The School should also act to increase student skills in oral and written communication and team work, and address individual differences in these skills among seniors.

Educational Goals and Assessment Techniques

Our mission states that the School of Business and Economics (SBE) "… shall offer high quality educational programs that provide functional knowledge, skills, and capabilities to a diverse student body and enable the students to succeed in a dynamic business environment …" To fulfill this mission, we use several techniques to assess student learning in terms of five educational goals.

1. Discipline specific Knowledge and Skills (Learn terms, facts, concepts and theories of the subject. Develop skills in using materials, tools, and/or technology central to this subject. Prepare for graduate study.) Over the past three years, the SBE has used the techniques of a capstone experience and a comprehensive post-test to assess student learning of discipline specific knowledge and skills.

2. Personal Development and Career Preparation (Develop a capacity to think for one's self, be responsible for one's behavior, and to make wise decisions. Develop ability to work productively with others. Develop leadership skills.) Over the past three years, the SBE has used the capstone experience to assess student learning of personal development and career preparation.

3. Basic Academic Success Skills (Improve writing, speaking, reading, math, computer, and listening skills; develop study and presentation skills.) Over the past three years, the SBE has used the capstone experience to assess student learning of basic academic success skills.

4. Higher order Thinking Skills (Develop analytical and problem-solving skills, the ability to think creatively, to synthesize and integrate, to think holistically, and to recognize fact from fiction.) Over the past three years, the SBE has used the capstone experience and the comprehensive post-test to assess student learning of higher order thinking skills.

5. Academic Values (Develop openness to ideas, a concern for social issues, an informed historical perspective, an understanding of the role of science and technology, and an appreciation of other cultures. Develop the ability to make ethical choices.) Over the past three years, the SBE has not conducted interviews of graduating students or a survey of employers to assess student learning of academic values. (Interviews of graduating seniors are being conducted this month.)

Instrumentation

The capstone experience for SBE undergraduate students is the required course in Administrative Policy (J401). This senior-level course addresses the broad question of how a business enterprise as a whole is managed as opposed to how any one functional area (e.g., accounting) is managed. So, the capstone course serves as a good way to assess student learning of discipline specific knowledge and skills from prerequisite business courses. The capstone course also requires students to integrate ideas from several business disciplines, and so provides a means to assess higher order thinking skills. Some course assignments require students to apply skills in analysis, group work, and communication-both written and live presentation. Consequently, the course offers an opportunity to assess student learning of basic academic success skills as well as personal development and career preparation.

The comprehensive post-test is a graduation requirement for SBE students. Over the past three years, the School has administered the Educational Testing Service (ETS) Major Field Test in Business Administration to all graduating seniors. This instrument consists of 120 multiple-choice questions, some of which require the student to use diagrams, charts, and tables of data. The post-test directly assesses student learning of discipline specific knowledge and skills in eight functional areas of business (e.g., accounting). The post-test also assesses learning of higher order thinking skills.

Analysis of Assessment Data

In reflecting on the capstone experience as an assessment of SBE undergraduate students, the instructor made the following points. Student learning of discipline specific knowledge and skills from prerequisite business courses has not changed markedly over the past three years. "While most students seem to recall having studied most of the concepts, their ability to explain and apply these concepts can improve much." Student learning of team work skills (part of Personal Development and Career Preparation) seems to have improved over the past three years, due perhaps to increased experience with team work in prerequisite courses. Student learning of the basic academic success skills of writing and oral presentation also improved over the past three years, but clear individual differences in skill levels remain among seniors.

The comprehensive post-test scores (see attachment) indicate that, relative to roughly 400 other participating institutions, SBE graduating seniors placed in the top 1-2 percent in 2002, the top 2-10 percent in 2003, and the top 5-25% in 2004. (Earlier results indicate that our graduating students placed in the top 5-32% in 1998, the top 7-11% in 1999, the top 12% in 2000, and the top 3-8% in 2001.) Overall, these data suggest that SBE undergraduates have mastered discipline specific knowledge and skills and higher order thinking skills, relative to students at other business schools. (Although the instructor reported that students have difficulty explaining and applying discipline specific concepts from prerequisite courses, the capstone course apparently restores and enhances student learning to a very high level, relative to other institutions.)

Because the exact set of participating institutions changes with time, longitudinal comparisons of scores must be interpreted with caution. So, the slight decline in relative standing over the past three years (from top 1-2% to top 5-25%) could be due to a real decline in relative level of learning, differences in the number and nature of participating institutions, other sources of statistical error in the tested samples, or some combination of these factors.

ETS also reports relative performance of students in terms of their knowledge of a specific discipline (e.g., accounting). Over the past three years, the relative standing of SBE students' knowledge of international business seems especially strong. (This might reflect addition of a required course in international business to the SBE curriculum about six years ago, and/or the international expertise of our faculty.) In contrast, percentile rankings of our students' knowledge of some other disciplines, e.g., accounting and business law, have been less outstanding relative to students at other business schools.

Actions Taken

Annual reports over the past three years reflect few program changes as a result of assessment. The SBE Planning and Assessment Committee has been preoccupied with planning activities related to new AACSB standards. The Associate Dean, whose duties include assessment, has acted as Internal Dean for the past year. Consequently, although the 2002-2003 annual report describes a careful revision of the graduating student survey, it has still not been administered.

We are proud of the consistently high rankings of our graduating seniors on the comprehensive post-test. At new student orientations, we now note the graduation requirement and the outstanding performance by previous students. New students are encouraged to keep notes from earlier classes to aid in their preparation for the comprehensive post-test.

Proposed Next Steps

The School should improve implementation of its existing assessment plans. Interviews of graduating students are again underway, after a hiatus of several years. A survey of employers is also past due according to the schedule set up at the School's last accreditation. A plan for assessment within each discipline area was approved by the faculty several years ago but has not been fully implemented.

The School should use the ETS results about knowledge of specific disciplines. Perhaps in conjunction with assessment plans for each discipline, the School should investigate how we can enhance graduating senior knowledge about accounting and business law. Extrapolating from the apparent success in international business, we might consider increasing the coverage of these disciplines across the curriculum, including the capstone course.

The School should increase the skill of our graduating students in written and oral communication and team work. This emerged as the highest priority in a recent faculty survey of possible strategic action steps. We might increase assignments that use communication and team skills in courses across the curriculum. A related challenge is to find ways to help individual students who continue to struggle with such skills.

Dave Vollrath, Chair
SBE Planning and Assessment Committee

Go to top