|
|
|
ASSESSMENT RESULTS
SUMMARIES
The SPEA Criminal Justice assessment protocol includes pre and post testing as well as surveys utilized with exiting seniors, alumni, and employers. All students in the eight required core courses of the major are administered a pretest the first day of class and a post test toward the end of the semester. The exiting senior survey is administered in the J439 class each fall and spring semester. The alumni and employer surveys are administered approximately every third year. For alumni mailing, the alumni office provides a list of the Associate and Bachelor of Science in Criminal Justice graduates over the past five years. For employer mailing, a list is compiled of employers who have hired our graduates over the past several years. The list is created from the responses indicated on previous rounds of alumni surveys. The list of employers not only includes the local area but other surrounding counties and other locations in neighboring states that have hired our students.
This report provides (1) a comparison of the results of all rounds of assessment to date: first round fall 1997-fall 2000, second round spring 2001 – spring 2003, and third round fall 2003 – spring 2007, (2) highlights of the main results, and (3) a statement of the ways SPEA has used assessment results.
Comparison of the Three Rounds of Assessment
Testing
Faculty teaching the required core courses met to determine objectives for each of the eight classes. They then developed short multiple choice tests (10-15 questions) to be used for pre and post testing in each of the courses, except J439 – Crime and Public Policy - that is taught as a capstone course. In that class we utilize a short answer test. We began using this course embedded student learning outcome assessment strategy fall of 2006. The goal of this strategy is to assist faculty in determining whether course objectives are being met as demonstrated through significant improvement in test results from pre to post. The analysis of the fall and spring test results indicates that, overall, course objectives are being met. (The complete report is available through the IUSB Assessment homepage through the “grant” link).
Senior Exit Survey
We only started using the exiting senior survey in the fall of 2000, therefore no “first round” data is available for this assessment strategy. We changed some of the questions on it starting fall 2003. It is predominately a student satisfaction survey of the whole program including questions about course content, time and days of offering of required and elective courses, advising, availability of faculty, participation in and opinion of internship program, and intended job choice.
Second Round (spring 2001-spring 2003 plus fall 2000)
There were a total of 103 completed surveys.
71 students indicated that they participated in an internship during their years in the program (68%).
Overall, students indicated the major strengths of the program were class size (98%), availability of faculty (84%), and advising (78%). Assistance in job placement upon graduation was indicated as the weakest component of the program (78%).
Third Round (fall 2003-spring 2007)
There were a total of 129 completed surveys.
30 students indicated that they participated in an internship during their years in the program (23%).
Overall, students indicated the major strengths of the program were class size (98%), course content (93%), and availability of faculty (86%). Assistance in job placement upon graduation was indicated as the weakest component of the program (61%).
Alumni Survey
The goal of the alumni survey is primarily designed to determine whether the graduates of the program have found work that they are satisfied with, whether the work is in the area of their degree, and to gather feedback about the SPEA criminal justice program. We want to know if they felt academically prepared for employment and/or graduate school.
First Round (fall 1997-fall 2000)
160 surveys were mailed of which 33 were returned (21%)
The majority of the alumni who were employed indicated that their jobs were in the criminal justice field (60.6%). 72% indicated that they felt academically prepared to enter the workforce. 46% indicated that they participated in an internship during their time in SPEA. They indicated that the greatest strengths of the program were availability of faculty, class size, and course content. They indicated that the major weakness of the program was assistance in job placement upon graduation (100%) followed by advising (42%).
Second Round (spring 2001-spring 2003)
168 surveys were mailed of which 66 (39%) were returned.
Less than 50% of the alumni who were employed indicated that their jobs were in the criminal justice field (48%). 72% indicated that they felt academically prepared to enter the workforce (40% sufficiently and 32% to a great extent). 71% indicated that they felt academically prepared to enter graduate school. They indicated that the greatest strengths of the program were class size, availability of faculty, and advising. They indicated that the major weakness of the program was assistance in job placement upon graduation (78%). Only 14% indicated that advising was a weakness of the program.
Third Round (fall 2003-spring 2007)
181 surveys were mailed of which 44 (26%) were returned.
A little more than half of the alumni who were employed indicated that their jobs were in the criminal justice field (52.3%). 72% indicated that they felt academically prepared to enter the workforce (54% sufficiently and 18% to a great extent). 16% indicated that they are currently enrolled in a graduate program. Of those enrolled in a graduate program, the overwhelming majority (85%) indicated that they felt academically prepared to enter graduate school. Alumni indicated that the greatest strengths of the program were availability of faculty, class size and course content. They indicated that the major weakness of the program was assistance in job placement upon graduation (59%).
Employer Survey
The primary purpose of gathering feedback from employers of our graduates is to determine whether or not they feel that our students are prepared for employment in their agencies as well as to determine how important they feel specific areas are for employee success. We also want to determine their use of interns and their hiring trends of our students.
First round (fall 1997-fall 2000)
47 surveys were mailed of which 21 were returned (45%).
Employers indicated that the most important areas of content knowledge were in criminal law, communications and teamwork. They indicated that the areas where IUSB students were the best prepared were in basic knowledge of corrections with communications being the lowest followed by team work and awareness of social responsibilities.
Only 33% indicated that they had used interns from IUSB criminal justice.
On a scale of 0-4 (with 0 being not at all and 4 being extremely), the respondents’ overall mean of IUSB criminal justice preparedness to be successful employees was 2.8.
In comparison to other graduates they hire, they rated IUSB criminal justice graduates 2.68 (with 0 being less and 4 being better).
Second round (spring 2001-spring 2003)
39 surveys were mailed of which 23 were returned (59%).
Employers indicated that the most important areas of content knowledge were teamwork, followed by awareness of social responsibilities and general education. They indicated that the areas where IUSB students were the best prepared were in basic knowledge of law enforcement with teamwork and awareness of social responsibilities rated the lowest.
73% of the respondents indicated that they had used IUSB criminal justice interns.
On a scale of 0-4, the respondents’ overall mean of IUSB criminal justice preparedness to be successful employees was 2.54.
In comparison to other graduates they hire, they rated IUSB criminal justice graduates 2.46 (with 0 being less and 4 being better).
Third round (fall 2003-spring 2007)
65 surveys were mailed of which 23 were returned (35%).
Employers indicated that the most important areas for employee success were general education, written and oral communications, computers, teamwork and awareness of social responsibilities. They indicated that the area of least importance was grant writing, math and science. They went on to indicate that the areas where IUSB students were the best prepared were in general education, written and oral communications, computers, teamwork, and awareness of social responsibilities. Employers indicated that students were the least prepared in grant writing, math, and science.
30% of the respondents indicated that they had used IUSB criminal justice interns.
On a scale of 0-4, the respondents’ overall mean of IUSB criminal justice preparedness to be successful employees was 2.76.
In comparison to other graduates they hire, they rated IUSB criminal justice graduates 2.06 (with 0 being less and 4 being better).
Highlights of Main Results
It was rewarding to determine that in the third round of survey responses from employers there is agreement between what employers stated were the most important areas for employee success and the areas where IUSB students were the best prepared (general education, communications, computers, teamwork, and social responsibilities. In the previous two rounds of assessment, there were disparities between what employers had indicated were the most important areas for employee success and the areas where IUSB students were the best prepared.
Unfortunately, in rating IUSB graduates in comparison to graduates they employ from other institutions, the rating has consistently been decreasing (2.68 to 2.46 to 2.06). This is an area that needs additional exploration in order for our Program to plan any corrective actions.
On a positive note, although assistance with job placement has consistently been noted as a weakness by exiting seniors and alumni alike, the percent indicating it as a weakness has steadily declined (exiting seniors: 78% to 61% and alumni: 100% to 78% to 59%).
Class size and availability of faculty were indicated to be the greatest strengths of the Program by exiting seniors and alumni alike with course content and advising also seen as strengths by more than 75% of the respondents.
Use of Assessment Results
As is demonstrated through the highlights of the results in comparing the three rounds of assessment, the changes that we made to improve on student preparedness to meet employer expectations have succeeded. Several years ago we had a faculty retreat that focused on how to improve student writing skills and, incorporating various strategies appears to have positive outcomes. Faculty incorporated a variety of assignments into the course requirements in a number of classes which focus on knowledge of current events specific to criminal justice and this appears to have improved student awareness of social responsibilities. Assignments in a number of courses also require demonstrations of proficiency in computer programs such as word and excel spreadsheets. Requirements for team activities have also increased in a number of courses.
Three years ago a career development class was added to the spring course rotation of classes. The Criminal Justice Association, with assistance from the Criminal Justice faculty, hosted a career fair specific to the field, in the spring of 2007. Four years ago we began placing job announcements on a bulletin board in a centralized location. All of these efforts appear to have improved the students’ (both exiting senior and alumni) ratings of the assistance they are provided in job placement. We will continue to explore ways through which we can provide better assistance in job placement.
The course embedded student learning outcome assessment strategy utilizing pre and post testing has only been in effect for one year. In three years, when we have more data, we will be able to report on the results of this strategy and compare the impact of corrective actions taking place to improve student learning outcomes where needed. Based on the first year of results the faculty have already met to incorporate changes in assignments and lecture to improve in specific areas noted, through data results, to need improvement.
Conclusion
The Criminal Justice Program incorporates a combination of direct (student learning outcomes) and indirect (student, alumni, and employer ratings) data collection strategies for assessment. We have found this to provide us with valuable information that can be used for continuous quality improvement. We will continue to explore ways through which we not only gather information relevant to assessment of our Program but will also focus on development of strategies to improve both student learning outcomes and student and employer satisfaction.
|
|
