English Undrgrad Third Year Review Report - May 2008

Evaluation Rubric

English Third Year Review–2008
Describe any changes to the program's educational goals since the last Third Year Review, and the rationale for those changes.
In addition to the specific outcomes goals for student reading and writing identified in our portfolio assessment  rubric and the five Higher Learning Commission objectives referenced in our 2005 Third Year Review,  the department has added three educational goals: to develop curriculum aimed more directly at preparing students for writing in the workplace and in the computer age; to develop the Creative Writing component of our curriculum; and to build a stronger student community of English Majors.

Describe any assessment techniques used for measuring the Educational Goals that have been added or discontinued since the last Third Year Review, and the rationale for those changes.
We have updated the rubric used to score student portfolios in order to reflect the greater number and sophistication of professional and creative writing papers being generated by students. These changes reflect positive growth in both aspects of the curriculum since our 2005 Third Year Review.

We have improved our methods for collecting portfolio papers by having each faculty member identify the English majors in a given course, compile a portfolio of papers produced by those students during the semester, include the faculty member’s own assignments in the portfolio, and give the portfolio to the Associate Chair, who passes the papers along to each student’s advisor for review. This more complex system was needed in order to collect a representative sampling of student papers for departmental review.

We have supplemented an analysis of faculty syllabi with an analysis of faculty assignments based on the rubric developed for assessing student portfolios. We felt that assignments offered a better source of data for measuring the extent to which faculty communicate the department’s objectives to students.

We have recently added an Alumni Survey to be sent out every five years as appropriate in order to supplement the annual English Major Survey. The alumni survey is designed to keep better track of our graduates’ chosen professions and the role the English major has played within and beyond those professions. We feel this will be useful not only in assessing program outcomes and developing curriculum that addresses students’ career objectives but also in improving our recruitment and advising of majors. Along with the Alumni Survey, we sent out a departmental newsletter to keep alumni informed about changes in faculty, career trajectories of graduates we’ve heard from, and recent and upcoming departmental events.

We have discontinued efforts to incorporate students’ own assessment of their portfolios because this requirement did not fit well with the content of our Senior Seminar Capstone course and because we have so far identified no other appropriate venue in which each senior could produce this piece of analysis.

Attach any assessment instruments that have been used during the past three years, and the data collected, (or, summarized data, if that is more appropriate.)

What analysis has been done with this data? What conclusions has your department drawn?

1. Analysis and Conclusions Regarding Student Portfolios: The total number of portfolio papers collected (83) and the total number of students represented by this work (37) has increased steadily since 2005 and slightly since 2007. This reflects an improved procedure for collecting portfolio papers, and we plan to improve that collection procedure even more in the coming year. We devoted the March departmental meeting to the assessment of student portfolios in order to involve all faculty teaching in the major, and we committed part of the April meeting to a discussion of data collected through that assessment and its implications for how well we are communicating our core goals to students. Overall we were pleased by a marked improvement in student’s understanding and use of the “concept of genre” and their effective incorporation of “social and/or literary history as context,” two reading objectives we had stressed in the previous year’s assessment discussion and which we have been emphasizing in our assignments and class discussions. At the same time, we noticed a slight decline in student’s use of theory, which may have resulted from that shift in emphasis and is something we need to balance more. We were pleased by the overall competency and strength in writing skills and by a slight gain in research skills, though we may need to work on incorporating research into more assignments. We feel that students may have benefitted from a gradual increase in the number and variety of writing courses offered at the 200 level. We also noted that the weaker scores for reading and writing tended to represent students in the early courses and the stronger scores often reflect students’ improved abilities once they have reached the upper-level courses. We are pleased by this correlation because it was not as clearly the case in our 2005 assessment and we feel that students may have benefitted from a more coherent and committed faculty grasp of the core goals of our curriculum gained through repeated discussions of student work in relation to these objectives. Recent improvements in the Creative Writing curriculum, including committee-led restructuring of shared course goals and a commitment to adopting a critical vocabulary that students could use to analyze the assigned readings and their own writing are not yet showing up in the creative writing papers collected. We felt that 8 papers (some of which were individual poems or short collections of poems by a given student) could not be considered a representative sampling of the work students are producing in these courses and we plan to improve the collection and the scoring of these papers. 
2. Analysis and Conclusions Regarding Faculty Assignment Objectives: Although this was a preliminary study based on a newly designed instrument, our measurement of the objectives specified in faculty assignments as compared to the resulting student papers seems like a useful addition to our assessment techniques. It became evident that virtually every literature assignment and the majority of creative writing assignments emphasized analysis and interpretation (in reading) and college level exposition and skill in literary analysis (in writing). The correlation between these clearly and repetitively spelled-out assignment goals and the resulting strengths demonstrated in student papers was both heartening and instructive. If faculty assignments were to incorporate the same emphasis on other objectives (such as use of theory in reading or research skills in writing) student papers would likely improve in those areas. At the same time, we noted that faculty have given priority to the objectives that in fact we most value in the major. In effect the goals of analysis and interpretation and the skill in exposition are considered even more crucial than a student’s ability to incorporate genre, theory, historical context, and research skills. At the same time, more than half of the assignments collected do successfully implement those other objectives and are very important to the overall balance and success of the major. We plan to work on incorporating more of those objectives into every assignment; yet at the same time need to continue our discussion about weighing and balancing those priorities. We were also pleased by the degree to which the assignments were responsive to Higher Learning Commission goals in their representation of multicultural literatures and diverse perspectives, and by encouraging students to explore the social, ethical, and philosophical issues in literary works as a way to understand and develop their own academic values. 
3. Analysis and Conclusions Regarding Student Major Survey: The survey of our majors was only completed after our two assessment meetings, so although the data has been circulated and discussed through e-mail and on our shared departmental “h” drive, we are still planning further analysis. However, we are pleased to have collected 8 responses, since it has been difficult to collect a representative sampling of responses from seniors in past years. Our preliminary discussions have noted positive responses regarding faculty commitment, material covered within courses, and students’ awareness of their gain in skill. We have also noted several requests for a greater variety of courses and will try to respond to those requests as our faculty resources allow. We recognize what a valuable tool this can be and plan to improve our collection and analysis of these surveys.
4. Analysis and Conclusions Regarding Alumni Survey: The survey of our alumni was only completed after our two assessment meetings, so although the data has been circulated and discussed through e-mail and on our shared departmental “h” drive, we are still planning further analysis. However, our preliminary discussions of the 11 completed have noted an interesting variety in the professions adopted by our majors and in the skills adapted from the major into those professional settings. We plan to analyze these survey results thoroughly and incorporate some of that data into our planning for the major and into our advising work. We also hope to make those alumni responses available to current and future students by including them on our departmental Website

What changes have been made to the program as a result? (Curriculum, classes offered, classes discontinued, scheduling, advising, faculty education etc. . .)

1. We have improved faculty education by devoting part of a follow-up meeting to a continuing discussion of our assessment data, by focusing their attention on the correlation between their assignment objectives and the objectives used to evaluate student portfolios, and by making the results of our surveys and our assessment reports available to faculty on a shared “h” drive.
2. We have created a new range of W231, W232, and W234 professional and business writing courses which allow student within and beyond the major to prepare themselves for the workplace, to practice the kinds of writing that will strengthen their employment opportunities and will improve their success in workplace writing. We have also reinvigorated W315, Writing for the Web, again to give our majors a greater variety of writing-based options, but also to better prepare them for writing in the computer age.  
3. The Creative Writing side of the department has developed considerably since our 2005 Third Year Review through the addition of a new tenure-track creative writing faculty member, two full-time lecturers with creative writing credentials, and through the active involvement of many more faculty members on the Creative Writing Committee. This has resulted in more developed and coherent curriculum being taught in creative writing classes and in a host of creative writing events–visiting poets and writers giving readings and offering student workshops, performances showcasing our own creative writing faculty’s recent work, “Open Mic” nights where students have the opportunity to perform their work, and a public gallery presentation of student projects.
4. In addition to the inclusion of past and upcoming events on the English Department Website, the Creative Writing Committee Chair has also created a Blog that incorporates student photos, quotations, and comments while keeping them informed about conference, workshop, contest, and publication opportunities, upcoming readings and performances, and links to related Websites: http://iusbcreativewriting.wordpress.com/. Student responses to this influx of talent and energy have been very positive in terms of increasing enrollments in creative writing courses and in demands for new courses. This is an area of growth that students requested in response to previous and current English Major Surveys, so we are particularly delighted to see this development. 
5. Also in response to previous year’s student surveys, we have worked on building a stronger student community of English Majors through the Creative Writing Events detailed above, through a number of Film Series jointly organized by the Film Studies Program, and by creating an English Club in 2006. Led by two consecutive student presidents working with a faculty advisor, the English Club has conducted monthly writing workshops for students, hosted an open Mic. night for poetry and fiction readings, and has hosted a film series each semester. The lively core of students organizing these events has attracted more majors (and non-majors) to spend time together on campus eating pizza, sharing their work, and conversing. The English Department Lounge has also attracted a regular crowd of students working on joint assignments, preparing for various classes, sometimes working on collaborative assignments, and occasionally sharing impressions and concerns with faculty visitors. 

How did assessment data and analysis support these changes?

• Curriculum developments in professional, Web-based, and creative writing were deeply influenced by student surveys and by anecdotal evidence gained through advising regarding students’ career needs. Improvements in student community-building and involvement in on-campus events was spurred on by survey responses, though it also reflected an influx of faculty energy and commitment. Improvements in faculty education have certainly been encouraged and supported by repeated years of shared assessment of student work using a developed rubric and have resulted in improved faculty ability to implement those goals in their assignments and teaching practices. 

What changes does the department plan to make in the coming years to the program and to assessment techniques, and why?

• Because of our past difficulties in collecting completed English Major Surveys from a representative sampling of our seniors we have decided to ask student to complete the survey during the Senior Seminar which, as the capstone course for the major, should prove to be a better venue for collecting this data 
• We plan to continue improving our data collection by identifying students in the major on every faculty roster and asking faculty to supply every paper produced by those majors and to supply every assignment used in courses on the major. 
• We plan to use more of our assessment meeting to analyze faculty assignments in relation to the core objectives demonstrated in student portfolios. We also plan to use more of the follow-up meetings to analyze student and alumni surveys. This will improve faculty education and advising regarding the implementation of our core goals and the ramifications of those goals for students’ experiences entering and succeeding in the workplace.
•  We plan to continue the tradition of an occasional alumni newsletter in order to keep graduates better informed and to encourage our graduates to keep us posted on their career trajectories and shifting professional responsibilities.
•  Finally, we plan to circulate copies of our assessment tools, the resulting data collected, and our conclusions about the data on the departmental Website to better inform current and prospective students about the program.

How were faculty, students, administration, alumni and other groups involved in assessment?

• All faculty involved in teaching courses in the major shared in gathering data, scoring portfolios, and analyzing the data resulting from student portfolio reading in comparison to an evaluation of faculty assignments–something we have recognized as a crucial step in implementing those shared goals and planning future improvements. Students were involved through surveys and through anecdotal communications during coursework and in advising sessions, and alumni were involved through surveys.

How were assessment data and results shared with faculty, students, administration and alumni?

• Data assessment and results were shared with faculty during two faculty meetings, by e-mail, and on a shared “h” drive. Many of these objectives and assessment tools were also shared with the administrative committee preparing our campus for Higher Learning Commission re-accreditation.

In one paragraph, please summarize the most important impacts of the assessment of student learning on the program.

• Assessment of student learning has resulted in greater faculty involvement in compiling a rubric of shared goals and in better faculty implementation of those goals in their assignment objectives. This has arguably led to improved student achievement of several program objectives including a better sense of genre and of historical and cultural contexts for literary analysis. Assessment surveys have resulted in more student involvement in the development of the major program and have supported growth in professional, web-based, and creative writing. Student involvement in the assessment process has also contributed to the development of a stronger English Major community and greater student participation in campus events.

Is there any other information that you would like included in this report?

Since our last third-year review in 2005, we have made substantial changes to our departmental assessment process. These changes were a direct result of the workshop with Barbara Walvoord held on campus in 2005 and of our attempts to make the assessment process more useful for our program. Copies of the 2005-06 and 2006 Assessment Reports are attached; the 2007 Assessment Report will be available by May 1, 2008.