Undergraduate Education Programs Third Year Review Written Report

Evaluation Rubric

Undergraduate Programs include Elementary, Secondary, and Special Education
1. Assessment contact person, and person preparing the report, (if different):
Program Coordinators: Dr. Lee Smith, Dr. Susan Cress, Dr. Kwadwo Okrah
Assistant Dean: Dr. Karen Clark
2. Attach 2005 and 2006 annual reports (See Attachment One: Annual Reports)
2005 Annual Reports: Elementary, Special Education, Secondary Education
2006 Annual Reports: Elementary, Special Education
3. Attach an updated departmental assessment plan
Each program area assesses undergraduate candidates (university students) at designated checkpoints during the program. Artifacts, grades, PRAXIS scores, GPA, grades in specific classes, field experience evaluations, and student teaching data are some of the data reviewed at these checkpoints. The National Council for the Accreditation for Teacher Education (NCATE) also requires us to assess professional dispositions. These are assessed in a class titled Exploring the Personal Demands of Teaching for undergraduate students. They are also assessed during the student teaching experience. Professional dispositions are monitored using letters of concern. Attachments outline each checkpoint for each program area. (Attachment Two: Checkpoint Information)
4. Describe any changes to the program’s educational goals since the last Third Year Review, and the rationale for those changes.
Undergraduate programs have been aligned for several years with ten IUSB Standards which are based on the INTASC Principles. Each program also assesses the professional dispositions of undergraduate candidates. Attached are a list of the IUSB Standards and Professional Dispositions. These lists represent the goals for undergraduate programs in the School of Education. (Attachment Three: IUSB Standards and Dispositions)
5. Describe any assessment techniques used for measuring the Educational Goals that have been added or discontinued since the last Third Year Review and the rationale for those changes.
During 2006-2007, the unit has adopted a new rubric for assessing artifacts from undergraduate or initial licensure candidates. (Attachment Four: New Rubric for Initial Candidates) Student teaching evaluation forms have been revised to show alignment with the standard (Attachment Four: Student Teaching Forms). Recommendations will be made in April 2007 about revising undergraduate checkpoints.

6. Attach any assessment instruments that have been used during the past three years, and the data collected, (or, summarized data, if that is more appropriate.)
Attached are charts that summarize PRAXIS I Scores, PRAXIS II scores, and student teaching data for undergraduate students through the spring 2006. Charts that summarize student teaching data for fall 2006 and spring 2007 are not yet available.  In addition, charts are attached that summarize assessments of artifacts for elementary, secondary, and special education students through summer 2006. Charts summarizing data for fall 2006 and spring 2007 are not yet available. (Attachment Five: Assessment Retreat Data)
7. What analysis has been done with this data?  What conclusions has your department drawn?
The attached charts show analysis of the data. Data are reviewed each August at an annual assessment retreat. At the retreat program faculty review the data, discuss implications, and begin discussions about programmatic changes.
8. What changes have been made to the program as a result? (Curriculum, classes offered, classes discontinued, scheduling, advising, faculty education etc. . .) 
Elementary program faculty reviewed the data and concluded that the PRAXIS scores over the last three years, in reading, writing, and math remain high; no further interventions are deemed necessary at this time.
Data contained in the Letters of Concern for elementary program students (19 for 2005/2006) have centered around one of three issues. The most frequent is absenteeism, the next is poor performance, and the third is plagiarism. In terms of absenteeism policies are clearly stated in each syllabus to communicate the importance of regular attendance. Poor performance is dealt with on an individual basis and does not seem to be reflected in any one elementary class. The Elementary faculty has focused on plagiarism issues by making sure that students in each upper level block of courses have passed the online IU plagiarism tutorial.
In reviewing the summary of student teaching issues over the past three years, classroom management is the most frequently noted problem. In the newly revised program the elementary faculty has made a concerted effort to address classroom management in terms of classroom environment, teacher behavior, positive learning experiences, and diversity in all methods classes.
Elementary education faculty have only one semester of checkpoint information covering the students in the revised program, but after the first semester (spring 2006), 60% of students were rated at the highest level (outstanding) after student teaching in all of the standards and 40% were rated at the next level (proficient). The faculty will continue to review data as subsequent cohorts of students complete the new elementary education program.
Secondary program faculty has also made changes to their programs. In fall 2006 the state of Indiana revoked its decision not to allow holders of Rules 46/47 licenses to add minors and endorsements. In response to this, the secondary education program has worked to reinstate minors and endorsements for those holding those licenses as well as those eligible under Rules 2002.
It was decided to reinstate these programs using the new licensing standards under Rules 2002. As a result, current major programs were reexamined at the same time to assure that majors, minors, and endorsements were appropriately realigned. Information is attached which provides content major matrices of standards and how four of our secondary education content areas currently align with these standards.
In addition, information on each of these areas-English/Language Arts, Social Studies, Science (Chemistry, Earth Space, Life Science, Physical Science, Physics), Mathematics, and Foreign Languages-is also provided in Bulletin form specifying in detail the content and requirements of each of the majors, minors and endorsements.
One other components of this segment is a (new to IUSB) matching course from Bloomington proposed and developed for English/Language Arts Secondary Education majors: EDUC M412 Methods of Teaching Writing.  
Special education program faculty view assessment data collected in 2005 to 2007, has provided a baseline for future improvement. Faculty evaluation of assignments and the subsequent rubric scores indicated nearly all students met the standards based criteria across the 10 standards areas. Results of assessment data from individual students and from the follow-up studies of undergraduate and graduate students indicated general satisfaction with the program outcomes; indicating that maintenance of the curriculum was an acceptable action.
Further evaluation mechanisms are being utilized in special education through a   continuous improvement model (as opposed to an objectives or standards based model) to ascertain the extent of two of the relatively weaker areas: collaboration and technology, and to determine possible areas of change or growth.  In the last few weeks (i.e. March 2007) data has been collected in focus groups of external stakeholders to inform the program faculty. This will be discussed with the special education faculty in the April program meeting.    
9. How did assessment data and analysis support these changes? 
All undergraduate program areas continuously analyze checkpoint data, letters of concern, issues resolutions, and student teaching issues at the yearly assessment retreat. As a result of this analysis, goals for the year can be reported, changes as outlined in #8 can be made, and then follow-up on changes can be assessed the following year.  The image of a circular assessment system where data and analysis, followed by changes in implementation, followed by data which is then analyzed in light of the changes is “best practice” in curriculum revisions and implementation. All changes were carefully tailored toward the standards as listed in our Unit Assessment System.
10. What changes does the department plan to make in the coming years to the program and to assessment techniques, and why?
Assessment is a continuous process in the School of Education. In the coming year, the School of Education Assessment Committee will make recommendations to simplify the unit assessment system and to implement a new database system. In addition, NCATE charges the SOE with establishing reliability, validity, and consistency of data. Steps will be taken in the coming year to meet expectations in these areas.
The elementary program faculty will continue to review data from artifacts, field experience evaluations, and student teaching data on an ongoing basis.
The Secondary Education Program has also had the opportunity to assess several semesters of students in two different placements at different developmental areas. We also now have experience with students attempting to meet the requirements of the IMAP portfolio, our Checkpoint 3 requirement for graduation. This experience has informed us that these multiple demands were excessive as our students can often be responsible for over 150 students in one placement alone, making it difficult for them to meet their students’ academic needs as well as their own student teaching requirements.  In addition, eight weeks in two very different school settings was not working well for student teachers to develop appropriate relationships with students or cooperating teachers or to have time to make such a major transition easily. As a result, the program faculty has voted to replace the two student teaching placements we are currently using:
EDUC-M 451 - Student Teaching: Jr. High/Middle School - 6 cr.
EDUC-M 480 - Student Teaching: Secondary School - 6 cr.
 With
EDUC-M 480 - Student Teaching: Senior High/Jr. High/Middle School – 10cr
EDUC-M 420 - Professional Development Seminar – 2cr
The faculty moved and voted to approve the following: Change Student Teaching to one placement for 10 credit hours. Students will experience both developmental levels thorough a combination of field experiences and student teaching with documentation on a new Field Experience Tracking Form. For example, a student who completes M480 in a high school would have had a middle school field experience with documentation earlier in the program. It will still be possible to combine or blend the one secondary content student teaching experience with special education, ENL or reading.
For the next year, students adding the special education license will do one placement of 9cr in content area, 3cr in special education and 2cr of student teaching seminar (M420). Similar blending will be true for ENL and reading with another content area.
Require M420 Professional Development Seminar for 2 credit hours. This course will primarily focus on the IMAP Portfolio and other practical teacher preparation issues. We are finding a need for more time to meet and support student teachers than the current zero credit seminar provides.
Student Teaching Semester (co-requisites) M480 Student Teaching: Senior High/Junior High/Middle School (10 cr.) M420 Professional Development Seminar (2 cr.) [Course new to IUSB campus]
In special education, this year, additional data was collected through focus groups with stakeholders in the special education program utilizing evaluation techniques more commonly used for program change and development within a continuous improvement model. As is required by the accreditation agency (NCATE) data will be collected to ascertain the effectiveness of meeting standards, as is paramount in objective based assessment systems. 
11. How were faculty, students, administration, alumni and other groups involved in assessment?
                        Faculty is actively involved with assessment activities. They are required to assess student artifacts at designated checkpoints. Students submit artifacts via LiveText. They receive completed rubrics from faculty. In addition, all student teaching assessments are discussed with students. Assessments of professional dispositions are shared in the class Exploring the Personal Demands of Teaching. By policy, any letters of concern written about a student must be shared with and discussed with the student.
12. How were assessment data and results shared with faculty, students, administration and alumni?
An Assessment Retreat is held each August for faculty and administrators in the School of Education. During the retreat data including the following are reviewed:

• PRAXIS I scores
• PRAXIS II scores
• Student teaching Data
• State report on First-year internship required of all new teachers
• Student teaching issues report
• Letters of Concern Report
• Summaries of Artifact Assessments

Data are shared with area educators as advisory group meetings. During the 2006-2007 academic year, the special education program has shared data with their advisory group. Elementary program faculty will meet with their advisory group before the end of the semester. Secondary have not scheduled advisory group meetings to date.
In the future, assessment results will be shared with faculty in the College of Liberal Arts and Sciences at schedule UTEC meetings. In addition group meetings will be schedule to share results with students in fall 2007.
13. In one paragraph, please summarize the most important impacts of the assessment of student learning on the program.
The curriculum of the School of Education is driven by assessment information in light of an accreditation process which looks at performance based assessment.  By providing defined checkpoints and looking at  individual assessments, department assessments, and unit assessments, faculty are constantly informed as to the strengths and weaknesses in the program.  Not only are faculty able to provide remediation for individual students, but they can recommend curricular changes and revise instruction to positively impact weaknesses in the program. Assessment of student learning outcomes has contributed to improvement of syllabi, and alignment of course objectives and performance standards. Faculty use of formative assessment in the classroom to provide feedback to both students and teachers.
The assessment system (also know as the performance measurement system) assist the special education department in refining both the curricular goals in its program and also the assignments that are utilized to evaluate the students on the standards and curriculum objectives.  Furthermore, by comparing the instructor’s evaluations across goals  about individual students and groups of students (i.e. means scores) with both the quantifiable score from follow-up surveys and textual narrative reports from the open ended questions and focus groups the assessment of student learning is supported by both internal and external types of validity. Survey data will be available at the presentation.

Attachment One:
Annual Assessment Reports

Attachment Two:
Checkpoint Assessments

Attachment Three:
IUSB Standards
IUSB Dispositions

Attachment Four:
New Rubric for
Initial/Undergraduate Candidates
Student Teaching Forms

Attachment Five:
Data from August 2006 Assessment Retreat

top