IU South Bend CAMPUS ASSESSMENT COMMITTEE
2000-2001

Project Description:
Project Planning:
What went well, what did not:
Results of the Survey:
Budget:
Conclusions:

Plan to Create an Assessment of Student Learning Confidence and Effectiveness of Course Delivery

A Clear Description of the Project

This project allowed the faculty involved in the development of an online assessment instrument a space to try new forms of course/instructor/instruction assessment through an online forum. The team (faculty involved in this project) developed a timeline, identified needed training, support materials and pilots to try out items to be included on a comprehensive online course survey template that faculty could use throughout the campus. Unfortunately, the time line did not include temporary setbacks in reality. By the end of the spring semester, we were at the point of evaluating the results of the midterm surveys and were not able to pull together the information to pilot the end of the semester course assessment survey as we had hoped. The team (with the exception of one member) has agreed to continue working on this project throughout the 2003-04 academic year. Our hope is to provide templates for faculty and or department use prior to the end of the spring semester.
These templates will be available through Oncourse. The template will contain 3-4 sections. The first part of the survey will include a general set of questions to collect data that is of interest institutionally, such as program of study, gpa, student rank, amount of time spent on class work, etc. (These have been identified in a draft format at this time.) Second, the department will have items to select from which were taken from their current paper course evaluations. In the third section, faculty members can/will create a set of survey items to assess the level of learning/student knowledge and confidence on the course topics at approximately midterm in the semester. A final set of questions will assess the effectiveness of the learning environment of the course to the midterm point, (such as effectiveness of lectures, handouts, tests, classroom activities, etc) These responses will provide the faculty member feedback on the course structure, be able to identify strengths and weaknesses in the course, presentation, required prerequisite skills or knowledge, in addition to the general information provided on paper and pencil course evaluations.

The templates and bank of questions are currently being developed in Oncourse through the test and survey tool IU uses for survey and evaluation projects. This type of surveying is now available through Oncourse since there have been upgrades to allow faculty to use this format to collect student data, process it and return the data to faculty without a direct link to any student. Additional controls can be placed on the survey to allow students access to the system only once or repeatedly depending on the faculty need.

The team working on this project included faculty from the Health professions/Nursing, Education, Business, Informatics, and Dental.

Planning a project

a. The general goal for this proposal was to bring together a group of faculty from disciplines across campus. This group was directed to develop a test bank of assessment survey items for faculty to use. A trainer was brought up from IUPUI to train the initial set of faculty on the use of the survey tool. Then faculty worked one on one with the UCET consultant to review questions to be used in their midterm review. Each member was allowed to select support materials and books for their use in the development and analysis of the midterm survey and for the development of their end of semester course evaluations.

b. To complete this project, the lead team member attended the Ed-Media conference (June 21-28, 2003). Workshops were offered on how colleges were using online resources to assess student learning and classroom teaching effectiveness as well as on-line assessment and evaluation tools. In addition a workshop was held by the IU South Bend faculty to share the project timeline, and results to date. Those in attendance were asked to provide feedback and insight as to how the project could be improved. Attendance was good at the session and there was much discussion about the project. Most faculty were interested in the process, and although some faculty there found it surprising we were even bothering putting time into the evaluation process, there were no feedback comments offering improvement (unfortunately). Overall, those in attendance were very impressed and shared email and business cards with the session leader requesting follow-up information as the project proceeded.
c. The extension of this project is to submit this pilot assessment project to an IU funding source for course assessment and evaluation through IUPUI. Our hope was to complete this process over the summer, but at this point, that plan has been moved to the summer of 2004 when we will have additional data to use in developing that proposal. If we can convince others of the value of this project, the project will be expanded each year by providing further training of faculty in various departments on this campus for use of the survey tool and template development.

What went well, what did not

Upon notification of the grant approval, a training session was scheduled with the university wide Flashlight program/online assessment specialist from IUPUI. Unfortunately, prior to our training, she took a position with another institution. When our team received notification of her departure from the university, we contacted Jay Fern, Oncourse/Flashlight Director to schedule with another specialist. A date was set for late January for training by either himself or another qualified trainer. Unfortunately, on the date of our training, Indianapolis was shut down by a snow storm. We rescheduled for February, and again Indianapolis was closed down by a second blizzard. Again we rescheduled for early March - realizing that the training would come after the midterm break but hopefully close enough to still complete a Mid-Term assessment. This time the session was cancelled due to university scheduling, when we finally received the training, most were into their 11th week of classes. We decided to go with one survey for the semester and begin again in the fall.

Results of the Survey

Faculty received mixed responses from students. This was in part because each member chose their own items, and created additional items on their own - without input from the team. Some items were better written than others, some were the same as items use on the paper surveys, and were found to be no more or less effective on-line for responses. Faculty members also were diverse in how they presented the survey to their classes. Some gave additional credit for completing it, others gave time in class, and a couple just put it up in Oncourse as a schedule item.

The team found that the response rate from students ranged from 95%-20%. The key difference being those who promoted the survey in class and encouraged students to complete the survey and those that just posted it and let the students take it if they wanted to complete the survey.

Faculty also considered the usefulness of some of the responses - finding that students gave no better responses to the weaker items on the on-line format than they had on paper surveys. (Something we had predicted, but wanted to try out anyway.) Many felt their items provided good feedback and that the students that did respond took more time to complete the survey and took advantage of the ability to type in the responses - perhaps influencing the length of the responses.

Conclusion

This team is on the right track. There are not enough hours in the week to accomplish such a large project in such a short time. This will be a growing and evolving project. There is sufficient interest to continue the project even though the funding is exhausted. We believe the Assessment committee will be impressed with the on-going progress of the project. We will continue to update the Assessment committee of our progress and the availability of the templates at appropriate times.

Back to top of page