CHANGING
PRACTICE BASED ON COURSE
EVALUATIONS RESULTING IN OUTCOMES
The Department of Nurse Education has created a computerized application, to facilitate the collection, aggregating and trending of data used to evaluate program effectiveness. The Nurse Education Computer Application (NECA) otherwise known as the computerized database is a total information system that provides for ongoing and immediate program analysis utilizing student and faculty evaluations to measure program effectiveness as outlined in the systematic plan for program evaluation and assessment of outcomes as required by the National League for Nursing Accrediting Commission.
Listed below are the goals and functions of the NECA:
- Provide
a systematic structure to discover, document, and analyze the Nurse
Education program’s many courses, labs and clinical affiliates.
In effect, evaluate the programs performance as viewed by the
students. The success of the
students after graduation is also a goal of the program and data on that
success must somehow be obtained.
- The
system must be easily accessible to students, faculty, the department
administrator and coordinators. It
must be secure to each group, allowing students access to information
gathering areas only. Faculty
access to view their group’s results only.
The administrator and coordinators of the program have access to all
aspects.
- The
system must be capable of storing and tracking each program student’s
demographic information, course grades, and a wide range of factors that may
be correlated to a student’s success or failure in the program.
This correlation may be used to predict a student’s potential
success allowing the program to better evaluate its admittance standards and
provides the ability to identify highly motivated students who may need
additional help to achieve their goals.
- All
evaluations provided by the students must be anonymous, yet the system must
record that the student did complete and submit an evaluation.
This insures full yet anonymous participation by the students.
- The
format of the discovery vehicle must minimize human intervention, allowing
the production of consistently viable results that will be analyzed
automatically by computer. Secondarily,
more general (open ended) information must also be gathered as a means to
allow students to express their thoughts on the various courses and the
program as a whole.
- The
system used must be capable analyzing the information gathered
automatically. The processed
information must be displayed in both a graphic and numerical fashion
allowing rapid visual evaluation.
- The
Information store must be searchable by many varied factors, but primarily
course, year and semester, thereby
allowing comparisons of the same course, year-to-year, semester-to-semester
and obtaining documentation of the cause and effect of any changes.
- A research committee must be in effect to evaluate the discovery results at the end of each semester or as the need arises to decide on whatever action must be taken to make an improvement.
Change in nursing practice is based on the evaluation process. All nursing students must evaluate the specific course in which they are enrolled including didactic, laboratory and clinical.
It is important that faculty review course evaluations completed by students as soon as possible so that recommendations for change can be made and implemented based on data. In the Nurse Education Program this is done utilizing three methods. Samples of all three are demonstrated below:
1. Objective Data Analysis
2. Trended and Aggregated Data
3. Subjective Data Analysis
Objective Data Analysis
A sample question of the course evaluation of Nursing I is demonstrated below. As the faculty reviewed the data, it was evident that question #8 was an issue for the students. This needed immediate attention!
QUINSIGAMOND
COMMUNITY COLLEGE
Nurse Education Lab
I Evaluation Results
Evaluations completed: Fall Semester, 2001
|
Example #1: Survey Question: |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
***8. |
Were there an adequate number of instructors available in the lab? | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
After review of the data (course evaluations), it was decided by the Nursing I Faculty, Lab Personnel, and Program Coordinator to increase the number of lab instructors and reevaluate the next semester. After discussion with Nursing I Faculty and Lab personnel and Program Coordinator , a schedule was developed and implemented in the Spring 2002 semester outlining the changes that were necessary to correct the problem of insufficient personnel in the laboratory setting. The plan was adopted for both Nursing I and Nursing II. See table below for specifics.
QUINSIGAMOND
COMMUNITY COLLEGE
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||
|
|
|
During the review of the Nursing I evaluations for the following semester ( Spring, 2002) , the response to #***8 is dramatically different after the increase in the number of lab instructors for that semester. Trended and Aggregated Data |
During the review of the Nursing III course evaluations, it was evident from the beginning of the implementation of the Nurse Education Computer Application that there were several problems within the Nursing III (NUR 211) course.
This section of the portfolio assessment will identify changes made to NUR 211 from 2000 through 2004 and the resulting trended data.
Fall 2000 and Spring 2001. The faculty identified that students needed to recognize this course as a comprehensive course covering three disciplines: Pediatrics, Obstetrics and Mental Health Nursing. As a result, they decided to host a one day orientation to Maternal Child Nursing. During this presentation, the Professors emphasized the correlation of the two disciplines.
Fall 2001and Spring 2002. The faculty agreed to combine their clinical evaluations into one document. Previously, each specialty completed separate clinical evaluations for each specialty, resulting in an inability to track student's progress. With the advent of the one clinical tool, Professors are more readily able to track strengths and weaknesses of a student in the clinical setting.
Fall 2002 and Spring of 2003. The Nursing III team decided to combine the Pediatric and Maternal Child text to help the students to understand the relationships of these two disciplines. They selected Waley and Wong. This is the first time a text for two disciplines had been combined.
Fall 2003 and Spring 2004. The department introduced a new faculty member to the team. Additionally, a more streamlined Psychiatric text was selected and required. In the Spring 2004, two new faculty members were added to the team, thus leaving one original member. The team reviewed previous semesters evaluations and tackled some very distinct and complex issues. One of the solutions that was proposed by the team is that when specific lectures were presented to the class, other Professors within the team would attend the lectures to help the students to recognize the common threads in each of the three disciplines.
Demonstrated in the graph below, is the trended and aggregated data from Spring 2000 until Spring 2004 .
Example #2
In the Fall of 2004 Nursing II students completed the course, clinical and laboratory evaluations. When the Professors were completing and reviewing the end of semester report, they noted that the trended and aggregated data for the Medical Surgical textbook had dropped significantly. See the data below:
| Question: | Medical Surgical Nursing Textbook |
|
|
|
|
|
|
|
|
Based on the findings of the evaluations and the remarkable difference in the trended data, the faculty decided to change the Medical Surgical Text.
Spring evaluations were done in May 2005. Results of the trended and aggregated data can be found below:
| Question: | Medical Surgical Nursing Textbook |
|
|
|
|
|
|
|
|
|
Subjective Data Analysis
The Nurse Education Computerized Database (NECA) includes a comment component aligned with each objective evaluation. This allows the student to openly comment on several areas of the course. After completion of the student evaluations, the specific nursing team and Program Coordinator read the comments and identify strengths and weaknesses as stated by the students.
( see sample below)
Nursing II students were introduced to a modular lab structure in the Fall, 2003 semester. This was originated with the cooperation of the Nursing II team and the Lab Coordinator. The premise for the decision was based on the changing healthcare environment and the need to allow students to "check -off" on their skills earlier in the semester and then utilize these skills in the clinical setting. As a result, subjective data was collected and a sampling of the students' responses are outlined below. The Fall, 2003 semester followed by the Spring, 2004 semester comments demonstrated for review:
Fall, 2003:
The days are long.....upstairs/downstairs every 45min. NOT FUN!!!I would have liked more frequent shorter days....longer than 45 minutes per topic in 2weeks and being able to come back and be signed off on topic....I forgot what I learned 2 weeks ago....open lab hours....difficult,it is packed and you can not get anything done...I went to both open labs at 9-10am and there were so many people I don't think I even spoke to Joan, so for me I did not get much out of the time....
I feel that this lab module was not prepared for. I liked only having lab 4 times but the display was bad. The teachers all taught us a different way to do things. testing us off was horrible...It was very unorganized. Lab instructors were helpful to a point, but.... Students need to be allowed more practice time. Open lab was ok, but I feel that teachers should have been able to sit and teach us if we did not understand.
The overall structure of the lab was a problem. Eight hours was an extremely long day and most of our attention was lost after lunch. Perhaps some kind of creative measure to figure a plan that is not one 8 hour.
The Nursing II team and the Lab Coordinator reviewed the pilot program and all evaluations of the students and made improvements based on the students evaluations and the instructors observations.
Spring, 2004
The full day labs for this semester are much better than 2hrs each week.
I liked having only four labs in the semester. The only negative thing is that you must come to open lab, because fitting everything into one day is not enough time, and by the end of the day your learning ability has decreased. Overall I enjoyed lab, and I felt it was a much better learning environment than Nursing I lab.
Overall, I was pleased with the lab. this semester, open lab times were offered at times I could attend and I found the sessions to be extremely helpful in practicing technique and preparing for competencies. The instructors were always accessible and more than happy to answer any questions I had.
Nursing 2 labs were great for my learning process. the time in which labs were scheduled were really a positive change to the nursing program. the staff in the labs were great and helped me.
Nursing 2 lab was much better than in Nursing I. I enjoyed the full day lab seminars, and felt it was a much better use of time than having it scheduled every week as in Nursing I. Lab instructors were wonderful and very knowledgeable and helpful. Non-judgmental critiques were instrumental in the learning process.