Thursday, July 28, 2005

A study on the rating given by the third year high school students of Dominican College of Sta. Rosa, School Year 2004-2005 on the Teaching Procedures

Introduction

Teacher quality is the single most important factor influencing students’ achievement. In fact, PAASCU (Philippine Accrediting Association of School,
Colleges, and Universities) consider Faculty and Instruction as the most important factors that determine school quality. In fact, as Jennifer King Rice reports , teacher compensation represents a significant public investment. In 2002 alone, the United States invested $192 billion in teacher pay and benefits. Given the size of this investment, there is remarkably little research to guide such critical decisions
as whom to hire, retain, and promote.

To be a teacher may be easy for some. Indeed, an individual aiding another to learn is in a sense, a teacher. James Garfield (20th president of the United States) once remarked “A good teacher on one end of the log, and an interesting student on the other will make a university anywhere”. Still, we must see to it that the teaching profession is equipped with all the trainings and studies, aimed towards the objective that education is really ushered towards development. Johann Heinrich Pestalozzi believed that education has the power for societal regeneration.

It is a given that we must train our teachers to be effective. This is in order to carry out the task of educating our children for the future. But, as William Osler remarked, “The successful teacher is no longer on a height, pumping knowledge at high pressure into passive receptacles...He is a senior student anxious to help his juniors”. Education, in our modern times, became a great task and responsibility.

The problem in assessing teacher effectiveness has been discussed in full detail by Michael Dunkin of University of Sydney. In his study, he proposed five main preliminary matters involved in arriving at a system for the evaluation of teachers. The first is the purpose of the evaluation; the second is the target category of teachers to be assessed; the third is the conception of teachers' work that is adopted; the fourth concerns the dimensions of teaching quality about which judgments are to be made; and the fifth is the approach to establishing the validity of the assessments.

He also criticized the traditional methods of assessing teacher quality by means of observational schedule or checklist, following Haertel (1991) , especially of unreliability across content areas and grade levels, poor conceptual bases, incompetence and lack of resolve by principals who apply them, negative teacher attitudes towards them, lack of uniformity of them within school systems, inadequate training of school administrators in their use, trivialization of teaching proficiency, and reinforcing a "single, narrow conception of effective teaching" (p.5).

On the part of the use of rating scales, Good and Mulryan (1990) found that the problems in their use had persisted from the early years of the twentieth century right up till the time of their writing (1988) . When Medley and Shannon (1994) reviewed the literature on the validity of observational rating scales for measuring teacher performance, they found that the best of them had high content validity. It is not clear what was meant by "content validity" in this case, but presumably it had a wider meaning than academic subject-matter knowledge and included "aspects of teacher performance known to be related to teacher effectiveness" (p.6018).

Medley and Shannon concluded concerning predictive validity that there is no empirical evidence that correlations between supervisors' ratings of teacher performance and direct measures of teacher effectiveness differ from zero. Thus, they apparently do not contain the information about teacher effectiveness they are assumed to contain. (p.6018)

Good and Mulryan (1990) invoked a professional development criterion and concluded as follows :

...[T]he key role for teacher ratings in the 1980s is to expand opportunities for teachers to reflect on instruction by analytically examining classroom processes. For too long rating systems ... delineated what teachers should do and collected information about the extent to which they did it, Ratings of teacher behavior should be made not only to confirm the presence or absence of a behavior but with the recognition that many aspects of a teaching behavior are important (quality, timing, context) and that numerous teacher behaviors combine to affect student learning.


Observation schedules, however, were less subject to halo effects on raters, and had been shown to have predictive validity. Good and Mulryan (1990) concluded that the identification of relationships between observed classroom behaviours and students' scores on standardised achievement tests and on criterion-referenced tests had been "small but significant." Stodolsky (1990) pointed out, "[u]sers must accept the limitations of observations as sources of evidence about teaching while recognizing that they provide a needed direct view of teaching processes in action" (p.185). Later, she concluded, "[c]lassroom observations are likely to be the centrepiece of a systematic evaluation.

Given all these intricacies and weaknesses, we cannot do away with teacher effectiveness assessment. In order for our school to be of good quality, we must focus towards the assessment of whether our system of education is indeed working or not. Dominican College of Sta. Rosa (DCSR) is now gearing up to be first-level PAASCU-accredited. Series of surveys have been made especially on the classroom instruction and the faculty in order to assess the right of DCSR to be accredited, based on the strength of its faculty and classroom instruction.

A study by Atento (2004) focuses on the Seniors and Juniors, and how these students feel about their teachers’ classroom instruction procedures. The questions he used are the very same questions PAASCU is using for its accreditation. His study inspected on the teachers themselves, the stating of objectives, the content of teaching, teaching procedure, medium of instruction, instructional tools and classroom management . For the part of the proponent, she surveyed the Juniors, in order to gauge the characteristics of their teachers, and how they feel about the quality of teaching instruction given to them.

Sample

For the sample, the study focused on the third year high school (N=35) students. This is about half of the whole Junior high school population, and about 10% of the whole high school population. The study chose the said year level, because the students of these levels were exposed to the teaching instruction in Dominican College for three years now, and mainly because the proponent’s advisory is third year. A class rating was made on the students, using the final average they have for the School Year. There are ninety-three juniors, and percentile ranking was used to determine who are included in class 1 and class 2. The median is at rank 46, and so, students ranked 1-46, are “Class 1” students; while those who were ranked 47-93 are “Class 2” students. This ranking style is not revealed to the students.

Also, there are 32 old students and 3 new students. Old students are those who were exposed to the teaching instruction at DCSR for more than two years.

Instrument

The instrument is written by Ramon George Atento and the proponent, and is a part of the proposed instrument for the assessment of the Science-Enriched Curriculum of DCSR. It is composed of forty-seven questions, attempting to find the attitude of the students towards their teachers. It is composed of questions pertaining to teacher qualities and characteristics, such as: Appearance, Clarity of Instruction, Classroom Management, Compassion, Competence/Content, Enthusiasm, Ethical/Moral/ Professional, Giving of Grades, Humor, Instruction Procedure and Quality, Insistence on Mastery and Learning/ Student-Centeredness, Using a Variety of Teaching Methods, Motivation, Organization, and Personal Relationship.

The students were asked to give their ratings on the statements bearing in mind their teachers in DCSR. The scale used is Interval, using the values of 5 for Strongly agree; 4, for agree; 3 for minimally agree; 2 for disagree; and 1 for strongly disagree After reading each of the statement, the student then rates his/her agreement on the same, bearing in mind their teachers at DCSR.

Variable grouping

The study followed the grouping of questions as was said above, in the following way:
Factors Statement numbers
Appearance Statement #13
Clarity Statement #14
Classroom Management Statements #5, 36
Compassion Statements #16, 32, 37, 40
Competence, Content Statements# 4, 8, 9, 45
Enthusiasm Statement#3
Ethical/Moral/Professional Statements# 10, 11, 12, 25, 26
Giving of Grades Statements # 34, 35
Humor Statement#29
Quality of Instruction,
Teaching Procedure Statements# 1, 18, 43, 44, 46, 47
Insistence on Mastery
and Learning/
Student-Centeredness Statements# 2, 17, 38
Using a Variety of Teaching
Methods and Strategies Statement#39
Motivation #15, 19, 20, 21, 22, 23, 24, 27, 28, 30, 31, 33

Organization of Classroom
Instruction/ Evidence of
Teaching Plan Statements# 6, 7
Relationship with Teachers Statements# 41, 42


Method of Research

The proponent used survey questions and distributed them to III-St. Thomas, her advisory class, and III-St. Catherine. The students were asked not to rate specific teachers but the classroom teaching procedures of all teachers.

Discussion and Results

Cronbach Alpha.

In order to test the consistency of responses to all items, and for the homogeneity of all items, the study used SPSS 11.0 to compute for the Cronbach Alpha. The result is a high consistency rating of 0.9545.

Overall Average

The overall average rating for all the questions by the respondents is 3.51126, with a standard deviation of 0.874786. This is found to be highly correlated with Atento’s result of 3.5438, with a standard deviation of 0.9483 . P-value for the comparison of the two means is at 0.8414 with t-obtained at 0.1959.

Comparing between means using Student’s t-test

The grand mean for the Class 1 students is higher, at 3.6787 (standard deviation at 0.400), while the Class 2 has 3.2867 (standard deviation at 0.5028). T-test for the two means reveal an obtained value of 2.4926 revealing a significance verdict for the two means. Hence, we reject the equality of means, and profess that Class 1 students give significantly higher ratings for their teachers than Class 2 students.

Now, ratings were higher for the new students, with a mean of 3.7447 (standard deviation at 0.425), as compared to the old students’ 3.489 (standard deviation is at 0.4872). However, comparing between means using the t-test obtained a low 0.85341 or a no-significance between the two means. This is partly because of a low number of new students (only three). We accept here the equality of means.

Comparing results among Factors

Below are the results of the means gathered per factor
Items Average
1 Humor 4.00
2 Relationship 3.91
3 Clarity 3.66
4 Competence/Content 3.62
5 Motivation 3.62
6 Compassion 3.54
7 Enthusiasm 3.54
8 Instruction 3.53
9 Methods 3.46
10 Ethical/Moral/
Professional 3.44
11 Classroom Mgmt 3.4
12 Mastery 3.36
13 Appearance 3.31
14 Grades 3.07
15 Organization 2.86


Discussion on the results per Factor

It is found out that the highest rating given to teachers in DCSR, is on their sense of humor. The students agree that their teachers use humor and fun in teaching instruction. This is important for the students, since this takes away the boredom of classroom set-up. Humor is also very much needed in DCSR classroom set-up, since the curriculum of DCSR High School is Enriched and students have to sit down for 12 classes a day, the maximum hours allowed by Department of Education.

Secondly, and almost understandably, the students believe that they have a good relationship with their teachers. They treat them as friends , and as second parents.

Thirdly, the students believe that their teachers are clear in their teaching. They speak clearly. However, this fact does not make them learn in a more positive way, as is clear later on.

Next, the students believe that the teachers they have are competent in their fields. This is also clear in the PAASCU self-survey evaluation made by Atento and Tejido on the competency of teachers. The students believe that their teachers are knowledgeable in the subject matters they are teaching , and that they are competent in their fields. In Atento , this was rated the highest, receiving 3.619. As is clear, this is very correlated with our 3.62 (see above).

Almost the garnering the same average is motivation. The students feel that they are motivated by their teachers. One student who gave scores of only 2s and 1s, gave just one grade of 4 to this question: “My teachers made me a better person”. The students feel that their teachers push them into learning , and that they expect highly of them . Their teachers also believe they can make good in life , and are supportive of them , encouraging them , and using praise and encouragement whenever they do good.

Next, the compassion of the teachers is judged 6th highest, with a rating of 3.54. The students feel that their teachers are willing to help them always , trying their best to understand students.

Enthusiasm ranks 7th with an equal rating. Here, the students believe that their teachers are enthusiastic in their teaching

At the lowest, we have organization of teaching. The students feel that the classroom instruction is not so organized , and that the instruction follows a plan. Basically, as a teacher, I feel this is due to the inability of the teachers to do their lesson plans, or come to class with an evident plan of instruction.

“Appearance” is ranked second lowest, which is a bit of a surprise. Anyway, the students feel that their teachers do not regard their appearance when they teach in front of the class.

Insistence on Mastery/Learning and Student-centeredness garnered 3.36 as average and is among the lowest. Here, the students feel that their teachers do not regard mastery in their teaching. The teachers move on to the next topic, even without mastery. Also, they do not believe that the instruction of their teachers is student-centered.

Again, classroom management received a low 3.40. This is below the average of 3.51 for all the factors rated by the students. In Atento , classroom management is the second lowest rated factor, receiving 3.447. Although this still translates to GOOD, in his study this creates a scenario where improvement is greatly needed. The students here feel that the teachers do not see to it that they are quiet in class. Also, the teachers do not maintain order in the classroom. Again, the proponent does not maintain that the teachers absolutely are not in control of their classrooms. Only that among the factors rated, this receives one of the lowest ratings.

Professionalism/Ethical Dealings is also judged one of the lowest. The proponent feels that due to the high ratings given in the relationship with teachers (see above), and also in motivation and compassion, the students are sometimes confused on the dividing line that separates teachers from themselves. Perhaps, they feel that teachers are family. Familiarity breeds contempt, as a famous proverb states. One especially bad feedback from the survey is that their teachers speak bad words.

Methods is also not at par with the other factors. This received only a meager 3.46. This is correlated with Atento, where Instructional Tools received 3.357, the lowest for his study. The students believe that their teachers do not vary their teaching methods in classroom instruction.

Individually, the students rated these statements the highest:
1. My teachers use humor and fun in classroom instruction. – 4.00
2. Our teachers believe we can make it good in life. – 3.97
3. Our teachers are supportive of us – 3.91
Our teachers are our second parents – 3.91
Our teachers are our friends – 3.91
6. Our teachers use encouragement and praise whenever we do good – 3.89
7. Our teachers expect highly of us – 3.8
Our teachers encourage us – 3.8
Our teachers try their best to understand students – 3.8
10. Our teachers are knowledgeable in the subject matters they are teaching – 3.77

Individually, the students rated these statements the lowest:
1. The classroom instruction is organized – 2.83
2. Our instruction follows a plan – 2.89
3. Our teachers do not say bad words – 3.03
4. I love my teachers because they give me high grades – 3.06
5. I love my teachers because they give me a passing mark – 3.09
6. Our teachers do not focus on our failures – 3.14
7. My teachers will not move on to the next lesson unless we have mastered the topic – 3.26
8. My teachers made me a better person – 3.29
9. My teachers regard their appearance when they teach in front of the class – 3.31
The teaching instruction I received in my school is student-centered. – 3.31

Summary and Conclusions

Based on the analysis of each item, we can deduce the following:
The main strength of the teachers in DCSR-High School Department has something to do with their personal relationship with the students. The students feel that their teachers are their friends and their second parents. However, because of this, the students tend to feel that their teachers are not professional in their dealings with them. The teachers sometimes forget that they are professionals, and that they should treat the students professionally. There is also a tendency to be relaxed in the midst of students. The students report that their teachers say bad words in front of them, and this destroys the teachers’ professionalism in front of them. However, the students still feel that their teachers are competent in their fields, and that these people are knowledgeable in their subjects. Only, the students feel that their classroom instructions are not organized. Nor do they follow a plan. As the proponent have mentioned, this reflects the inability of teachers to prepare their lesson plans. The students have a way of knowing whether their teachers came in to class prepared or not. I also believe that, as Atento concludes in his study , the teachers of DCSR, is high on the content of their teaching. However, they must also vary their classroom activities, focusing on different methods and tools in instruction.

Finally, as in Atento , classroom management is still a weakness. Perhaps, because of their inability to be professional in their dealings with students, and their insistence on personal ties and relationships with students, they do not earn the respect they do deserve even inside the classroom. I believe that teachers should balance their personal relationships with students with their insistence on discipline and their right to be treated as professionals. This is also the cause of disarray, and lack of organization in instruction. Hence, as is clear, the weaknesses and strengths of teachers are all inter-connected.

Here are some of my recommendations regarding the study:

1. It is a limitation of the study that the proponent surveyed only the juniors, for reasons aforementioned. Actually, the plan of Mr. Atento and I is to have a standard for all the students of Dominican College, not only in their attitude towards teachers, but also towards the curriculum, their parents, study habits and so on. Hence, although he already surveyed the fourth year, and some second year students, I cannot present the results here yet, since we are trying to complete the whole High School Department. It is therefore recommended to include all the other levels.

2. It is also the same weakness, as in Atento on the part of the study that the students were asked to rate the teaching instructions, not only of their present teachers but the teachers that they have had since their first year. Here, the new students’ responses are a little bit more reliable, in the opinion of the researcher. This is for the reason that the memory required from them is lesser, and they have a few teachers to rate. Hence, it must be made clear to the respondents that they are rating their teachers as a whole.

3. Some students do not wish to rate their teachers as a whole, but individually. They mentioned to the researcher that if they will be rating the teachers as a whole, they would only remember the bad ones. This must be assumed by the study for the moment. Later on, the proponent wants to administer the same survey for individual teachers and inspect whether they are at par with the majority, and are thus, effective, or not.

Glόria tibi Dómine!