The instrument, SETES-CE used for this study was developed by the researcher, which proceeded in a number of identifiable stages in accordance to the principles guiding the development of valid instruments. The stages are planning, preparing, focus group discussions, pilot testing and analysis of draft instrument. The purpose of this scale is to evaluate teaching effectiveness in Colleges of Education.
56 3.5 Procedure for the Development of the Scale 3.5.1 Generation of Items
Generation of items involves the initial writing and collation of draft items into a pool.
The items were generated from reviews of relevant literature, focus group discussions and written assignments in form of essay by 100 students randomly selected across different schools in Emmanuel Alayande College of Education, Oyo. The students were to write an essay on what they perceived as the qualities of a good lecturer. On Tuesday, April 6th, 2010, five lecturers from Emmanuel Alayande College of Education who had been earlier informed participated in a focus group discussion. The purpose of the discussion was to deliberate on what they regarded as indicators of teaching effectiveness.
At the end of the discussion, it was gathered that effective teaching can be considered from the following: mastery of subject matter, teaching methods, communication skill, instructional materials, course material, motivation/ reinforcement techniques, strategies for individual differences, methodology, classroom management and evaluation style, among others.
Lecturers‟ personality (intelligence, hard work, friendliness, punctuality, firmness, self discipline, dressing, commitment, patience, tolerance and approachability. Also, materials preparation which includes the quality, adequacy and relevance of instructional material was considered. Curricular preparation was also examined to include the appropriateness of the content of the lesson. Finally, the issue of environment as a factor of effectiveness was also raised. The location of the college, lecture rooms arrangement, ventilation and physical facilities were also identified as the measure of teaching effectiveness. At the end, a total of 113 items were generated to form SETES-CE.
The scale was divided into two sections, sections A and B. Section A sought information about the students‟ background and the name of the lecturer identified by the researcher to be rated by the students while section B consisted of the 113 generated items.
3.5.2 Establishing the Face and Content Validity
The researcher critically examined the items for correctness to ensure that the indicators are well represented in the draft items. To establish content validity, five experts in the area of educational evaluation including my supervisor examined the extent to which the items measured teaching effectiveness. Their suggestions were used to restructure and polish some of the items.
57
There was an outright deletion of some items where they were duplicated, and new ones were generated where necessary. The face validity, in terms of the appearance, organization and typing format of the scale was examined at this stage. The items were again given back to these experts for final screening. After the initial content validation, the pool of items was printed and then administered to the target participants for pilot testing.
2.5.3. Pilot Testing
The scale was administered to the selected sample of 640 respondents representing the four Colleges of Education sampled for this study by the researcher. Four research assistants were trained by the researcher in each of the sampled Colleges of Education to assist in administering the scale. The training was considered to be very important in order to ensure objectivity of data collection and control for possible procedural bias. Also, to ensure uniform administration conditions across the colleges, the research assistants to be involved in the data collection were given guidelines detailing specific instructions on how to administer the scale.
Letter of introduction was collected from the Institute of Education for ease of entry into the Colleges of Education used for the study.
3.5.4 Establishing the Subscales of SETES-CE
The purpose of the factor analysis in the construction of the scale is to “examine the stability of the factor structure and provide information that will facilitate the refinement of a new measure” (Hinkin, 1955: 977). At this stage, the researcher established the factor structure or dimensionality of the construct. Principal component factor analysis with orthogonal varimax rotation was used to analyse the data in order to determine the number of underlying factors. A scree test was also used to determine the number of meaningful factors. Two criteria were used to identify major factors of the scale:
The eigen value of the factor must be greater than 1 (Kaiser, 1960);
The scree plots for the factors must be on a slope. (Steven, 1992).
58
3.5.5 Determination of SETES-CE’s Psychometric Properties
The internal consistency of the entire scale was estimated using the Cronbach reliability coefficient alpha (R). A high alpha is desirable since it reflects that the items are homogeneous and thereby measuring the same underlying property. Nunnally and Bernstein (1994) suggest that an alpha level of at least .80 is sufficient for most purposes, given that correlations are attenuated very little measurement error at that level. Furthermore, DeVellis (1991) stated that alpha values between .80 and .90 should be considered very good. Therefore, in this study the suggestions of Nunnally and Bernstein (1994) and DeVellis (1991) were considered.
3.5.6 Use of Final Version of SETES-CE
The final scale after the validation identified four components namely preparation with eight (8) items, personality having fourteen (14) items, followed by classroom interaction with thirty – eight (38) items and evaluation forming thirteen (13) items. A total of seventy – three (73) items form the final scale. The final scale was administered on 200 students each selected from the remaining two Colleges of Education namely, Adeniran Ogunsanya College of Education, Lagos and College of Education Ikere - Ekiti, Ekiti State. In each of the colleges, one particular general course each in education was selected in year II and III. Also, the researcher obtained information about the lecturers teaching the selected courses from the office of the dean, school of education of each college in order to consider their rank, age, gender and year of experience before the administration of the scale. After identifying the courses, and based on the information obtained from the deans‟ offices, four lecturers among those teaching the courses were selected and rated by the students from the five schools of the two colleges.