Assessment of Student Performance in the Learning Physics as one Nation Project

Assessment of Student Performance in the Learning Physics as one Nation Project

Christopher C. Bernido,  Ma. Victoria C. Bernido and Dr. Carolina Porio

Abstract

     We report on the assessment of student performance of participating schools in the pilot implementation of the Learning Physics as One Nation (LPON) project in School Year 2008-2009. The LPON project is an initiative of the Fund for Assistance to Private Education, funded by the Department of Education of the Philippines, and designed as a possible solution to the nation’s chronic lack of  qualified physics teachers. Of the 32 LPON schools, only five had teachers who had an undergraduate degree in physics or physics education. The LPON project components include a specially designed Physics Essentials Portfolio of 239 learning activities to be independently accomplished by students in the classroom during one school year, and 18 DVD volumes of associated video lessons by physicist-educators. Being conceptually based on the Dynamic Learning Program (DLP) of the Central Visayan Institute Foundation (CVIF) in Jagna, Bohol, the LPON didactic method follows the limited-teacher-intervention rule, i.e., on average, students independently accomplish written work 70%-80% of the allotted physics time, with the rest for strategic teacher intervention or discussion which, in the LPON case, is through the video lessons. For assessment of performance, a pre-test and post-test were independently conducted by the Center for Educational Measurement. Results are indicative of a positive effect on student learning, in particular, in terms of gain in pre-test-post-test scores and in number of students achieving given performance levels, compared to benchmark performance levels characteristic of the norm group and accredited Philippine private schools.

I. Introduction

      The Learning Physics as One Nation (LPON) [1] initiative of the Fund for Assistance to Private Education (FAPE), with a special grant from the Department of Education (DepEd), was motivated by the alarming decline in number of qualified physics teachers in the Philippines. A study of the Department of Science and Technology (DOST) in the 1990′s revealed that only 27% of high school physics teachers were qualified to teach the subject (with an undergraduate or graduate physics or physics education major). In spite of massive teacher training programs organized to address the problem, recent DOST studies [2] show that this number has dropped to 8%, while the website of the Southeast Asian Ministers of Education Organization (SEAMEO), accessed in early 2010, indicates a more alarming 4%. This decline is partly attributed to migration of science teachers to foreign countries. The usual crisis response of training more teachers only serves to make them more marketable to foreign countries. It is therefore reasonable to anticipate the continuing scenario of lack of qualified physics teachers in the country. This means that majority of over one million high school seniors yearly are being taught physics by teachers lacking formal training in the concepts and methods of physics. (In School Year 2008-2009, the total enrollment for secondary schools was 6,763,858 with 5,421,562 in public schools and 1,342,296 in private schools [3]. Of these, based on previous years’ statistics, over 1.2 million were fourth year high school students. Physics is the required fourth year science subject with an allotted time period of 6 hours per week.)

     The LPON initiative is a response to the crisis through strategic intervention on a national scale. For implementation of the LPON project, the Central Visayan Institute Foundation (CVIF) of Jagna, Bohol, home institution of the lead project consultants, worked with the National Institute of Physics of the University of the Philippines in Diliman, Quezon City, the Samahang Pisika ng Pilipinas (Physical Society of the Philippines) and the Samahang Pisika ng Visayas at Mindanao (Physical Society of Visayas and Mindanao).

     The LPON Project provides a technology-assisted framework for fourth year high school students to learn basic physics concepts and methodology even if their classroom teacher has little or no formal training in physics. The project is based on the Dynamic Learning Program (DLP) [4] of the CVIF. The LPON thus addresses the students directly so as to foster independent learning in the DLP way. With this arrangement, the LPON project simultaneously provides teachers with a year-round, in-school training in physics content as they go through the learning activities and watch the accompanying video lessons.

     We also note that the LPON project is designed for progressive improvement in physics performance of the whole spectrum of learners from challenged to fast learners. Moreover, the LPON instructional materials have been prepared such that a project lead team can monitor student progress on average, and address questions from the field through e-mail, mobile phone text messages, and fast courier services.

     For assessment of student learning outcomes and evaluation of the effectiveness of the LPON project, a repeated measures design was followed. A pretest was conducted by the Center for Educational Measurement (CEM) in July 2008 and a posttest over seven months later in March 2009 for 32 schools in the LPON project.
     
     In this paper, we highlight some results of the assessment and present some recommendations that would induce and enhance student learning in high school physics up to globally competitive levels in the shortest possible period of time (two to three years) in spite of severely limited human and material resources.

II. Projects components and Methodology

The Physics Essentials Portfolio (PEP) and the associated DVD video lessons are the major components of the LPON project. The PEP contains 239 Learning Activities to be accomplished in the classroom during the physics period, covering a whole school year of the high school physics curriculum. Eighteen (18) DVD volumes (1,511 minutes or around 25 hours) of video lessons go with the PEP. Being conceptually based on the CVIF-DLP, the LPON didactic method follows the limited-teacher-intervention rule, i.e., on average, students independently accomplish written work 70%-80% of the allotted physics time, with the rest for strategic teacher intervention or discussion which, in the LPON case, is through the video lessons. The video lessons are to be watched by the students after they have accomplished the associated written learning activity. (Watching before the written activity would reduce the video lesson to ordinary entertainment or educational videos on TV channels, watched passively by the students.) A unique feature of the LPON video lessons is the deliberately detailed analyses and step-by-step mathematical derivations or worked-out examples. There is also much written board work for remarks and transitional phrases to allow for foreign language learning (i.e., English) and to help a broader spectrum of learners.

     For the LPON PEP learning activities, we were permitted to pilot a physics syllabus that is a hybrid of the traditional physics curriculum (TPC) and the more recently adopted DepEd 2002 Basic Education Curriculum (BEC) energy-theme-based physics curriculum (ETBC). The TPC follows the standard sequence and topic emphasis of methods of physics, mechanics and dynamics, electromagnetism, optics and modern physics. The ETBC has a major rearrangement of topics with optics coming early, while kinematics and dynamics are taken up in the last semester. The ETBC includes historical topics such as the steam engine and the industrial revolution, among others [5]. The hybrid LPON physics syllabus is designed to enhance students’ learning while giving a solid preparation for college entrance examinations and college course work. It basically follows the TPC and excludes some ETBC topics which are time-consuming but do not directly improve student performance in core physics topics. Crossover topics in different subjects are indicated in the PEP.

     We also note that there are rules for strategic learning in the LPON project, e.g., absolutely no calculators, explicitly and neatly written step-by-step solutions to problems, physical units included in all steps of calculations, among others. The pedagogical bases for the rules are included in the PEP.

III. Profile of participating Schools

     Thirty-four (34) private high schools joined the pilot implementation during the School Year (SY) 2008-2009. These include thirty-two (32) private schools under the FAPE-Educational Service Contracting (ESC) program representing fourteen (14) regions of the country, Davao Christian High School (DCHS) in Davao City, since it has implemented the CVIF Dynamic Learning Program for three years, and the CVIF. Four schools had both LPON and non-LPON classes, with the school principals themselves deciding on which classes are to implement the LPON. Eight schools had accredited status with the Philippine Association of Accredited Schools, Colleges and Universities (PAASCU) by the time they implemented the LPON. Accreditation is an assurance of quality facilities, school programs and environment, and faculty profile. The number of students per school who were involved in the LPON project ranged from 41 up to 343 students. The total number of students in the LPON project for the thirty-four schools is 4,882 students.

     For the pilot implementation, five teachers had undergraduate physics majors, the rest majored in General Science, Math, Chemistry, Engineering, English and Forestry. The physics teacher in the school with the highest mean Percentage Correct Score in the post test (First Asia Institute of Technology and the Humanities, Tanauan City, Region IVA) in the LPON project majored in Forestry in college. The teacher in the school with the highest statistical gain in learning of the students in the LPON project (Christ the King College, San Fernando, La Union, Region I) majored in General Science in college and graduate (masteral) school.

IV. RESULTS AND ANALYSIS

     Twenty-nine (29) schools were given tests following the traditional physics curriculum (TPC) of the DepEd, as requested by the Project Consultants. It should be noted, however, that the correspondence between the CEM test topics and the LPON hybrid syllabus is only 86%. (With the limited budget, the standard CEM physics test was conducted. No new test was designed specifically for the LPON.) For the 29 schools, 5000 students took the pretest and 4751 students took the posttest. Four schools had both LPON and Non-LPON classes, with a total of 379 students in non-LPON classes. The paired t-test for dependent means was done for students with both pretest and posttest, a total of 4,363. Those lacking either the pretest or the posttest are excluded in the analysis presented here.

     Three schools were given the more recent energy-theme-based (ETBC) test. Here, we note that, being more conceptual and historical in approach the energy-theme-based curriculum has a lower degree of mathematical rigor and physical analysis compared to the standard or traditional physics curriculum. This may be partly seen from the norm group mean of 49 for the standard physics curriculum, and the norm group mean of 56 for the energy-theme-based curriculum test.

     The CVIF and DCHS were not included in the LPON testing, although the latter already has yearly CEM pre and post tests.

A. Cohort Comparative Descriptive Statistics for Pre-test and Post-test Percent Correct Mean Scores

     The total group of 29 schools given the TPC post-test had a statistically significant gain in Means for percent correct (PC) scores. Twenty-seven schools individually posted a significant gain in PC Means (2.06  tobserved  15.51). Note that t is the ratio of the difference between the Means to the standard error of the difference between Means. Statistical significance of the gain in PC Means from pre-test to post-test is obtained at 95% (p = 0.05) confidence level (c. l.) for three LPON schools, and 99% (p = 0.01) confidence level for 20 LPON schools. One LPON school had a statistically non-significant increase in the PC Mean. For only one LPON school, the post-test PC Mean was lower than the pre-test PC Mean by two points, but the decrease is statistically not significant.

     The top performing school in the LPON cohort (not PAASCU accredited, Region IVA) which took the standard physics curriculum test had a gain in PC Mean of 9 points (from 39 to 48), with a post-test performance level frequency distribution closely approaching the normal (bell-shaped) distribution locally compared to the CEM norm group. This is not far from the PC Mean of 49 for the CEM best school (PAASCU accredited, non-LPON, Metro Manila) for school year 2007-2008. The LPON school with the highest statistical gain in pre-test-post-test Means is in Region I, and is not yet PAASCU accredited.

     The three schools (all PAASCU accredited) which took the energy-theme-based physics curriculum test had an average PC Mean of 54. The best performing LPON school (Region XI) had a post-test PC Mean of 60, comfortably above the CEM norm group mean of 56. The CEM All Schools (Non-LPON) PC Mean was 53 for 10,905 examinees for school year 2008-2009. The best performing CEM School (PAASCU accredited, non-LPON, Metro Manila) for the same year had a PC Mean of 71.

     Table 1 shows comparative descriptive statistics for schools which implemented the LPON for all physics classes, and with the TPC post-test. All Means shown in the Table are for Percent Correct scores. The Table also shows standard deviation (SD) and critical values of t for statistical significance at 95% and 99% confidence levels (c.l.).

     Tables 2 and 3 show comparative descriptive statistics for schools which had both LPON and non-LPON physics classes, and with the traditional curriculum post-test.

lpon1
lpon2
lpon3

Tables 4, 5, and 6 show the statistics for non-PAASCU-accredited and PAASCU-accredited LPON schools. It is interesting to note the range of Percent Correct Means. Non-PAASCU accredited LPON schools: 30.0 < PC Mean < 47.7. PAASCU-accredited LPON school: 35.5 < PC Mean < 41.8. PAASCU-accredited non-LPON: 38.3 < PC Mean < 41.3. It is easy to see that the performance of LPON schools is within the range of quality benchmarks in the Philippine setting, even at a very early prototype stage.lpon4
lpon5
lpon6

B. Comparison for Two School Years of LPON Schools with the Traditional Physics Curriculum Post-test
lpon7
C. Comparison Based on Cohort Performance Levels

Results of the pre-tests and post-tests show that there was a signaaificant gain in the number of students in the upper 6 performance levels of the CEM tests (excellent, superior, above average, high average, average, low average), and a significant decrease in the number of students in the lowest 3 performance levels (below average, poor, very poor). Table 8 shows the standard score range for the nine CEM performance levels, while Table 9 shows the percentage increase/decrease for each performance level. This is further illustrated in Figures 1 and 2. This indicates the acquisition of competencies by students in the whole spectrum from slow to fast learners.
lpon8
lpon9
lponf1
lponf2

For benchmarking, we compare the LPON distribution by performance level to that of the CEM All Schools performance for the same school year, SY 2008-2009. The results are again for the traditional physics curriculum post-test. Note, however, that schools which avail of the services offered by CEM (middle column in Tables 10 and 11, i.e., CEM All Schools) generally belong to the more affluent schools which can afford the fees for the external tests, or are consistent in maintaining quality standards. On the other hand, LPON pilot schools include small private schools with limited resources, and which are subsidized by the government under the FAPE Educational Service Contracting scheme giving tuition fee assistance of P5,000 per student for one school year. From Tables 10 and 11, the distribution by performance level benchmarks are not far from that of the LPON schools, and projected to be attainable within two to three years of LPON implementation.
lpon10
lpon11
D. Comparison of Best Schools with Traditional Physics Curriculum Post-test

Again, for benchmarking purposes, one may also look at Table 12 which shows the best school performance under the TPC post-test. Even at an early stage, the performance of the LPON best schools, based on highest statistical gain and highest PC Mean, is nearly comparable with the CEM best school in MetroManila, especially on the Mean PC and the highest and lowest scores obtained. These numbers should be noted since in the LPON, students are learning independently most of the time through the written PEP activities and the video lessons.
lpon12
E. Comparison of LPON and non-LPON Classes in the Same School with the Energy-Theme-Based Curriculum Post-test (PAASCU-accredited School)
lpon13
Table 13 shows that a higher percentage of students under the LPON entered the Above Average to Excellent quality indices as compared with the non-LPON students. Interestingly, for the Poor and Very Poor quality indices, there were more non-LPON students, percentage-wise, than LPON students.

Table 14 shows the distribution by quality index for the CEM All Schools which can be compared to the LPON school of Table 13 for SY 2008-2009 with the ETBC post-test.
lpon14
V. Project Evaluation by pilot Schools Administrators and Teachers

     All teachers who responded to the questions in the survey form accomplished during the LPON Evaluation Workshop on 2 May 2009, indicated a gain in confidence and competence (25 teachers). Twelve out of twelve principals/academic coordinators also indicated a gain in teacher confidence and competence. Nine out of nine principals/academic coordinators who answered the relevant questions (with available data) indicated an improvement in students’ mean performance in external and/or national assessments. Two principals remarked on parents being happy about the improvement in performance and behavior of their children who may be slow to average learners.

     Because of generally positive feedback on the effectiveness of the materials, for the school year 2009-2010, the materials (PEP Learning Activities and 18 volumes of DVD video lessons) are still being used by most of the original pilot schools, and an additional one private and 26 public schools in the province of Camarines Sur, as requested and funded by the congressional representative of the district. For School Year 2010-2011, FAPE is expanding the project to more than 200 private schools in the country.

VI. Implementation problems encountered and addressed

     The most common concern expressed by administrators and teachers involved in the project is the difference in pace of covering the LPON PEP material due to suspension of classes for various reasons such as bad weather, extra-curricular programs, and adjustment to the LPON project, among others. Some schools were able to cover only about half of the planned activities. During the Project Evaluation Workshop held on May 2, 2009, the project consultants gave tips and strategies for increasing content coverage for the next school year.

     The most common complaints from students were against the amount of writing involved and the rule that calculators should not be used. The pedagogical value of the rules of the project had to be emphasized. The generally serious video lessons elicited various reactions in the students, ranging from boredom to heightened interest in pursuing physics as a career. (As of the writing of this paper, five students from one pilot school, and three from another, chose B.S. Physics as their major in college for Academic Year 2010-2011.) Students who appreciated the value of the course in preparing them for college studies were generally more attentive to and appreciative of the video lessons.

     For teachers, the most common complaint was against the amount of paperwork for checking. These issues and problems were addressed by the project consultants in the LPON circulars, and in the LPON Evaluation Workshop.

VII. Conclusion and Recommendations

     Analysis of the results of assessment shows that the LPON classes posted a significant gain in learning basic physics topics even at a very early pilot (prototype) stage. This is in spite of the normal initial resistance by students and teachers typical of new programs. (There were considerable complaints against the amount of writing in the LPON, the ban on use of calculators, among others.) Moreover, most LPON schools were able to accomplish only a little more than half of the LPON coverage for various reasons including adjustment to the method, suspension of classes and extra-curricular activities, and minimal cross-subject integration. We also reiterate that the LPON students learned physics independently in class 70%-80% of the physics period. Furthermore, the LPON is designed so that students need not be given any homework. (The CVIF DLP includes a zero-homework policy in all subjects [4].)

     It is then remarkable that the performance levels of LPON classes are within the range of local norm groups and benchmark (PAASCU-accredited or top-performing CEM non-LPON) schools. This is true for the comparison of pre-test to post-test gain, mean Percent Correct scores, and distribution by performance level quality indices from Excellent down to Very Poor.

     With experience and diligent use of the LPON materials following the LPON rules for strategic learning, the prognosis is then good for high school physics classes even if there is a lack of qualified physics teachers and lab facilities. Indeed, performance levels in physics can be significantly improved by the LPON approach of having a well-designed physics curriculum fit for the purpose of training Filipino students in a Filipino setting (taking into consideration the math and technical language deficiencies and present aptitudes of Philippine high school seniors), step-by-step and simplified discussion of physics concepts and problems by experts, and integrated review of math concepts and skills enhancement. However, the most important factor that boosts learning may be the requirement for the students to be task-engaged (hands-on, minds-on) more than 70% of the allotted time period for physics, instead of being overly dependent on the teacher, classmates, or family and relatives.

     In order for LPON schools to achieve targets deemed attainable by the lead project consultants (up to globally competitive levels after two to three years of diligent implementation), the following actions and policies are highly recommended:
          1) An LPON-based Physics Curriculum for the Fourth Year level should be adopted in the context of a Science/Math Curriculum for all year levels in regular high schools. The proposed curriculum indicates specific topics where purposeful cross-subject integration can be done without losing mathematical rigor for physics topics necessary to be globally competitive in the 21st century.
          2) The LPON video lessons should be viewed by the students, and not only the teacher.
          3) Teachers should note that there are non-LPON physics class “fun” activities which do not serve to improve student performance and yet take up time and energy. The LPON approach minimizes such activities so as to make room for more coverage of essential topics.
          4) School administrators, especially in a country which lacks qualified physics teachers, should discourage approaches to Physics teaching that emphasize “conceptual” learning over rigorous mathematical discussions [6]. These approaches require a high level of competence, maturity in physics teaching, and English language proficiency. Otherwise, students (and the teacher) may end up more confused. These approaches may lead to poor performance in college physics and mathematics, and the overall poor performance of the country in international assessments.
          5) School administrators should review their school calendar to cut out unnecessary extra-curricular activities.
          6) There should be a national focal team competent in curriculum design, didactics, testing and measurement so as to have a field-evidence-based unified coherent approach to improvement of student performance in physics. The team may have at least two physicists (minimum credentials should include a Ph.D. in physics and at least five international publications), two physics educators (with a doctorate degree in education and at least five international publications), and two high school physics teachers (licensed by the Philippine Regulatory Commission and with at least five years of classroom high school physics teaching experience).
Finally, we note that the LPON model can naturally be adopted for addressing teacher shortage in other Science, Technology, Engineering and Mathematics (STEM) disciplines. The shortage is imminent even for advanced countries as they experience a decline in number of students taking up the STEM courses in college and graduate school.

Acknowledgment

We are grateful to the Department of Education for the funding support, to Ms. Ruby Javier, the LPON Project Coordinator, and to the LPON pilot schools administrators and physics teachers.

References

[1] Carpio-Bernido, M. V. and Bernido, C. C. (2005). “Learning as One Nation,” Philippine Daily Inquirer (Nov. 20, 2005, p. A16); “On a Low-budget Large-scale High-impact Science Learning Program for Philippine Schools,” Invited talk at the “Science Summit,” University of the Philippines, Baguio City, 3-4 April 2008; adapted for Pisika 1 (Samahang Pisika ng Pilipinas, April 2008) p. 13.

[2] Ogena, E. B. (2008). “Status of Philippine Science and Mathematics Education,” Science Education Institute, Department of Science and Technology.

[3] Basic Education Statistics Fact Sheet, Department of Science and Technology (2010).

[4] Carpio-Bernido, M. V. and Bernido, C. C. (2004). “Science Culture and Education for Change, Part I: Innovative Strategies for Secondary Education in the Philippines,” Transactions of the National Academy of Science and Technology (Philippines), 26 (2), 243-267.

[5] Carpio-Bernido, M. V. (2008). “In Situ Curriculum Adaptation and Reform: Balancing World-Class Targets, Meager Resources, National Policy and Culture in a Developing Country,” Invited Lecture at the International Conference on Science and Mathematics Education, University of the Philippines National Institute for Science and Mathematics Education Development, Diliman, Quezon City.

[6] See e.g., Hewitt, P. G. (2010). Conceptual Physics, llth Ed. (Addison-Wesley).

One Comment »

  1. Chris Bernido May 6, 2013 at 4:02 PM - Reply

    Dr. Carolina Porio is the third author of the article entitled, “Assessment of Student Performance in the Learning Physics as one Nation Project.” Thanks in advance for including her name as co-author. All the best to PERJ.

Leave A Response »