An Inquiry into Retention and Achievement Differences in Campus Based and Web Based AP Courses

A decade ago the Advanced Placement (AP) program was introduced into the Canadian province of Newfoundland and Labrador. Initially, schools embraced this opportunity to provide their students with opportunities that were previously unavailable. However, within a few years the AP program was relegated to urban and large regional high schools. Few smaller, and particularly rural schools, were able to offer AP courses only to the brightest one or two students taking it as an independent study. In 1997-98, schools began to delivery AP courses in a web-based method. The purpose of the study is to examine the retention rates and student achievement in AP courses in the province between different delivery models.


Introduction
The province of Newfoundland and Labrador (pop. 540,000) is situated on east coast of Canada. The province has approximately 340 schools, the majority of which may be classified as small schools (i.e., average enrolment > 200). Most of the schools are located in rural places and a significant number are considered to be remote and isolated. Curriculum provision has always been a challenge in part because of the scale of schooling but also because of the difficulties in attracting and retaining qualified teachers especially in areas such as math, science and foreign language. Newfoundland and Labrador is not unique in this regard. These challenges are shared by rural schools the world over (Mulcahy, 2003).
In 1992-93, the province endorsed the Advanced Placement (AP) program for use. Initially, both rural and urban schools eagerly embraced AP and the unique opportunities it afforded students to take college level courses while still in high school. However, within a few years of its introduction AP had been relegated exclusively to urban and large regional high schools. Smaller rural schools found that it was not feasible for them to offer AP in their schools. They did not have enough students, they could not attract qualified teachers, and in many instances attempting to offer AP courses for the few compromised the general education of the many. If the opportunity to take AP courses remained dependent on traditional in-school provision, such courses would remain unavailable to rural students.
In order to address this issue of accessibility of AP courses for its rural students the Vista School District in 1998-99 initiated a pilot program that would make AP courses available via web-based distance education. "The Vista District Digital Intranet: The Delivery of Advanced Placement Courses to Young Adult Learners in Rural Communities" (VDI) was a federally funded project of the Vista School District and the Centre for Tele-learning and Rural Education at Memorial University of Newfoundland that allowed any student in the district to enroll in AP biology, chemistry, mathematics and physics. Prior to the creation of the VDI, only three of the ten schools with secondary grades were able to offer AP courses to their students. Two years later, a teacher and an administrator at Discovery Collegiate in Bonavista, spearheaded a second initiative to offer the AP social studies courses (i.e., the Centre for Advanced Placement Education). These projects were shared with the Burin Peninsula School District until 2001-02.
Other rural AP web-based initiatives have been initiated by the Baie Verte-Central-Connaigre and Lewisporte-Gander School Districts which shared a web-based AP Physics course during the 1999-2000 school year and the Avalon West School District began its web-based AP program. The most recent development in web based AP programming has been the urban Avalon East School District pilot of a web-based AP Physics course during the 2002-03 school year, with courses in AP Psychology, AP European History, and AP Environmental Science in development for web-based delivery for the 2004-05 school year.

The Study
The purpose of this study is to examine the retention rates and student achievement on standardised (e.g., College Board) exams in the AP curriculum across the province. The primary aim of this study is to explore the similarities and differences between retention rates and student achievement between different models of AP delivery (i.e., classroombased, web-based, or independent study).
This led to the following research questions. 1. Is there a difference in retention rates in Advanced Placement course by delivery model? 2. Is there a difference in student achievement on the AP exam by delivery model?

Literature Review
Over the past half decade, the College Board and the Educational Testing Service (the organization that administers the AD tests on behalf of the College Board) have conducted studies on the differences in examination performance based upon ethnicity and gender (Stricker 1998;Brelan & Oltman 2001;Buck, Kostin & Morgan 2002); however, there has been little analysis of differences in both course retention and examination performance based upon method of course delivery or based upon urban-rural distinctions.
As was the case in Newfoundland, for many students in rural jurisdictions the only opportunity to access AP courses is through some web-based alternative, such as APEX© or a virtual high school. In fact when considering the opportunities and challenges of virtual education, the Wisconsin Department of Public Instruction stated that what seems to be the greatest potential benefit is the increased opportunity for students in small, rural districts that cannot offer the full range of courses provided by larger and wealthier districts. If only one or a handful of students are interested in an AP course or a particular enrichment course, most school districts cannot afford to offer it. (Saunders, 2001) In addition, in his study of trends and issues in virtual high schools Clark (2001) found that 60.6% of virtual high schools in the United States offered AP courses (p. 8).
While there has been little analysis of differences in both examination performance and course retention based upon method of course delivery or based upon urban-rural distinctions by the College Board, there has also been little research on course retention based upon method of course delivery for secondary schools in general. However, there has been research conducted at the post-secondary level on this topic. For example, Bauman (2002) indicated that "in the Instructional and Performance Technology (IPT) online master's degree program [at] Boise State University… from 1989 to 1996, 44 percent of the program's master's degree candidates dropped out" (p. 8). It is even more interesting to note that Western Governor's University, who were able to achieve 80 percent retention rate in this online distance education courses, were considered something of an anomaly worthy of additional study (Santovec, 2004, p. 4).
At the secondary level, the data that have been reported tends to be unreliable. For example, the Virtual High School (VHS) in its third annual evaluation stated "the dropout rate was low for all of the courses and somewhat different between VHS and face-to-face in three of the four courses" (Kozma, Zucker, Espinoza, McGhee, Yarnall, Zalles & Lewis, 2000, p. 27). Having said this, in the previous year's evaluation Epinoza, Dove, Zucker & Kozma (1999) also stated that "it was found that VHS was serving a fairly narrow range of students, those who were academically advanced and college bound" (p. 48).
Even fewer studies have been published on differences in achievement in AP examinations between course delivery methods. To date, the only reference that the researchers have been able to discover is that "of the 137 students who took Advanced Placement courses through the Florida Virtual School in 2000-01, more than half earned scores of 4 or 5 (on a scale of 1 to 5) on the exams. About 65 percent of the students earned scores of 3 or higher, compared with 61 percent nationally" (Thomas, 2002, p. 3). This research study is an attempt to fill the gap that exists in the literature with reference to the differences in course retention and AP examination performance based upon method of course delivery and based upon urban-rural distinctions for secondary schools.

Data Collection
Early in 2004 the researchers examined the Newfoundland Department of Education's Annual General Report from 1992-93 to 2002-03. Through this examination, eighty-two schools were identified as having offered AP courses. At the time, sixty-six of these schools were still open (eleven had changed names and/or buildings) and sixteen had closed altogether. The sixty-six schools were sent a survey that indicated the AP courses that were taught and the years that each course was taught for that school and asked that the school complete the number of students enrolled in the course in June, the number of students who took the AP exam and the number of students obtaining each of the five possible scores on the AP exam. In instances where the school had closed down, the survey was sent to the school that was now responsible for students from that geographic region.
In addition to data from the Department of Education, the researchers obtained data from the College Board. This data included the number of students who took the exam, along with the number of students who obtained each of the five possible scores on the AP exam. However, there were forty-nine classes, representing twenty-seven schools, for which the College Board was unable to provide data, due to a number of reasons. In some instances there were no students at the school who actually took the AP exam. In other instances, students at a particular school wrote their exams at a neighbouring school that was already registered with the College Board. Finally, was unavailable also because the school had changed names and/or buildings and the College Board identification code was no longer associated with the original school. Another difficulty with the data from the College Board was due to the lack of response from the schools.

Results and Discussion
The data from the school surveys and from the College Board is organized into three different comparisons. The first comparison is simply an urban-rural comparison. Table  1 presents the data that was obtained from the school surveys, while Table 2 presents the data obtained from the College Board. Due to the discrepancy in data sets, researchers were unable to determine the method of delivery for approximately twenty courses.  Based on the school survey data, Table 1 indicates that there was a 7% difference in the retention rate between the urban schools and the rural schools with the latter having the higher figure. Conversely, the percentage of students taking the AP exam at urban schools is approximately twice that of students taking the AP exam at rural schools. This indicates that while rural schools may be better at retaining students in their AP courses, they have much less success in having their students challenge for the AP exam (and the college/university credit that comes with it).
It should also be noted that there is a slightly higher percentage of students scoring 4 and 5 on the AP exam at rural schools than at urban school. However, this is not supported by the College Board data, as Table 2 indicates that a significantly higher percentage of rural students scored 1 or 2 on the AP exam. This suggests that there may be no true pattern in the differences in exam results between urban and rural students.
The second comparison of the school surveys and College Board data is based upon the delivery method of the course. Table 3 illustrates the data obtained from the school surveys based upon classroom delivery, web-based delivery or independent study, while Table 4 provides the same comparison using the data from the College Board. As is illustrated in Table 3, there are a significantly higher percentage of students in web-based AP courses who took the exam compared to courses that were offered in the classroom. Interestingly, the majority of students in the province who took web-based AP courses were students from rural schools. However, this finding is contrary to the finding from Table 1 that the percentage of urban students taking the AP exam was almost twice that of rural students.
While the school survey data in Table 1 indicated that there were a significantly higher percentage of rural students who had scores of 1 or 2 and a significantly lower number of students who had a score of 3 on the AP exam, this is not supported by the data provided by the College Board, which indicates that there was little different in the performance of students between the classroom and web-based delivery. While both data sets present significant trends for the independent study group, at present this population is not large enough for these numbers to be statistically reliable.
The third and final comparison is based upon both an urban-rural distinction and the method of delivery. Once again, Table 5 illustrates the data from the school surveys, while Table 6 provides the data from the College Board. It is interesting to note that Table 5 appears to shed some light on the contradictions noted above with regards to the percentage of rural students who take the AP exam and the percentage of web-based students that take the AP exam. As the table illustrates, those urban students who take a webbased AP course have a 100% rate of students taking the AP exam, while the rural students who take a web-based AP course have a little more than half of the students take the AP exam. So while larger numbers of rural students take web-based AP courses, it appears that web-based AP is more successful in urban schools in getting students to challenge for the AP exam (and the college/university credit that comes with it).
In addition, the percentage of classroom urban students who take the AP exam is almost three times higher than that of rural students. Once again, the rural schools are better at retaining students in the AP courses, at least in the classroom. In web-based AP courses, it appears that rural schools are less successful than urban schools at retention. Finally, there appears to be stronger student performance by students in the rural web-based courses, as Table 5 indicates that two thirds of the students scored 4 or 5, while Table 6 confirms a slightly lower percentage of about half.

Conclusion
While web-based delivery methods have traditionally been used by rural schools to provide students with opportunities to take more advanced or enrichment courses, such as Advanced Placement, it appears that urban students have become more successful at both retaining their webbased students and having them challenge for the AP exam. However, overall it appears that rural schools do have the advantage of being able to retain more students in their AP courses, largely due to the strength in their classroom delivery numbers. Finally, while a smaller percentage of rural students complete their web-based AP course, and even fewer challenge the AP exam, those that do challenge the AP exam tend to perform better than their rural classroom and their urban counterparts.
As data become available from the Newfoundland Department of Education that fill in the errors that have been identified in the Department's Annual General Report the researchers will be able to make more definitive statements about the retention and the percentage of students who take the AP exam. In addition, once the method of delivery has been determined for the remaining classes, along with the addition of another approximately fifteen school AP codes, the researchers will be able to obtain a more complete data set from the College Board. This complete data set will allow them to make stronger arguments regarding the performance of students on the AP exam.