What Counts as Evidence in Rural Schools ? Evidence-Based Practice and Practice-Based Evidence for Diverse Settings

Despite a century of research searching for what works in teaching and learning, a project that has benefited from political support in the form of the Elementary and Secondary Education Act (ESEA) of 1965, any individual child’s experience of learning in school is marked by inequalities based on socioeconomic status, race, immigrant status, and geospatial location (Eppley & Shannon, 2017). The search for what works has remained unchanged in philosophy and design despite long-standing evidence of persistent and growing inequalities among groups (Reardon, 2011). Evidence Based Practice (EBP) has undergirded contemporary education policies and the educational outcomes that have resulted from this research. As such, EBP directs teachers’ and students’ experiences in school. Education policy in the United States favors a nearexclusive emphasis on evidence-based practice to guide policy and practice about what educational interventions work (Walsh, Reutz, & Williams, 2015). What “works,” however, ignores the relevance of context. Context of place, particularly of rural places, is an important consideration for research whose aim is to identify and promote effective and culturally sustaining instruction for students (Gruenewald, 2003; Lester, 2012). This policy column considers the limitations of EBP as the foundational philosophy of contemporary education policy in rural schools and suggests Practice Based Evidence as a socially just alternative.

Despite a century of research searching for what works in teaching and learning, a project that has benefited from political support in the form of the Elementary and Secondary Education Act (ESEA) of 1965, any individual child's experience of learning in school is marked by inequalities based on socioeconomic status, race, immigrant status, and geospatial location (Eppley & Shannon, 2017).The search for what works has remained unchanged in philosophy and design despite long-standing evidence of persistent and growing inequalities among groups (Reardon, 2011).Evidence Based Practice (EBP) has undergirded contemporary education policies and the educational outcomes that have resulted from this research.As such, EBP directs teachers' and students' experiences in school.Education policy in the United States favors a nearexclusive emphasis on evidence-based practice to guide policy and practice about what educational interventions work (Walsh, Reutz, & Williams, 2015).What "works," however, ignores the relevance of context.Context of place, particularly of rural places, is an important consideration for research whose aim is to identify and promote effective and culturally sustaining instruction for students (Gruenewald, 2003;Lester, 2012).This policy column considers the limitations of EBP as the foundational philosophy of contemporary education policy in rural schools and suggests Practice Based Evidence as a socially just alternative.

Evidence Based Practice in Rural Schools
Evidence Based Practice is teaching practice supported by scientific, "true" knowledge generated as an outcome of randomized control trials (Biesta, 2010).Evidence Based Practice is encoded in policy.For example, the Every Student Succeeds Act (ESSA; P.L. 114-95) mandates that schools spend federal funds to purchase programs for which there is "strong," "moderate," or "promising" evidence (USDOE, 2015) as defined by the types of studies that have provided evidence on an intervention's The characteristics of rural schools, in particular, challenge the generalizability of EBP (Eppley, 2011;Roberts, 2014).Programs with "strong" evidence may fail to translate into the intended outcomes for students in rural contexts.Rural schools enroll 9.7 million children in distinct contexts for teaching and learning where standardized curricula and instructional practices may not meet the needs of learners.Rural schools share in common some characteristics such as low population density, distance from urban centers, and an intersection between place and culture (Johnson et al., 2014).However, rural communities are also diverse and vary greatly in terms of student demographics, local economies, access to resources, and geography, among other features (Howley & Howley, 2010;Flora, Flora & Gasteyer, 2015).The application of EBP is particularly problematic in rural schools because 'what works' as determined as the outcome of randomized control trials is based on deterministic ideas about human behavior (Biesta, 2010).The lived experiences of rurality such as long bus rides, community composition, and even the limited access to the internet, challenge deterministic outcomes.Interpretations and understandings of and between students and teachers in any one rural community make for problematic applications of randomized control trial data.The following example illustrates this point.Summer 2018

An Evidence-Based Intervention in Rural
Mississippi: READ 180 READ 180 is a commercial reading curriculum distributed and promoted by Scholastic, Inc. used widely across the U.S. The program includes whole and small group instructional units, independent reading, and computer-based instructional activities and is cited by the WWC as "effective" for improving literacy outcomes.Mr. Charles was required to implement READ 180 in his seventh grade classroom at Pine High School, a small, rural school in Mississippi serving just over 150 students in grades 7-12.Mr. Charles described his students "like trees in an aspen forest -branches touching, roots entwined.The vast majority of the children I teach have known each other all of their lives."From the beginning, Mr. Charles attempted to implement Read 180 with fidelity.After participating in training and reading about the program, he said, "If I'm dutiful to the program, it should work." The first unit in the program focused on immigration, a topic Mr. Charles was eager to engage with his students.While the topic and the readings sparked engaging conversation, both the teacher and the students struggled with the final writing prompt.At the end of the unit, students were asked to brainstorm, draft, revise, and edit a paragraph focusing on how immigration had changed their neighborhood.The students wondered how to respond to the prompt.Students who attend Pine High School ride the bus 30 to 40 minutes one way, and live off of unpaved roads near farms or forested land.Mr. Charles said, "They think of their 'neighborhood' as the three or four houses around them.Everyone said, there are no immigrants [where I live]."In an effort to have the students produce the required text, Mr. Charles asked them to think about times they went to the larger town 25 minutes away and ate at restaurants there such as the Chinese buffet or Mexican restaurants.The students wrote short texts about eating "weird" food, wearing "bright" clothes and being "nice."The students were generally unwilling to revise their responses.Mr. Charles' desire to implement the program with fidelity conflicted with local evidence about students' learning.From that day forward, Mr. Charles deviated from his earlier fidelity to EBP by adapting the program in order to attempt to account for his knowledge of his students and their needs.

Practice-Based Evidence: Attending to Context
Practice Based Evidence (PBE) offers a viable alternative to Mr. Charles and his students.PBE is immediately relevant, contextually based data collected to address the particular: this student in this context.It begins with the premise that human action never has universal consequences, and we therefore should expect a range of outcomes to any intervention across time and space (Biesta, 2010).As Bryk (2015) wrote, "Every student is not the same, nor is every context.The complexity is real, and it cannot be sidestepped by standardizing all activity in an effort to teacher-proof instructional environments" (p.474).The creation of PBE is a pragmatic exercise because continuous necessary adjustments in teaching practice, like those made by Mr. Charles, are understood as good teaching practice rather than ruptures in fidelity.PBE may be of particular use to teachers and learners in unique and complex rural settings.PBE does not disregard EBP findings, but, instead, judges EBP according to the contextualized understandings in which the teaching and learning occurs.This is a critical point of understanding.
While NCLB drew clear lines around the randomized control trial and quasi-experimental studies as only acceptable means to produce evidence, the Every Student Succeeds Act (2015) makes space for PBE.ESSA lists four possible forms of evidence: • Strong: randomized control trials • Moderate: quasi-experimental • Promising: correlational • Ongoing: active evaluation The "Promising" and "Ongoing" types of evidence appear to acknowledge the difficulty of establishing strong or moderate evidence that transfers effectively across cultures, time, and space (Balu et al., 2015;Gamse et al., 2008) These strategies paraphrase Bryk's (2015) distinction between EBP and PBE: "the difference between knowledge that something can work and knowledge of how to actually make it work reliably over diverse contexts and populations" (p.469).Erickson (2014) argues that the latter requires careful design of, attention to, and reflection upon "the what and how of local practices in order to determine specific local mechanisms of cause -why what is working does so, why it sometimes works better, why it sometimes falters" (p.5).In this way, local context contributes to a continuous aggregation of understandings of the potential for instructional methods in "complex and challenging networks of social interaction" (p.5).Said another way, considering how to make something work in local contexts suggests an important distinction between rurality as a factor to be overcome as opposed to rurality as a site from which to contribute to more nuanced understanding of instructional practices.

Practice Based Evidence in Diverse Rural Virginia Districts: Gifted Education
A study examining alternative identification processes for identifying underrepresented rural students in high poverty rural districts for gifted education services is an example of using PBE in rural schools.In previous work examining a curricular intervention, fidelity of implementation, and student outcomes, researchers tested the effectiveness of an integrated, curricular model developed by the National Research Center on the Gifted and Talented.In the "What Works in Gifted Education" study, researchers tested the effectiveness of the model in a randomized cluster design in more than 200 classrooms (see Callahan, Moon, Oh, Azano, & Hailey, 2015).Researchers also evaluated the relationship between teachers' fidelity of implementation and student achievement (Azano et al., 2011).In stratifying teachers' fidelity of implementation across all sites (urban, suburban, rural), they found that rural teachers were overrepresented in the "low fidelity" categorymeaning teachers did not adhere to the curriculumand, yet, their students were not outperformed by non-rural teachers with "high fidelity"-an anomaly in the findings.
A closer look at this phenomenon (Azano, Callahan, Missett, & Brunner, 2014) prompted further inquiry (see Azano, Callahan, Brodersen, & Caughey, 2017) explicitly focusing on the unique experiences of gifted education teachers and students in high-poverty, rural districts.The project set out not only to increase the number of identified gifted students in rural districts and impact language arts achievement, but also to positively influence affective outcomes, such as reducing stereotype threat and increasing academic self-efficacy.To meet these goals, the project established partnerships with rural school districts to better understand how place is conceptualized in their rural communities and to consider how identification processes might be context driven.In partnership, researchers and district leaders determined student inclusion for gifted programming based on local rather than national norms using universal screening for all students, as well as teacher ratings of student behaviors informed by rural focused professional development for teachers prior to rating students.The project has increased the number of students in gifted education programs across 13 high-poverty rural districts participating in the study.Moreover, preliminary findings suggest that students identified as gifted by the project in rural school districts using alternative identification processes are not outperformed by students identified as gifted using pre-existing processes in those same school districts.
This example provides preliminary, practicebased evidence informing an alternative process for identifying students for gifted services according to a local standard, responding to local needs, and applying relevant and place-conscious interventions to capitalize on local assets.Evidence was produced within a highly contextualized setting.Specifically, PBE suggested alternative ways for thinking about and assessing giftedness in rural communities.The inquiry began as an analysis of how and why evidence based practices weren't working in a particular setting.Rather than concluding that rural teachers were unable or unwilling to deliver a curriculum with fidelity or that, based on traditional measures of giftedness, that there were simply no gifted students, researchers generated PBE based on considerations of contextual responsiveness.The inquiry was based on negotiated understandings of what counts as gifted, an analysis of the relevance of EBP, and the development of locally produced interventions and data.The project, at its essence asked, "What works for this student in this place?"