Sustained positive effects on graduation rates

12 Pages·2012·0.17 MB·English

Sustained positive effects
mdrc
on graduation rates
Building Knowledge
To improve Social policy
produced by new york city’s Small
public High Schools of choice
By Howard S. Bloom and Rebecca Unterman
D
uring the past decade, New improving academic progress and
York City undertook a district- substantially improving graduation
wide high school reform that is perhaps prospects, particularly for disadvantaged
unprecedented in its scope, scale, and pace. students. This report included data for
Between fall 2002 and fall 2008, the school over 21,000 students from four cohorts
district closed 23 large failing high schools who entered ninth grade between fall
F
(with graduation rates below 45 percent),1 2005 and fall 2008.4 Findings in the report
opened 216 new small high schools (with were based on a series of randomized
different missions, structures, and student lotteries that were used when SSCs had
E
selection criteria), and implemented a more applicants than seats, in order to
centralized high school admissions process determine who got offered an opportunity
that assigns over 90 percent of the roughly to enroll in an SSC and who did not. SSC
I
80,000 incoming ninth-graders each year lotteries are a little-known byproduct
based on their school preferences. of the current district-wide high school
R
application processing system (HSAPS).5
At the heart of this reform are 123 small, These lotteries make it possible to
academically nonselective, public high rigorously estimate the effects of enrolling
B
schools. Each with approximately 100 in an SSC versus enrolling in some other
students per grade in grades 9 through 12, New York City public high school, based
Y
these schools were created to serve some of on a comparison of subsequent outcomes
the district’s most disadvantaged students for lottery winners and lottery losers and
C and are located mainly in neighborhoods a procedure that accounts for who among
where large failing high schools had been them enroll in an SSC.6
closed.2 Hence, they provide a realistic
I choice for students with widely varying In a large sample, like that used for the
l
academic backgrounds. MDRC researchers MDRC study, lottery winners and lottery
call them “small schools of choice” (SSCs) losers are the same, on average, in all
because of their small size and the fact that ways before they enter high school.
o
they do not screen students based on their Consequently, it is valid to attribute any
P academic backgrounds.3 differences in their future academic
outcomes to their access to an SSC.
In June 2010, MDRC, a nonprofit, Because students who lose an SSC
nonpartisan research organization, lottery attend over 200 high schools that
released a report on the effectiveness vary widely in their size, age, structure,
J a n u a r y of 105 of the 123 new SSCs, which academic programs, and effectiveness,
2 0 1 2 ddeemmoonnssttrraatteedd tthhaatt tthheeyy aarree mmaarrkkeeddllyy the MDRC report judged SSCs against the
MDrC PoliCy Brief
overall effectiveness of a diverse group of resources during start-up.8 By integrating a
other high schools.7 The results released in demanding and comprehensive academic
2010 indicated that, on average, the 105 SSCs curriculum, personal attention to student
studied increased student progress toward academic progress, and real-world experiences
graduation during their first three years of with community partners, the new small
high school and increased students’ four-year schools intended to prepare students for both
graduation rates. college and career.
Given data that were available at the time, In other cities, small high schools are often
the first MDRC report could only follow fashioned by reconfiguring large existing
the first student cohort (that entered in fall schools into smaller units in the same
2005) through four years of high school to buildings with the same teachers and students;
examine its graduation rates. This policy in contrast, the typical SSC was created largely
brief extends the analysis by a year, which “from scratch” with a principal, teachers, and
adds information on high school graduation students who were new to the school. At the
rates for the 2006 cohort and provides a fifth same time, many SSCs are located with other
year of follow-up for the 2005 cohort. This SSCs in buildings that previously housed a
information makes it possible to address the large public high school that was closed.
following three questions:
• Was the positive average effect of SSCs on SSCs are not just small in size. They are
four-year graduation rates for the study’s purposefully organized around smaller
first student cohort sustained through the educational units that are designed to give
second cohort? students a better chance of being known by
• Was this positive average effect sustained their teachers and other adults in the building.
across subgroups of students with different For example, many SSCs have created special
prior academic proficiency, family income, student advisory structures that provide
race/ethnicity, and gender? common planning time for teachers to discuss
• Was the average four-year graduation effect their students’ progress and problems.
sustained after five years?
SSCs received start-up funding from
The answer to all three of these questions is yes. philanthropic organizations plus technical
assistance and policy support from the New
w Ha T ar e S Sc s ? York City Department of Education and
SSCs were developed through a competitive intermediary organizations that were often
proposal process that was designed to experienced in managing the launching of
ensure that school founders met specified new schools — which helped to facilitate
conditions and to stimulate innovative ideas school leadership development, staff hiring,
from a range of stakeholders and institutions. and program start-up. Most SSCs began with
SSCs emphasize academic rigor and strong only a ninth-grade class, adding a subsequent
and sustained personal relationships among grade each year. Finally, SSCs received special
students and faculty. In addition, most were allowances with respect to serving English
2 founded with community partners who offer language learners and special education
students relevant learning opportunities inside students during their first two years of start-up.9
and outside the classroom and provide school The creation of SSCs was supported by a
faculty with additional staffing support and consortium of funders, led by the Bill &
January 2012
Melinda Gates Foundation, the Carnegie academic achievement.13 Prior research on
Corporation of New York, and the open small schools has produced mixed results
Society Institute. SSCs were implemented about their effectiveness. Some studies have
in collaboration with the United Federation found that they improve student outcomes,
of Teachers and the Council of School such as increasing levels of academic
Supervisors and Administrators.10 Unlike achievement and lowering rates of school
charter schools, which typically operate dropout.14 In addition, some studies suggest
independently of the school district in which that these effects are most pronounced
they are located, SSCs are directly responsible for disadvantaged students.15 on the other
to the New York City Department of Education. hand, a major study of a large number of
small schools initiatives suggests that they
w Hy STu d y S Sc s ? do not improve student academic outcomes
Almost all major urban school districts in the appreciably.16 In all of these cases however, the
U.S. and all but four states have attempted nonexperimental nature of the research limits
to create new small schools or attempted its ability to support causal conclusions.
to transform large schools into campuses
with “small learning communities.”11 This MDRC’s findings about SSCs are relevant
movement had its beginnings in the 1960s, to current federal policy on high school
when urban educators and community reform, particularly the U. S. Department of
organizations began to implement smaller Education’s School Improvement Grants
school structures as an alternative to the large (SIGs) for failing schools. Reforms funded by
high schools that dominated the landscape, SIGs include school transformation, school
many of which were failing. By the late 1990s, restart, school closing, and school turnaround.
small schools had emerged as a national SSCs straddle several of these categories since
reform strategy championed by such groups they are typically replacements for schools
as the Coalition of Essential Schools and the that have closed and they operate as regular
National Association of Secondary School public schools. However, it is important to
Principals. This strategy proliferated through recognize that SSCs represent far more than
district- and foundation-led initiatives in major just changes in school size and structure.
cities like Chicago, Philadelphia, Boston, They also represent innovative ways to use
oakland, and New York. In 2000, the Bill & these structural changes to leverage human,
Melinda Gates Foundation launched a national financial, and curricular resources.
campaign to improve failing urban high
schools, with small schools as a key strategy.12 The coupling of small schools with a
system of choice for all entering ninth-
The common theory of change underlying graders also has relevance for the national
both small schools and small learning debate about charter schools. on the one
communities specifies that smaller hand, SSCs share many similarities with
educational units promote stronger charters — they are small and personalized;
relationships among students, among most began with the help of intermediary
adults, and between students and adults. sponsors that resemble charter management
These relationships, in turn, are expected organizations; the students and teachers 3
to increase student engagement and help are there by choice; and the curricula are
teachers respond to students’ academic and demanding. on the other hand, SSCs are
social needs, thereby increasing students’ overseen by the Department of Education
MDrC PoliCy Brief
Table 1. estimated average effects of SSCs on Four-Year Graduation Rates: Student Cohorts 1 and 2
TargeT conTrol p-value For
eSTimaTed
ouTcome (%) SSc group eSTimaTed
eFFecT
enrolleeS counTerparTS eFFecT
graduaTion
graduaTed From HigH ScHool 67.9 59.3 8.6 ** 0.000
local diploma granted 19.9 19.6 0.3 0.406
regents diploma granted 41.5 34.9 6.5 * 0.041
advanced regents diploma granted 6.2 4.7 1.6 0.154
college readineSS
passed math regents at 75 or higher 23.3 22.5 0.8 0.682
passed english regents at 75 or higher 37.3 29.7 7.6 ** 0.002
NoTES: There are 13,064 student observations for cohorts one and two combined. There are no statistically significant differences between
estimated effects for the two cohorts. Because of rounding error and the complex calculations involved, results by diploma type do not sum
exactly to those for overall graduation rates.
with unionized teachers and principals; The report found similar positive effects on
the schools are nonselective and open to progress toward graduation in ninth grade
students regardless of academic proficiency; for subgroups of students who varied in
and the school choice process was designed terms of their reading and math proficiency
to engage all entering ninth-graders, not just before entering high school, their eligibility
the most motivated parents and children that for free or reduced-price lunches, and their
might seek out charter schools. race/ethnicity and gender.
H o w mu cH do S Sc s At two and three years after students entered
H e l p STu d e nT S? high school, the study found that SSCs
previous Study Findings: Because students increased their progress toward graduation by
became part of the study sample in four roughly the same amount as they did during
annual cohorts, MDRC’s first report on the the first year. And at four years after students
effects of SSCs was based on four years entered high school, the study found that
of follow-up for the first cohort of entering SSCs increased high school graduation rates
ninth-graders, three years of follow-up for the by 6.8 percentage points (to 68.7 percent for
second, two years of follow-up for the third, target SSC enrollees from 61.9 percent for
and one year of follow-up for the fourth cohort. their control group counterparts).
Based on data for all four cohorts, the report new Findings on graduation and college
found that SSCs increased the percentage readiness for Two Student cohorts: Findings
of students who were on track17 toward in Table 1 indicate that SSC effects on
4 graduation in ninth grade by 10.0 percentage four-year high school graduation rates are
points (to 58.5 percent for “target SSC sustained through a second cohort of entering
enrollees” from 48.5 percent for their control ninth-graders. Enrolling in an SSC instead
group counterparts).18 of another type of New York City public high
January 2012
school increased average four-year graduation subgroups examined that had samples that
rates for the first two cohorts combined by 8.6 were large enough to support findings.21
percentage points (to 67.9 percent for target
SSC enrollees from 59.3 percent for their Consider the results for subgroups defined
control group counterparts).19 Almost all of by students’ prior academic proficiency, as
this graduation effect reflects an increase in measured by their eighth-grade state test
receipt of New York State Regents diplomas. scores in reading and math. New York State
For this type of diploma, students must pass reports this proficiency in four levels. levels 1
a minimum of five Regents examinations and 2 (did not meet or partially met standards)
(English language Arts, Mathematics, represent student performance that is below
Science, Global History, and American grade level. levels 3 and 4 (fully met standards
History) with a score of 65 points or above or met standards with distinction) represent
and must pass all courses that are required performance that is at or above grade level.
by the state. None of the SSC graduation Since very few sample members score in the
effect appears to reflect an increase in receipt top category, no findings are reported for it.
of local diplomas, which have less stringent The findings indicate that SSCs increased four-
standards for scores on Regents examinations year graduation rates appreciably for students
and are being phased out by New York State in levels 1, 2, and 3 of reading and math prior
for the graduating class of 2012. likewise, proficiency.
little if any of the SSC graduation effect
reflects an increase in receipt of Advanced Although specific estimates vary somewhat
Regents diplomas, which have the most (from 7.7 to 11.3 percentage points for reading
stringent standards and are received by very and from 8.6 to 11.4 percentage points for
few students who apply to SSCs. math), all of the effects are appreciable and
statistically significant, indicating that every
Table 1 also indicates that enrolling in an group benefited substantially from attending
SSC increased students’ college readiness an SSC. For example the 11.4 percentage point
in English but not in math, as measured gain in high school graduation rates produced
by the percentage who scored 75 points or by SSCs for students with a level 1 eighth-grade
higher on the corresponding New York State math proficiency score implies a 33 percent
Regents examinations.20 This threshold increase (11.4/34.5) in graduation rates for this
is used by the City University of New York subgroup of students with especially weak math
to exempt students from taking remedial preparation. However, there is no clear pattern
courses in these subjects. to the variation in results across proficiency
subgroups; so while the impacts are statistically
new Subgroup Findings on graduation significant, the reported differences between
for Two Student cohorts: Adding a second subgroups are not statistically significant.
cohort of students to the analysis of SSC Therefore, one should not conclude, for
effects on four-year graduation rates instance, that SSCs worked differently for level 1
increased the sample size by enough to students than for level 2 students, even though
make it possible to examine how these the point estimates differ.
effects vary across subgroups of students 5
with diverse backgrounds. Findings in Table These subgroup findings by academic
2 indicate that SSCs increased four-year high proficiency are especially striking given
school graduation rates appreciably for all the dramatic differences that exist in the
MDrC PoliCy Brief
Table 2. estimated effects of SSCs on Four-Year Graduation Rates for Student Subgroups: Student Cohorts 1 and 2
TargeT conTrol p-value For
eSTimaTed
STudenT cHaracTeriSTic SSc group eSTimaTed
eFFecT
enrolleeS counTerparTS eFFecT
8TH-grade reading proFiciency
not meeting State learning Standards (level 1) 37.3 29.6 7.7 * 0.044
partially meeting State learning Standards (level 2) 66.7 55.5 11.2 ** <.0001
meeting State learning Standards (level 3) 81.1 69.8 11.3 ** 0.003
exceeding State learning Standards (level 4) -- -- --
8TH-grade maTH proFiciency
not meeting State learning Standards (level 1) 45.9 34.5 11.4 ** <.0001
partially meeting State learning Standards (level 2) 65.1 53.8 11.3 ** <.0001
meeting State learning Standards (level 3) 85.9 77.3 8.6 ** 0.001
exceeding State learning Standards (level 4) -- -- --
low-income STaTuS
eligible for free/reduced-price lunch 66.9 55.7 11.2 ** 0.000
not eligible for free/reduced-price lunch 74.3 67.4 6.9 ** 0.001
race/eTHniciTy, By gender
Black male 65.2 55.3 9.9 * 0.021
Black female 69.1 60.8 8.3 * 0.018
Hispanic male 61.3 53.7 7.7 0.065
Hispanic female 68.7 60.0 8.7 ** 0.032
other male 78.9 69.3 9.6 0.359
other female 85.4 78.4 7.0 0.409
Known/unKnown STaTuS
Known 68.8 61.1 7.6 ** <.0001
unknown 67.0 56.9 10.1 ** <.0001
NoTES: Estimated subgroup effects are not statistically significantly different from each other within any category of subgroups. Because of analytic
constraints, the total number of student observations varies across categories. Respectively, the number of student observations is 651, 8,223, 2,221,
and 114 for reading proficiency levels 1 through 4; 2,612, 5,963, 2,768, and 53 for math proficiency levels 1 through 4; 9,841 and 2,409 for eligible and
not eligible for free/reduced-price lunch; 1,708, 2,452, 1,866, 2,130, 283, and 261 for black males, black females, Hispanic males, Hispanic females,
other males, and other females; and 6,389 and 4,820 for known and unknown.
January 2012
Table 2. estimated effects of SSCs on Four-Year Graduation Rates for Student Subgroups: Student Cohorts 1 and 2
TargeT conTrol p-value For
eSTimaTed
STudenT cHaracTeriSTic SSc group eSTimaTed graduation prospects for each subgroup, as experienced a 7.6 percentage point increase
eFFecT
enrolleeS counTerparTS eFFecT measured by its control group counterpart’s in four-year graduation rates; the roughly 43
graduation rate. For example, rates range percent of students in the sample who were
8TH-grade reading proFiciency
from 29.6 to 55.5 to 69.8 percent for levels 1, not known experienced a 10.1 percentage
2, and 3 in reading, respectively. Therefore, point increase. Both findings are statistically
not meeting State learning Standards (level 1) 37.3 29.6 7.7 * 0.044
SSCs increase high school graduation rates significant although they are not statistically
partially meeting State learning Standards (level 2) 66.7 55.5 11.2 ** <.0001
appreciably for students with extremely weak significantly different from each other. Thus,
meeting State learning Standards (level 3) 81.1 69.8 11.3 ** 0.003 graduation prospects, modest prospects, contrary to what might be expected, students
and stronger prospects. who make an effort to become known to their
exceeding State learning Standards (level 4) -- -- --
SSC do not benefit more from attending it
Findings for subgroups of students who were than students who do not make this effort.
8TH-grade maTH proFiciency
and were not eligible for free or reduced-
not meeting State learning Standards (level 1) 45.9 34.5 11.4 ** <.0001 price lunch tell a similar story. They indicate new Findings on Five-year graduation
that SSCs increased graduation rates for rates for the Study’s First Student cohort:
partially meeting State learning Standards (level 2) 65.1 53.8 11.3 ** <.0001
both subgroups (by 11.2 and 6.9 percentage Competing hypotheses exist about the likely
meeting State learning Standards (level 3) 85.9 77.3 8.6 ** 0.001 points) even though their graduation effects of SSCs on five-year graduation
prospects differed substantially. rates. For example, SSCs might be especially
exceeding State learning Standards (level 4) -- -- --
effective at keeping struggling students
low-income STaTuS Findings for subgroups of students by race/ engaged in school for a fifth year in order
ethnicity and gender further reinforce the to meet their graduation requirements. If
eligible for free/reduced-price lunch 66.9 55.7 11.2 ** 0.000
preceding results. They indicate that SSCs so, then SSC effects on five-year graduation
increase graduation rates appreciably for rates might be greater than their effects on
not eligible for free/reduced-price lunch 74.3 67.4 6.9 ** 0.001
all subgroups with a sample that is large four-year graduation rates. on the other
race/eTHniciTy, By gender enough to detect these effects.22 Specifically, hand, SSC effects on four-year graduation
SSCs increase graduation rates for black rates might reflect their ability to “speed
Black male 65.2 55.3 9.9 * 0.021
males and females by 9.9 and 8.3 percentage up” graduation for students who otherwise
Black female 69.1 60.8 8.3 * 0.018 points and for Hispanic males and females would need five years. In addition, due to
by 7.7 and 8.7 percentage points.23 their small staffs, SSCs might not have the
Hispanic male 61.3 53.7 7.7 0.065
capacity to provide the additional resources
Hispanic female 68.7 60.0 8.7 ** 0.032 The final subgroup findings in the table needed to adequately assist students in their
are based on whether or not students were fifth year. For either or both of these reasons,
other male 78.9 69.3 9.6 0.359
known ahead of time to the SSC in whose SSC effects on five-year graduation rates
other female 85.4 78.4 7.0 0.409 lottery they participated.24 A student can might be less than their effects on four-year
become known to an SSC by contacting it graduation rates.
Known/unKnown STaTuS
in person or by telephone, by visiting it, by
meeting with its representatives at a high Table 3 presents estimates of SSC effects on
Known 68.8 61.1 7.6 ** <.0001
school fair, or in other ways. To promote graduation rates at five years after the study’s
unknown 67.0 56.9 10.1 ** <.0001
informed school choice, the New York City first student cohort entered high school, as
Department of Education encourages eighth- well as their four-year graduation rates (from
NoTES: Estimated subgroup effects are not statistically significantly different from each other within any category of subgroups. Because of analytic
graders and their parents to do all of these MDRC’s first report). These findings indicate
constraints, the total number of student observations varies across categories. Respectively, the number of student observations is 651, 8,223, 2,221,
and 114 for reading proficiency levels 1 through 4; 2,612, 5,963, 2,768, and 53 for math proficiency levels 1 through 4; 9,841 and 2,409 for eligible and things, which require initiative and effort. that between year four and year five, graduation
not eligible for free/reduced-price lunch; 1,708, 2,452, 1,866, 2,130, 283, and 261 for black males, black females, Hispanic males, Hispanic females, rates for target SSC enrollees and their control 7
other males, and other females; and 6,389 and 4,820 for known and unknown.
Findings in Table 2 indicate that the roughly group counterparts increased by almost exactly
57 percent of students in the subgroup the same amount. Hence, the effect of SSCs on
sample who were known to their SSC graduation rates did not change.
MDrC PoliCy Brief
Table 3. estimated average effects of SSCs on Four-Year and Five-Year Graduation Rates: Student Cohort 1
TargeT conTrol p-value For
eSTimaTed
ouTcome (%) SSc group eSTimaTed
eFFecT
enrolleeS counTerparTS eFFecT
Four-year graduaTion
graduaTed From HigH ScHool 68.7 61.9 6.8* 0.013
local diploma granted 24.6 21.9 2.8 0.261
regents diploma granted 39.5 34.6 4.9 0.074
advanced regents diploma granted 4.4 5.5 -1.1 0.366
Five-year graduaTion
graduaTed From HigH ScHool 75.2 68.1 7.1* 0.018
local diploma granted 27.3 24.4 3.0 0.265
regents diploma granted 44.9 40.0 4.9 0.081
advanced regents diploma granted 4.7 5.8 -1.1 0.368
NoTES: There are 5,363 and 5,428 student observations for the four-year and five-year analyses, respectively. The difference represents
65 students who reenrolled in a NYC public high school between their fourth and fifth years. Because of rounding error and the complex
calculations involved, results by diploma type do not sum exactly to those for overall graduation rates
Specifically, graduation rates increased from 1. The 105 SSCs studied serve about
68.7 percent in year four to 75.2 percent in year 45,000 students at full capacity. This
five (or by 6.5 percentage points) for target SSC is equivalent in size to the entire high
enrollees and from 61.9 percent in year four to school population of Houston, Texas,
68.1 percent in year five (or by 6.2 percentage the seventh largest school district in the
points) for their control group counterparts. United States.
The effect of SSCs on five-year graduation rates
(a 7.1 percentage point increase) therefore was 2. The estimated 8.6 percentage point effect
almost identical to the effect of SSCs on four- of SSCs on four-year graduation rates
year graduation rates (a 6.8 percentage point is equivalent in magnitude to about 43
increase).25 In addition, at five years, the majority percent of the gap in graduation rates
of the SSC impact comes from receiving Regents between white students and students of
diplomas, as was the case at four years. color in New York City.26 Furthermore,
this effect is relative to currently existing
w H y a r e T H e S e schools, not to the 23 large failing high
F i n d i n gS im p o rT a nT? schools that were closed, schools that had
To place these findings in perspective, graduation rates that were much lower than
consider: (1) the scale of the intervention the rates of the control group schools that
evaluated, (2) the size of the effects it the SSCs were compared with in this study.
produced, (3) the nature of the population it
8 served, (4) the diversity of subpopulations 3. SSC effects reflect a student population
it benefited, (5) the institutional nature of that is 93.2 percent black or Hispanic,
the intervention, and (6) the rigor of the 83.9 percent eligible for free or reduced-
evaluation conducted. price lunch, 21.8 percent overage for
January 2012
Table 3. estimated average effects of SSCs on Four-Year and Five-Year Graduation Rates: Student Cohort 1
TargeT conTrol p-value For
eSTimaTed
ouTcome (%) SSc group eSTimaTed grade, and 70.5 percent below grade level • Do SSCs increase college readiness and
eFFecT
enrolleeS counTerparTS eFFecT in reading and 63.9 percent below grade success in postsecondary education?
level in math, based on state tests in • How were SSCs created? How do they
Four-year graduaTion
these subjects in eighth grade. operate? What obstacles do they face?
graduaTed From HigH ScHool 68.7 61.9 6.8* 0.013
How are they dealing with these obstacles?
local diploma granted 24.6 21.9 2.8 0.261 4. SSC effects were experienced by students What are their most important sources
with widely varying graduation prospects, of financial and human resources? What
regents diploma granted 39.5 34.6 4.9 0.074
by students who were and were not factors do they believe are most important
advanced regents diploma granted 4.4 5.5 -1.1 0.366 eligible for free and reduced-price lunch, to their success?
by students who were male and female, • How do SSC effects vary by school and
Five-year graduaTion
black and Hispanic, and by students who what factors predict this variation?
graduaTed From HigH ScHool 75.2 68.1 7.1* 0.018 were known and not known to their SSCs. • What role does teacher selection, training,
and quality play in producing SSC effects?
local diploma granted 27.3 24.4 3.0 0.265
5. New York City’s SSCs were developed
regents diploma granted 44.9 40.0 4.9 0.081 through a demanding proposal process co n c l uSi o n
that was designed to ensure specified In summary, the present findings provide
advanced regents diploma granted 4.7 5.8 -1.1 0.368
conditions and to stimulate innovation highly credible evidence that in a relatively
NoTES: There are 5,363 and 5,428 student observations for the four-year and five-year analyses, respectively. The difference represents from a range of stakeholders and short period of time, with sufficient
65 students who reenrolled in a NYC public high school between their fourth and fifth years. Because of rounding error and the complex
institutions. This differs markedly from organization and resources, an existing
calculations involved, results by diploma type do not sum exactly to those for overall graduation rates
many small schools in other jurisdictions school district can implement a complex
that are created by reconfiguring existing high school reform that markedly improves
schools. Furthermore, SSCs operate graduation rates for a large population of
within the school district, in contrast low-income, disadvantaged students of color.
to charter schools, which operate more
independently. en d n oTeS
1 The closure of the 23 large high schools happened over
time — as they no longer accepted entering ninth-graders
6. These findings are based on data for a
and, thus, phased out over four years.
very large sample that was created by a
2 Fifty SSCs are in the Bronx and 31 are in Brooklyn. Many
series of naturally occurring randomized of them are in the city’s poorest neighborhoods.
lotteries. Hence, they provide unusually 3 other new small secondary schools created by New York
City during this time period include 38 general high schools
rigorous evidence about the causal
for grades 9-12, which screen students based on their
effects of SSCs. And instead of relying academic backgrounds; 21 transfer schools that are designed
on a single test score as a proxy for to help students who are overage for grade and under-
credited; 33 middle/high schools for grades 6-12 or 7-12; and
later life outcomes, the study measures
one specialized high school for high-performing students.
actual grade progression, high school
4 Bloom, levy Thompson, and Unterman (2010).
graduation, and Regents diplomas
5 SSC lotteries are not public events, but rather the result of
attained and scores achieved. the rules by which HSAPS assigns students to SSCs. of the
123 SSCs created in New York City, 105 had at least one lot-
tery for one of the four student cohorts in the present study.
wHaT are THe nexT STepS
6 Appendix A of Bloom, levy Thompson, and Unterman
For reSearcH on SSc s?
(2010) describes how HSAPS creates SSC lotteries, how
over the next three years, with funding from an SSC lottery creates a treatment group of SSC lottery
winners and a control group of SSC lottery losers, and
the Bill and Melinda Gates Foundation, 9
how this information was used, together with information
MDRC and its partners will expand and about the schools in which students enrolled, to estimate
deepen their research on SSCs to address the effects of enrolling in an SSC relative to enrolling in
some other New York City public high school.
such questions as:
MDrC PoliCy Brief
Corresponding findings for the effects of winning an SSC SSCs worked better for the second cohort than the first.
lottery are presented in Appendix B of the original report.
20 These findings on college readiness for cohorts one
These findings are consistent with those for the effects of
and two combined tell the same story as those for cohort
enrolling in an SSC. The analysis in this policy brief uses
one in MDRC’s first report.
the same methodology.
21 To estimate SSC effects for student subgroups, the
7 Because MDRC’s study examines only one component of
sample for each SSC lottery is stratified by the student
New York City’s overall high school reform initiative (that
characteristics that define each subgroup. Because not
is, the small schools of choice), it does not attempt to de-
all lotteries contain students from each stratum, the total
termine the overall effect of these reforms. Neither does
sample represented by each type of student characteristic
it attempt to determine the effect of closing the 23 large
(e.g., by reading proficiency or race/ethnicity) varies.
failing high schools on students who would have attended
22 The sample for “other males” comprised only 283 stu-
them had they not been closed.
dent observations; for “other females,” the sample was
8 For example, a law firm provides internships and em-
only 261 student observations.
beds real-world examples in the curriculum for the Urban
23 The estimated SSC effect for Hispanic males is not sta-
Assembly School for law & Justice, and the Urban Assem-
tistically significant at the 0.05 level, although it is statisti-
bly New York Harbor School is growing one of the largest
cally significant at the 0.10 level, which is used by many
oyster beds in the region.
researchers. Furthermore, this finding is consistent with
9 As a result of a court decision, the New York City
those for the other three subgroups of students of color
Department of Education ended this practice.
and is not statistically significantly different from them.
10 As described by Randi Weingarten at a forum on small Hence, the data suggest that SSCs increase four-year
schools sponsored by the Alliance for Excellent Education graduation rates for Hispanic males, although the evi-
in November of 2010, the United Federation of Teachers dence for this subgroup is weaker than that for the others.
participated in the planning and approval process for
24 Bloom, levy Thompson, and Unterman (2010) de-
many of the new small schools in New York City.
scribe how each SSC lottery includes either students who
11 By 2008, 46 of 50 states and the District of Columbia were known to the SSC or students who were not known
had received multimillion-dollar federal grants for small to it, but not both. lottery winners and losers are thus
learning communities (U.S. Department of Education, perfectly matched in this regard.
n.d.).
25 The slight discrepancy between the reported difference
12 While the Gates Foundation’s stated goal was to sup- in changes over time of graduation rates for target SSC
port the creation of schools that possessed seven “Attri- enrollees and their control group counterparts and the
butes of High Performing Schools,” it was the structural change over time in the estimated impact of SSCs on the
focus on school size (not to exceed 100 students per graduation rate is due to rounding.
grade) that became a key strategy of the foundation’s
26 MDRC computations based on 2011 data from the NYC
grantmaking (Evan et al., 2006).
Department of Education indicate a gap between citywide
13 See Finn and Voelkl (1993); lee and loeb (2000); Was- four-year graduation rates for white students and students
ley et al. (2000); Klem and Connell (2004). of color equal to 21.0 percentage points for the fall 2005
cohort and 18.8 percentage points for the fall 2006 cohort.
14 For student achievement, see Haller, Monk, and Tien
The average gap is thus 19.9 percentage points.
(1993); Howley (1989); Howley and Huang (1991); lee
and Smith (1997). For dropout rates, see Pittman and
Haughwout (1987); McMullen, Sipe, and Wolf (1994). reFerenceS
15 See lee, Bryk, and Smith (1993); lee and Smith (1995);
For all analyses in this brief, MDRC’s calculations use
lee and Smith (1997).
High School Application Processing System data from
16 See Evan et al. (2006). eighth-graders in 2004-2005 to 2006-2007, as well as
data from New York City Department of Education atten-
17 The “on-track” measure was developed by the Consor-
dance, course credits, Regents exam, and enrollment files
tium on Chicago School Research. High school students
from the 2005-2006 to 2009-2010 school years.
who are on track to graduate by the end of their first year
— meaning that they have earned at least 10 credits and Allensworth, Elaine, and John Q. Easton. 2005. The On-
are failing no more than one core subject — are three and Track Indicator as a Predictor of High School Graduation.
a half times more likely to graduate in four years than are Chicago: Consortium on Chicago School Research.
other students (Allensworth and Easton, 2005).
Bloom, Howard S., Saskia levy Thompson, and Rebecca
18 Target SSC enrollees are sample members for whom Unterman. 2010. Transforming the High School Experience:
SSC effects are reported. Appendix A of Bloom, levy How New York City’s New Small Schools Are Boosting Stu-
Thompson, and Unterman (2010) describes how this dent Achievement and Graduation Rates. New York: MDRC.
group is defined and how findings for it are obtained.
Evan, Aimee, Jamie Shkolnik, Mette Huberman, Becky
10 19 The estimated average effect of SSCs on four-year Smerdon, Barbara Means, Mengli Song, Karen Mitchell,
graduation rates was 9.9 percentage points for the second Charles Storey, linda Shear, and Kazuaki Uekawa. 2006.
cohort and 6.8 for the first cohort. Both of these estimates Evaluation of the Bill & Melinda Gates Foundation’s High
are statistically significant, although the difference between School Grants Initiative. Washington, DC: American Insti-
them is not. Hence, one cannot say with confidence that tutes for Research.