Interpreting the Academic Dashboard for Grades 3-8

The Academic Dashboard presents a set of measures that demonstrate a school’s progress in meeting the English language arts (ELA) and mathematics goals required of each SUNY authorized charter school in its Accountability Plan.  Right now, all the data in the dashboard are taken from the New York State assessments and therefore only reflect those grades tested by the state’s ELA and math testing program (grades 3-8).

The Dashboard displays data for each year available within the past five years.  It is important to note that for a variety of reasons, data for a particular school might not be available.  In these cases, the school would not have an Academic Dashboard or would have an incomplete Dashboard.

First, schools in their initial years of operation may not have students enrolled in grades where the state administers tests. Similarly, the state administers tests at the end of a school year and does not release results until late summer or early fall. A school would need to complete an entire school year with students in the testing grades before data would become available for the Dashboard.

Second, one of the measures used in the Dashboard, the growth score, compares the performance of the school from one year to the next. Until a school generates two years worth of data, the school’s growth cannot be calculated.

Lastly, although some data does exist for SUNY authorized charter high schools, the SUNY Charter Schools Institute is still developing a separate high school Dashboard.

Understanding the Measures

The Dashboard incorporates two types of measures: comparative measures and growth measures.

The Comparative Measures

The Institute compares the English language arts and mathematics performance of its schools to schools within the local district and throughout the state.

As part of each school’s Accountability Plan, the Institute compares a charter school’s score on the state’s English language arts and mathematics assessments to the scores of all other schools serving the same tested grades throughout the local school district (or the community school district (CSD) in New York City).  This comparison only includes students from the charter school enrolled for at least two years.

For each year when data are available, the blue dots show the percentage of students at the charter school who scored proficient and the grey dots show the percentage of students from the school district who scored proficient on the exam.  Proficiency is defined as a score of 3 or 4 (the highest possible score). You can interpret the horizontal distance between the dots as the extent to which one group is outperforming the other.  The graph shows the overall performance of the school and district but you can mouse over each data point to see a grade level breakdown of the percentage of students who were proficient on the state exams.

It is important to note that this measure does not take into account student demographics like poverty, special needs status, or students’ language proficiency.  Any of these factors can impact the performance of one school relative to another school. For example, a school that serves a larger population of economically disadvantaged students may face additional challenges compared to a school that serves a smaller population of economically disadvantaged students.  In order to understand how a SUNY authorized charter school performs compared to schools that serve a demographically similar population of students, the Institute provides a measure referred to as an effect size.

The effect size measure compares the scores for a SUNY authorized charter school to other public schools statewide that serve a similar population of students in poverty. To do this, the Institute uses a very common statistical analysis called a regression equation.  The equation uses the scores from every public school across the state that serves a similar percentage of students who are as economically disadvantaged as a SUNY authorized charter school. Using this data enables the Institute to determine how a school should score based on its demographics and compare that to how the school actually scored. (Update: The Institute also recently began the use of mean scale score to measure student achievement in the effect size calculation.  Please refer to the document on changes to the Institute’s effect size analysis for more details.)

SUNY authorized charter schools that perform higher than expected have a positive effect size, and SUNY charter schools that perform lower than expected have a negative effect size.  In order to meet the goal outlined in SUNY’s Accountability Plan, a school must post an overall effect size of 0.3.  A school performing exactly as expected has an effect size of zero.

It should be noted that for each school year up to and including 2011-12 the Institute used the percentage of free lunch eligible students to measure poverty. In the 2012-13 school year, the Institute opted to use a measure of economically disadvantaged students.  The New York State Education Department uses this measure more commonly as its measure of student poverty.

The Growth Measure

Finally, the growth measure shows how well the school does in growing its students’ knowledge each year. The measure tracks how much the school grows what students know and are able to do each year.   SUNY authorized charter schools that are able to grow the scores of their students more than comparable students in other schools statewide have a greater growth score than a school that simply maintains the scores of its students over time. When possible the calculation can look back over as many as 3 years of assessment scores. You can mouse over each annual data point to see the growth scores by grade level (where the data are available). The higher the growth score, the more a SUNY charter school was able to grow what students know and are able to do.