How Broome Street Academy is Using Active Ingredients to Innovate and Improve Its Mental Health Services and Supports

“We believe that one of our Active Ingredients or the secret sauce for us is that when kids have access to mental health services, they will… eventually do better in the world, which includes doing better in school.” – Melissa Silberman, Former Head of School, Broome Street Academy

What is the Active Ingredients Project?

Launched in 2017 with funding from the Walton Family Foundation and the Bill and Melinda Gates Foundation, Active Ingredients aims to help charter schools develop credible systems for measuring and reporting meaningful student outcomes from supports such as wraparound services, community partnerships, service projects and social-emotional learning.

Engaging Broome Street Academy

Broome Street Academy is a one-of-a-kind charter school located in Manhattan “formed on a foundation rooted in social justice.” Started by The Door, a nonprofit organization, 50% of Broome Street students are considered opportunity youth —students experiencing housing insecurity, in the foster care system, in the juvenile justice system, or students with disabilities. To meet their unique academic and nonacademic needs, the charter school, working alongside The Door, runs a full service wraparound high school program with a particular focus on mental health services and supports.

The SUNY Charter School Institute approached Broome Street Academy about participating in the Active Ingredients project in spring 2019 to provide them with the opportunity to measure an aspect of its program – access to those mental health services and supports – that it wasn’t necessarily getting credit for within its existing accountability plan. In particular, the Institute wanted to gain a more comprehensive understanding of how the charter’s services impact students and their overall performance and share those data and best practices with other schools.

From Broome Street’s perspective, Active Ingredients would provide an opportunity to gather program outcome data that the school and the Door could use to ensure the program meets its mission and gain additional philanthropic funding to support hiring more mental health providers. Above all, the Institute and Broome Street were excited to tell the full story, with the credible, verifiable, and relatable data, of how the charter set students up for success in different ways.

How the Process Started

Broome Street brought together a team made up of the school leader, lead social worker, and data manager. This team worked with the Institute to clearly catalog and define the various elements of the program that the team believes has the most important impact on students using a program evaluation tool called a logic model. The charter eventually chose to focus on Broome Street’s mental health services, and more specifically, measuring student attitudes and perceptions about the services, including how open students are to accessing them. As a team, they chose to measure this Active Ingredient using attendance data, a regularly administered mental health screening tool—The Ohio Scales, and three survey questions aimed at determining student attitudes toward accessing and receiving mental health services. These student outcomes directly related to the Active Ingredient and their larger goal of improving access to the available mental health services.

“For us, what was critical was getting our school social workers involved in the project because they absolutely understood that this was our Active Ingredient before we did,” said Melissa Silberman, Broome Street Academy’s Former Head of School.

Adapting and Innovating

Broome Street began collecting data to measure their Active Ingredient in fall 2019. However, the onset of COVID halted data collection in spring 2020 once schools closed for in-person instruction. Shortly after, Black Lives Matter protests spurred leadership and staff to reflect on the effect racism has on their entire student population. They decided that all Broome Street students needed to engage in mental health services in order to respond to the racism they regularly face.

“We started with this idea that some people in the school need social work and others do not. And then coming from watching a large, kind of loud, and uncomfortable debate about systemic racism unfold through COVID, we really pushed ourselves to acknowledge systemic racism as a reason people need social work. And all of this was reflected in the documentation of our program elements within the logic model,” said Silberman.

Moreover, the Active Ingredients team realized their use of attendance data was too far removed from their Active Ingredient, and they needed a more proximal outcome. The team found they could analyze redacted social worker’s notes to determine students’ concerns and mental health issues. Most recently, this data helped Broome Street determine that social workers needed additional training on body image issues and eating disorders to address these needs in their students. All of these adaptations are responses to how BSA’s goal changed for their mental health services despite keeping this as their Active Ingredient.

Authorizer – School Collaboration

The Institute and Broome Street engaged with one another on a monthly basis throughout the Active Ingredients project. The collaboration provided an opportunity for the Institute to learn more about Broome Street Academy’s program and the school’s rationale for prioritizing certain activities. The Institute staff’s deeper understanding of Broome Street made it possible for them to better work with the school. Conversely, working alongside their authorizer helped Broome Street feel more comfortable reaching out to the Institute staff when they had questions or needed assistance. The collaboration also helped inform the Institute’s approach as it scaled the project to encompass the diverse array of Active Ingredients at other schools.

Next Steps

Broome Street and the Institute continue to establish a cogent and credible data collection, analysis, and reporting structure that allows Broome Street to include its Active Ingredients measures in its charter Accountability Plan – which is the document that states Broome Street’s performance goals. Both the Institute and Broome Street are eager to include data on their Active Ingredient to paint a fuller picture of the school’s outcomes for students that includes a better understanding of how mental health services bolster students’ engagement in their academic pursuits. The Institute believes that including these facets of Broome Street will help the school’s mental health services continue to improve both in quality and in the number of students who access them.

Not only is the Broome Street team collecting data to inform their authorizer and stakeholders, they are collecting new data to help their staff understand the mental health challenges students are facing due to racism, the pandemic, and social media. By doing so, Broome Street can assess the effectiveness of the school’s existing key design elements and how they support students’ ability to manage challenges.

“We really feel that once we were able to document how we use mental health to help young people feel safer at school, we got this by-product that it’s not just that they feel good there. We’re doing something different than other places, and it’s making kids feel good,” said Silberman.