Measuring impact at Skateistan: Social Return on Investment

At Skateistan we talk about ‘impact’ a lot. But what do we actually mean, and how do we measure it? The challenge is, impact is multi-dimensional and there are lots of ways to understand, analyse and communicate it. But essentially we want to know what sort of difference and how much of a difference we are making. Over the past few years we worked with three volunteers who rose to this challenge and explored a unique way of looking at impact. We are very excited to show you what they found.

It’s called ‘Social Return on Investment.' What, you say? Yes, it’s a mouthful. So we shortened it to SROI. SROI is a measurement tool that helps organizations to understand and to tell the story of how social, economic and environmental value is being created by their programs. Social values are translated into monetary values so that a ratio for return on investment can be calculated. For example, an investment of $1 USD could deliver $X USD of social value. But SROI is primarily about value, rather than money. The process requires careful consideration of all levels of all program operations, from the planning of activities to their execution and evaluation.

“What I enjoy about the project is the story that it tells, SROI is not only a number, it tells the story of change.”

Elisa

Skateistan Volunteer

“I was never able to visit the skate schools in Afghanistan, but this project helped to paint a pretty clear picture of what it may be like. I got to learn things about Afghanistan, the children attending the skate schools, and the teachers making it all happen. Learning about each child’s story and seeing what they value about the program really helped to build a strong connection with what I was doing, hence why it was such an enjoyable (but challenging) process.”

Zach

Skateistan Volunteer

Our volunteers, Zach Lynn, Elisa Likovic and Adèle Guinaudie, each focused on a Skate School location (Afghanistan, South Africa and Cambodia respectively) to write up a SROI report for the year 2018. Methodology was chosen from an established guide for SROI analysis developed by Social Value UK and the New Economics Foundation (Social Value UK, 2012). We were supported throughout the process by Statistical Evaluation Consultant, Samantha Chui, as well as a team of 10+ independent volunteers from ‘Statistics Without Borders’ to help us analyze and verify our data.

Below are some of the main steps followed:

Who was involved?
Identifying key stakeholders such as students, families, staff and indirect stakeholders like governments, schools, partners, and even yourselves.
How much was invested?
Identifying the monetary inputs - including in-kind donations.
What were the activities?
Understanding Skateistan’s sport and educational program activities, as well as staff training activities.
What were the changes?
Identifying major outcomes; improved wellbeing and skill development including life skills, leadership skills, and literacy/numeracy skills. As well as improved child care capacity for our staff which takes into account an improvement in employability as staff develop necessary skills to succeed and excel in the workplace. These outcomes were evidenced from qualitative and quantitative data analysis from our Skate Schools.
How did we value these changes?
Noting various assumptions and estimations from qualitative interviews with staff at our Skate Schools, and identifying suitable financial proxies to our program activities. We asked ourselves; how long will the impact last?; how much change would happen if we didn't exist?; and what services (financial proxies) can we compare our programs with?
What did we find?
Calculating the SROI ratio based on a 5-year and 10-year forecast by incorporating all of the above information.

We explored each location Skateistan works in individually, because each country’s context is very different, meaning that a comparison between them is not appropriate. For example, 2018 was the year Skateistan Cambodia moved locations and built a new Skate School, so a smaller number of programs were running than usual. In reality, the values we found are likely to be understated, given the unique nature of the services offered by our sport and educational programs. Similar services are extremely scarce and don’t necessarily offer the same quality of guidance for youth, or accessibility to such a variety of activities.

“For this project, I had to refer to various international and government databases. While I knew certain socioeconomic data would be difficult to find, I was pretty shocked to see how scarce it actually was and, when it was there, how skewed the information seemed to be. Next to this, having to read about the history of Afghanistan as well as the situation that the Afghan people live in today was hard to believe – at the same time though, it shows how valuable the skate schools truly are.”

Zach

Skateistan Volunteer

Key staff members from each Skate School were involved in the SROI process, but the analysis is not without its limitations. With more freedom in geographic locations, time and resources, more people could be involved in the valuation process. Overall, the study highlighted some lessons learned that we now want to focus more on. One of our volunteers, Elisa, explains,

“Conducting focus groups with more stakeholders, like the parents, was, for example, one of the recommendations that we made.”

Elisa

Skateistan Volunteer

“Data is empowering and can yield more accurate and efficient decision making. Quantitative analysis is a catalyst for change. Skateistan has utilized original data collection, data science, and statistics to ensure that their internal policies and external community engagement programs are transparent, reproducible, and reliable. Skateistans’ SROI models set the standard for best practices in measuring programmatic impact and social change."

Samantha Chiu

Lead Data Scientist at Booz Allen Hamilton

An important lesson was about our understanding of long-term impact. Making forecasts for 5 or 10 years into the future got us thinking about how we can build on our programs and data collection system to stay connected with our students even when they leave Skateistan and graduate from high school. Many of our students have grown up to become educators at Skateistan themselves, but there’s also many unheard and untold stories out there. That’s why this year we are piloting an alumni project, supporting our graduating students to take action in their local communities.

“For me, it was a valuable process because it helps to understand the value of Skateistan’s programs in the context of kids who would not ordinarily have access to such opportunities. In South Africa, extra murals (organized activities outside of school) are often regarded as an opportunity for the privileged and are sometimes so expensive that they seem exclusionary. It’s great to know that what we offer to kids has a value that can, for better or worse, be given a monetary figure.”

Mbali

General Manager, Skateistan South Africa

We are very grateful to everyone who was involved in this research. If you are interested in reading one of our full reports on a Skate School location, please contact us at info@skateistan.org. We are in discussion about third party evaluation plans for the future, and we appreciate all constructive feedback to ensure the ongoing improvement of our impact evaluations.

“I am really grateful to have taken part and contribute to such an in-depth and important study and I believe many other organisations would benefit from it.” - Adèle,

Skateistan Volunteer

If you'd like to stay up to date with all of Skateistan's activities, make sure you subscribe to our monthly email newsletter. You can also support Skateistan's work by making a donation today.