Relevance
Research suggests that girls value the opportunity to use computing in real-world applications. The Relevance intervention was designed by Apps for Good and the Raspberry Pi Foundation, and aimed to test whether seeing computing as relevant to solving real-world problems in contexts that matter to them would increase female pupils’ interest in studying computing.
A series of lessons for Year 8 pupils (aged 12 to 13) was developed for teachers in secondary schools in England to deliver for up to 13 weeks. These were used in a pilot study with three secondary schools between April and August 2021. The pilot study was evaluated independently to produce a set of recommendations and modifications, which were then incorporated into the resources for a full trial. The randomised controlled trial was implemented between January 2022 and April 2022, and involved 97 schools. Participating schools were divided randomly into two groups: a ‘control’ group who taught their usual computing lessons, and a ‘treatment’ group who delivered the Relevance intervention using the resources written for this study.
Before delivering the intervention, teachers completed a free online training course to introduce them to the resources. During the lessons, the teachers supported pupils to understand some common ways that technology can be used to solve real-world problems, then helped them to identify problems in their own lives and communities that could be solved using an app. Pupils worked in groups to develop their technical skills and design, code, and test their own apps in App Lab.
“I really enjoyed delivering it, and learning new things myself with the pupils, and am planning on using the resources again with next year’s Year 8 groups across the school.” – Secondary school teacher (report, p. 53)
The intervention was evaluated independently by the Behavioural Insights Team. The evaluators collected survey data from all participating schools at the start and the end of the intervention, with a final sample of almost 4,000 Year 8 female pupils. They also interviewed five teachers, conducted four pupil focus groups, observed four lessons, and collected survey data from 28 teachers delivered the intervention. Once the evaluators had analysed this data, they shared their findings and recommendations.
What we learnt
From the randomised controlled trial itself:
- The trial evaluation did not find statistically significant evidence that taking part in the intervention increased girls’ intention to study GCSE Computer Science. However, exploratory analysis using data from a survey question asking about pupils’ reported intention to study GCSE Computer Science in a slightly different way found a statistically significant effect for some (but not all) of the analyses conducted. Further information about this exploratory analysis is available on pages 21 to 22 and pages 35 to 36 of the evaluation report.
- The results suggested that the intervention had a possible small positive effect on the perceived relevance of and interest in computing amongst girls, though this effect was not statistically significant.
From the teachers’ and pupils’ feedback about the intervention:
- Some teachers reported finding the resources useful and stated an intention to use them again with future Year 8 classes.
- Some teachers reported challenges in delivering the resources, and there were indications that not all pupils at schools in the ‘treatment’ group received the lessons as intended, which could have limited the intervention’s impact. Although assessing the impact of the coronavirus pandemic on delivery was outside the scope of the evaluation, the start and the end of the intervention implementation took place during periods in which there were high numbers of cases of coronavirus nationally, with increased pupil and staff absences in schools across the country.
- Some female pupils reported increased engagement and enjoyment in computing lessons, particularly with regards to the creative elements. However, this increase may not be enough to lead to an expected increase in girls’ perceived relevance of computing.
- Some teachers reported that they were implementing a number of interventions in order to increase girls’ uptake of GCSE Computer Science. It may be that situating computing in more relevant contexts may not be enough to overcome the systemic barriers such as societal stereotypes and views about computing that prevent female pupils from choosing GCSE Computer Science.
From the independent evaluators’ further recommendations:
- It could be beneficial to combine using the intervention resources with other approaches to increase girls’ interest in computing.
- Some modifications to the resources would help increase pupil engagement and make the intervention easier to deliver, including reducing the number of lessons, refining the app examples, and making it easier to create apps collaboratively.
- The evaluators also recommended further adaptations including allocating more time to practical and creative elements and making it easier and more interactive for teachers to access additional support.
- Setting up systems to track whether schools are able to implement the intervention as planned and to use school administrative data to measure actual GCSE subject choices would help to better understand the longer-term impacts of the intervention.