Informal Learning
Non-formal learning settings often see a higher proportion of girls taking part and typically offer a focus on creativity and opportunities for contextualised learning. Research suggests that girls are often unaware that the computing skills learnt in extracurricular or non-formal settings can help them in school and formal computing studies. The Informal Learning strand of the Gender Balance in Computing programme aimed to overcome this barrier by using two interventions, one in primary schools and the other aimed at secondary schools.
In both interventions, existing non-formal learning resources were adapted to help children and young people better understand how the skills they were using in non-formal settings could help them when studying computing more formally. The adapted resources drew on the behavioural science concepts of endowed progress and self-persuasion and the adaptations were made jointly by the Behavioural Insights Team, Apps for Good, and the Raspberry Pi Foundation.
Secondary school pupils took part in non-formal learning using an adapted Apps for Good course and primary schools ran after-school Code Clubs using adapted materials. The first run of these interventions began in Autumn 2019 and was disrupted by the outbreak of the coronavirus pandemic in March 2020. The interventions were repeated: the secondary school intervention was repeated from April to July 2021 and the primary intervention ran again from October 2021 to February 2022.
Secondary school intervention: Apps for Good
In the secondary school intervention, year 8 pupils from 57 schools took part in twelve, hour-long, teacher-led sessions adapted from one of the courses offered by Apps for Good. In these sessions, learners developed an app to solve a problem that was relevant to them. An example of a self-persuasion activity from this intervention (shown in Figure 1) was for pupils to create a poster to promote the positive aspects of studying computing.
This intervention was evaluated by quantitatively measuring pupil attitudes towards computing through the collection of survey data directly before and after the intervention. Interviews with staff and pupils were also conducted to understand their experiences with the intervention and how the intervention may or may not have fulfilled the intended outcomes.
Primary school intervention: Code Club+
The primary school intervention was evaluated using a randomised controlled trial (RCT), with participating schools assigned randomly to either a control group who used existing Code Club projects or a treatment group who used adapted Code Club projects, referred to as Code Club+. Both groups ran clubs for 12 weeks in after-school sessions with pupils aged 7–11 years old, and a total of 54 schools took part across the two intervention runs. An example of how the adapted resources drew on the concept of endowed progress is a skills rating game (shown in Figure 2), which pupils played at the start and the end of the intervention to see how their skills had improved.
Quantitative data on attitudes towards computing was collected through pupil surveys. Observations of Code Club sessions, interviews with staff, and pupil discussion groups were also conducted to understand the mechanisms of change and the diversity of programme delivery.
What we learnt
From the intervention evaluations:
Secondary school intervention:
- Girls reported a small but positive change in their attitudes towards computing, but a small negative change in boys’ attitudes was found between the pre-intervention and post-intervention survey. It is not possible to determine the extent to which these changes were due to the adapted resources or other factors, as there was no comparison group of pupils who used the original Apps for Good materials.
- There was no statistically significant change in the reported intention to study GCSE computer science.
Primary school intervention:
- Relative to regular Code Clubs, the estimated effect of the Code Club+ intervention on attitudes towards computing and stated intention to select GCSE computer science was positive, but was small in size and not statistically significant.
The added demands COVID-19 placed on schools likely resulted in significant challenges to the delivery of both interventions and might have thereby limited their effectiveness.
From teachers’ and pupils’ feedback about the interventions:
Secondary school intervention:
- Interviews with teachers and pupils suggested that the relevance to real-world problems in their communities made computing more purpose-driven and appealing to girls.
- Some teachers suggested that the intervention’s focus on creativity and collaboration helped girls’ attitudes to computing.
Primary school intervention:
- Both the existing Code Club projects and the Code Club+ resources were well received by the teachers and pupils who were interviewed.
- Teacher feedback suggested that the Code Club+ materials may not have been delivered consistently and as intended across all schools, which may have limited any additional impact compared with the existing Code Club materials.
From the independent evaluators’ further recommendations:
Secondary school intervention:
- The small increase in positive attitudes towards computing was not accompanied by an increase in reported intention to study GCSE computer science. Linking the intervention content even more explicitly to computer science as a GCSE subject in the adapted materials and providing further guidance to teachers could help to convert positive attitudes into the confidence and intention to choose GCSE computer science.
- The adapted materials could be further refined to balance the focus on engaging activities and technical skills to improve girls’ confidence.
- As some teachers reported difficulties in fitting content into the time available, reducing the lesson content or identifying core and optional content might help to make the implementation of the intervention more feasible, and to ensure teachers understand the key aspects of the content.
- A RCT of a refined version of the intervention could be conducted to obtain estimates of the impact of the intervention.
Primary school intervention:
- Based on the feedback from interviewed teachers, it would be beneficial to make the activities in the Code Club+ materials that link informal and formal learning even more salient and to allow teachers to view their pupils’ progress through the materials.
- Ensuring that all teachers are familiar with the Code Club+ activities ahead of the intervention, for instance by holding a mandatory training session, could help ensure they understand the purpose and delivery of the extra content and thus deliver the intervention as intended.
- For future evaluations, identifying strategies to measure longer-term outcomes targeted by the intervention, such as GCSE subject choice, would help to better understand the impact of early interventions targeting gender imbalance in computing.
Read the full evaluation report for Informal Learning: Secondary intervention here.
Read the full evaluation report for Informal Learning: Primary intervention here.