DfE-funded RCT: SSIF Reading Attainment Study
Executive Summary
Seventeen infant and primary schools were involved and took part in a UK government-funded randomised controlled trial to investigate the impact of the ReadingWise Decoding Programme on reading ages. ReadingWise Decoding is an online, personalised programme that uses a range of techniques to accelerate reading proficiency. It trains teaching assistants to work with groups of 10 pupils.
The cohort of children involved was composed of those struggling to read at the expected level. Further qualitative measures were used, including staff and pupil surveys.
At the evidencing stage, the programme delivered a statistically significant improvement to the intervention group’s (n=163) reading ages when compared to the control (n=131). The mean increase in reading age in the intervention group was 79% greater than the control group (9.5 months compared to 5.3 months). If it were the case that the divergence continues, the impact would quickly double.
The survey data suggested overall satisfaction with the training and the programme with all staff considering supporting the further use of the programme and 94% of pupils indicating that they would recommend the programme to other pupils. 50% of staff running the intervention had received reports of the programme improving pupils’ work back in mainstream lessons. 87% of pupils said they enjoyed reading more after doing the programme.
In summary, the programme delivered strong, statistically significant impact and was well received by both staff and students. The teaching assistant and small groups model allows considerable cost efficiencies and best use of existing resources.
Background and Objectives
ReadingWise Decoding
The ReadingWise Decoding Programme is an online, personalised reading intervention that harnesses a range of techniques to deliver a rapid increase in reading proficiency in the shortest possible time. Teaching assistants are trained and work with groups of 10 pupils in regular, 20-minute sessions. Since 2015, over 20,000 pupils in UK schools have completed the programme.
Participants
Initially fourteen infant and primary schools in Liverpool were taking part in this government-funded Strategic Schools Improvement Fund (SSIF) project. This number has grown to seventeen as word got out and as the government requested the project extended its scope to include younger pupils.
The aims of this project are to improve outcomes in literacy by:
- increasing teaching assistants’ capacity to improve decoding among a sample of age 7-11 pupils (KS2) and age 6 pupils (KS1) pupils identified as having an attainment gap;
- reaching more pupils more cost-effectively;
- increasing collaboration between local schools;
- improving assessment to support on-going evidence of impact.
The project is running from January 2018 to March 2019, with this interim report coming at the midpoint. Each school has received training and has delivered the programme to a total of 150 pupils. The next phase in this project (September 2018 – March 2019) is to scale-up to a total of 300 - 500 pupils dependent on need.
Methodology
Initially fourteen schools in the Everton area were invited to participate by the North Liverpool Teaching School Partnership, with the total increasing to seventeen. A within-school randomised controlled trial design was selected to measure impact along with a staff survey and a participant to gather qualitative data.
School briefings
A headteacher or deputy headteacher at each school received a briefing from the ReadingWise project lead. This included an outline of the project and its aims, the trial design, and information designed to ensure the school was aware of the time and staff commitment necessary to deliver. These briefings were conducted over the phone, lasting approximately 30 minutes. All schools maintained their commitment to the project.
Training
Once the briefings were concluded, training sessions were scheduled over two days, the 14th and 15th March 2018, at the North Liverpool Teaching Schools Partnership buildings. It was agreed that this venue and number of sessions would make attendance as easy as possible for those involved. Six sessions ran, each lasting 2 hours, and schools were invited to choose any one of the sessions to attend. The expectation was that one member of the senior leadership team (or equivalent) and one teaching assistant would attend, together, from each school.
The training focused on the background to ReadingWise, the optimum environment in which to run the programme, and a thorough introduction to the activities within the programme.
The aim of the training session was to ensure that teaching assistants could target support and deliver the programme effectively to groups of ten pupils, and that they had the support of at least one senior member of staff. All twenty-nine staff responding to the survey rated the training as good to excellent (Good: 18%, very good: 61%, excellent: 21%.).
Pupil selection and pre assessment
To measure the impact on pupils’ reading, a randomised controlled trial design was chosen. By the 30th March 2018 school staff had selected 20 pupils from each school with low reading ability to take part in the project, basing their decision on a combination of teacher judgement and existing assessment data. Once the pool of candidates had been identified, they were randomly assigned into control and intervention groups through blind selection.
Children in both groups were then assessed using the GL SWRT reading assessment, creating a pre-intervention benchmark. This assessment was selected as a suitable measure for decoding (as opposed to other aspects of reading), because it is widely accepted as a robust, standardised assessment, and because it is not too burdensome on the schools’ time.
Analysis of pre-assessment scores shows the randomisation worked: there is no statistically significant difference in the pre-assessment scores between those randomised to receive ReadingWise and those who were not within schools.
Intervention and post assessment
The intervention pupils then began the ReadingWise Decoding intervention programme, consisting of regular (minimum three times weekly) sessions of 20 – 30 minutes with 10 pupils in each session and the teaching assistant supervising. Participants were placed on a personalised lesson plan based on their pre assessment data, and they worked independently at their own pace. The control group continued to participate in the school day in a ‘business as usual’ way.
Participants completed an average of 6 hours 19 minutes of the ReadingWise Decoding programme over a 6-week period. At the end of this period, both intervention and control groups were once again assessed with GL SWRT. The post assessments took place in July 2018. Participants were also invited to complete a survey, which received 149 returns.
The resulting pre and post scores were then analysed to give impact data, and the survey returns to yield qualitative data.
Results
Quantitative impact
Overall, results indicate that the intervention group’s average reading age increased by 9.5 months, while the control group’s average reading age increased by 5.3 months (see table 1). It is notable that the gap between reading age and chronological age closed from 15.4 months to 9.0 months in the intervention group, and from 13.1 months to 10.6 months in the control group, suggesting not just faster, but much faster progress among the participants (see figure 1). Participants improved 79% faster than those in the control.
Formal statistical modelling of the interim data confirms that, even after taking account of a pupil’s school, pre-treatment score, age in months, gender, language, and poverty status (i.e. school fixed-effects regression modelling), students receiving the ReadingWise programme made greater progress that those in the same school who were not, and this difference is statistically significant.
Qualitative data
Staff survey returns
Did teachers report ReadingWise helped pupils in regular lessons?
50% of staff running the intervention reported that either pupils’ class teachers, or the pupils themselves, felt ReadingWise had helped participants back in whole class lessons. Comments include:
“Teachers noticed a huge difference in the abilities of the children to engage in whole class work, and saw the improvement in the children as significant.”
“John’s teacher reported that his fluency had improved. Sammy and Patrick’s teacher also reported that their fluency had improved.”
Engagement and perceived quality
Staff found that pupils engaged well with the programme, with 90% stating that pupils displayed either ‘good’ or ‘high’ engagement (‘no engagement’ and ‘low engagement’, 0%; average engagement, 10%). A similar percentage, 88%, rated the ReadingWise Decoding intervention as either ‘good’ or ‘excellent’.
Ease of deployment and continuation
When asked about the ease of timetabling the sessions, 84% found timetabling sessions moderately, very, or extremely straightforward. All staff would consider supporting the continuation of ReadingWise in their school (definitely not: 0%, probably not: 0%, possibly: 30%, very probably: 35%, definitely: 35%).
Participant survey returns
Participants in each school were invited to complete a survey. 149 pupils responded.
Reading for pleasure
One of the more important findings is that an overwhelming number of pupils (87%) said they enjoyed reading more after the intervention. This is important as there is a clear correlation between enjoyment of reading and attainment in reading:
“The data shows that 10-year-olds who enjoy reading have a reading age 1.3 years higher than their peers who don’t enjoy reading, rising to 2.1 years for 12-year-olds and 3.3 years for 14-year-olds.”
95% of participants reported enjoying the ReadingWise programme, while 98% reported thinking their reading had improved since using ReadingWise (a little bit: 49%, a lot: 49%).
How focused did pupils think they were?
95% of participants felt that they were either ‘quite’ or ‘very’ focused when using ReadingWise (this correlates well with the staff perceptions, where 90% of staff found that pupils engaged well).
Do pupils think ReadingWise helped them back in class?
When asked whether ReadingWise sessions helped them in their normal lessons, 92% reported that they had helped either a little bit or quite a lot (not a lot: 8%, a little bit: 42%, quite a lot: 50%).
Would participants recommend it to other children?
Participants were asked whether they think other children should use ReadingWise, with a yes or no reply. 97% said yes, they think other children should use ReadingWise.
Would they like to continue?
83% of participants stated yes, they would like to continue to use ReadingWise.
Discussion
At this evidencing stage we can conclude that the project is working towards its aims. Children in the intervention are progressing almost 80% more quickly than those in the control group; staff are happy with their training and are able to successfully deliver sessions to groups of ten pupils; and participants report feeling a higher level of enjoyment with their reading, and report noticing an improvement in their reading in their day-to-day lessons. 50% of staff running the sessions indicated that improvement in participants’ performance back in the classroom had been reported.
ReadingWise in the Classroom
Since this DfE-funded RCT: SSIF Reading Attainment Study, ReadingWise has helped over 100,000 pupils in primary and secondary school advance their reading abilities, fluency, and enjoyment of reading.
Read our other studies:
What’s missing from the government’s reading strategy report?
University of Cambridge, Reading More Wisely
"Reading Wise has been great for our school. It was recommended to us from a cluster school and what a difference it has made in such a short time! We have used Reading Wise as a targeted intervention for 10 children as a trial and have been blown away by the results.”
“In only one term, the majority of our children made at least 10 months of progress in their reading age- some even more! We have decided to purchase Reading Wise for the whole school and are very excited to see the progress throughout the school.”
We are extremely impressed with the impact Reading Wise has had in our school and are very excited to continue our journey. I would recommend Reading Wise to any school!"
Hannah Boardman, Irlam Primary School, Salford, UK
See what other headteachers and literacy leads say about ReadingWise in our case studies and frequently updated testimonials.
Boost your school's literacy results: Arrange your 20-minute demo at a time to suit you.
We highly recommend you spend 20-minutes running through the programme with our friendly and knowledgeable team by booking a demo to suit your diary
Technical appendix
Dr Rebecca Allen
Research design
The study relies upon randomisation to achieve comparable intervention and control groups of pupils, balanced on both measured and unmeasured factors, so that the difference between them can be given a causal interpretation. Seventeen is too few to implement a cluster randomised trial, so ReadingWise asked the participating schools to provide long-lists of 20 pupils.
Ten pupils from each of these school lists were selected at random to receive the ReadingWise programme, with the others asked to continue ‘business as usual’ reading support. The randomisation was a success: there was no statistically significant difference in the pre-test scores between the two groups.
Making direct comparisons between intervention and control pupils within the same schools is only valid if we can be sure that the intervention programme does not spill over to affect the education of the control group pupils.
Background characteristics of pupils in study
Pupils in the study were in a variety of year groups, from Year 2 to Year 6, but were most typically eight and a half years old. They were more likely to be boys (55%), 15% had English as a second language and 54% were eligible for free schools.
At the start of the intervention, they were an average of 14 months behind their reading age. There is considerable variation in this pre-test deficit (the IQR is -27 to -4 months), but this is partly because the test is relatively short so has a large standard error on the score.
Attrition of pupils from study
Post-intervention scores were obtained on 91% of all the students starting the trial. ReadingWise does not hold full records on why post-intervention scores were not obtained, but schools who did provide this information recorded the student had moved schools or was off sick on the day of the test.
No school had more than two pupils missing from either the intervention or the control group. However, there appears to be greater missing data from the control group: 20 pupils overall, versus just 8 with missing data who received the programme.
Results
We analysed the impact of the programme using the changes in the reading test scores and changes in the child’s reading age. The substantive findings are the same, but slightly different simply because there is a non-linear mapping of test scores to reading age. We report here the results on changes in reading age because they are easier for lay-readers to interpret.
Table 2 reports the overall mean change in reading age between the start and end of the programme, separately for the intervention and control groups. The intervention group saw a larger rise in the reading age over this period, but we perform more formal analysis to see whether the magnitude of their greater increase is statistically significant.
