12. Schools Infection Survey

"One of the largest and most comprehensive longitudinal research studies undertaken globally" (students not in school on the day, so symptomatic infections and close contacts weren't included)

Sep 14, 2023

After the ad hoc study on teachers risk had been misrepresented in the media, wrongly describing a lack of evidence as no evidence of transmission; limitations that the UK Statistics Authority and the ONS recognised, the next important study conducted on behalf of the government was the first round of the Schools Infection Survey which was published on 17 December.

The survey continued into 2022 and is one of the main sources of evidence on transmission in schools and children cited by UK government agencies and administrations in other countries. The survey is led by Shamez Ladhani, the UK’s clinical lead on covid in children who has said on a number of occasions that preventing infection in children would simply prolong the pandemic.

ADR UK (Administrative Data Research UK) which describes itself as “a partnership transforming the way researchers access the UK’s wealth of public sector data, to enable better informed policy decisions that improve people’s lives.” Described the survey as “one of the largest and most comprehensive longitudinal research studies undertaken globally in primary and secondary schools during the pandemic”, and explains how vital the work has been in regards to government policy.

Research outputs from SIS were a core component of the approaches informing policies to keep schools open safely in the earlier parts of the pandemic. They also informed factors supporting recovery during the main period, reducing collateral impacts of the pandemic on children’s educational attainment and wellbeing.

The data collected by SIS in the second half of the 2021 spring term, shortly after schools reopened following national lockdown, was key to the government’s assessment of risk and planning for the ongoing policy that kept schools open. Knowledge of the lower antibody levels in young children in late Autumn 2021 was also crucial, as this informed vaccine recommendations.

Similarly, data around implementation of Covid-19 preventative measures has been widely used across government in their approach to managing the ongoing risk of infection and transmission in schools, particularly at the start of the new academic year in September 2021.

The rich dataset generated will also provide useful insights for future pandemic preparedness.

It was one of the key studies in the UK’s decision to remove all measures from England’s schools including isolation in September 2021. Since then parents have been advised not to send children into school if they have a fever, however they are allowed to send their children to school even if they know they have covid. In fact, in an effort to improve attendance the DfE is encouraging parents to send children to school unless they are so sick they can’t get out of bed.

The first publication of the School’s Infection Survey came at a time when the debate on schools was at fever pitch, while the school holidays were starting, the big question was if they should return at the start of January, and if they did, what measures should be in place?

The study made headlines across the UK media and abroad with its findings that "infections in schools reflects community transmission." Although this reassuring headline was repeated regularly, criticism of the studies methodology didn’t receive much media attention

The study is done by going into schools and testing the children on a number of occasions, this is then compared to test results in the wider population. However, only those students who are in school on the day are tested, meaning that the study omitted any child isolating after having tested positive for covid and any students who had been identified as close contacts.

What the survey is actually tracking are the cases test and trace missed, not the total number of cases in schools, and what this showed was that just as many cases were being missed as were identified in the local community. Due to this the statements put out regarding the survey are incredibly misleading.

Award Winning

The easily apparent issues with the paper were ignored and not rectified in later rounds which continued into the summer of 2022. In August 2022 former deputy CMO Jennifer Harries, now Chief Executive Officer of UKHSA celebrated the Schools Infection Survey citing its importance.

"The study partnership of ONS, The London School of Hygiene and Tropical Medicine (LSHTM) and UKHSA, working closely with DfE and the wider scientific community via the Schools Scientific Advisory Group, has provided critical insight into the impact of coronavirus in schools. Your approach has been a tremendous example of pan-Government working alongside academia – true teamwork across organisational boundaries."

The work also won a Civil Service award for its implementation and impact.

"The Schools Infection Survey and the Covid Infection Survey have jointly been recognised in this Civil Service award. These awards recognise the successful delivery of large scale projects by the Civil Service recognises and pays tribute to outstanding individuals, teams, managers and leaders who make up the Civil Service profession. Awards are presented to inspirational teams or individuals who deliver complex and ambitious projects across Government." Emma Rourke Director, Health Analysis and Pandemic Insight, Office for National Statistics.

We also came third in the Government Project Delivery Profession Award on the same date. This was an award by HM Government (UK).

"The Covid Infection Survey (CIS) and Schools Infection Survey (SIS) came third in the Government Project Delivery Profession Awards for Excellence in Delivery. The Government Project Delivery Awards are presented to inspirational teams or individuals who deliver complex and ambitious projects across Government." Emma Rourke Director, Health Analysis and Pandemic Insight, Office for National Statistics.

Results Misrepresented

For a full explanation of the flaws with how the Schools Infection Survey has been reported and used, I have included the analysis of Sarah Rasmussen, a mathematician who was a Dr at Cambridge University at the time but is now an Associate Professor at .

18 December 2020

In case anyone wanted to see some examples of prevalence curves that do and do not “mirror what’s happening [in the general population] outside the school gates,” here’s a graph.

Hint: parents—same trend; kids—qualitatively different.

Here’s the weighted fortnightly estimate version.

By contrast, the PHE/LSHTM/ONS School Infection Survey was never meant to be used as an estimate of overall infection prevalence among students or staff.

How do I know that? The ONS report told me so

So if I were an @ONS or @LSHTM collaborator on this Schools Infection Survey project—which clearly represents a tremendous effort—then I’d be a bit annoyed with the take of the @PHE_uk press release, which misinterprets the data in ways the ONS report warns the reader against.

To be clear: this is not dodgy data. The ONS Schools Infections Survey report did an admirable job with presenting the data and describing methodology, limitations, and uncertainties of the study. It’s the @PHE_uk press release that misinterpreted it and presented it badly.

What these data capture are the totally rogue infections that were neither suspected from symptoms nor from contacts with known/suspected cases.So, much of what these high prevalences represent is a devastating failure of testing and contact tracing efforts.

We already knew from school attendance records that at any given time, only about .2% of the student population was staying home after testing positive, compared to the 1.5% and 2.2% prevalences of primary and secondary students then (mean for 3-19 Nov) according to ONS data.

But given the significant fraction of students isolating at any given time, we could have hoped that most of the “missing” cases were isolating at home and simply hadn’t been tested.

According to this new data, that’s wrong: most of the “missing” cases were still at school.

As for staff, if we wanted to use this data to try to estimate the *total* prevalence of school staff infected (ie including those isolating at home due to symptoms or contact), then we’d need a handle on the ratio of rogue at-school infections to home-isolating infections. That’s a difficult ratio to estimate.

But that ratio is almost certainly smaller than the analogous ratio for primary school students (due to high proportion of asymptomatic/cold-like cases for young kids), and is likely at least as small as the ratio for secondary students.

Since we need to multiply by 1+ the *inverse* of that ratio—ie (rogue+home)/rogue = 1+ home/rogue—we then need to multiply by a *larger* correction factor for staff than we would for the corresponding student groups (or at least as large, in the case of secondary students).

So given this lower bound on correction factors, the closeness of student and staff prevalences in the Schools Infection Survey corresponds to an estimate that total staff prevalences are at least as high as student prevalences computed from community ONS *ISP* survey data.

But many cautions should be applied to such an estimate, because there are so many sources of uncertainty, not just in the correction factor but also in the Schools Infection Survey data, which was unweighted, not representatively sampled, and very low sample size.

Still, those limitations are okay, because the ONS very carefully spells them out in the report.

Also, this is a work in progress. They’re working on increasing sample size, getting weighted versions of the data, and generally doing what they can to conduct a useful survey.

Ah, wait! I forgot to tell you the offending punchline from the press report. It was the *context* in which they placed Dr Ladhani’s “mirror” quote.

After reporting “rogue” in-schools prevalence figures, they reported a similar-looking percentage (1.2%) for the *actual*total* prevalence in the general population, followed by Ladhani’s quote that the school survey data mirrors what’s happening in the general population.

Placing his quote in this context makes it look like the in-school prevalence data is somehow comparable to the general population total prevalence, when it isn’t at all, since the in-schools data only represents a fraction of total prevalence of school-related infection.

Moreover, since far more children are asymptomatic or have infections indistinguishable from the colds so common for children, the rogue/circulating-vs-isolating ratio for the general population is much lower than that for children. These data just aren’t comparable at all.

There’s one sentiment of the PHE press release that does ring true, and that’s that community infection levels do impact school infection levels. That impact is crucial.

But the use of the term “mirror” right after the mention of similar-sounding numbers is very misleading.

Lastly, I wrote this thread in response to a number of followers who contacted me with concerns about the schools survey and particularly its press release. I think the schools survey can be a positive, and hopefully future press releases will be written with more care.

Summary

The omission of students isolating was almost entirely omitted from all reporting of the survey. The claim that this proved infections in schools reflected the community continues to be cited by the UK government, UKHSA, RCPCH and many others including the WHO advisory group on children and the JCVI when considering the vaccination of younger age groups.

The survey and its influence on policy raises a number of important issues. Why was the decision made to not include students who were not at school because they knew they had covid? The survey’s methodology and the reporting around each round of results seems almost designed to create a more reassuring narrative around transmission in schools than those working in education experienced considering that recent ONS statistics show education workers are the profession with the second highest rate of infection and over 60,000 children have long covid. A paper published by the UKHSA’s CLoCK team concluded that children faced ongoing and substantial harm from Omicron infections and reinfections.

Despite its flaws, how has the survey come to be seen as flagship study to the point that criticism from experts in relevant fields has been summarily dismissed? When the BMJ’s Inquiry Series paper on schools concluded that the UK’s relaxed approach to transmission in children was an outlier relying on a narrow evidence base there was a vigorous response from members of the RCPCH working with the government, who questioned why the piece had been published without their involvement. A response was rushed out, however it mainly consisted of invalid criticisms, for instance complaining that a specific study hadn’t been included when the BMJ paper contained a detailed table evaluating the study. The criticism gained more media attention than the contents of the BMJ paper.

Ladhani, the UK’s clinical lead on covid in children still claims that over 90% of infections in children occur outside of school and that children are less likely to be the index case in a household, a claim many parents dispute.

Thank you for reading Counter Disinformation Project. This post is public so feel free to share it.

11. ONS ad hoc teachers study, Nov 2020

Counter Disinformation Project

August 28, 2023

Published at the start of November 2020, the ONS ad hoc teachers study was an analysis of a snapshot of data taken from 2 September to 16 October. The paper was provided by government to justify its decisions on schools. On 3 Nov, the Department for Education tweeted a video of Chris Whitty saying:

Read full story