Here's why you can't directly compare coronavirus vaccines

From Novavax, Gamaleya, Oxford/AstraZeneca, Pfizer/BioNTech, Moderna, Valneva, and more, recalling the many coronavirus vaccines has become a Generation Game-style memory quiz.

But which of this ever-growing list of jabs is most effective? Which offers our best shot of ceasing COVID-19 completely?

Unfortunately, even with the phase 3 trial results and emerging real-world data, there’s still no telling which vaccine is most effective for every demographic. And here’s the thing: we may not know for many years.

Why compare coronavirus vaccines is harder than you think

It’s easy, oh so easy, to merely compare the jabs based on their initial efficacy rates, with the Moderna, Pfizer and other mRNA vaccines claiming to be over 90 per cent effective at preventing coronavirus infections.

However, these numbers don’t give the full story. To get that, you have to needle out exactly how they're calculated.

How COVID-19 vaccine trials work

To get an efficacy rate, scientists enlist several thousand volunteers (for instance, 43,448 people signed up to thePfizer/BioNTech phase 3 trial).

Next, these people are secretly randomly assigned into two groups: the first receives the actual vaccine, while the other half, the control group, receives a placebo.

Then scientists wait until a certain number of confirmed coronavirus cases from the total trial population emerged (170 people in the Pfizer trial). It’s then discovered how many of these people had the actual vaccine and how many the placebo.

For instance, the Pfizer trial examined 170 people who became infected with COVID-19. If 85 people were in the control group and 85 were in the vaccine group, the Pfizer jab would have an efficacy of 0 per cent – you’d be just as likely to get coronavirus with or without it.

However, in the actual trial, of these 170 infected people, only eight of them had received the vaccine, with 162 the placebo.

However, in the actual trial, here’s how the 170 infected people were split:

• 162 from the placebo group(these people received the placebo and became infected with COVID-19).
• Eight people from the vaccine group(these people received the real vaccine and become infected).

From these numbers, the scientists could calculate that those in the vaccine group were 95 per cent less likely to be infected than the placebo group. Therefore, the vaccine has an efficacy rate of 95 per cent.

This doesn’t mean that you have a five per cent chance of getting coronavirus if you have the vaccine. Instead, this means you, as an individual, are 95 per cent less likely to become infected if you had the vaccine than if one wasn’t available.

While the mechanics behind them are fairly simple, it’s still difficult to directly compare the results from two trials. This is because most vaccines were tested with a different group of people in a different geographical location.

For instance, with a phase 3 trial efficacy rate of 94.1 per cent, the Moderna vaccine may look much better on paper than the Johnson & Johnson (Janssen) jab, which was only shown to be 72 per cent effective in its trial.

While both vaccines were tested on US populations, the Moderna vaccine was carried out from August to November 2020. Johnson & Johnson? From September to January 2021, a time where infection rates were much higher, meaning participants may be more exposed to the virus.

Interestingly, the Johnson & Johnson vaccine was also trialled in other countries simultaneously, which generated different efficacy rates. Most notably, the jab was shown to be 64 per cent effective in protecting against coronavirus in a South African population at a time when a new B.1.351 variant was emerging.

Currently, there is no large-scale data available for how effective the Moderna vaccine is against the B.1.351 variant.

“This is an important point: we know that some of the vaccine trials in South Africa conducted at the same time as those vaccines trialled in the US have different results because the virus itself different,” explains Dr Connor Bamford, virologist at Queen’s University Belfast.

“The trials can be dependent on the population and density. But also the actual virus, which has changed with new variants. If you get another variant of the virus, then the effectiveness is going to change.

"This all means there is no hard and fast efficacy rate you can rely on and compare with others.”

Read more about coronavirus:

• Does the Oxford/AstraZeneca vaccine cause blood clots?
• Ventilation and viral loads: the key misunderstandings of how coronavirus spreads
• Pandemics past: What we can learn from threats to the human race

Just to make things even more complicated, real-world data has served up some surprising results. For instance, judging by its phase 3 trial efficacy rate of 94 per cent, you’d probably bet the Pfizer vaccine would be more effective than the Oxford/AstraZeneca jab (with an efficacy of 70 per cent) at preventing elderly people from being admitted to hospital with severe COVID-19.

However, this may not be the case. Recent real-world studies have indicated that the Oxford/AstraZeneca jab is slightly moreeffective at preventing over 80s being admitted to hospital(80.4 per cent), compared to the Pfizer vaccine (79.3 per cent).

“As trials are always quite specific they might not always translate into the real world. Plus, in many trials, participants were given two doses, three or four weeks apart. And now we're not doing that it’s not absolutely clear how things will change, either,” says Bamford. "Vaccines are always aiming a moving target."

As they have to protect vastly different demographics of people, vaccines effectively have several moving targets to hit simultaneously.

But while many vaccines understandably trialled their vaccines in vulnerable elderly groups, without more data it’s difficult to know which jab is more effective in younger people.

“It’s going to be a long time before the post-match analysis where we work out exactly what the precise risks and benefits were to every conceivable group of people with every conceivable pre-existing medical condition were,”saysDr David Matthews, virologist from the University of Bristol.

“A certain vaccine may conceivably have one advantage or disadvantage that we just don’t know about yet.”

So which vaccine is best?

While comparing vaccines can get very messy, this question has one clear answer: whatever one you’re offered.

While it's unclear which vaccine is most effective at stopping the spread of COVID-19 in every demographic, all the vaccines have been shown to be incredibly effective in averting serious illness and worse.

In fact, while some participants sadly died from coronavirus during trials of the Johnson & Johnson, Moderna, Oxford/AstraZeneca and Pfizer vaccines, not a single one of those people were from the vaccine group.