Are all COVID-19 vaccines 100% effective in preventing severe illness and death? No, each vaccine is different

On March 23, 2021, AstraZeneca was exposed to a scandal. The U.S. Data Safety Monitoring Board (DSMB) believed that the AZD1222 clinical trial data released by AstraZeneca the day before was "outdated and potentially misleading", saying that AstraZeneca selectively displayed the most beautiful data, "such a decision would undermine the public's trust in the scientific process." [1]

The design of clinical trials and the omissions in the data actually collected will affect the correct interpretation of the data. Media interpretations are not necessarily completely correct. Everyone should clearly understand the vaccine trial data and its meaning, weigh the pros and cons, and make the best choice for themselves and the people they care about.

Compilation | Kestrel

On February 27, 2021, the U.S. FDA approved the emergency use of the COVID-19 vaccine (Ad26.COV2.S) developed by Johnson & Johnson for people over 18 years old; on March 11, the European Medicines Agency (EMA) also conditionally approved the Johnson & Johnson vaccine for marketing. Because this new vaccine only requires one injection and the storage and transportation conditions are not so high, it has attracted the attention of public health departments in various countries.

However, the effectiveness of the Johnson & Johnson vaccine in preventing COVID-19 infection is still slightly lower than that of the mRNA vaccine. Clinical trials in the United States have shown that its effectiveness is about 72%, while the protection rates of the vaccines developed by Moderna and Pfizer-BioNTech are 94% and 95%, respectively. Despite this, many experts point out that the evaluation of the quality of the vaccine cannot only focus on these numbers, but should also note that the Johnson & Johnson vaccine is as good as the other two in preventing the most serious consequences of infection. When reporting on the Johnson & Johnson vaccine, the New York Times interviewed the vaccine coordinator in Virginia, who described himself as "chicken blood" and "100% prevention of hospitalization and death - that's what I want to hear."

The news that "all new coronavirus vaccines are equivalent" also gave public health officials a shot in the arm. They have been working hard to promote vaccination: if people understand that the vaccine they can get the fastest is the best vaccine, the nightmare will be over sooner. News and commentary media Vox urged readers to pay attention to this: there were zero hospitalizations and deaths in the clinical trials of these vaccines. CNN medical analyst Dr. Leana Wen said that these vaccines are indeed 100% effective in preventing hospitalization and death. Several former members of Biden's COVID-19 Advisory Committee wrote in a USA Today column: "The claim of varying effectiveness ignores the most important point, that all vaccines tested are 100% effective in preventing hospitalization and death."

Vaccine equivalence is also popular on social media. A USA Today column by a former member of the Biden team linked to a data table on Twitter (Figure 1) made by infectious disease doctor Monica Gandhi. The table shows the results of various trials of 6 different vaccines. The "hospitalization/death prevention" column is highlighted in yellow, and each box below it says 100%, which looks like a straight line.

Figure 1
https://twitter.com/MonicaGandhi9/status/1356818441064206337

Ashish Jha, dean of the Brown University School of Public Health, also tweeted a similar table - an array of zeros consisting of hospitalizations and deaths, which also expresses the same meaning. Eric Topol, a well-known doctor and researcher, also posted his own clinical trial statistics, highlighting a column of "100%". These tweets were later forwarded more than 20,000 times.

These data are indeed encouraging. Based on this, we can assume that the vaccine has a very high protection rate against the worst-case scenario. But how high is "very high"? Obviously, hospitalizations and deaths among the vaccinated population will not be eliminated.

Experts are certainly aware of this. Gandhi has been updating the data in the table. As more clinical research data is added, the protection rate of Moderna's vaccine has become 97%. Jha also said, "Nothing is 100%, but these vaccines are definitely close." Topol said that the data in his table cannot be used as a sufficient basis for measuring the effectiveness of the vaccine, although they all show that the situation is very optimistic. Despite this, the "optimism" expressed in the data they released earlier is still widely circulated and repeatedly misinterpreted.

To understand why these data are unreliable, it’s important to look at how the vaccine development process begins.

Last April, the World Health Organization set a target efficacy for the vaccine—50%—with optional statistical efficacy options: the efficacy of the vaccine could refer to the efficacy of reducing the risk of symptomatic infection, the efficacy of reducing the risk of severe infection, or the efficacy of reducing the risk of transmission. In June, the FDA issued a similar guidance, and other regulatory agencies followed suit. Among the options, the most feasible is the protection rate against symptomatic infection, because there are many cases that meet this standard and it is easy to determine through large-scale trials. If asymptomatic infection is also included, there should be more cases that meet the standard, but there was not enough manpower and material resources to support this plan in the early stages of the epidemic. In the end, the vaccine trial was designed according to this standard: to make the "efficacy" of preventing symptomatic infection higher than 50%.

At that time, the trial may also choose the number of severe cases, hospitalizations or deaths to calculate the effectiveness of the vaccine, but after all, such cases are far less numerous than those that simply show symptoms. Collecting a certain number of cases will inevitably slow down the progress of the trial. For example, in the United States, about 1 in 200 infected people dies. So to obtain the data needed to verify that the vaccine is more than 50% effective, it will take more time and recruit more subjects. It is true that some researchers have used severe illness as a secondary outcome for statistical analysis, but these trial protocols are designed so that there is no way to draw conclusions about secondary outcomes. Even fewer trials use hospitalization and death as secondary outcomes.

Therefore, these data are not sufficient to support the conclusion that "vaccines can 100% prevent serious infections." The exaggerated data table (Figure 1) circulating on Twitter involves 6 vaccines, and only the vaccines from Oxford-AstraZeneca and Johnson & Johnson include data on hospitalized cases. Novavax's vaccine data also includes hospitalized case data, but it is incomplete.

It is conceivable that when ordinary people read the summaries of these clinical trial reports on social media, they may get the impression that no one needs to be hospitalized after vaccination. In fact, these clinical trial data are published in different journals and regulated by different agencies. In addition, some trials did not even include the step of "statistical effectiveness in preventing hospitalization" when designing the program, and only released a small amount of data from the early stages of the trial. On the other hand, hospitalization is not synonymous with severe infection. Some people may choose to stay at home even if their blood oxygen level drops sharply, while others may choose to be hospitalized for treatment just in case even if they only have mild symptoms.

In the two clinical trials that explicitly used the number of hospitalized cases as a measure of vaccine efficacy, one person in AstraZeneca's control group developed severe disease, but eight were hospitalized; 34 people in Johnson & Johnson's placebo group developed severe disease, but only five were hospitalized. It is true that there were zero hospitalizations in the experimental group, but these numbers are so small that such a high efficacy conclusion is unreliable. Diana Zuckerman of the National Center for Health Research pointed out that Johnson & Johnson's claim that no one in the experimental group was hospitalized was misleading to the public unless it was also stated that only five people in the placebo group were hospitalized. In addition, clinical trial results cannot be directly extrapolated to the general population because the proportion of subjects over 60 in some trials is too low.

If you look at the number of severe cases, you'll understand how fragile those numbers are. In Pfizer's trial, there was one severe case in the experimental group and three in the placebo group. If you remove one of those cases, the efficacy value would become 66% or 100%. In the Novavax and Oxford-AstraZeneca trials, there were zero severe cases in the experimental group and one in the control group. The change in efficacy is even more exaggerated if one person is more or less. The problem is even greater when considering the number of deaths. Only Moderna and Johnson & Johnson's vaccine trials reported deaths in the control group when analyzing efficacy.

Table 1. Comparison of the number of people in the trial results of the four vaccines. [2-5]

These are only early trial data, because the conditions of those who joined the trial late have not been reported in time. In December, the FDA pointed out that in Moderna's trial, one subject was hospitalized for severe illness two months after the second shot, and the data of this person's group is still being analyzed and has not been reported.

We can learn more from the public vaccination programs that are underway. In a recent report[6], researchers compared data from 600,000 vaccinated people in Israel with 600,000 unvaccinated people (the two groups were matched on demographic indicators such as age), and calculated that the Pfizer vaccine's overall effectiveness in preventing hospitalization was 87%. Jha (dean of the Brown University School of Health) tweeted that the Pfizer vaccine performed "fabulous" in real-world situations. Two reports from Public Health England (a government organization) estimated that hospitalizations were reduced by 50% and 43% respectively in people of the same age who received only one shot of the Pfizer vaccine.[7][8] That's great - the vaccine really works - but it's not right to claim that the vaccine is perfect.

Although it is inspiring to say that all vaccines are 100% equally effective, this statement is problematic. It is true that all vaccines approved by the FDA are great, and people should not think that there are better vaccines and not get vaccinated. However, the fact is that there are differences between vaccines. For example, some vaccines have higher protection rates, some have fewer side effects, some are more convenient to administer, and some are developed using traditional technology. As for the effectiveness of different vaccines in preventing hospitalization and death, there are also differences.

Knowing this, how do we make a choice? Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases and chief medical adviser to the president, said in an interview with The New York Times: "We now have three highly effective vaccines." - Yes, that's all he said. In other words, getting any vaccine will be of great benefit to you. If the people around you are also vaccinated, your risk of infection will be lower. No matter which one you get, you may have to get a booster shot (regardless of the brand) soon to deal with immune protection that may weaken over time or new mutant strains. Getting a shot of a local existing vaccine can also prevent you from feeling like you're running naked when the infection rate around you rises sharply.

There’s another layer to consider besides efficacy. While the Pfizer and Moderna vaccines have an advantage in preventing symptomatic infection, the Johnson & Johnson vaccine has advantages in other ways — it doesn’t have stringent refrigeration requirements and is easy to distribute; the company is offering it at cost worldwide; and it only requires one shot.

For individuals, Johnson & Johnson's vaccine also has advantages. On the one hand, it is more convenient to get just one shot. It has a lower rate of side effects than Moderna's vaccine. Looking closely at the results of these trials, you will find that about 2% of Johnson & Johnson vaccinations have recorded side effects, such as fatigue, myalgia, fever, and even serious enough to affect daily activities. For people who have received the second shot of the Moderna vaccine, the incidence of side effects is higher than 15%. People who are still waiting and watching may be inclined to get just one shot after learning about this difference; people who are skeptical about the new technology of mRNA vaccines may trust Johnson & Johnson's technical approach (using non-replicable viral vectors) more, after all, this method has been used to develop Ebola vaccines and has been approved in Europe.

Given these issues, it is dangerous to convince the public that every vaccine is equally effective and that you never have to worry about serious infection after getting vaccinated, no matter how well-intentioned. Vaccination is a public health imperative and for the collective good, but it is also a personal choice. After all, people want to protect themselves and their loved ones, but they also care about the side effects of vaccination, rather than just being satisfied with not being hospitalized or dying.

Admittedly, drawing public attention to these issues may also cause widespread anxiety. Gandhi, the producer of Figure 1 above, admitted that it is important to focus on severe cases; to get out of the shadow of this plague, a detailed and concerted academic dialogue is needed. Topol argued that he had repeatedly emphasized that the "effectiveness" of the vaccine was for symptomatic infection, but it was always taken out of context. Jha insisted on his original point of view, believing that hospitalization and death were rare in the experimental group of clinical trials after all, and that entanglement with these minor differences was tantamount to nitpicking.

Considering these nitpicking details is not nitpicking. It’s sometimes critical to choose the right vaccine for different people and different situations. These differences should not be swept under the rug, especially now that more contagious variants have emerged and the demand for vaccines in the United States will soon be outstripping supply. Now that states are rescinding mask mandates or easing restrictions, some may think a simple, crisp message is exactly what’s needed, no matter how exaggerated it may be. But what if that simple, crisp statement is wrong? Imagine what kind of psychological expectations high-risk people will have when they go to the movies if everyone assumes that they’re safe as long as they’re vaccinated? Will employers still feel the need to invest in adequate hygiene in the workplace? What will they think and do if they find out that even with the vaccine, they’re only 85% sure they won’t get a serious infection?

Exaggerated propaganda will ultimately harm the collective interest. Those who believe that everything will be fine after getting vaccinated may lose their trust in experts if they find that this is not the case. In the first place, people’s trust in the information about the new crown vaccine has become a problem. Anti-vaccine elements may also use this to attack experts.

In late February, Jha tweeted that it was wrong to think that the public could not distinguish nuances, which was paternalistic. I couldn't agree more. Treating the public like adults is a basic principle: we don't need to exaggerate. It's usually not easy to discuss how to weigh the pros and cons of drugs and vaccines, but we've been working on it, and this is also the case with the new coronavirus vaccine.

Compiled from:

https://www.theatlantic.com/author/hilda-bastian/

References

[1] https://www.statnews.com/2021/03/23/astrazeneca-may-have-used-outdated-information-in-announcing-covid19-vaccine-results/

[2] FDA Briefing Document. Pfizer-BioNTech COVID-19 Vaccine. Vaccines and Related Biological Products Advisory Committee Meeting. December 10, 2020 https://www.fda.gov/media/144245/download (Accessed on December 09, 2020).

[3] Voysey, M. et al. (2021). Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomized controlled trials in Brazil, South Africa, and the UK. Lancet, 397(10269), 99-111. https://doi.org/10.1016/S0140-6736(20)32661-1

[4] FDA Briefing Document. Janssen Ad26.COV2.S Vaccine for the Prevention of COVID-19. Vaccines and Related Biological Products Advisory Committee Meeting, February 26, 2021 https://www.fda.gov/media/146217/download (Accessed on February 24, 2021).

[5] https://ir.novavax.com/news-releases/news-release-details/novavax-covid-19-vaccine-demonstrates-893-efficacy-uk-phase-3

[6] https://www.nejm.org/doi/full/10.1056/NEJMoa2101765

[7] https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/968977/COVID-19_vaccine_effectiveness_surveillance_report_February_2021.pdf

[8] https://khub.net/documents/135939561/430986542/Early+effectiveness+of+COVID+vaccines.pdf/ffd7161c-b255-8e88-c2dc-88979fc2cc1b?t=1614617945615