Statistics lie

Statistics can be misleading. When drug companies market their products, the actual benefit message is not always as good as the glowing endorsement. While drug companies endeavor to do good (find new medicines/cures/therapies), they are tasked with selling their products. Selling involves telling one’s story in the best light possible. It’s not dishonest to do so. But it does mean that doctors need to be thoughtful, wary users of information.

Case in point: Prevnar, a vaccine to protect against Streptococcus pneumoniae (aka pneumococcus, a bacteria that causes ear infections and meningitis in children and pneumonia in adults). The vaccine has been adopted as a standard of care in the immunization schedule of children. It works. It has cut rates of disease. Vaccines (this and others) are one of the great medical innovations of our society.

Because adults also suffer from pneumococcal infections (pneumonia), Pfizer, the maker of Prevnar asked the very relevant question, “If we vaccinate adults, will we cut down the risk of pneumonia.”

At face value, the answer is a resounding “Yes.” The study published in the New England Journal of Medicine (March 19 2015) showed a 45.6% reduction in the risk of sero-type specific pneumonia. Astounding right?

But wait. I hear you asking “serotype-specific pneumonia…. What’s that?”

It turns out this study is really telling us that if you vaccinate against the specific strains of Strep pnuemo in this particular vaccine, you’ll cut the rate of pneumonia caused by this specific vaccine by 46%.

Ok. Still great, right? Maybe not so much. Let’s take a look at the actual numbers:

This trial called “CAPiTA” enrolled 85,000 patients. That’s a huge trial. Huge trials are more likely to be accurate, and they’re also likely to find really small benefits. Here are the numbers. There were about 42,000 patients in each arm (half the patients got vaccine, half got placebo). Patients were then followed for about four years. In the vaccine arm, there were 49 cases of “serotype-specific” pneumonia. In the placebo arm there were 90 cases. So yes, dropping from 90 cases to 49 cases is a reduction of 46%, but maybe percentage terms aren’t the right way to express the benefit. In numeric terms, Prevnar prevented 41 cases of pneumonia. For emphasis, by vaccinating 42,000 patients, 41 cases of pneumonia were prevented over a four-year period. Stated another way, the medical community needs to vaccinate 10,000 people to prevent one case of pneumonia per year.

Here are the numbers in table form for easier visualization:

vaccine placebo
number 42,240 42,256
# cases of “vaccine-specific” pneumonia 49 90
Total # cases, all pneumonia types 747 787
death 3,006 3,005
death from pneumonia 6 7

Looking at the numbers, there’s something more to notice. Remember that this vaccine only prevents serotype-specific cases of pneumonia. Does the vaccine cut down total cases of pneumonia? The answer is no. While the number is numerically higher for total vaccine cases in the placebo arm, the difference is noise. Statistical analysis says the total number of pneumonia cases are the same. This is because a lot of different bacteria can cause pneumonia. Strep pneumo is just one of these (about 13% of total cases of pneumonia). Cutting down a few cases of Strep Pneumo doesn’t make a difference.

How about deaths? Pneumonia can be fatal. Did Prevnar cut the number of deaths? As you can see from the table, not at all. Deaths were identical in both groups in the study.

There’s even one more confounder. . . . Who carries around Strep pneumo bacteria to infect adults? It’s little kids. The CAPiTA trial was done in the Netherlands. There is not widespread use of Prevnar in kids in the Netherlands. In contrast, Prevnar vaccination of kids is widespread here in the US. Indeed, after Prevnar was introduced in the US, it was observed that serotype-specific Strep pneumo infections in adults fell by about 70% (see here and here and here) once kids were uniformly vaccinated. This is called “herd” immunity. Immunize the herd (in this case kids, the carriers) and the unvaccinated adults also don’t get sick from lack of being exposed to sick kids.

Because Prevnar use is limited in the Netherlands, it’s likely that there is more Strep pneumo pneumonia in adults in the Netherlands. We can’t know for certain but it seems reasonable that pneumonia here in the US caused by Strep pneumo is even more rare than the 13% rate in the Netherlands. This means whatever tiny benefit was seen in the Netherlands CAPiTA study is probably even smaller in the US (if it exists at all).

In summary, the headline “46% reduction in serotype-specific pneumonia” upon further evaluation to me reads “is there really any benefit at all?”

Now don’t get me wrong. I’m a huge proponent of vaccines. I strongly believe this does no harm, and perhaps there is some miniscule benefit. In a perfect world, every intervention with any benefit, no matter how small, would be used. However, this is not a perfect world, and interventions have costs. Sometimes the costs are in the form of side effects. Sometimes the costs are monetary.

In this case the costs are monetary. Vaccines cost money, and this one is expensive. Prevnar costs about $150 per dose. This means that from the Netherlands study, immunizing 10,000 people costs $1.5 million dollars to prevent one case of pneumonia per year (and again, the benefit might be even smaller in the US). That one case of pneumonia, by the way, could probably be treated with generic antibiotics for $10. I don’t think as a society we should consider that good bang for the buck.

Remember, however that while drug companies do great research, their marketing teams have only one mission: sell, sell, sell. Advertising to doctors that Prevnar vaccination of adults cuts pneumonia risk by 45%…. that sounds impressive. Does it sell vaccine doses? You bet. On their 2015 first quarter earnings call, Pfizer stated they sold $300m worth of adult Prevnar vaccine. Good job, Pfizer!


Sources:  New England Journal of Medicine “Polysaccharide Conjugate Vaccine against Pneumococcal Pneumonia in Adults”;   CDC “Manual for the Surveillance of Vaccine Preventable Diseases”;   U.S. National Library of Medicine “Changing Epidemiology of Invasive Pneumococcal Disease Among Older Adults in the Era of Pediatric Pneumococcal Conjugate Vaccine“;   New England Journal of Medicine “Decline in Invasive Pneumococcal Disease after the Introduction of Protein–Polysacharide Conjugate Vaccine”


Print Friendly

Leave a Reply

Your email address will not be published. Required fields are marked *