Search Peter’s website:

The CDC Tells More of the Truth than Usual
about Flu Vaccine Effectiveness

The title says it all: On January 11, 2013, the U.S. Centers for Disease Control and Prevention (CDC) held a routine flu update press briefing that was anything but routine. Even as the briefers continued to urge Americans to get vaccinated against the flu, they were more candid about the low effectiveness of the vaccine they were recommending than I can ever remember them being.

In this column I’m going to annotate that unprecedented press briefing. Seasoned influenza aficionados (“flubies” or “flu geeks”) are welcome to skip down to “The Annotated Press Briefing.” Normal people may want to read the following orientation first.

Flu Vaccine Limitations

When U.S. public health agencies acknowledge the limitations of the influenza vaccine – which they do much less often than they should – they tend to use a phrase like “less than perfect.” “Less than adequate” would capture the situation better.

Especially compared to the dramatically successful childhood vaccines that too many of us take for granted, the flu vaccine is ineffective in several ways:

• The vaccine works only about 60% of the time under optimal conditions – that is, when the vaccinee is a healthy adult 65 and under and the vaccine is a good match to the flu strains currently circulating.
• The vaccine works a lot less than 60% of the time on the people who need it most, the elderly and the sick. Some studies even suggest that it may do no good whatever in preventing influenza among the frail elderly, who are likeliest to die if they catch it. Other studies show reduced hospitalizations and deaths – that is, milder cases – when members of this vulnerable population are vaccinated but still catch influenza.
• The vaccine is manufactured using a decades-old technology that’s slow, clunky, and vulnerable to all sorts of delays. (It relies on millions of fertilized chicken eggs.) We’re out of luck if we suddenly need more vaccine than we geared up to manufacture more than six months previously.
• The vaccine targets specific flu strains, the ones the experts predicted would be most common, months earlier when that clunky manufacturing process needed to get started. But flu viruses mutate endlessly. That means we must annually urge people to get vaccinated again against the newest strains. And if new variants start to circulate during the current flu season, we can end up with a vaccine that’s much less than 60% effective. Or we can be stuck for months with no vaccine at all against a never-before-seen pandemic strain.
• The immunity conferred by the flu vaccine (when it works) wanes. So even when the circulating flu strains are the same ones that circulated the previous year, most experts think people should still get vaccinated again.

Despite these and other serious limitations, every year the flu vaccine saves thousands of lives, hundreds of millions of days of misery, and billions of dollars in productivity. That’s because influenza is both a very common disease and a very often serious one, even though many people erroneously dismiss it as little worse than a bad cold.

The flu vaccine could accomplish a lot more – even this less-than-adequate vaccine – if we vaccinated more people. The U.S. does a better flu vaccination job than any other country (with Canada a close second), managing in recent years to vaccinate over 40% of the population. Most other developed countries do a lot worse; some barely try. And of course health departments in the less developed countries have more severe infectious diseases to worry about.

The flu vaccine is by far the most effective weapon against influenza. Getting vaccinated against flu helps protect the vaccinee; it helps protect others the vaccinee comes into contact with; and it helps build the market for flu vaccine, thereby increasing the supply in future years and manufacturing capacity in bad years (or pandemic years).

Other weapons of less but still significant value against influenza transmission to yourself or others include: staying home when you’re sick; trying not to touch your face (eyes/nose/mouth); taking Tamiflu as quickly as possible if you have flu symptoms; avoiding crowds and keeping your distance from people who are showing signs of respiratory disease, especially during flu season; covering coughs and sneezes with your sleeve instead of your fist; and washing or sanitizing your hands whenever you can. I have tried to list these in descending order of effectiveness, but it’s something of a guess; effectiveness research for most of them is scanty, inconsistent, and inconclusive.

The flu vaccine has an excellent safety profile – not perfect, but excellent. It’s not a dangerous vaccine, just a comparatively ineffective one.

Long History of Overstating Effectiveness

Until very recently, the CDC has tended to overstate the effectiveness of flu vaccination, presumably in order to persuade more people to get vaccinated. My wife and colleague Jody Lanard and I have argued against this long tradition of flu vaccine hype in many articles – here, here, and here, for example – and even more often in emails to public health officials, journalists, and each other.

Ethical principles aside, Jody and I believe CDC flu vaccine hype does actual harm. Though our evidence is weak, we have hypothesized four main downsides:

• CDC hype threatens the sustainability of the flu vaccination effort. Whenever people find out that they have been misled – especially if it’s intentional – they tend to overreact. Even a valuable public health tool can end up widely rejected when people realize that it has been oversold.
• CDC hype threatens the credibility of vaccination more generally, and even of public health itself. The strongest argument in the arsenal of vaccine opponents is the hype perpetrated all too often by vaccine proponents.
• CDC hype undermines the case for a better flu vaccine, especially in the minds of potential investors. If the current vaccine is as good as the CDC has made it sound, why prioritize funding the search for a better one?
• CDC hype sets the stage for a cascade of far worse hype on the part of state and local health departments, individual doctors, and journalists – all of whom take their cue from the CDC, and many of whom oversimplify the CDC’s overstatements without realizing they are overstatements, and thereby unintentionally drive them even farther from the truth.

It is in this context that I want to offer enthusiastic praise – and some criticism – of the CDC’s January 11, 2013 flu press briefing. Entitled “CDC Update: Flu Season and Vaccine Effectiveness,” it was presented by CDC Director Dr. Tom Frieden and CDC flu epidemiologist Dr. Joe Bresee. It is the most candid CDC seasonal flu briefing I have ever seen with regard to flu vaccine effectiveness.

What follows is the CDC transcript, interspersed with my commentary. I have left out parts of the transcript unrelated to vaccine effectiveness, and I have re-paragraphed the transcript in order to put my comments immediately after the relevant quotes. The press briefing text is in sans-serif type on a blue background. My annotations are indented in serif type.

The Annotated Press Briefing

This is already a stunning beginning: the mere fact that an MMWR article on flu vaccine effectiveness is a major focus of a CDC press briefing. In the past, the CDC didn’t normally publish early-season effectiveness studies of the flu vaccine. And it didn’t normally emphasize publicly the effectiveness data it had – especially not when it was still trying to get people vaccinated and the data revealed lower effectiveness than most prospective vaccinees might otherwise assume.

For years, in fact, the CDC claimed 70–90% effectiveness for the flu vaccine in healthy adults 65 and under, and encouraged state and local agencies to do likewise, in the face of accumulating data, well known to flu experts, that the real number was significantly lower. Now the CDC has its own study showing 62% “overall” effectiveness, and instead of burying the number it’s showcasing it. This is a remarkable change.

Ideally, the CDC would say it’s a change. But expecting a government agency (or anyone, really) to acknowledge prior dishonesty is expecting a lot. Abandoning prior dishonesty is praiseworthy enough.

Whether prior dishonesty needs to be acknowledged is a contentious question. Some think doing so is a precondition for forgiveness and a bulwark against recidivism. Others think it is feasible to transition quietly from falsehood to truth without ever paying the price of your prior dishonesty – diminished reputation and damaged credibility. Jody and I are in the former camp. Most organizations choose the latter option if they can, and the CDC is no exception here.

I don’t think giving reporters information as soon as it gets the information is actually what the CDC “always” does. During the swine flu pandemic, for example, the CDC had information about which age groups were most at risk for severe outcomes. It bent over backwards not to give reporters that information, because the information conflicted with the CDC’s earlier decisions about vaccination prioritization, which it had decided not to reverse. (See “Why did the CDC misrepresent its swine flu mortality data – innumeracy, dishonesty, or what?”)

I do think giving reporters information as soon as it gets the information is genuinely what the CDC is doing in this January 11 press briefing.

Frieden’s first sentence here is a beautifully matter-of-fact way of suggesting that nothing reported in today’s briefing is especially unusual – not the high incidence of flu and not the low effectiveness of the vaccine.

In the December 3, 2012 press briefing, by contrast, the CDC culpably claimed that the early start to the season might signal it would be a bad season. Tom Frieden began that briefing by saying: “This is the earliest regular flu season we’ve had in nearly a decade, since the 2003–2004 flu season. That was an early and severe flu year, and while flu is always unpredictable, the early nature of the cases as well as the specific strains we’re seeing suggest that this could be a bad flu year.”

I’m not aware of any data showing a relationship between when the flu starts to spread widely and how deadly it turns out to be. Claiming such a relationship on December 3 seemed to me like just a convenient rationale for urging people to get vaccinated. In late seasons officials tell us it’s not too late so get vaccinated. In early seasons they tell us it looks like a bad year so get vaccinated.

But today Frieden is saying simply it’s a flu season pretty much like many others, except it started a month or so earlier than usual.

Frieden’s summary of the “state of the season” – only the beginning of which I have retained in this excerpt – strikes me as admirably free of hype. If anything it errs in the other direction. There’s no emphasis at all, no effort to explain which facts the CDC thinks are more interesting or worth reporting than other facts. “Here’s everything that’s happening this week. It has no real predictive value regarding what might happen next week.” But I don’t want to start complaining about CDC under-interpretation of flu data. The absence of hyped over-interpretation is wonderful.

In the past, studies of seasonal flu efficacy were rarely highlighted publicly by the CDC. (They weren’t secret; you could find them if you looked for them.) As a rule, mid-season estimates and even end-of-season estimates were publicly discussed only when vaccine mismatches with circulating strains occurred. Except when there was a mismatch that needed explaining, all we’ve usually seen is that inflated 70–90% figure, often without even qualifying it: “…in healthy adults 65 and under in years with a good match.” Frieden is introducing his unprecedented candor about flu vaccine effectiveness with the pretense that it is business as usual.

Most estimates of vaccine effectiveness distinguish healthy children and adults (on whom the vaccine works best) from the sick and elderly (on whom it works less well). But this preliminary CDC estimate includes everybody. Many flu experts have noted that the end-of-season overall number is likely to be lower, for a variety of technical reasons. If Frieden thinks so too, he would be wise to say so now – rather than risk surprising people later with a “worse than we thought” assessment. Similarly, if he thinks the small size of the preliminary sample means the end-of-season number could easily be lower or higher, he should say that. The risk communication term for this advice is “anticipatory guidance” – that is, forewarn people now so there are no surprises later.

Still, the big news is the 62%. I’d have liked to see Frieden immediately explain two crucial facts about the 62% figure:

Frieden does get to the first point later. He never makes the second point.

Frieden doesn’t make a big deal about this definition of what it means for a flu vaccine to work. But it’s actually a pretty big deal. If he explained it, a lot of people (even a lot of doctors) would find it surprising.

This CDC measure of flu vaccine effectiveness looked exclusively at people who sought medical attention for an acute respiratory infection (ARI). Their infections were tested to see if they had the flu or some other influenza-like illness. The patients were asked if they had gotten a flu vaccination this season (at one of the five study sites, patient records were checked instead). Then the percentage of flu-positive ARI patients who were vaccinated was compared with the percentage of flu-negative ARI patients who were vaccinated, and an odds ratio was calculated.

In the early part of this year’s flu season, among the U.S. doctors whose patients were included in the CDC study, people with medically attended ARIs that turned out to be flu were 62% less likely to have been vaccinated than people with medically attended ARIs that turned out not to be flu.

What the CDC means here by flu vaccine effectiveness is effectiveness in keeping people from ending up in a doctor’s office or hospital emergency room with influenza.

This conflates two possible effects of the flu vaccine: preventing you from getting influenza at all, and making your case of influenza (if you get it) milder so you’re less likely to see a doctor about it. The CDC and other public health agencies have not usually kept this distinction clear in their public communications. They talk about “preventing the flu” when sometimes – like here – what they mean is preventing a case of the flu that requires a doctor’s attention.

This measure of flu vaccine effectiveness leaves out three cohorts that some people might argue deserve attention:

All or nearly all flu vaccine effectiveness studies leave out the first two cohorts, people with non-flu ILIs and people with asymptomatic flu cases. But leaving out the third cohort, people with mild cases who never saw a doctor about them, is far from universal; it depends on the study.

Many reporters missed this methodological point. And many who probably didn’t miss it chose to ignore it. The American Pharmacists Association website, for example, reported that “the 2012–13 influenza vaccine has been shown to be 62% effective at preventing the virus.”

One of the most prominent of the researchers contributing to the MMWR report that the press briefing was focusing on is Edward Belongia of the Marshfield Clinic. In the clinic’s own January 11 news release about the study, Belongia is quoted with the same over-simplification:

“The take home message is that the flu vaccine is moderately effective this year, and people who are vaccinated have about a 60 percent lower risk of getting the flu compared to someone who is not vaccinated…,” said Dr. Edward Belongia, an epidemiologist and a lead researcher on the report.

Similar Belongia quotations appeared in U.S. News & World Report and on Fox News stations, among others.

Other reporters were careful to capture the fact that “effectiveness” in the CDC study meant reduced likelihood of having to go to the doctor with influenza, not reduced likelihood of catching the flu.

And some reporters, like Ryan Jaslow at CBS News, tried to have it both ways:

The CDC also released a new study Jan. 11 … that found this year’s flu vaccine is about 62 percent effective. That means a person who takes the shot is 62 percent less likely to have to go to a doctor to get treated for flu….

The vaccine has been about 60 to 70 percent effective at preventing the flu in recent years.

Frieden explained the study’s methodology clearly, but he didn’t emphasize the distinction between preventing flu and preventing medically attended flu, and many reporters missed it. Some experts, like Belongia, didn’t bother to explain it.

The first sentence of this passage is worth parsing – that “we have known for a long time” that the vaccine is “far from perfect.”

“We have known for a long time” is Frieden’s first chance to say “…but we haven’t been as candid about it as we should have been, until now, so a lot of people (including reporters in this room and on this call) are surprised at how low the number is, even though we at CDC are not surprised.” He says nothing like that, here or anywhere in the press briefing. That is my principal criticism of Frieden’s overall good job in this briefing. He is candid about flu vaccine effectiveness, but he is not candid about the prior absence of candor, the history of hype.

“Far from perfect” is a far from perfect description of 62% effectiveness. “Effective in just over half the people vaccinated” would be more accurate – and would sound a lot less effective, of course. Still, I’m highly critical of public health officials who say “far from perfect” without giving a number. As long as Frieden is up-front about his 62% finding, I’m okay with him describing it as far from perfect.

As for Frieden’s “best tool we have” message, it’s accurate and super-important. But I think he jumps to it too quickly. It would work better if he dwelled more first on how disappointing that 62% figure must be to people who thought vaccination was virtually a sure thing … as it is for many other vaccines.

The middle of this passage is wonderfully honest about the fact that the flu vaccine works less well on those who are sick or elderly than it does on healthy young people. Frieden could easily have skipped saying this, since the study he’s discussing doesn’t analyze these differences. To his credit, he says it anyway, signaling pretty clearly that 62% is an average and the vaccine is less effective than that for the sick and elderly.

What about “our basic approach at CDC is to get information as quickly as we can and then share it openly and transparently”? This is the second time Frieden has made this pat-yourself-on-the-back claim. Would that it were always as (close to) true as it is at this press briefing!

“What you can do to protect yourself against the flu” is very conventional CDC messaging, as it should be. Especially at a press briefing that reveals that the flu vaccine isn’t that great, it is important to stress that it’s still “by far the best tool we have to prevent influenza.” And I really like “better late than never.” Unlike “it’s not too late,” “better late than never” almost acknowledges that earlier would have been preferable.

Frieden’s list of other flu-prevention and flu-mitigation measures is unobjectionable. I like the fact that hand-washing is deemphasized. Like vaccination, hand-washing is all too often hyped as a way to prevent catching and spreading the flu – especially when there’s no vaccine to hype (as was the case when H3N2v started spreading to humans at agricultural fairs in the summer of 2012). Hand-washing is demonstrably effective against many contagious diseases, but there’s surprisingly little evidence that flu is one of them. The evidence is strong that hand-washing gets rid of flu virus on your hands when you have a chance to wash them – but that’s not the same thing as reducing transmission. Most experts think flu spreads chiefly via droplets; washing your hands is minimally helpful after someone has coughed in your face. Nor is it very helpful in places like buses, offices, shops, and classrooms, where people have no choice but to re-contaminate their hands repeatedly.

I also like Frieden’s advice that people should get to a doctor “if you’re very ill or if you have an underlying condition.” Most flu messaging talks about underlying conditions in terms of their impact on your flu case: Your underlying condition makes your flu likely to be more severe because you’re debilitated. What’s often neglected is the other side of the coin: Catching the flu, even a mild case of the flu, can destabilize your underlying condition – so people with underlying conditions should seek medical care if they get a mild case of flu, not to treat the flu so much as to make sure the underlying condition doesn’t go out of whack. Frieden doesn’t quite say this, but he comes closer than usual.

The first question asked for more detail on which parts of the U.S. were currently flu hot spots and which were comparatively flu-free. The second question dealt with spot vaccine shortages. The third question asked why Frieden had said (in a passage I deleted) that the holidays had made it harder to interpret flu trends. I have nothing to say about any of these answers.

Serrie is right to want to get this clarified, since the commonsense interpretation of the effectiveness of a vaccine is keeping you from catching the disease, not just keeping you from catching a bad enough case of it to send you running to your doctor. Frieden gives the right answer, again. I assume plenty of people other than Serrie were having trouble with it. It would have helped for Frieden to “go meta” on his answer. The optimal way to correct a misimpression takes four steps:

Here is how Serrie ended up covering the answer to his question: “The CDC estimates that this year’s seasonal flu vaccine is 62% effective, meaning that if you got the shot you are 62% less likely to have to go to the doctor with flu-like symptoms.” Serrie understood the distinction between preventing you from catching the flu and preventing you from getting a bad enough case of the flu to send you to the doctor. But “flu-like symptoms” is wrong. He missed the point that the flu vaccine has no value whatever against ILIs other than influenza.

This is very good. We wish we had a vaccine that lasted a long time, that covered all flu viruses and didn’t need to be reformulated, that worked 90% of the time or better, that worked that well even on old people and sick people….

The “we wish…” formulation is an excellent way to show you share your audience’s concerns about what’s gone wrong or what you can’t make right. It was explicitly recommended in a May 2011 presentation on “Overcoming the Complex Challenge of Influenza Vaccination Messaging” by Glen Nowak, then the CDC’s top respiratory disease communication professional. Nowak recommended saying: “We wish the vaccine was more effective – but it’s the best step one can take to protect themselves from flu.” Frieden has followed Nowak’s recommendation almost word for word.

Frieden is on solid ground in saying the CDC’s preliminary study has too small a sample to conclude reliably that the vaccine is more effective against some flu strains than against others. Even so, I don’t think he should be quite so dismissive of Deborah Cox’s point that the vaccine seems to work least well against the most dangerous circulating strain, A(H3N2). And in his shoes I would qualify the statement that “you have to get revaccinated each year” because the strains keep changing. It’s an overstatement. Some years the vaccine has the same strains as the year before; then the case for revaccinating leans less on changing strains than on diminishing vaccine potency (another oft-stated claim about which there is surprisingly little evidence).

This is beautiful, much stronger than “less than perfect” – it captures the sad truth that the flu vaccine works least well on the people who need it most.

I do wish Frieden had explained both reasons why sick people are more endangered by flu than healthy people: They’re likely to have worse flu complications (that’s the point everyone makes) and even a mild flu bout can worsen their other conditions (that’s the point everyone neglects).

I could quibble that saying the elderly and frail are “less likely to get the benefit that others get” still overstates the flu vaccine’s value. Only sixty-some percent of those “others” get any benefit. And the evidence is equivocal about whether the flu vaccine works at all for people over 65. Still, most experts think it probably does help some, and stressing that it works less well is huge progress.

It remains to be seen whether Frieden’s candor about vaccine effectiveness is a one-off or a new policy. The CDC’s parent agency, the U.S. Department of Health and Human Services, runs the www.flu.gov website, which mails out periodic bulletins to subscribers. On January 23, 2013, as I was writing this column, I received a flu.gov piece on “Flu Risks for Seniors.” It explains why flu is deadlier for people over 65 than for younger people, and urges seniors to get vaccinated. It says nothing to suggest that the vaccine might not work that well on seniors.

Bresee’s added point is very important. Even though the flu vaccine works less well on vulnerable people, it is likelier to save the life of a vulnerable person than of an otherwise healthy person, who could almost certainly endure a bout of flu without dying. So there are three truths here:

Public health officials usually make only the first point, neglecting the other two. Frieden shows unusual candor in making the second point. And Bresee rightly adds the third point, lest Frieden’s candor unduly undermine flu vaccination enthusiasm among vulnerable populations.

This is one of several questions at the press briefing that reveals reporters’ surprise at how ineffective the flu vaccine actually is (“barely sort of how you define effective”), and their trouble believing that it’s always roughly that ineffective. This is testimony to how successfully public health agencies have given the media (and the public) a false mental model about the flu vaccine: that it works really well.

So enduring is this false mental model that in some cases it survived the briefing. Every reporter got the message that the CDC says the vaccine is only 62% effective this year, but not every reporter understood that that’s typical, not extraordinary. I found a few stories that included quotations from Frieden or Bresee saying it was typical, right along with paragraphs in the reporters’ words suggesting it’s a bad year for the flu vaccine.

Still, most reporters wrote stories that showed they had actually learned from the briefing that 62% is pretty typical. To reporters’ credit, I couldn’t find any stories that said this year’s vaccine was only 62% effective whereas usually the flu vaccine is 70–90% effective. To reporters’ discredit, I also couldn’t find any stories (except one opinion piece in the Huffington Post) that said 62% is typical and it’s a good thing the CDC has finally abandoned its discredited 70-90% estimate.

Alice Park, who asked this question, got the answer just right. Her story begins as follows:

Based on early data from flu sufferers, health officials say the current influenza vaccine is 62% effective in reducing symptoms of the disease.

That means that those who are vaccinated are 62% less likely to need to see a doctor for their illness compared to those who are not vaccinated. While that’s a relatively low rate compared to those for childhood vaccines, which hover closer to 90% effectiveness and above, Centers for Disease Control and Prevention (CDC) director Dr. Thomas Frieden said Friday that 62% is “in line with what we expect” with influenza shots, which have to be reformulated every year from best guesses about which virus strains are likely to be circulating during the winter.

Veteran flu reporters, of course, already knew that the flu vaccine never works very well – though they may have been surprised to see the CDC showcasing the point. Precisely because it’s old news to them, they wouldn’t have considered the 62% finding as newsworthy as the newcomers did. Perhaps in an effort to get the 62% well-covered, the CDC seems to have intentionally called on more reporters who are new to the flu story than is customary at its flu briefings. (Jody pointed this out to me.) Helen Branswell of the Canadian Press, for example, is widely considered the dean of flu reporters; January 11 was one of the few times I can recall her not getting to ask a question.

If this was an intentional strategy, it worked. Repeated questions about the 62% from non-medical reporters gave Frieden and Bresee multiple opportunities to make their points. As a result, the veteran reporters wrote stories explaining the flu vaccine’s low effectiveness more clearly than they would have done if they’d been allowed to focus as usual on their more sophisticated, more knowledgeable questions. But a few of the newcomers still missed that the 62% wasn’t a shocking number. And to the best of my knowledge no mainstream journalist wrote a story about what really was shocking: the CDC’s unusual candor.

Frieden is absolutely right that the glass is 62% full. If we had been enduring the seasonal flu every year without any flu vaccine at all, the invention of one that works 62% of the time would be huge – like the miracle of the shingles vaccine, which prevents about 50% of that horribly painful condition. But it’s no accident that so many people find themselves comparing the 62% to 100% instead of to 0%. The CDC and its partner agencies created the 100% frame by giving us the impression of a very effective vaccine. This is a good example of one key way hype hurts. When people learn the truth, they’re likely to be shocked that 62% is so much worse than they’d thought, not grateful that it’s so much better than nothing.

So if the CDC now plans to come out of the closet about a vaccine that works only about 62% of the time, it needs to help people get over the shock – and still get vaccinated – by replacing that misleading 100% frame as quickly as possible. The best way to do that is to be brutally frank, not just about the 62% but also about how the public got a much-higher-than-62% misimpression in the first place. I’d love to have seen Frieden say something like this:

62% effective is enormously better than nothing, but compared to 100% effective it looks really bad. If a lot of people are comparing it to 100% instead of to 0%, that’s our fault, not theirs. Over the years we have given an inflated impression of how effective the flu vaccine is, so of course people are bound to be surprised when we become more open about the fact that it works a lot less well than we wish it did. Every time a public health official says the flu vaccine prevents flu, and stops there, he or she is sowing the seeds of people being surprised, shocked, and disappointed when they learn that it only prevents a little over half the disease bad enough to need medical attention. That’s why we’re trying to teach our own people to stop doing that.

Bresee is right to stress that the 62% finding isn’t surprisingly low. Given that some news stories still missed this key fact, he should have stressed it even more. Ideally, he would have had a graph showing flu vaccine effectiveness data for past seasons. Not that reporters are likely to use a graph in their stories – but it would have helped them absorb the point: no big dip for 2012–2013, just random variation (depending mostly on match) averaging in the high fifties or low sixties.

When Bresee says that the CDC’s 62% figure is “in line” with “some recent reviews of vaccine trials,” the main review he has in mind is a meta-analysis of prior research, published in the January 2012 issue of The Lancet Infectious Diseases. This meta-analysis looked at thousands of studies, but set very tough methodological criteria for which ones to include, ending up with 31 eligible studies to reanalyze. Its key conclusion: The average efficacy of the flu vaccine in adults 18–65 was 59%. There was a lot of variation from season to season; the 95% confidence interval for the 59% bottom line was 51% to 67%. That was for adults 65 and under. In children 7 and under, the live attenuated influenza vaccine (the FluMist nasal spray) did better, averaging 83% efficacy (69% to 91%). The meta-analysis authors didn’t find any studies that met their inclusion criteria for children 8–17 or for adults over 65 – though it’s universally agreed that the flu vaccine works least well in seniors.

The CDC started abandoning its prior 70–90% effectiveness claim about a year before the Lancet I.D. meta-analysis was published. About two weeks before the online publication of the meta-analysis (which the CDC had seen in draft form several months earlier), it changed its flu vaccine efficacy estimate to a much more supportable 50–70% estimate. That enabled some CDC officials to comment blithely after the meta-analysis was published that it was nothing new.

But as I pointed out in an October 2012 analysis: “Now, sadly, CDC and many lower-level public health officials often provide no flu vaccine effectiveness estimate at all in their public communications, having learned that 70–90% is scientifically unsound but fearful that the more accurate 50–70% might undermine public acceptance. This is a small example of officials not trusting the public, which is a very large risk communication problem in public health.”

To the best of my knowledge, the January 11 press briefing is the first time the CDC has made a big deal publicly of the fact that the flu vaccine works (at reducing medically attended cases of influenza) only about sixty percent of the time. It would have been gracious for Frieden and Bresee to give more explicit credit to the Lancet I.D. meta-analysis – and to a follow-up report by many of the same authors entitled “The Compelling Need for Game-Changing Influenza Vaccines.” Without those two documents (both spearheaded by Michael Osterholm of CIDRAP, the University of Minnesota Center for Infectious Disease Research & Policy), I doubt the January 11 press briefing would have said what it said.

Abandoning hype and starting to be candid is a virtue even if an organization can’t bring itself to acknowledge the history of hype and the pressure it took to force the change. But people get through the adjustment reaction faster if you explain that their prior misimpression is your fault, not theirs.

Bresee passes up the opportunity here to tell people that pandemic H1N1 (swine flu), like the old pre-pandemic H1N1, is less deadly than H3N2 (though it skews younger). Also, much of the initially susceptible younger population now has some degree of immunity to pandemic H1N1. This season is mostly H3N2 so far, and is therefore more severe.

Bresee is reiterating his key talking points: It’s not a perfect vaccine, it’s only “60, 50, 70 percent” effective, but it’s the best tool we’ve got. I approve of those talking points, and I approve of reiterating them. But he should also have answered the questioner’s concern about the relationship between the severity of the flu season and the inadequacies of the vaccine. He should have said that the flu vaccine is about 60% effective on healthy adults 65 and under when the vaccine match is pretty good, whether it’s a mild year or a severe year. For sure in a severe year we have more reason to regret that the vaccine doesn’t always work – but it works just as badly in a mild year.

Despite Bresee’s (and Frieden’s) efforts, here is how Robert Lowes ended up explaining “62% effective” in his story:

The current flu vaccine is 62% effective, which means that 38% of Americans who get vaccinated could still get the flu.

Not.

I have no quarrel with what Bresee says here, but he passes up a golden opportunity to explain that a lot of influenza-like illnesses (ILIs) aren’t flu – which makes the flu vaccine seem to work even less well than it actually does. If the vaccine is 62% effective at keeping you from going to the doctor with the flu, it’s a lot less than 62% effective at keeping you from going to the doctor with something that feels like the flu … and that normal people routinely call the flu even if their doctor bothers to test and tells them it’s something else. So when people say they got vaccinated and got the flu anyway, it could be because they’re in the 38% whose vaccine didn’t work to prevent a medically attended case of influenza, or it could be because they’re in the even larger group whose “flu” wasn’t influenza in the first place.

Whether 37% is high or low is a terrific question, especially since news stories almost always make the U.S. flu vaccination rate sound low: “only 37%.” This is an opportunity for Bresee to explain that the U.S. flu vaccination rate is the highest in the world. He might also mention that the CDC puts a higher priority on flu vaccination than the vast majority of comparable agencies elsewhere. A lot of countries – including countries as developed, wealthy, and health-conscious as the U.S. – pay comparatively little attention to influenza vaccination. Some European experts see it as a U.S. obsession.

I don’t know if this is hype or not. Bresee makes it sound like finding a new generation of better flu vaccines is a hot, high-priority effort, and maybe it is. The CIDRAP study I cited earlier, “The Compelling Need for Game-Changing Influenza Vaccines,” says it isn’t a high-priority effort but it should be – and says the widespread misimpression that the current flu vaccine works well is a big piece of the low level of interest in finding better one.

Bresee’s answer also skirts Weise’s question about whether there are ways other countries do better. Some other countries do have more effective flu vaccines than the U.S., because they use adjuvants – chemicals added to the vaccine in order to boost its effectiveness. The U.S. is afraid to add adjuvants, not because we’re worried about their safety but because we’re worried about arousing additional public concern and controversy. Now that the CDC is coming out of the closet about the low effectiveness of the flu vaccine, perhaps it can also come out of the closet about the possibility of using adjuvants to make it more effective.

Reporters keep asking for historical vaccine effectiveness data – and rightly so, to nail whether 62% is typical or surprising. Why didn’t the CDC have a graphic ready to roll out?

As I noted earlier, 50–70% is now the CDC’s standard flu vaccine effectiveness estimate – though this press briefing is by far the CDC’s most aggressive effort to date to publicize that estimate. Simply for saying 50–70% (or 62%), and saying it over and over, I’d give this briefing (and this answer) a B, and the “most improved camper” award.

But Bresee really should acknowledge that the 50–70% estimate is for healthy adults 65 and under in years with a good match.

Moreover, in this answer and throughout, the briefing fails to acknowledge that as recently as a couple of years ago CDC officials were still saying 70–90%. And sometimes the effectiveness hype was worse than that. In an October 26, 2009 interview on PBS, Carol Baker, then head of the CDC’s Advisory Committee on Immunization Practices (ACIP), said that “in perfect match years, you should get at least 90 percent protection or better.” I have to think Baker knew better and was intentionally hyping the vaccine’s effectiveness. The possibility that I’m wrong and the ACIP head was actually that ill-informed is terrifying.

Bresee’s explanation of two of the current flu vaccine’s defects – the virus keeps changing and our antibodies keep weakening – is solid. And this time he doesn’t sound so confident about the short-term prospects for developing a universal flu vaccine that evades these defects.

Bresee is resisting the temptation to say that an early start to a flu season means it will last longer or be more severe. But he’s not saying that there’s no connection – that the severity of a flu season is independent of when it starts, that an early start can mean either an early end or a long season, and that the only stable thing here (stable for a while, anyway) is that H3N2 seasons tend to be more severe than H1N1 seasons.

And he’s certainly not admitting that at the start of the season the CDC did try to connect “early” to “severe” as part of its pitch to get vaccinated.

Martin’s story in the Wall Street Journal accurately captured what the 62% figure meant. In addition, Martin did some research of his own to compare the most recent other “early flu season,” 2003–2004, with this season. Though far from claiming that an early flu season necessarily means a severe flu season, Bresee had raised the possibility that the high mortality in the 2003–2004 early flu season might presage high mortality again this time. Martin found and added a fact that weakened the comparison, a fact left out by Bresee at the briefing: “Dr. Bresee said the early start to this flu season reminds him of 2003–04, when about 48,600 people died from influenza. But that year, the flu shot was an inaccurate match for that year’s dominant strain.”

This kind of journalistic enterprise – testing what a source tells you against other sources or against the documentary record – is less common than it used to be. It’s lovely when it happens. If it happened more routinely, it would help rein in hype, from the CDC or from any other source.

As I noted earlier, the right way to correct a misimpression is first to identify it and take responsibility for it. “People hear ‘epidemic’ or ‘above the epidemic threshold’ and think that means it’s a really bad flu season. That’s our fault; we have taught people that epidemics are really bad, and then we don’t usually explain that that’s not necessarily what we mean when we say flu is epidemic. We probably shouldn’t use the word ‘epidemic’ to talk to the public about flu unless we take the time to explain it better.”

This is a lovely example of self-policing the temptation to hype. Bresee starts to say the vaccine gives you a milder case, then changes it to “could” give you a milder case, and then adds that the data are sparse even though it “makes sense” to him.

The hypothesis that the flu vaccine gives people milder cases even when it fails to keep them from getting the flu at all has been around for years, typically rolled out when the CDC or some other agency felt obliged to mention that sometimes the vaccine does fail to keep people from getting the flu at all. After the 70–90% effectiveness estimate was discredited and the CDC switched to 50–70%, “gives you a milder case” became a far more common flu vaccination messaging meme. As agencies cut “50–70%” out of their copy, they compensated with the “lower severity” claim.

Good for Bresee for saying it makes sense to him but there’s not much evidence it’s true!

As recently as October 17, 2012, a CDC web page ironically entitled “Misconceptions about Seasonal Influenza and Influenza Vaccines” conceded that the flu vaccine is less likely to work for the elderly and people with weakened immune systems, then added: “However, even among these people, a flu vaccine can still help prevent complications.” The revised “Misconceptions” page online as I write, updated January 10, 2013 (the day before the press briefing), omits that unproven claim.

Bear in mind that the CDC study featured in this news briefing – the one with a 62% effectiveness bottom line – sampled only patients with influenza-like illnesses sufficient to get them into a doctor’s office. As I pointed out earlier, “62% effective” here means 62% effective in keeping you from ending up with a medically attended case of flu – not 62% effective in keeping you from getting the flu at all. This CDC study doesn’t distinguish a vaccinee who didn’t get the flu from a vaccinee who got it but didn’t seek medical attention for it.

Not all flu vaccine effectiveness studies are designed this way. Some have related vaccination to microbiological proof of influenza without regard to whether the people in the sample ever sought medical attention. But when Bresee rightly points out that the evidence is sparse on how flu vaccination affects flu severity when it fails to prevent flu altogether, he should add that the CDC’s most recent study sheds no light at all on how often the flu vaccine prevents a case of flu altogether versus how often it only makes the case mild enough that no doctor visit is needed.

But I don’t want to end with a criticism, even a mild one. The main thrust of the CDC’s January 11 press briefing is to tell reporters and the public that the flu vaccine works a lot less well than most people imagine, even though it’s still our best tool against influenza. This is unprecedented candor about flu vaccine effectiveness.

I hope CDC critics notice this huge improvement, praise it, and start holding the CDC to this higher standard of honesty.

And I hope CDC officials pay close attention to the outcomes of the January 11 press briefing. There may be some short-term downsides to candor, as people react (and possibly overreact) to the belated revelation that the flu vaccine works less well than the CDC had led them to believe. But so far I haven’t seen any editorials urging people not to get vaccinated, or any sudden increases in anti-vaccine activism. Mostly, people seem to be slowly absorbing the truth that we need a better flu vaccine, but meanwhile the one we’ve got is a lot better than none at all – exactly the truth the CDC wants them to absorb.

Are people learning to mistrust the CDC because of its previous hype? Or are they learning to trust it because of its newfound candor? Neither, for the most part – they’re just learning the truth about the flu vaccine. But to the extent that the January 11 briefing affects what people think of the CDC, I hope it makes them think better of it. It certainly has had that effect on me.

My wife and colleague Jody Lanard contributed to this column.

Copyright © 2013 by Peter M. Sandman

Contact information page:    Peter M. Sandman