Misreporting of Recent Flu Vaccine Research: Headlines Sure To Spark Trouble

TOPIC: recent flu vaccine research, science in pop media

A recent headline on ScienceDaily.com announces that "Children Who Get Flu Vaccine Have Three Times Risk Of Hospitalization For Flu," and sure enough a quick Google search on that exact title produces hundreds of hits, including respectable sites such as ScienceDaily.com and medwirenews and a title at the U. S. Dept of Health & Human Services site www.healthfinder.gov:

Hospitalization Rates Higher in Kids Who Get Flu Shots

and of course at anti-vaccine propaganda sites such as www.ageofautism.com, that site trumpeting that "Flu Vaccine Triples Child Hospitalizations."

Wait — what?

Where is all of this coming from?

It turns out that the headlines are terribly misleading, especially for a unscientific general population that (1) easily confuses correlation with causation and (2) doesn't understand the difference between an experimental study and a quasi-experimental (or even correlational) study.

Most of the material being published out there under these headlines consists of simple regurgitations (or sometimes only pieces of) a premature and irresponsible press release issued 5/19 by The American Thoracic Society in promotion of that 'Society's 105th International Conference being held in San Diego, the press release having been timed to coincide with the specific symposium in which the related research was first being presented.

Premature? Yes. The American Thoracic Society press release concerned research not yet ready for general public consumption, concerning research not yet peer-reviewed (except minimally for conference presentation) or published in a peer-reviewed journal and involving a relatively small and special population of participants.

Irresponsible? Yes, in part for the reasons mentioned above, but also for the breath-takingly cavalier attitude with which they threw out for pop-media consumption poorly-explained research results under a heading sure to catch attention and sure to appear to have widespread and profound public health policy implications. Add to that the fact that the press release was based on an already-poorly-written abstract with questionably-worded conclusions, and the news release becomes an incredible fiasco.

Here's the original press release:

News Release

FOR RELEASE MAY 19, 2009 at 1:30 p.m. PDT

FOR MORE INFORMATION, CONTACT:
Keely Savoie or Brian Kell
ksavoie@thoracic.org or bkell@thoracic.org
ATS Office: 212-315-8620 or 212-315-6442 (until May 13)
Cell phones: 917-860-5814 (KS) or 516-305-9251 (BK)
ATS Press Room: 619-525-6323, 619-525-6324 or 619-525-6325 (May 15 to 20)
Mini-Symposium time: May 19: 1:30 p.m. to 4 p.m.
Presentation time: May 19: 3:20 p.m.
Location: San Diego Convention Center, Room 3 (Upper Level)

Flu Shot Not Effective in Preventing Flu-Related Hospitalizations in Asthmatic Children

ATS 2009, SAN DIEGO— The inactivated flu vaccine does not appear to be effective in preventing influenza-related hospitalizations in children, especially the ones with asthma. In fact, children who get the flu vaccine are more at risk for hospitalization than their peers who do not get the vaccine, according to new research that will be presented on Tuesday, May 19, at the 105th International Conference of the American Thoracic Society in San Diego.

Flu vaccine (trivalent inactivated flu vaccine—TIV) has unknown effects on asthmatics.

“The concerns that vaccination maybe [sic] associated with asthma exacerbations have been disproved with multiple studies in the past, but the vaccine’s effectiveness has not been well-established,” said Avni Joshi, M.D., of the Mayo Clinic in Rochester, MN. “This study was aimed at evaluating the effectiveness of the TIV in children overall, as well as the children with asthma, to prevent influenza-related hospitalization.”

The CDC’s Advisory Committee on Immunization Practices (ACIP) and the American Academy of Pediatrics (AAP) recommend annual influenza vaccination for all children aged six months to 18 years.

The National Asthma Education and Prevention Program (3 revision) also recommends annual flu vaccination of asthmatic children older than six months.

In order to determine whether the vaccine was effective in reducing the number of hospitalizations that all children, and especially the ones with asthma, faced over eight consecutive flu seasons, the researchers conducted a cohort study of 263 children who were evaluated at the Mayo Clinic in Minnesota from six months to 18 years of age, each of whom had had laboratory-confirmed influenza between 1996 to 2006. The investigators determined who had and had not received the flu vaccine, their asthma status and who did and did not require hospitalization. Records were reviewed for each subject with influenza-related illness for flu vaccination preceding the illness and hospitalization during that illness.

They found that children who had received the flu vaccine had three times the risk of hospitalization, as compared to children who had not received the vaccine. In asthmatic children, there was a significantly higher risk of hospitalization in subjects who received the TIV, as compared to those who did not (p= 0.006). But no other measured factors — such as insurance plans or severity of asthma — appeared to affect risk of hospitalization.

“While these findings do raise questions about the efficacy of the vaccine, they do not in fact implicate it as a cause of hospitalizations,” said Dr. Joshi. “More studies are needed to assess not only the immunogenicity, but also the efficacy of different influenza vaccines in asthmatic subjects.”

###

Session # C94: “Viral Infections in Childhood Respiratory Disease”

Abstract # 561: “Flu Vaccination in Asthmatics: Does It Work?”

It's interesting to look at the deterioration of the information as it flowed from Joshi, et al's original conference abstract, to the press release, then on to various pop media outlets:

Original title of Joshni, et al's conference abstract:

Flu Vaccination in Asthmatics: Does It Work?

(which, given the actual description of the research actually seems a poor title)

Then to this in the press release from the American Thoracic Society:

Flu Shot Not Effective in Preventing Flu-Related Hospitalizations in Asthmatic Children

Then to something like this in the mainstream pop-media (this one appearing on ScienceDaily.com):

Children Who Get Flu Vaccine Have Three Times Risk Of Hospitalization For Flu, Study Suggests

Then further mangled by less mainstream media to things like this:

Flu Vaccine Triples Child Hospitalizations

That last one of course now making it sound like the very existence of the flu vaccine is somehow leading to dramatic increases in child hospitalizations, which is utter nonsense.

So what's really happening here? And why does the first author Dr. Avni Joshi herself remark that the findings “… do not in fact implicate it [the vaccine] as a cause of hospitalizations” ?

In part, she is acknowledging what this study was NOT: the study was not an experimental study allowing cause-effect conclusions. In other words, the study was NOT something like that outlined in the figure below:

Given such a design, we might reasonably compare the conditional probabilities of being hospitalized with and without having earlier received a flu vaccine, something like

p(hospitalization | flu vaccine) vs. p(hospitalization | no vaccine)

Such an experimental design would allow us to fill in all of the cells of an incidence table like the one shown here at right:

But again, that's NOT the study Joshi and colleagues did. Instead we have the following description from Joshi and colleagues in their conference abstract:

METHODS: We conducted a cohort study of all pediatric subjects( 6 months to 18 years age) who were evaluated at Mayo Clinic, Rochester, MN, USA who had laboratory confirmed influenza during each flu season from 1999-2006 to evaluate the efficacy of TIV. A case control analysis was performed with the cases and the controls being the subjects with asthma who did and did not required hospitalization with the influenza illness respectively.

and this slightly elaborated description from the press release:

… the researchers conducted a cohort study of 263 children who were evaluated at the Mayo Clinic in Minnesota from six months to 18 years of age, each of whom had had laboratory-confirmed influenza between 1996 to 2006. The investigators determined who had and had not received the flu vaccine, their asthma status and who did and did not require hospitalization. Records were reviewed for each subject with influenza-related illness for flu vaccination preceding the illness and hospitalization during that illness.

So, to be clear: this appears to have been a retrospective study of 263 participants already diagnosed with flu, some of whom had received a flu vaccine before their illness, some of whom had not, and some of whom were later hospitalized for their illness. The design of the study looks more like this:

and instead of giving information about the probability of hospitalization if you get a flu vaccine, the results give us information (with limited generalizability) about the probability of hospitalization if you were vaccinated but still came down with the flu. The phrasing makes this sound like some arcane, subtle distinction, but the information being communicated is dramatically different.

The pop media headlines and coverage lead one to believe that getting a flu vaccine somehow triples one's likelihood of being hospitalized for the flu. That is wrong. Instead, the research being reported implies only that if you get the flu (and have serious enough symptoms that you go to a medical clinic to be checked out) despite having self-selected into getting a flu vaccine (which means you're already part of an at-greater-risk population in general) then you're more likely to be hospitalized than someone who was also quite sick from the flu but without having had the earlier immunization.

Some of the possible explanations for such a result lie with the special patients themselves, and this is why Joshi points out that the results do not imply that the vaccine causes an increase in hospitalization rates. The special patients who get a serious case of the flu despite being vaccinated may be special (for example) in having particular health difficulties already or some other constitutional predisposition for the illness, and/or the patients may be special in having contracted particularly dangerous forms of the illness.

RELATED REFERENCES

Children Who Get Flu Vaccine Have Three Times Risk Of Hospitalization For Flu, Study Suggests. (5/20/2009). Science Daily, accessed 5/21/2009 at http://www.sciencedaily.com/releases/2009/05/090519172045.htm

Flu Shot Not Effective in Preventing Flu-Related Hospitalizations in Asthmatic Children. (5/19/2009). New Release from the American Thoracic Society, accessed 5/20/2009 at www.thoracic.org/sections/publications/press-releases/conference/articles/2009/abstracts-and-press-releases/joshi.pdf

Heckenlively, K. (5/21/2009). Flu Vaccine Triples Child Hospitalizations, but Won’t Turn Them into Horned, Hairy Apes, say Experts! Age of Autism: Daily Web Newspaper of the Autism Epidemic, accessed 5/21/2009 at http://www.ageofautism.com/2009/05/flu-vaccine-triples-child-hospitalizations-but-wont-turn-them-into-horned-hairy-apes-say-experts.html

Joshi, A. Y., Iyer, V. N., Hartz, M. F., Volcheck, G. W., Patel, A. M., & Li, J. T. (2009). Flu vaccination in asthmatics: does it work? Presentation at The American Thoracic Society's 105th International Conference, San Diego, CA, 5/19/2009. Abstract available as part of the press release available at www.thoracic.org/sections/publications/press-releases/conference/articles/2009/abstracts-and-press-releases/joshi.pdf

Lyford, J. (5/21/2009). Flu vaccine fails to reduce hospitalization in asthmatic children. medwirenews, accessed 5/21/2009 at http://www.medwire-news.md/48/82539/Respiratory/Flu_vaccine_fails_to_reduce_hospitalization_in_asthmatic_children.html

RELATED RESOURCES

American Thoracic Society. http://www.thoracic.org/

CDC - Influenza (Flu). http://www.cdc.gov/flu/

Centers for Disease Control & Prevention (CDC). http://www.cdc.gov/