Monthly Archives: September 2010

There is a political faction that for years has been pushing for regulators to require that babies on planes be secured in a car seat, thus requiring that they be in a seat of their own rather than on an adult’s lap.  It is not clear why anyone actually wants this.  The usual explanation for weird regulation (it helps the industry profit) does not make sense — after all, the airlines could change their own rules themselves.  The only thing I can come up with is that there is a cabal of frequent fliers who are really sick of 18-month-old lap babies kicking the backs of their seats and want to restrain them.  The reason this is such a mystery is that health concerns about the babies cannot possible explain their motivations, because — as has been pointed out in numerous analyses — this would kill more babies than it saved and injure many many more, as well as killing a lot of grownups.

Why is that?  Because the extra cost of another airplane seat for a young family would be just enough to tip some of them into driving rather than flying, and driving is far more dangerous than commercial airline travel.  And not only is air travel a lot safer than car travel, but most of the fatalities that do occur are in crashes in which a safety seat could not provide any protection (roughly half of all fatalities in crashes are on flights where there are no survivors).  Thus, unlike safety restraints in cars, which save the lives of roughly half of all the people who would have died in unrestrained crashes, the potential benefit is low.  Of course, occasionally someone is saved from injury by restraints on a plane, but it is rare enough that the benefit would be outweighed by the shift to driving.

There is pretty good data for calculating what would happen (data about price, response to price, accident statistics), so this makes a great teaching example that I have used many times.  It is a wonderful lesson in how policies do not necessarily have a particular effect just because who ever proposed them says that they have that effect.  One year, using one particularly good version of my class’s analysis, a student and I presented the results at the annual epidemiology meetings and she won an award for the work.  That version is not currently online, so no link, but anyone who is interested can find someone else’s version:  every few years, going back to when I was a student, someone repeats this analysis and concludes that the policy would kill far more people than it saved.

In spite of how there is no upside for anyone to this policy (other than that kicking thing), it is the official position of the American Academy of Pediatrics that not only should the regulation be put in place, but pediatricians should take desperately valuable time during office visits to advise new parents to spend the extra money to buy their baby her own airplane seat.  This was published in Pediatrics about a decade ago, in an article that basically said “yes, the economic analyses say more young families will drive and die as a result of this, but we do not believe in economic evidence.”  Keep in mind that this is long before Pediatrics decided to strive for the title of Dumbest Health Science Journal with their articles about “third hand smoke” and such.

The latest is a push by the U.S. National Transportation Safety Board to implement the safety seat regulation.  They do this periodically, and the Federal Aviation Administration that makes the rules pays attention to the evidence about what would really happen and refuse.  But what is remarkable is the basis of NTSB’s latest recommendation:  They cite the recent crash of a small commercial flight in which everyone was killed and no one could realistically have survived, but where the bodies of some of the children on board were flung from the wreckage, and thus were apparently not restrained.  Horrible without a doubt, but how does this possibly support the call for safety seats?  Is it because if the babies had been restrained then they only would have been killed once due to the crash and not killed again because they were then flung away from the wreckage?  Being flung from a vehicle is often the cause of death in car crashes, but not plane crashes.  Perhaps NTSB should have put airplane experts in charge of this one rather than their highway guys.

More than a decade ago, some researchers at the U.S. CDC created an incredibly complicated model that was highly dependent on what were basically wild guesses, and came out with the estimate that there were 76 million cases of foodborne disease in the U.S. annually.  Several researchers, including one of my students and me, pointed out that this estimate was ridiculously over-precise, given the complete guesswork that went into it.  Also, it was almost certainly biased downward (i.e., a lot too low).  Our article was a general analysis about accounting for errors in estimates and just used that estimate as a particularly glaring example of researchers who apparently do not understand the concept of error bars.

The reason I bring it up is that every time there is an outbreak of foodborne disease that makes the U.S. newspapers (i.e., it had a lot of victims in the U.S.), this number gets toted out again, like in this New York Times article, written by Walecia Konrad yesterday.  Granted, I have never heard of Konrad and she(?) is not one of NYT’s expert health writers, but I have seen the big names make this mistake also, as well as numerous others.  And sure enough, the CDC website has that figure (and some others from the same original paper, which are also repeated by Konrad) in multiple places on its website.  Several of these have date stamps as recent as 2009 and as far as a few minutes of searching revealed, there is no acknowledgment that this is a very rough estimate and the only way a reader would even learn the age of the statistic would be to find one of the rare invocations that actually cited the original paper.

Why is this such an embarrassment for both health reporters and epidemiologists?

Even if we assume that most of the inputs into the model were unbiased (i.e., really were the best possible guesses – which is an incredibly charitable assumption), given the uncertainty of the guesses, it would be bold to assume that the estimate was correct within a factor of two.   (If you want details, they are in our paper.)  But think about the implications of reporting that “76 million” figure that has been repeated a zillion times since then (plus or minus half a zillion).  If they had claimed the round figure “about 75 million”, it could be read as something as imprecise as “more than 50 million but less than 100 million”.  That would probably have still been too precise a claim, based on the quality of the model used to derive the estimate, but it would at least have been in the ballpark.  If they had rounded their estimate to 80 million it would have implied something like “between 75 and 85 million” (because 74 million would round to 70, while 86 would round to 90).  That would be way too precise a claim, but at least representing some vague awareness that their estimate is not perfect.  By claiming 76 million and offering no indication of uncertainty bounds around it, however, they are effectively saying “we are quite sure the answer is between 75.5 million and 76.5 million”.

So the reported figure was ridiculous the day it was originally written.  But it is even more absurd to continue to make the same quantitative claim.  This figure, even if it had been correctly estimated in the late 1990s (using data that often was years older), is obviously not a scientific constant like the speed of light.  This should be fairly obvious to reporters who use this number, since the reason they are writing the story is because there is an unusual event happening, as in the present case, that did not happen the year before.  More important, food handling practices change over time, some for the better and some for the worse.  At the very least, I would like to think that newspaper reporters and the CDC are aware of the little matter of the U.S. population increasing by more than 10% since the original analysis was done, so unless improvements in food safety perfectly balanced out the expanding population with just the right decrease in per capita disease rates, year after year, the number could not possibly remain constant.

In other words, pretending that this number is accurate is like estimating the number of Americans that voted Republican or were Hispanic with one study in the 1990s, and then continuing to claim the number is exactly that value forevermore.  I would assume that political scientists would not make those mistakes.  It is a real shame that CDC’s epidemiologists lack equivalent understanding of the way the world works.

The source of the error hardly lets Konrad and other reporters off the hook however.  The NYT article, like many others, presents the figures as if they were God’s Own Truth, not even reporting the source let alone including caveats like “CDC claims…”.  You would think that following the huge embarrassment of playing stenographer for the U.S. government’s lies that started the Iraq war, the NYT’s instructions to its reporters would include “never repeat something the U.S. government claims as if it were an indisputable fact.”

Why does this matter, and why did I decide to write about it today since reporters have been repeating this error for years?  The answer to the latter was that I was tipped over the edge by seeing on the front page of the newspaper an article about back-to-school shopping that declared that “on average, fathers are expected to spend 23 percent more on their … children than mothers”.  It boggles the mind to think that anyone would believe that this prediction could be made with that kind of accuracy (not “about one quarter more than”, but down to the last percentage point).  Do people not realize that even if we had a complete cash register data from every purchase made during the season, we would not be able to estimate that figure so accurately retrospectively?  We could not figure out how many were made by dads rather than moms that precisely (e.g., if they are both at the store, who gets credit?) or what constitutes a back-to-school purchase.  Do people not realize that we probably cannot even estimate how many schoolkids have dads who buy them anything with that degree of precision (due to uncertainty about who constitutes a dad, whether they are actually in the kids’ lives, etc.)? 

Actually, I guess the answer is that most people do not realize.  Therein lies why this matters.  As soon as something has a number attached, reporters and others seem to turn off even their most basic critical abilities and common sense.  It never occurs to them to ask even the simple “how could anyone possibly know this (e.g., exactly what portion of back-to-school purchases will be made by fathers)?”  People seem to think that those who cook up these numbers have some magical sciences available to them.  Modern science lets us measure the concentration of cadmium or BPA in biological samples down to the parts per billion level, so it stands to reason that they must be right when they claim that many people are being sickened by it, right?  If they tell us that 47,318 people die each year in the U.S. from environmental tobacco smoke then it must be true, right?  Surely a science so precise could not coexist with genuine controversy about whether there is any major mortality risk from ETS exposure. 

And so, since we know that the U.S. government can count foodborne disease cases (which are almost never definitively diagnosed, by the way) within 1% accuracy, surely when they flatly declare “salmon genetically engineered to grow quickly is safe to eat and poses little risk to the environment”, they must be right.  Right?  The first bit seems likely, but the declaration by the Food and Drug Administration that there is basically no chance of ecological pollution from the new genes rings a little hollow.  Maybe the country’s regulators of pain relievers and pacemakers somehow know more than the rest of us about the risk of introducing novel agents into the ecology, but I do not share the New York Times’s faith in the government (at least they attributed the claim to the FDA rather than just declaring it true).  Perhaps if New York Times reporters read the New York Times, they might have noticed their several articles this year about the spread of the “Roundup ready” engineered gene from crops to weeds, making the latter resistant to what had been the best available herbicide.  This is not to take a position on genetic engineering, mind you, just to speak up in favor of not being so gullible about any declaration made using the language of science.

As I have written about several times this year, the claims about smoking in movies causing a large portion of smoking initiation are a great example of the “if only…” claims made by anti-tobacco extremists to support more and more draconian manipulations of society.  The usual script is “if only X were changed then smoking/tobacco use would be reduced by Y%”.  Of course, because the extremists wield so much power, X is often changed in just the way they demand and then …… nothing changes, much to the surprise, of, well, no one at all.  Why no surprise?  Because the extremists who made the claim conveniently forget they ever did so, and rewrite their history to exclude it.

These are not the grossest most malicious type of lies (see the comments from this TobaccoHarmReduction.org blog post for an interesting discussion related to that point), but it is probably the second-most harmful set of extremist claims from the narrow perspective of public health (the most harmful being the lies that attempt to discourage smokers from adopting harm reduction).  To make it a bit more difficult for them to pretend they never said it, I and others are making a habit of documenting these claims.  So in case there is any confusion about it in the future, Physicians for a Smoke Free Canada recently published the claim (pdf) that baldly stated that just under half of all smoking among 15 to 19-year-olds in Canada was caused by them seeing smoking in movies.   Note that this would mean that pretty close to half of all smoking was caused by film portrayals among those in their 20s and 30s and however long movies have had this thrall, since most smokers start during this age range.

So, they are effectively claiming that once smoking in films is gone, smoking incidence (initiation rate) will have dropped by almost half compared to the bad old days, even after controlling for the effects of all other anti-smoking efforts, and so prevalence (usage rate) will be lower by half in the cohorts that were teenagers after smoking was removed from films.  It is not necessary to go into the use of statistics (either worthy of a failing grade in an intro class or highly skilled, depending on one’s take on how statistics should be used) employed to cook up this claim.  They made the claim, clearly an unambiguously, burying the statistics that were used to rationalize it.

Anyone want to place a little wager in support of their prediction?

I am sure they would not do so.  They are assuming that by the time their prediction is shown to be wrong they will have skated on to some other, quite possibly equally nutty, claim that they can trick the mainstream media into repeating.  Indeed, it is not necessary to wait very long to show they are wrong.  Since most smoking has been eliminated from movies over the last decade, we should have seen a huge drop in smoking initiation according to their prediction, even after subtracting the reduction attributable to the effects of increased taxes, education campaigns, etc.  Since the actual drop has been quite a bit smaller than the movie model would predict, it must mean that in spite of all the other anti-smoking efforts, smoking initiation would have dramatically increased had it not been for getting rid of it in movies.  Whew!  Canadian youth sure dodged a bullet there.

I expect that some of you are saying “so what?”, since Physicians for Smoke Free Canada is one of the more dishonest anti-tobacco extremists groups.  Granted, this publication may reflect their particularly poor judgment: other groups who have touted similar claims have avoided publishing a 40 page report in PDF, which is much harder to make disappear than an html page or a billboard.  But this claim has been made by most of the other Tobacco Drug War types also, and even by the otherwise respectable health groups who are willing to squander their credibility on anti-tobacco extremism. 

However, I have to say that  this one strikes me as strange even for them.  I understand why they might make up any claim they can think of to get taxes increased — they get to keep a lot of the money, after all.  I have written about why they are so desperate to discourage harm reduction.  But why risk burning quite so much credibility over what appears in movies?  Even if it really mattered, there is not really much fight left to fight:  They have already succeeded in bullying studios into removing almost all smoking.  Are they really that desperate to find an excuse for why their last decade or two of promises to dramatically reduce smoking have failed?