Today Robert West’s research group will publish[*] in his journal, Addiction, a study about success rates for smokers’ most recent cessation attempt, as a function of what cessation aid, if any, they used. The results are further information that puts the lie to the claim that e-cigarettes are not helping people quit smoking. However, the way the results are already being interpreted by laypeople, and how they will inevitably be reported in the future is not accurate. Note that this post is about how to properly interpret the scientific meaning of the study. It is not about the policy implications of the results, and most definitely not about whether spinning the results in a particular way useful or proper in pursuit of a particular political goal.
[*At the time I am writing this, though probably not by the time you read it, this verb tense is correct. In an example of a very disturbing trend in academic publishing, this study was made available to the press in advance of making it available to scientists who could review it. This results in reporters rushing to write stories and asking for comments of researchers who have not even read the paper, much less had a chance to assess it. The reporters want to run whatever quick story they can the moment the embargo is lifted (the first minute the paper appears in the journal) – or in this case even before the embargo is lifted, since the stories are already running. Since most of the scientific community has not had a chance to peer review the paper before their stories run, the only comments they can offer are uninformed, and thus the reporters have nothing to work with other than the assertions in the press release. And, yes, in case it is not obvious: The fact that I am writing this before the paper is officially out means that I have a bootleg copy of it in advance – but I only got it after the first press reports already appeared, and many of those who are commenting on it already apparently have not seen it at all.]
The study is very similar to a 2008 study by Brad Rodu and myself (which, strangely, is not cited in West et al., even though it is – to my knowledge – by far the most similar study in the literature to date). In that study, Brad and I looked at the method employed by smokers (Americans, c.2000) in their most recent quit attempt. We found that those who attempted to quit by switching to smokeless tobacco were by far more likely to have succeeded than those using any other cessation aid. In the new study, West et al. did the same thing for British smokers who used either e-cigarettes or NRT. Both studies also looked at those who quit unaided.
As with our study, those using tobacco harm reduction (THR; i.e., switching to a low-risk alternative to smoking) were more likely to have quit smoking than the others. It turns out that the success rate for those using smokeless tobacco was much more impressively higher in our study than those using e-cigarettes in West’s, though for the reasons explained below, not too much should be read into that.
The West study is being interpreted as showing something like “e-cigarettes are 60% better for smoking cessation than NRT” based on a comparative rate of successful cessation among those using the particular products. This is undoubtedly true – indeed, it is probably a gross understatement: NRT barely works, if at all, while e-cigarettes are tremendously effective for many people. However, that is not a proper interpretation of the study results.
The equivalent claim about Rodu and Phillips would have been “smokeless tobacco is more than 600% more effective for smoking cessation than NRT.” This would be an inappropriate conclusion and we took pains to make that clear. West et al. also are careful about phrasing their conclusions: “E-cigarette users were more likely to report abstinence than either those who used NRT bought over-the-counter…” (quoted from the abstract of the bootleg version of the paper I have).
The reason that this is not the same as saying “X% more effective” is embedded in the quoted sentence, in the reference to the people themselves. People choose the cessation method that they think will be most effective for them. They do not always assess this correctly, of course, but there is clear self-selection. The misinterpretation “X% more effective” is based on the common error of treating people as if they were molecules – all the same. This is an attitude that is all too common among many people working in tobacco research (though, I do not believe, among any of the authors of the papers mentioned here), and a simplification that is inevitable in laypeople’s interpretation of the science (unless they are severely warned against it).
In the case of Rodu and Phillips, we can be fairly sure that most those who switched to smokeless tobacco had, at the time they decided to quit smoking that way, already tried it and decided they liked it enough to switch to it. Moreover, many of them were probably among the small (at the time) portion of the population that understood that smokeless tobacco was low risk. Thus, they were much more likely to succeed for reasons that had more to do with the people than the product per se. Indeed, it is a safe bet that if those who quit using NRT or some other method had been handed smokeless tobacco and told to try to quit by switching to it, very few would have done so – far fewer than actually quit using their chosen method.
The situation with the West results is almost certainly not so extreme, but the same principle applies. Those who switched to e-cigarettes were those who chose to try to switch to e-cigarettes. Many had already tried them and decided they were appealing. Though the number who had not previously tried is probably higher than the switchers who had never tried smokeless tobacco in Rodu-Phillips, in both cases everyone who tried to switch was willing to switch. Many of the would-be quitters were not willing to switch.
Similarly, those who tried to use NRT included those who believed that using NRT was a good way to quit. One reason for the poor performance of NRT among such people is that they have been convinced by aggressive marketing that NRT will magically make them not want to smoke anymore after they taper off of the NRT. For most smokers, that is not true. NRT is a viable substitute for smoking, as a THR product, for some smokers. Getting a bit of aid to get away from smoking forever is just what a (very) few smokers need. But for the most part, NRT does not perform as marketed. That aside, the people who chose NRT rather than e-cigarettes intentionally did not choose e-cigarettes, suggesting that they did not think the latter was the best option for them.
The issue becomes even more clear when we compare those who quit unaided, as most ex-smokers did. Smokers who are genuinely ready to quit all tobacco products forever just quit. That is why the method is so successful – because it is self-selected by those who know they do not want a substitute and do not need a magical drug to make them no longer want to smoke. Giving such people any of the substitutes or other cessation aids would not necessarily increase their cessation rate at all. West et al. had somewhat richer data than did Rodu and I, and thus were able to show that those who quit unaided had a slightly lower score on a “strength of urges to smoke” index, just as we would predict.
To sum up the implications of this: It is not an accurate interpretation of cessation success data to say that a particular method is better, generically and across the population. An extreme possibility illustrate this. The following is consistent with the study results:
- Those who quit smoking using NRT consisted of every single individual in the study sample for whom that would have been successful. That is, the use of NRT was successfully selected by everyone for whom it would work, and no one who did not try it would have succeeded in quitting had they tried it.
- But for the unsuccessful quitters who did not try e-cigarettes (those who tried NRT or unaided cessation, but kept smoking), e-cigarettes would have often been successful. That is, had those unsuccessful quitters been handed an e-cigarette and told “try this instead”, many more of them would have quit smoking
Under that hypothetical (which, again, is perfectly consistent with the study results), we would say that e-cigarettes are enormously more effective for smoking cessation than the “60% more” that is the common misinterpretation of the results.
Note that this is not an argument for running clinical trials, in which everyone is forced to try a particular method, to avoid these complications. That is a tempting solution to the self-selection problems. But clinical trials introduce much bigger problems, and thus are an inferior research method for smoking cessation, for reasons I have discussed at length before. Clinical trials do directly address (albeit badly) the question “which product is X% more effective across the population”. But the biggest problem with them is that this is not actually an interesting question.
Why? Because the self-selection is not a problem, but rather part of what matters in the real world. There is no possibility that every smoker in a population will be assigned one method to quit. Thus, there is no reason to try to figure out which would be the best single method to assign to everyone (which is basically what a well-done clinical study would show). Rather, in the real world, each of the cessation methods is available to everyone, and (since it does not matter which one someone uses, from the perspective of health) it is best if they can find which one works best for them (which refers to both effective smoking cessation and ongoing happiness).
So what matters from the West results and other data is the observation “people are successfully quitting using e-cigarettes.” That is an important and legitimate interpretation of the results. It is also good that many people are successfully quitting using NRT and many more are just quitting because they do not want to use any product any more. These methods are not in competition with each other, so we should not care which one works for more people. Any of them that works for anyone who wants to quit is a good thing. This is good, because there is no way to legitimately interpret the results in terms of which one works better.
EP-ology 