Michael Siegel puts himself in a hole, and then keeps digging

Those of you who do not like “drama”, stop reading now.

So in the previous post I revealed how Mike Siegel put out an announcement about an ill-advised e-cigarette study with vague protocol and an enormous price tag, and then about a week later figured out he could not possibly fund it and spiked the whole thing. In between those two events, he started asking the e-cigarette consumer community for donations, and CASAA recommended against providing them. I wrote a few analyses (somewhat coincidentally, since the first was already in my queue) about why the study design — a randomized controlled trial (RCT) — was inappropriate for measuring the role of e-cigarettes. And I wrote a private letter to Siegel pointing out why his crowdfunding was borderline unethical (at best) and created a serious risk of scandal for him. (My communication was partially to try to end the harm the crowdfunding might do to the community, of course, but I was also trying to do him a favor.)

He ended up cancelling the study, and this was clearly entirely due to the fact that he belatedly realized his funding goal was hopeless. But instead of trying to quietly move on from this fiasco, he put out a blog post (and the same statement on the project website) in which he tried to blame his failure on others. In doing that, he wrote claims about CASAA, and perhaps me, that were arguably defamatory. (He did not have the decency to mention the names of those he was making accusation about, but most everyone familiar with the issue interpreted his claims as being about CASAA. Study personnel also sent at least one tweet specifically to CASAA supporters requesting they contact them directly.) So I responded with the previous post. The details are all there.

[Note, in case it is not obvious: Like the previous post, this is just me talking, not CASAA.]

Not content to dig himself into a hole with his original post, instead of taking the opportunity to quietly walk away, Siegel decided to redouble his digging with accusations and misleading claims in some comments on that post. I would never have responded to those, nor perhaps even heard about them, were it not for this claim he made:

We did receive pressure and threats to alter the study design in order to try to create more positive results for e-cigarettes. For us, that was just unacceptable. At the end of the day, we could just not work under such conditions, which are not conducive to the practice of objective science.

This is simply unfathomable and, I think, represents just how incredibly out of touch he is with scientific discourse. He grew up in tobacco control and has since been conducting a monologue from a public health school, and so perhaps just never encountered critical scientific analysis.

Okay, perhaps he really did receive threats or pressure. But what was threatened and by whom? If they were threats of criminal activity, he should report them to the police. It seems like these would be the only threats that could actually cause him to alter his plans.

What other threats or pressure could there be with any teeth whatsoever? Did someone threaten to complain about this to his university about the study plan itself? If so, presumably he knew that the complaint would be groundless and ignored. Did someone threaten to complain to the university about the ethics of the crowdfunding? If so it was not I, and, as far as I know, I was the only one who was particularly concerned about the ethics perhaps violating university rules of conduct. Besides, that would be about the community crowdfunding itself, not “objective science”, and the response to that (valid) threat, if it were actually made, would be to just end the community crowdfunding, which he acknowledged was unimportant (see below).

Was the threat to point out the flaws in the study once it was underway or after the results were reported (we are now in the fictitious scenario where he actually managed to fund it)? If so, that’s science, baby. You should always assume that is going to happen. (And note that I never said such a thing, though he could certainly be confident I would subject it to critical analysis if it was carried out, so that is hardly something that could be called a threat even if I said it.) Was the threat to keep belaboring this fiasco to hurt his reputation? If so, it was apparently he who made the threat to himself, perhaps writing it on his own bedroom mirror like some scene from a horror movie.

No, I am pretty sure he was referring to the multiple knowledgeable and well-meaning people (not just me and CASAA) who urged him to do a different study that would be both more useful and more than an order of magnitude cheaper. Perhaps from his ivory tower the statement “we could get behind that and try to gather support for it” seems like pressure, though most of us would call it a friendly promise of collaboration. I cannot conceive of how it could be taken as a threat. A threat is a statement of “if you do X then I am going to do Y” designed to make someone not do X. No one tried to stop him from doing the study (we knew he could never fund it, after all) — we just tried to convince him it was a bad idea.

CASAA came out against his plans; it did not threaten to come out against his plans. (CASAA expressed its concerns to him before doing that, without making any threats, and asked him some questions. He did not respond to the substance of our concerns.) I pointed out the ethical dubiousness of his fundraising, not only not threatening anything, but making clear that I preferred to keep that conversation private. (I went public with it in my previous post only after he made some allegations that were pretty clearly about that communication.)

If that is what he is calling threats, it is defamatory. If it is not, he should really produce some examples of what he is referring to. Just the anonymized text would be fine — it would not confirm that they were real or credible, but would at least tell us what he is making allegations about.

Finally, I have to address the naivety of that last bit. There is no such thing as objective science. All science is designed by people and interpreted by people. It is only as good as the skill and integrity of those people. Someone pointing out that a particular study design is misguided is no less “objective” than someone thinking it is a good idea. Debate about methods is not any less “objective” than anything else in science. A better, cheaper study is no less “objective” than an expensive white elephant. His claim is not just self-serving, but shows a deep lack of understanding about the scientific process.

So since I took the time to comment on Siegel’s latest defamation, I will go ahead and look at the rest of what he wrote.

First, we would NEVER ask the vaping community for $4.5 million.

As I made clear previously, this misrepresents the concerns expressed about his crowdfunding. Obviously everyone knew this. Which is why we pointed out that if he raised even a few percent of that much from the community, it would crowd out all other community-supported research and activism.

This campaign was directed ALMOST ENTIRELY at electronic cigarette companies, including the largest independent (non-tobacco) companies. With something close to $10 billion in annual sales, it is not unreasonable to expect that one could raise $4.5 million.

As I pointed out previously, this is just naive. Gross turnover tells us little about what tiny fraction of that is either net equity (this, not gross revenue, is what someone might fund research to protect) or available cash (which could provide the funds). About half of total sales come from the majors, who he refused to take funding from. The rest comes from privately-held small (and a few medium) businesses, so he was basically asking the owners for personal donations. If the industry really thought his study would do a lot of good (difficult to believe), the sum probably would have been worth it to them collectively. But then the free-rider problem would kick in (better to let someone else pay for the study that benefits everyone), and there is not enough coordination to overcome it. It was obvious from the start that he had no prayer. Any number of people could have told him this. It only took him a week to figure it out after pitching his requests to the industry.

In fact, we did not send out any appeals directly to the vaping community. Most of our appeals for the funding are going to the very largest companies. However, we didn’t want to exclude the vaping community completely and thought that allowing them to contribute even a small amount toward the overall goal would give them a sense of participation and ownership.

This is just not true. The project webpage had a huge “Donate” button on it. The project twitter feed sent out instructions about how to donate. These appeals were obviously not directed at corporate funders. Consumers certainly interpreted the appeals that way, with many posting on social media that Siegel was asking for contributions in order to be able to do the study. These methods for “allowing” people — people who mostly could not understand the inaccuracy of his promises about this project — to give some of their own money to him were apparently in place from the start despite his claims that they were in response to popular demand.

Second, regardless of anyone’s views, there is no question that the FDA is going to require some sort of clinical or behavioral trial, with randomization of subjects, before it endorses electronic cigarettes as a bona fide smoking cessation or harm reduction tool. In other words, the FDA will never view e-cigarettes as “appropriate for the public’s health” – the key requirement for new product applications – in the absence of clinical trial-type evidence. Surveys are just not going to cut it. That’s just reality. Anyone who believes that the FDA is going to be convinced by survey data is simply not accepting reality.

This just demonstrates how little he understands the role of e-cigarettes, the current FDA battle, or how FDA works. There is relatively little interest in having FDA endorse an e-cigarette as a smoking cessation device (i.e., have it an approved pharmaceutical). If there is such interest, it comes from one of four major companies, not the community. The concern in the community is FDA threatening a de facto ban of 99.99% of all currently-available products — which would come from the Center for Tobacco Products, not the drug regulation arm of FDA. An RCT would not address this threat at all. Indeed, if a product were approved as a smoking cessation device, that would actually increase the threat because FDA might use that approval as an excuse for banning all other products. However, a pharmaceutical application would have to be about a product. Generic trials of some other product do not substitute. The company that wanted pharmaceutical approval would have to do its own tightly-controlled RCT with its own product.

As for showing that the products are good for the public health as a whole, he is completely backwards. An RCT of a cessation experiment in a clinic could tell FDA nothing about the public impact of e-cigarettes, which they are extremely interested in, while a good survey could. Again, Siegel could have learned this if he had only asked for advice and critical review.

Third, a randomized study, despite some limitations, is the only design that can address the problem of inherent differences between smokers who choose to use different products. It is clearly not the only study design that needs to be used, but it is one of the designs that is needed. No one study will provide the answer to this research question. Multiple studies are needed that use multiple designs.

Um, no. An RCT is almost perfectly designed to not tell us about differences among people. The whole point of that approach it is to pretend that everyone is interchangeable and see what happens when you act upon them. Yes, multiple studies are needed to answer all questions (duh!). But his proposed study did not look like it would answer any useful questions — but it would cost more than all those other useful studies combined. (Copy and paste remark about “ask for advice” here.)

Finally, the BSCiTS study was going to avoid many of the limitations of the RCT approach by not assigning a particular product to each smoker. Instead, our plan was to give subjects a choice of multiple products, including not only cig-a-likes [sic] but also egos and a few more advanced products. We were planning to include different choices of nicotine level and flavorings as well. I doubt that any other completed or proposed RCT on electronic cigarettes would have used such an approach. Our goal would be to simulate the real-life situation. Again, I doubt any other RCT study will do that.

I noted in the previous post that he seemed to be making this up as he went along. Each new communication he received about the rigidity of RCTs resulted in more claims of what he was going to do. (At one point he claimed that there was no chance an IRB would allow him to use open systems, so he was not proposing it. Here he seems to claim it was the plan to do so. Or perhaps he is claiming he can test “advanced products” but without them being open, which makes little sense. Maybe he wanted to use big batteries with pre-filled cartomizers. But that would severely limit the liquid options, unless the researchers will planning on filling tens of thousands of cartomizers themselves. Which would mean they were refillable, and thus open. I am not sure he understood any of this.) Notice also that he claimed above that this study would support a smoking cessation claims — which must be about a single product, not the category — but here he talks about trying to test the whole category at once. He is just making it up.

And he still misses the most important scientific points I tried to explain. No matter what regimen is used in an RCT, it is still just one regimen — particular choices (no matter how many), particular instructions, etc. — and it is still a clinical intervention that does not resemble the real world of e-cigarette adoption. If the fundamental problem with an RCT were the narrowness of the product options in the treatment arm, then maybe it would be possible to do something about it. But it is not. E-cigarettes are not a medical treatment. An RCT would tell us almost nothing about their real benefits.

I think we all learned something today. If you want to undertake a really ambitious project, take baby steps and ask for advice; do not just commit and then ask for support. And after you ask, listen. Don’t bristle at friendly advice as if it were a threat to your integrity, let alone lash out and defame those who offer it. If you do, you might run out of friends. Don’t mistake writing a lot for reading a lot. And finally, when you find yourself deep in a hole of your own digging, see if there is anyone who will swap you a ladder for your shovel.