Mike Siegel inappropriately blames the failure of his ill-advised research plans on others

Recently Michael Siegel announced a crowdfunding initiative toward a planned $4.5 million budget to do a randomized controlled trial (RCT) of switching to e-cigarettes. The Consumer Advocates for Smoke-free Alternatives Association (CASAA) came out against this and recommended that its constituents not donate (you can read the statement here). This was based on (a) the assessment that an RCT is simply the wrong method for measuring the phenomenon of interest (which I detailed more here and here), (b) the assessment that Siegel had no basis for his claim that the results of this study would substantially influence FDA or other critical actors, and most importantly (in my opinion), (c) it was far too expensive to fund via crowdsourcing. If even only a few percent of the total budget were collected from the e-cigarette community, it would take away enough funding to crowd out every other advocacy, education, and research the community wants to support.

[Note that I am an officer of CASAA, as anyone reading this presumably knows, and I was obviously involved in what is described above. However, I am writing this as a personal statement, not speaking for CASAA, and indeed covering some points and information that CASAA did not consider in anything it said.]

What CASAA did not mention, but I wrote to Siegel about (speaking just for myself) is that what he was doing also appeared to violate the spirit, and perhaps the letter, of rules of public research ethics. It is extremely dicey to ask a threatened community for research funds. E-cigarette users are terribly worried about the political climate, and thus are likely to respond to any request for funds that comes with a promise of doing something about the regulatory threats to their ability to choose to vape. Thus anyone asking for such funds needs to be very careful to make sure they are not promising too much. CASAA is very careful about what it promises and very careful to communicate what it does and plans to do, even though it seeks much more modest funding from the community. The ethical considerations are even greater when we are talking about university or university-style research, which comes with a strict set of ethical rules. The one time CASAA sought community funding for a scholarly research project (the Burstyn study, whose budget was 1/2 of 1% of that for Siegel’s project), we presented a precise description of the study we were going to fund, exactly how much money was needed, and a realistic assessment of how important it was (which was “very”, and we were right).

By contrast, Siegel presented only a hand-waving description of what he planned to do. When confronted with concerns with potentially problematic details of the (unspecified) protocol he or his staff always (to my knowledge) responded with some equivalent of “oh, we will take care of that — don’t worry.” This is the behavior of a shady corporate consultant, immediately responding to every question with reassurance in spite of apparently never having thought about it before, not of a careful researcher. I am certainly not calling either Siegel or his staff shady, but when your behavior looks like that of someone who is shady, it should be a red flag.

Even worse, Siegel said that if they did not raise the whole budget, they would do “something” else with the funding, some other research that was even more vaguely described. But this should not have been presented as a vague fallback because there was never any remotely realistic chance his plan would be fully funded. Siegel ruled out seeking funds from the two sources that could have funded it, the federal government and the major tobacco companies. That left the rest of the e-cigarette industry, who do not have nearly enough net equity or cash to support such an enormous project (even if they agreed it were wise), and the consumers whose total donations to all community funded causes over five years of activism would cover only about 5% of the budget.

Indeed, I told Siegel that the “consultant” metaphor was charitable because what it really smelled like was the behavior of a scammer. Of course, I made it clear that neither I nor presumably anyone else actually thought that he might be embezzling money or anything like that. But if your behavior looks like that of a scammer, especially when you are a public researcher, then you are probably skirting the boundaries of ethics (at best). A scammer would promise that the money would solve some major problem that the mark desperately wants to believe can be solved with a magic bullet (which Siegel basically did, though he had no reasonable basis for giving such assurances), would respond to every stated doubt immediately and with complete confidence but nothing concrete, and would say “give me whatever you can — if it is not enough, I will still do something good with it.” As I said, no one thought he was scamming under the guise of funding research (though there is a precedent for that happening in the e-cigarette community, so that is not without a touchstone), but if it feels like it, then something is ethically wrong. Indeed, I described the situation to researcher colleagues outside the e-cigarette community and they agreed. None of us could put our finger on a per se ethical violation (though depending on the exact rules at Boston University, there might be one), but we all agreed it felt very, very sketchy.

Related to this is the problem that the project is already heavily staffed. I suggested to Siegel that this seems to mean (I might be wrong, but he did not deny it) that any donations coming in would be quickly spent on staff, even if the project is destined to fall far short of its funding goal. (It could be that his department was paying for his staff until some funding goal was reached, though there was no indication of this, and department chairs are not usually big fans of spending their own money when they could be spending someone else’s.) So if, say, the project raised $200K from non-major companies and the community — a huge sum by their standards — it would have been enough to do a very nice observational research project (e.g., the first ideally-designed representative survey), but most of it would likely have already been spent on administration before that could happen.

I further pointed out to Siegel, in case he was not aware, that there are those (myself not among them, though I understand their motivation) who are opinion leaders in the e-cigarette community and who have very strong negative feelings toward him for his support of paternalistic public health policies, support of anti-smoking restrictions, and opposition to tobacco harm reduction (THR) via the use of smokeless tobacco. Thus if he collected enough money from the community to be scandal-worthy and then could not do anything big with it because it was already spent, there are those who could turn it into a scandal that might hurt him.

With all of the above in mind, I urged him to abandon the hopeless and misguided plan to do an RCT and instead create an observational protocol that could be done with the funding he could reasonably hope to get and, indeed, would be a scientifically better measure of the effects of e-cigarettes on smoking behavior than an RCT (for the reasons explained in the above-linked blogs). I even offered to help him with it. But whether or not he did that, I urged him to immediately end the crowdfunding based on the ethical and optics concerns I cited.

All of the above except the public statements were in a letter to him where I made clear (1) I was writing to him personally as a fellow independent researcher and not on behalf of CASAA, and (2) I was intentionally not sharing the letter with others. If he had complied with the request to end the crowdfunding, I never would have said a word to anyone else about the dubious ethics. Indeed, if he had just responded to me at all, offering some rebuttal, I might have sat on my hands about it (though I am not sure — I have very strong feelings about research ethics). So why am I writing this? Because of how he did respond, in this blog post (indented text is quoted from that).

In that post, he announced that he was discontinuing crowdfunding of the project — great! — and that he was returning donations that had already been made, which is more than I was calling for, but obviously fine. Presumably the latter was because he also announced he was cancelling the proposed study. That seems like a reasonable concession to reality and a good choice given the fundamental inappropriateness of an RCT in this context. He could have announced plans to design and seek funding for a more appropriate study instead but did not — his choice, of course. He noted that his group at Boston University had spent more than a year planning this (which is frankly troubling in itself given the vagueness of the plan and the apparent failure to think through the realities and ethics of funding), and made the expected reasonable-sounding statements about believing it would have been great and thanking people for support.

Had he stopped there, I would be praising him for recognizing the realities and for making a good effort which, like the majority of good efforts in the world, just was not going to happen. But he did not, and the problems began when he decided to “explain our reasons for discontinuing” the project. Notably absent was any recognition that the study design was a poor choice from a scientific perspective or that there was never any hope of getting enough funding. Nor was there any expression of concern about trying to collect huge sums from a community with much greater spending priorities.

He began with “divisiveness within the e-cigarette community”:

we discovered that there is great divisiveness within the e-cigarette community regarding the role of research—including what type of research should be done, who should do it, and who should pay for it. We welcomed feedback and suggestions about creating an optimal study design for BSCiTS [the working title of the study], and we sought to be responsive to concerns about study methodology. However, the tone of feedback directed at our team quickly became hostile. Our integrity as researchers was questioned, and criticism quickly shifted from concerns about study methods to personal attacks.

Let’s break that down. I am aware of no substantive divisiveness within the community over the assessment that this study was a bad use of limited funding, which is ultimately the crux of the issue. When CASAA made our assessment and reluctantly decided to explicitly recommend against donating, we expected quite a bit of pushback. The e-cigarette consumer community is, like most communities, ready to argue about most anything, and there are people who make a hobby of trolling attacks on anything CASAA does. But there was nothing but support. I have never seen such a lack of divisiveness in this community. No other opinion leader in the community came out with an opposing view. Various individuals questioned some of what we were saying, of course, but upon further explanation ended up saying “yes, you convinced me.” Now obviously there are some people out there who were not convinced — there are tens of thousands of activists vapers after all. But there was basically no one speaking up in opposition to our recommendation. It was just too clearly right.

I have already addressed the point about their being “responsive to concerns”. That came across as making it up as they went along, saying whatever people wanted to hear. That perception may be harsher than the reality deserves, of course. But the answers to those queries should have already been given in the initial communications, given that year of planning. Random new promises — like saying they will be testing lung function — popped up on social media without having been part of the published plan. If they really were not making this up as they went along, it should have not been flowing out so quickly and nonsystematically.

Most important, I am aware of no hostile feedback that anyone directed at Siegel. If there was any, it was not coming from those of us who led the response to his fundraising. CASAA was extremely polite and made clear our reluctance to be negative about someone who is usually a friend of the community. My blog posts pointed out the scientific error of pursuing RCTs in this context, a scientific analysis that was not the least bit personal or hostile. I was firm but completely polite in my letter, and it was all about seeking his recognition of my concerns. Perhaps someone actually did question his integrity, but I saw no evidence of that. I am guessing he is referring to my letter. But when I wrote to him, privately, expressing concerns that something smells bad and suggesting he stop doing it because of that, I was appealing to his integrity, expressing my faith in it, not questioning it.

BSCiTS was sparking further divisiveness. Our campaign was only public for a week, but in that short period of time it became evident it was exacerbating other, deeper fissures and ideological differences within the e-cigarette community. We were disturbed to find one organization in particular actively undermining our campaign efforts. It became apparent there is disagreement in the e-cigarette community about the need for rigorous research of the caliber that the FDA will require.

There were no apparent fissures as I noted. This is self-serving tripe. Yes, he found that CASAA (who he apparently did not have the nerve to mention by name) was trying to keep him from depleting the community’s very limited resources. Calling that “undermining” is a misleadingly inflammatory way to put it. CASAA openly stood up and explained our reasons for that to the community, letting individuals decide whether we were right or not. We did nothing sneaky, as implied by Siegel’s wording. It is “undermining” only to someone who assumes he is entitled.

I am also aware of no disagreement in the community about the need for rigorous research that the FDA will respond to. CASAA’s point was that what Siegel was ostensibly proposing did not constitute such research in any way, and would not accomplish what Siegel claimed. I am aware of no one in the community (Siegel is not part of the community) who disagreed with that assessment.

Indeed, CASAA privately communicated about FDA with Siegel. He apparently did not realize that navigating the FDA labyrinth in terms of new product approvals is not simply a matter of providing any old study. It is a massive undertaking with an almost unending set of hoops to jump through. We specifically asked Dr. Siegel whether he was working with experts who are knowledgeable about the information the FDA would consider acceptable and relevant and whether he had consulted with FDA Center for Tobacco Products scientists to get some assurance that they would pay attention to this. He did not respond to our questions. But the fact that he now asserts that we were questioning whether FDA-relevant research was needed, when what we actually did was ask what made him think that his was FDA-relevant research, is clearly disingenuous.

Initial reception from e-cigarette industry was not positive enough. Though we received much encouragement to develop and launch the BSCiTS campaign, the response from industry to our funding appeal was lukewarm. We are not confident that BSCiTS could receive enough financial support from independent e-cigarette companies to fund a rigorous study design.

I am not sure who he received encouragement from. He did not talk to me or, I am pretty sure, any of the other expert researchers who are close to e-cigarettes. He did not talk to very many people in the industry, because I am sure I would have heard about it if he did.

But the response to the funding appeal was entirely predictable. It took me about one minute to figure out that he could not possibly raise anywhere close to that sum from the industry (outside the majors, who he refuses to accept funding from). If he had asked me for input any time during his year of planning, rather than just springing this on the community with a request for personal donations, I would have told him so.

Pressure to compromise study objectivity. Finally, and most importantly, we received pressure from a major e-cigarette advocacy group to alter our proposed study design in order to produce a more positive outcome.

Now this is where I really call complete and total bullshit.

He is obviously talking about CASAA, but CASAA is not an e-cigarette advocacy group. CASAA is a consumer group devoted to promoting tobacco harm reduction. A large part of what we do is advocating to protect consumer access to e-cigarettes, yes. But notice how he tries to spin this as if it were some corporate influence.

Siegel is notorious for being part of the cabal that blames evil corporate influences for never-specified nefarious impacts on research and policy. Funny how that does not seem to bother him when he wants to fund his own research with corporate donations. But even setting that aside, he is accusing CASAA — a tiny-budget consumer group whose influence comes entirely from the trust we have earned in the community and who could not possibly influence what he chose to do (our advice to individual consumers to not donate is not why he could not fund this) — of threatening his integrity. Oh, please.

Attempting to exert influence on independent researchers to produce more favorable results is deeply unethical and extremely problematic.

Presumably he is referring to the fact that I pointed out in my scientific analysis that an RCT cessation trial was basically guaranteed to produce results that, contrary to what we know, would be interpreted as “e-cigarettes fail!!!!” because people do not understand the limits of the methodology. I further pointed out that an RCT like the one he is proposing (though I was writing in generic terms, not about his project) would be a great tactic by those who wish to cast doubt on the benefits of e-cigarettes.

Now, if someone did such a study, it would show what it showed, and I would be quite willing to explain why the naive interpretation of “failed!!!!” was incorrect (as I did about the Bullen study and about NRT trials — see the linked posts), and to extract what information was genuinely available from it. But let us not forget how Siegel was selling this study to the desperate people whose money he was taking. He was implicitly — almost explicitly — promising that it would show the true value of e-cigarettes (which I explained it would not) and would influence FDA’s decisions favorably (which it could not do no matter what it showed — it is irrelevant to the main decisions that are currently on the table — and would not be helpful given the misleadingly tepid results that are inevitable). It was he who was promising favorable results that he could not possibly deliver.

But I should emphasize that my point about how the results would be tepid and misinterpreted was the last point in my analysis. I made that point only after I had already explained why an RCT did not measure what anyone really wants to know.

My position, and that of CASAA, was that there are study methods available that give the true measure of e-cigarettes. If he really wanted to do something that was scientifically useful, he would do that. It just so happens that this would also be good for e-cigarette advocacy because the good science is on the side of e-cigarette advocacy. Siegel is desperately trying to construe this as some kind of conspiracy to cook the results. We proposed that he do a representative survey that would examine switching and quitting behavior, exactly as any expert in this area of research would (and as people like Robert West are already working on).

How exactly does that resemble his inflammatory charge of “exert influence on independent researchers”? Indeed, if he is so damn independent, why is he coming to the community to get money?

Moreover, this is hypocritical given the e-cigarette community’s rejection of biased research studies produced by tobacco companies and public health professionals alike.

WTF? We noted that his study is scientifically misguided and based on uninformed assumptions about regulators, and that it costs too much, and somehow he construes that as being at odds with rejecting blatantly political research?

We are unwilling to compromise our scientific integrity in order to conduct a study on behalf of the e-cigarette community.

Double WTF? Recognizing that a study design is misguided compromises his scientific integrity? Being ethical enough to not ask for money from a desperate population based on false promises compromises his scientific integrity? Doing some other research that any knowledgeable researcher would tell you is more useful compromises his scientific integrity?

To reiterate, in case it got lost here, if he actually had the funds to do the study he proposed, no one was going to try to stop him from doing it, even though the results would give ammunition to e-cigarette detractors (who could easily misinterpret them). But he was the one who claimed to be doing this study on behalf of the e-cigarette community in the first place.

We came to realize that the current climate within the e-cigarette community is not conducive to conducting objective behavioral research at the level of rigor upon which we insist. Ultimately, we have to place our own scientific integrity ahead of the desire to successfully secure research funding.

So what he is saying is basically: We proposed an unsuitable study design that we had not really thought through despite a year of planning. We inappropriately asked the community for money, and they balked at that. We figured out — as should have been obvious from the start — that we could not actually fund it at all. We did not consult with anyone who might have pointed out the various problems in our plan, but instead just sprung it on them. And then we quickly discovered that it was hopeless, which many people could have told us if we had only asked. So we are blaming our problems on people who want to compromise our integrity.

Yeah, you just keep telling yourself that.

[Update: more on this here.]