The two-day advisory committee meeting this week to review the safety of GlaxoSmithKline’s Avandia diabetes pill is about much more than any one drug or even the larger issue of drug safety - it’s also a referendum on the ability of a much-maligned institution to fulfill its mission as a public health agency. To examine what’s at stake, we asked Daniel Carpenter, the Allie S. Freed professor of government at Harvard University, who has just written “Reputation and Power: Organizational Image and Pharmaceutical Regulation at the FDA,” to share his views. Here is his guest column.

For the FDA, the Avandia decision is likely to be seen as a hallmark case that will determine whether the agency breaks from the Bush Administration past. For this reason, this week’s advisory committee meeting is the single most important event in the young tenure of the new FDA Commissioners, Margaret Hamburg and Joshua Sharfstein. The agency is under considerable pressure to more strictly regulate Avandia and a number of experts are pushing to have the drug removed altogether from the market. The Nissen meta-analysis of 2007 (“Nissen One”) and the update of 2010 (“Nissen Two”) are well known, as is David Graham’s newest study published in JAMA. Meanwhile, GlaxoSmithKline has sponsored a number of randomized trials in which the safety profile of Avandia appears better.

I tend to view these kinds of issues in light of the larger social canvas on which pharmaceutical regulation plays out. In my book, I try to understand the FDA as a social creature, a vessel in which society, companies, members of Congress, the media, and patients place their trust or distrust, at which they direct their fear or praise. Reputation has a lot of dimensions – among them performative (did the agency get it right?), technical (does the agency have the know-how and methods to get it right?), procedural (does the FDA follow accepted procedures suggested by law and science?) and moral (does the agency show compassion to those affected by its decisions? is it captured or inappropriately influenced?).

The agency’s reputation for consumer protection and scientific vigilance, especially before the 1990s, supported three kinds of power – directive, gatekeeping and conceptual. Directive power is the ability to order a company or a researcher to do something. Gatekeeping power comes from the FDA’s stature as veto player in the game of international drug development. Conceptual power is the power to shape the methods, vocabularies and notions that we use when we study and talk about drugs. All three kinds of power, and a number of dimensions of the agency’s reputation, are at stake here.

Let’s look at the following four dimensions of the Avandia controversy:
1. The Battle of Methods
2. Marciniak’s report and How Regulation Empowers Replication
3. Reputation, Power and Transparency
4. The Ethics of a New Trial

1. The War over Method. There is a long and ongoing struggle in the human and medical sciences over the value of observational data versus the value of experimental data, and the Avandia debate is one tumult in a much larger war. The Graham analysis is based upon Medicare beneficiaries and, speaking roughly, compares those who take Avandia to those who take Actos. The Nissen analyses examined randomized-controlled trials, but a meta-analysis is, in part, observational because it is not prospectively designed – in other words, Nissen did not plan ahead of time to run dozens of randomized trials, let the trials run, and then pool the results. Put differently, the trials that Nissen pooled were not intended to be analyzed together. This is not to say that they can’t be pooled and analyzed; it is just that non-trivial assumptions about comparability of treatment and control groups need to be made in order to support inferences from a meta-analysis of RCTs.

The all-too-easy and simplistic conclusion is to take the side of randomized clinical trials, as in theory there can be no chance for confounding variables to invalidate causal inferences (the internal validity issue). But as most people know, the issue is far more complicated than that. For one, aggregated findings from observational data can be quite powerful. I tell my students what is arguably the most important public health finding of the past century – that cigarette smoking is a major contributor to lung cancer and heart disease – came entirely, wholly, from observational studies. In other words, never did any scientist set out and randomize people to the smoking group versus the control group in a long-term randomized study (differential attrition would have made such a study impossible).

Another complicating factor is that a lot of the randomized clinical trials don’t get at certain questions. The main question here is not Avandia’s safety per se, but its relative safety-benefit profile compared to other new-class oral anti-diabetic treatments, most obviously Actos. Some of the most recent trials are randomized and reasonably well-powered, but they do not compare Avandia to Actos.

A third problem is what one might call the social imbalance of modern clinical experiment. RCTs have become the dominant criterion of evidence in medicine worldwide, but are expensive, and their prohibitive costs mean that the companies that can afford to run them have some asymmetric advantages over organizations and individuals (Nissen, Graham and the FDA epidemiologists, other skeptics of the drug) lacking the financial resources to conduct RCTs. This asymmetry may be an unintended consequence of American pharmaceutical regulations, namely shifting the balance of power to those with the resources to conduct the most costly and rigorous tests.

The deeper implication of this battle for the FDA is that the eventual decision will end up empowering one set of actors, perhaps at the expense of another. This is one reason why the TIDE trial, an RCT the FDA urged Glaxo to run and is designed to examine cardiovascular risks but is being criticized as unethical, is so important. If the FDA withdraws Avandia and goes further and closes down the ongoing TIDE trial, it will be a methodological victory for observational methods in pharmaceoepidemiology, not just at the FDA but in the global scientific community. Within the FDA and among its audiences, an Avandia withdrawal will be perceived as a reputational boost for the Office of Drug Safety (and Graham).

If on the other hand absolutely nothing is done and the status quo is upheld – Avandia remains on the market as is, with the TIDE trial continuing – it will be interpreted as a blow for the Office of Drug Safety, a partial thumb-off-the-nose to Senator Grassley and the Senate Finance Committee, and as a partial repudiation for Steven Nissen.

2. Marciniak’s Report: Another problem with clinical trials flows from the social imbalance issue. Clinical trials have to be designed and analyzed rigorously, and if they are not, then their informational value is in doubt. Since companies run and analyze the clinical trials, the credibility of the trial information is shaped by the credibility of the company’s clinical trial operation. It’s for this reason that FDA reviewer Thomas Marciniak’s memorandum severely criticizing the Record trial is so important. He thinks that the RECORD trial was badly designed and badly analyzed, and in a particularly striking (perhaps damning) remark, he thinks the flaws amount to “serious flaws with trial conduct.” It’s clear that Marciniak thinks these problems are more than methodological; he remarks that there are “problems with the study conduct that also limit any reassurances that RECORD can provide regarding the CV safety of rosiglitazone” (emphasis added).

This report could change the Avandia game entirely. For one, if Marciniak’s view is widely held at the FDA (or it’s widely shared among its advisory committee), then we no longer have a case in which the clinical trials point one way and the observational evidence points the other way. More important, because Marciniak is a CDER reviewer, we no longer have a case in which the FDA’s ‘trialists’ are entirely at odds with its ‘observationalists.’ In other words, Marciniak’s memo may create doubt among many CDER medical reviewers who favor randomized clinical trials but who may come to doubt GSK’s credibility in running them. That kind of loss of trust can be severely destructive for a company. As far as I can tell, Marciniak’s voice is a new one in this debate, and combined with the Senate Finance Committee’s concerns and the New England Journal of Medicine’s concerns about GSK’s conduct in running clinical trials, his memo could spell serious trouble not only for Avandia but also for GSK.

The final issue to keep in mind is that the Marciniak memo would not be possible in a world where the FDA did not have all of the trial data, including the case report files for each and every one of the patients in the RECORD trial. (In a revealing note to one of his slides, Marciniak refer to these case report forms as “the real RECORD” [slide 3, see also slide 20].) Since at least the 1950s, FDA rules have required NDA sponsors to turn over all of the individual case reports for data in their clinical studies, not just the statistical summaries but the reports that permit re-analysis of how each case was coded. This was neither a natural nor inevitable development; for many years, the British equivalent of the FDA required only statistical summaries. The EMEA now relies heavily upon case report forms, and if the Marciniak critique holds up, even partially, it will stand as a powerful demonstration of the value of the FDA’s rigorous approach.

3. Reputation and Power. That the FDA’s reputation has withered in recent years will be no surprise to readers here. I document the evidence for this development in chapter 12 of Reputation and Power, but I also reflect on how the FDA’s declining image shows us, in some sense, how deeply and broadly respected the agency was in the period from the 1940s through the 1980s. The sources of this decline are numerous; there are concerns that the FDA is dominated by the companies it regulates, that the user-fee program has generated a form of capture, or that the agency got caught up in the general deregulatory urge that has swept our country for the last 30 years (starting with Jimmy Carter, in fact).

More specifically, the decline in the FDA’s reputation became acutely visible in the Bush II presidency. There were a number of laments about the quality of the Bush Administration’s appointments to the agency. Yet where the FDA got into the most trouble in the Bush years was in its perceived lack of transparency. A lot of controversial decisions were made, but controversial decisions have been made by the FDA for a half-century and more. What was different was the impression that the usual procedures were not followed, and that dissent within the agency was squelched. It was the agency’s technical and procedural reputation that suffered as much as its performative reputation.

For instance, the infamous “Plan B” decision was not so much the triumph of politics over science; it was, but it was more than that. It was also the toppling of long-established norms of sub-delegation of drug approval decisions to the career civil servants at the FDA. When the Bush Administration ‘s FDA refused to permit an over-the-counter pill, they overruled civil servants who by federal code have some authority of these matters, also over-ruled an advisory committee recommendation, and did so without a publicly stated or transparent justification.

Hamburg and Sharfstein have a new transparency initiative, and in some ways the Avandia case will be its most severe test.The slides page for the meeting reveals a vigorous internal debate within CDER about methods, about the value of the RECORD trial, and about the risks of Avandia versus Actos. The agency has allowed Graham to publish his epidemiological study in JAMA, and it will hear his (and Nissen’s) testimony in the case. (The agency’s failure to allow drug safety officials to testify at advisory committee hearings was noted with disapproval in a GAO report that focused on Arava [leflunomide]).

My sense is that agency leaders want to demonstrate a commitment to listening (even incorporating) suggestions from numerous parties, and to make sure that every argument and concern is laid fully on the table. Whatever action is taken will need to be backed by data and quite possibly by a report that offers some rationale for the eventual action taken. The FDA does not always do this. Nor would I say that the FDA always needs to do this, as over-explanation of decisions could delay many important public health decisions and, in some cases, lead to confusion. Yet in this case I foresee a more elaborate justification for any decision rendered.

I don’t have a precise or credible forecast to offer, in part because the Marciniak memo complicates things, and in part because some new evidence may surface after I write this. I think FDA leadership knows that soft action on Avandia is likely to lead to some censure of the FDA’s leadership from Congress, not just Grassley but also DeLauro, Waxman and others. Yet it is still possible that some sort of intermediate action might be taken here, in part because the extremes just discussed are both (a) difficult from the standpoint of agency reputation and (b) irreversible at some level.

One possibility is that Avandia is temporarily withdrawn from the market but the TIDE trial is allowed to continue (with an IND exemption issued for this purpose). Another possibility is leaving Avandia on the market but only as a “second-line therapy,” namely a treatment of last resort. This would leave Avandia with a presence that GSK could try to enlarge through marketing, but doctors would be reluctant to prescribe it, not least because they would court more tort liability in doing so.

It’s an oversimplification, but my approach to agencies like the FDA is that, if their officials can do so, they like to take actions that do not box them in with respect to future actions. Let me be clear that I am not saying that the FDA wants to fudge the issue; preserving space for future discretion can be an optimal strategy under situations of high uncertainty, repeated interaction and complexity. Indeed scholars in game theory and decision theory have argued that ambiguity has important optimality properties in dealing with complex problems.

4. Is there an ethical issue in running an RCT with Avandia now? Let me finally weigh in briefly on the question of whether continuation of the TIDE trial is unethical. It would clearly be unethical and inappropriate to allow a clinical trial for a drug with strong and abundant evidence against its safety, and where there were acknowledged safe alternatives. But the point of running a trial is to reduce uncertainty and come up with better estimates of an average treatment effect. In other words, uncertainty is the whole point, and even if we placed a large amount of faith in the Nissen One study, it was a meta-analysis and I would not regard it, alone, as dispositive.

It is here where I may disagree with David Graham. Graham is undoubtedly controversial, as just about anyone in his position is going to court controversy. Yet Graham has made some ‘bank-shot’ calls with FDA-approved drugs in the last few decades. He has sounded early alarms on a number of
drugs that were later withdrawn in the United States and worldwide. I respect and value his point of view.

That said, I don’t think I agree with him that the TIDE trial should necessarily be stopped (though again Marciniak’s memo makes me wonder). I think a clinical trial of Drug A is clearly unethical only when there is reasonable certainty that drug A is unsafe, that there are no alternatives (a “Drug B”) that are known with high certainty to be safer, and the potential uses of Drug A are not sufficient.

If the evidence from a directly comparative trial could plausibly overturn previous studies that were not dispositive, then you have a strong argument that (a) the human benefit from getting the information outweighs the risk to patients in the trial arm who take Drug A, and (b) that the risk to Drug A patients is itself subject to considerable uncertainty (which is to say that we may not be able to make concrete, non-contingent ethical judgments about that risk).

Given the evidence available at the outset of the TIDE trial, it seems hard to characterize the evidence against Avandia (and in favor of Actos) as so overwhelming that it makes a direct, randomized-clinical comparison unethical. Remember that the TIDE trial was started after the Nissen One study, not after the more recent studies (Nissen Two and Graham in JAMA) that show additional worries for Avandia.

So the ethical implications of an informative clinical trial should be considered here. But here’s the rub: Marciniak’s memo casts doubt on GSK’s clinical trial operations. So too, other recent reports and laments, voiced by the Senate Finance Committee and the NEJM editors, also raise doubts about GSK’s clinical trial practices. The informational benefits of a comparative clinical trial are important, but they might be compromised and substantially reduced if society can’t principally trust the methods and results. On top of that, the TIDE trial has had difficulty enrolling patients, and the FDA might judge that its potential contributions to the debate over Avandia’s safety are not likely to be material.

The politics of Avandia are multidimensional. They involve struggles over scientific method, a battle between two drugs (and their associated sponsors) vying for the same market, and continuing global tilt about drug safety and efficacy – who gets to decide it, and how important it is. Ultimately, the politics of Avandia involve struggles over whom to believe. For this reason, there is almost no way that the FDA’s leadership can avoid sending strong signals with its eventual decision.

Avandia pic - Getty Images