Comparative Effectiveness & A Lack Of Comparisons
7 CommentsBy Ed Silverman // March 12th, 2010 // 8:41 am
An analysis of comparative effectiveness studies finds that only 32 percent, which were published in leading medical journals, compared the effectiveness of existing drugs. Instead, few compared meds with non-pharmacologic interventions, while only 19 percent examined safety and 2 percent eyed cost effectiveness, according to the study in this week’s Journal of the American Medical Association (here is the abstract).
The analysis examined 328 studies, of which 104 were CE studies, and discoved that only 11 percent compared existing drugs with existing therapies not involving drugs, such as surgery or lifestyle changes. Also worth noting - 87 percent were funded by nonprofit or government institutions, not drugmakers. And of the 212 randomized trials analyzed, 46 percent used an active comparator (active therapy), and the rest used an inactive control (such as a placebo). Active-comparator trials were less likely (44 percent) than trials with inactive controls (66 percent) to report positive results.
Of course, this is all the more interesting because, as the authors note, Congress recently passed legislation to provide more than $1 billion to support CE studies. “Our findings suggest government and noncommercial support should be increased for studies involving nonpharmacologic therapies, for studies comparing different therapeutic strategies, and for studies focusing on the comparative safety and cost of different therapies,” they write, adding that “regulatory agencies like the FDA (should) require active-comparator trials for medication approval whenever feasible.”
But if changes in diet or exercise might lead to a better outcome than a medication, how likely is it that a drugmaker would pursue such a study?
Lisa Van Syckel
Not Likely!
John
Ed, it seems to me that placebo and not “lifestyle changes” is in fact the most scientifically relevant control group.
The purpose of a clinical trial is to predict what will happen in the general population “with or without” drug treatment. The general population (including clinical trial participants) is highly exposed to messages about the importance of healthy lifestyles, but compliance is low. A statin clinical trial in which the control group is encouraged to eat a vegetarian diet and run 5 miles a day will undoubtedly show the latter treatment to be more effective, but this is comparing a drug treatment to a theoretical state that is not representative of what happens in the field. If the statin is not approved, the average American will not begin running 5 miles a day.
So I think it is very important that 1) doctors and the government redouble their efforts to encourage healthy lifestyles, and 2) clinical trials should continue to use placebo groups that are reflective of the actual current habits of the American public.
Mike Wokasch
The issues: 1) Congress obviously had no idea what it will take, cost-wise ($1 billion doesn’t get you much in clinical trials), to actually do the number and type of trials they are looking for. 2)scientists and clinicians have long held you need two well controlled trials to draw conclusions and therefore, a single large study won’t get the traction some might think 3) many of those who espouse quick and dirty comparative trials or meta-analysis to establish comparative effectiveness would never accept such data unless it supports their bias 4) just because comparative effectiveness studies will be costly, challenging to execute, and the results will be difficult to interpret doesn’t mean the market and the FDA won’t continue to expect them. 5) pharmaceutical and biotech companies expecting premium pricing on their newly launched products better be prepared to meet this market expectation, especially as healthcare reform (however it turns out) drives the market to become increasingly cost conscious
bariatric surgery
A statin clinical trial in which the control group is encouraged to eat a vegetarian diet and run 5 miles a day will undoubtedly show the latter treatment to be more effective, but this is comparing a drug treatment to a theoretical state that is not representative of what happens in the field.The concern several issues which come under the section of the comparative Effectiveness.Clinical trials should continue to use placebo groups that are reflective of the actual current habits of the American public.
Michael Kirsch, M.D.
Comparative effectiveness research (CER) will be very difficult to get airborne. I support the concept unreservedly. However, this process is designed to pick winners and losers and the stakeholders will fight to preserve their incomes and their industries. If CER determines, for example, that cardiac stents are overutilized, the their will be a volcanic reaction from the device folks and cardiologists claiming that the research is flawed. The same would be true of my own field of gastroenterology. Conversely, if CER supports a certain medical practice or procedure, the the proponents will hail the research as groundbreaking. http://www.MDWhistleblower.blogspot.com
pharmavet
Dr. Kirsch is on the right track. To be precise, in the industry we use the term “non-inferiority” studies (formerly called “equivalence” trials) instead of comparative effectiveness trials (which is a lay term). Statistically speaking, this are much different than placebo-controlled trials because of the obligatory way the null hypotheses must be set up. Without going into detail, the demands of hypothesis testing require much larger sample sizes and more expensive trials for equivalence studies.
It’s also a matter of strategy. FDA does not require comparative efficacy trials for approval. If I, as a Project Manager can spend $20,000,000 for a placebo-controlled trial that wins approval vs $100,000,000 on a comparative efficacy trial that gets me no more mileage (except perhaps with oncology drugs) and may actually hurt my drug if shown to be inferior to the comparator, that one is a no brainer.
The government is now in the tens of trillions of dollars spending mode. Certainly they can find a few billion for these types of studies. To paraphrase the late Senator Everett Dirksen: “A few billion here and a few billion there, pretty soon you’re talking real money”.
patrons99
There is a serious methodologic flaw with increasing usage of active comparator trials and the non-inferiority margin as the basis for approval. This methodology tacitly assumes that the active comparator is a “safe and effective” drug that should have been approved in the first place. Therein lies the flaw and it is insidious. Many of the approved drugs and biologicals in the marketplace should never have been approved. Many dangerous drugs are simply left in the marketplace, until the body count becomes great enough to demand their withdrawal.
New biological approvals are often granted expedited market approvals under the PDUFA with the non-inferiority margin and an active comparator as the choice of control. Gardasil’s market approval is a case in point, where aluminum adjuvants (hardly inactive controls) were used as the control group. Several of the pivotal studies upon which FDA based Gardasil’s approval actually used hepatitis B vaccine as the control arm. Surrogate efficacy endpoints, levels of antibodies instead of numbers of infections, were used.
Pfizer’s Prevnar 13 (acquired from Wyeth) is another case in point. Wyeth studied whether levels of antibodies that fight infection were equivalent, or “non-inferior,” between Prevnar and Prevnar 13, Reuters writes.
“For three of the types, “the non-inferiority criterion was not met” for some people in the study, FDA reviewers wrote…”. Apparently, this “FDA-speak” for saying that all that was really required for Prevnar 13’s approval was a showing of “inferiority”, arguably a very easy showing to make! As for safety, all that was required for Prevnar 13’s approval was a showing that Prevnar 13 was no more acutely toxic than Prevnar.
http://www.pharmalot.com/2009/11/pfizer-vaccine-missed-some-goals-fda-review/
http://www.pharmalot.com/2008/12/overseas-drug-trials-come-under-scrutiny/
http://www.pharmalot.com/2009/11/pfizer-vaccine-banned-by-dutch-over-infant-deaths/
http://www.pharmalot.com/2008/11/india-halts-wyeth-vaccine-trial-over-infant-death/