Industry Funded Clinical Trials & Biased Publication
14 CommentsBy Ed Silverman // August 3rd, 2010 // 9:50 am
A new study finds that clinical trials funded by the pharmaceutical industry are more likely to report a positive outcome, but less likely to be published shortly after completion than trials funded by other sources, such as the federal government, non-profit groups or academia. Unlike previous studies on this topic, the researchers say their study broadens the debate because they made a point of examining 546 trials that were registered on ClinicalTrials.gov and involved five widely used classes of meds for treating depression, high cholesterol, high blood pressure, heartburn and schizophrenia.
All of the trials occurred between 2000 and 2006. The study, which was published in The Annals of Internal Medicine, found 346 trials, or 63 percent, were primarily funded by industry, 74 or 14 percent were funded by government sources, and 126 or 23 percent by non-profits or other organizations. The upshot: Industry-funded trials were more likely to be Phase III or IV trials (88.7 percent); to use an active comparator in controlled trials (36.8 percent); to be multicenter (89 precent) and to enroll more participants. Overall, 362 trials or 66.3 percent had published results.
Meanwhile, industry-funded trials reported positive outcomes in 85.4 precent of publications, compared with 50 percent for government-funded trials and 71.9 percent for non-profit and other trials funded by non-government organizations. Trials funded by non-profits and other groups with industry contributions were also more likely to report positive outcomes than those without industry funding - 85 percent versus 61.2 percent. The rate of publication within 24 months of completion ranged from 32.4 percent among industry-funded trials to 56.2 percent among nonprofit or nonfederal organization–funded trials without industry contributions (read the abstract).
“This raises concerns that the findings may, in part, be related to biases in trials – the design or the way they are reported to the public,” Florence Bourgeois, a faculty member in the division of emergency medicine at Children’s Hospital in Boston, and one of the researchers, tell us. “We don’t know for sure, it may be other things that could sway results. It could be the questions that investigators choose to ask or the type of patient selected or how long the patient population is followed. All of these may impact results.
“The findings raise questions about whether the system is providing sufficient oversight,” she continues. “More oversight is needed and additional information is needed about the way the trials are designed. This would be critical in examining these factors. It would be helpful if study protocols were posted in ClinicalTrials.gov or comprehensive FDA review material of trials reviewed in the drug approval process for every medication were made available, although just providing study protocols may not be sufficient.”
A PhRMA spokeswoman send us a statement which, in part, says the trade group is striving for greater transparency and notes that the study “demonstrates one more way that patients and the public health can benefit from increased transparency: The study would not have been possible without the large volume of information about ongoing and completed industry-sponsored clinical trials provided through ClinicalTrials.gov and supported by PhRMA and its member companies.
“While our review of the study continues, it is important to note that the authors acknowledge that industry-funded trials tended to be for later stages in the lengthy drug development process. As the authors note, ‘Later-phase trials may be more likely to have positive outcomes, because there is more certainty about the drug’s efficacy and safety at this advanced stage in the drug-development cycle.’
Jim
I don’t think industry can win on this one. If the results had gone the other way, it would have been taken as evidence in support of the popular myth than “90% of drugs originate in academia”.
Statistics published in Nature Drug Discovery Reviews (volume 3 page 711)show that phase 3 clinical trials are 50% more likely to be successful that phase 2 trials. The results of this paper appear to be explicable in terms of co-linearity.
Cynic
Perhaps it is becuase the positive pharma trials get published, while the negative ones do not!
Doc
If the average physician had any idea about the degree of control that pharma marketing departments have over trial design, funding and execution - they would be shocked.
You can argue whether it is good or not all you want, but it is important to know that non-science/non-medical MBAs exert an unusual degree of power in the area of trial design.
David Miller
Academic studies are often not sized large enough to generate positive outcomes on clinically-marginal treatment differences. Bio-statisticians and the FDA are in this unholy alliance that says bigger trials are better and a p=0.0001 with a HR of 0.85 is a “better” outcome than a p=0.05 and a HR=0.50. Heck, I’ve even seen a prominent bio-statistician argue with a straight face that if the power of a trial is low going in, the outcome is sketchy regardless of the HR or p-value.
With the FDA mandating ever-larger trials, the law of large numbers explains why industry trials are more often successful than academic trials.
Nathan
Did it occur to anyone that the better outcomes might be due to the fact that we (in the pharma industry) are simply better at developing effective drugs than our counterparts in academia and non-profits? We have more resources at our disposal and far more experience. (this certainly does not seem to have occurred to the authors of this study)
David
One possible reason on why industry trials work is that their is much more profesionalism and diligence put into assessing whether the clinical hypothesis being tested is a reasonable one. The industry is much more likely to not invest millions of dollars in testing hypotheses with low liklihoods of success. It does not make business sense.
It drives me crazy that their is so much naivety in the academic world about how the industry that researches, develops, and supports drugs actually works. Instead of questioning the ethics of the industry sponsored trials, we should question the intelligence and diligence that is going into our tax payer sponsored trials! Maybe thats where our problem is.
Elmore
Industry sponsored pubs are done to spread good news to increase sales. Marketing rules. If they had big news that would increase market share they would announce it in an nanosecond. Most pubs are carefully crafted from the main message to the supporting themes, from the references to the target journals. This is a business. It employs thousands, or used to.
Jake
For all those concerned about the “conspiracy” by pharma to only publish good studies, don’t worry. You can take comfort in the fact that the FDAAA legislation requires that all Phase 2-4 clinical studies be listed on clinicaltrials.gov and the data by law must be posted after completion. Why would industry publish negative studies when you are now required to share the data anyway?
LF Velez
Spending time to write up a trial has costs in opportunities [you could be moving on to the next, potentially more successful or interesting, project] or money [outsourcing the writing to other members of your lab or to professional writers].
Few people get famous disproving hypotheses, and there have so far been few viable incentives to inspire people to spend time, energy, and resources on writing up trials that haven’t produced “positive” results.
pharmavet
David M., I have no problem with small, well designed academic trials similar to Phase II industry trials, as long as everyone is on the same page that they are useful for hypothesis testing. In fact, a well designed small trial should yield reliable enough data on magnitude of treatment effect and variability so that it should not be necessary to “overpower” a Phase III trial to show efficacy.
In my work, I always recommended that Phase III trials be powered at 90% with alpha et at 0.05, even if powering them at the “standard” 80% appeared to provide suffiecient margin against Type II error. My reasoning was that given the large expense of most Phase III trials, having the additional “insurance” provided by the extra statistical power seemed like a decent investment.
On the issue of clinically marginal differences, the example often used is a new antihypertensive drug. Let’s say that a small Phase II trial shows that your new drug only beats placebo by 2 mmHg. The decision then becomes as to whether to kill the drug based on a clinically insignificant drug-placebo difference, or go ahead with a highly powered Phase III trial that will show this marginal difference to be “statistically significant”. Logic would say to kill the drug, but the company may decide to go ahead with Phase III based on other attributes, i.e., safety profile, mechanism of action, etc.
David
I would interpret the findings of this report to show that industry runs higher-quality studies.
There are hundreds of millions of dollars at stake when a pharma company runs a phase III trial. There are teams of monitors, quality audits, and other steps to ensure proper study conduct. The company knows that FDA or EMA will critically examine the study for quality. Conduct of trials run by academic centers typically might be much looser. The fact that academic studies are not sufficiently powered also shows the lack of commitment to “doing it right.”
PS: I’m not the same David as the other illustrious commenters.
pharmavet
David, you are right. I’ve done a fair amount of business development and licensing on the pharma side. In many instances academicians come to us to license the latest and greatesr new drug based on a 12 patient, non-controlled, unblinded observational study. When we reject their idea based on inadequate data they get offended. They want to spend as little money as possible in hopes of a big pharma jackpot.
Christopher
Too true PV - my experience too - but you know what MD, PhDs can be like.
Ellen
No one seems to have made the connection to medical journals’ general lack of interest in, or at least a lower priority for, publishing negative trial results.