Journal Ads & Accurate Claims: Spielmans Explains
Yesterday, we wrote that a new study found the accuracy of ads for antidepressants and antipsychotics often make claims that can’t be verified and attempts to obtain data cited in the ads from the drugmakers were rarely successful. The study, which examined the accuracy of 69 ads that appeared in four widely read medical journals in 2005, was conducted by Glen Spielmans, an assistant psychology professor at Metropolitan State University in St. Paul, Minnesota, who previously conducted a study showing advertising for Lilly’s Cymbalta was skewered. We spoke with him about his reasons for doing the study and the take-away message. This is an excerpt…
Pharmalot: What made you look at this in the first place?
Spielmans: I was just reviewing drug ads for that period - 2004 and 2005 - because I was curious to see if they were accurate. The findings about ads for other kinds drugs advertising in journals showed that many were not accurate. I suspected the same thing would be the case (with antidepressants and antipsychotics). I thought it was strange that no one had looked at ads for that particular type of drug, given how widely used they are. So when I had some time, I had some students help me work on it.
Pharmalot: So why is this important?
Spielmans: Physicians frequently read medical journals - at least we hope they do. Market research says they do. And we know physicians can be influenced by an ad. So if they’re going to prescribe, perhaps based in part on advertising, we hope the advertising is accurate. But when claims are made that aren’t supported, the impression created is overly optimistic. It’s possible that the claims are true, but because data wasn’t reported and isn’t easily obtained or verified, we don’t really know.
Pharmalot: How difficult was it to verify or obtain sources cited in these ads?
Spielmans: It turned out not to be so easy. If we could get the source cited, maybe a study, we’d search to verify the claim in the ad. We’d look for the data, for instance. But there wasn’t a lot of cooperation when we asked for verification. Legally, the drugmakers don’t have any obligation to cite source or cite them accurately. As for providing references, it’s not required.
Pharmalot: You noted that Wyeth’s ad suggests doc look at data, but they wouldn’t give you any data. What happened?
Spielmans: I’m trying to think of a punchline. I chalk it up to the power of marketing. They made something and just do what they want to do. It’s somewhere between ironic and ridiculous.
Pharmalot: What do you think can be done to enhance the accuracy?
Spielmans: When I look at the FDA budget and staffing relative to the number of ads to be reviewed, well, they simply don’t have the number of people needed to do the work. They don’t pre-screen (ads), so it seems to be pretty difficult to catch ads early before they go out.
Pharmalot: But are doctors so easily fooled?
Spielmans: It’s difficult to know how easily fooled they may be. I think it’s safe to say that some ads are influencing prescribing habits. The ads may use a lot of footnote to make things look more impressive, but just because the footnotes are there doesn’t mean they support the claim. I think it would behoove journal editors to engage in some sort of oversight. You know, on one page there’s an article on medicines and evidence, and the next page there is an ad with unsubstantiated claims. They could check that the ads are accurate, although I’m sure it would be difficult and become an ugly tug of war.
Pharmalot: Of course, you’re a psychologist. So are you predisposed to oppose prescribing psych meds for these ailments? And if so, why?
Spielmans: In general, I have some problems with psych meds. But I’m certainly not 100 percent against them either. It makes sense to prescribe meds in some cases. I’m not saying that meds should never be prescribed. I don’t think evidence supports that kind of position. But there’s also evidence that psychotherapy works as well if not better in the long term for many conditions.
Bruce Grant
So let me be sure I have this straight.
In one round of analysis, a single analyst was unable to find verification for claims in some instances and unable to obtain data (presumably unpublished “data on file”) from one or more companies.
A relatively slender reed to base any conclusions about the accuracy of ads, especially given that the author is a psychologist, not a psychiatrist or other medically qualified professional, and thus his ability to analyze psychopharmacological and clinical psychiatric data is at best unestablished.
Analysis by a panel of analysts, where inter-rater concordance can be assessed, would have come closer to enabling a more definitive statement about claim accuracy to be made.
Considering the coals that ads of this type get raked over by internal medical/legal/regulatory review boards (Ask any pharma marketer whether MLR welcomes a fast and loose approach to proof.) I’m inclined to read this study as a statement more about the author’s skill-set and biases than about the accuracy of the ads themselves.
Glen Spielmans
Bruce,
While I appreciate your interest in the topic, I disagree with you on several points. Indeed, some of your comments are demonstrably false. Let’s go point by point.
1. “In one round of analysis, a single analyst was unable to find verification for claims in some instances and unable to obtain data (presumably unpublished “data on file”) from one or more companies.”
*I coded the data along with two advanced undergraduate students. So there was not just “a single analyst.”
*Correct: many claims were unsupported by their cited sources and requests for data on file were usually ignored.
2. “A relatively slender reed to base any conclusions about the accuracy of ads, especially given that the author is a psychologist, not a psychiatrist or other medically qualified professional, and thus his ability to analyze psychopharmacological and clinical psychiatric data is at best unestablished.”
*Saying that one has to be a psychiatrist or a “medically qualified professional” in order to analyze data is a commonly held and utterly incorrect belief. Analyzing relatively straightforward data from clinical trials can be done by anyone with a background in stats and research methods. Psychologists receive a great deal of training in stats and research methods.
*Compare doctoral psychology curricula with med school curricula and note which profession receives more training in research/stats. That is not meant as an insult to physicians, just to note that psychologists are well-versed in understanding stats and research methods used in clinical trials.
3. “Analysis by a panel of analysts, where inter-rater concordance can be assessed, would have come closer to enabling a more definitive statement about claim accuracy to be made.”
*Agreed – and that is exactly what we did! Please read the method section of the study prior to criticizing the methodology!! We checked interrater agreement.
4. Considering the coals that ads of this type get raked over by internal medical/legal/regulatory review boards (Ask any pharma marketer whether MLR welcomes a fast and loose approach to proof.) I’m inclined to read this study as a statement more about the author’s skill-set and biases than about the accuracy of the ads themselves.
*I admit to knowing relatively little about the internal review process at advertising and/or drug firms. How you choose to read the study is certainly your choice. Please note, however, that our findings were similar to findings from other research on the accuracy of drug ads, some of which was conducted by physicians (if that makes it more credible to Bruce).
Bruce Grant
Glen –
I appreciate the clarifications. I support good science, and, in my 25+ year career on the agency side of pharmaceutical marketing, have always insisted on rigorous proof of claims as the only sure way to maintain the trust of the professionals who prescribe our clients’ products.
Assuming that your methodology was, in fact, sound and your rating informed, the question, then, is where did the lack of substantiation come from?
Most pharma marketers complain pretty bitterly about the standards of evidence to which they are held by their internal MLR review boards (substantially higher, in many cases, than required by the FDA). In my own experience, inability to substantiate claims from the references supplied in an ad is more often the result of inadequacies in the preparation of references by the ad agency than due to the non-existence of the evidence itself.
The FDA requires “at least two adequate, well-controlled studies” to support the addition of new claims to a drug’s approved labeling, and I think that’s a pretty good standard to apply here as well. I await the results of confirmatory studies by other investigators to bring a more definitive foundation of evidence to this question.
Glen Spielmans
Bruce,
The lack of substantiation came from a claim being made that was not supported by the cited source. In some cases, efficacy was claimed for a particular patient subgroup, but the cited study found that the drug did not demonstrate efficacy for that patient group, or data were not reported for that specific patient group. However, there were other types of claims that were not supported in the cited sources. You can see a few examples in table 3 in the article.
I’d also welcome more research in the area. However, I would be surprised if future researchers found substantially different results. The research in this area has consistently found that medical journal advertising claims often lack a firm footing in evidence.
You may well be correct that the preparation of references by the ad agency is problematic. An interesting hypothesis. Thanks for your interest.
Grieving
I wonder where the Lilly internal MLR review boards were when Zyprexa was put on the market. Or where the FDA was when evidence of harm came rolling in, or when Japan acted swiftly and required a black box warning on Zyprexa almost two years before the FDA did. The drug was advertised as safe for the longest time. Does every Pharma have to have an internal MLR review board? Why are the claims for psyc. drugs in medical journals often lacking a firm footing in evidence as opposed to other classes of drugs? What could the reason be?
Laurie
Glen, thank you for your time and attention to this issue.
“I’m inclined to read this study as a statement more about the author’s skill-set and biases than about the accuracy of the ads themselves”
LOL!! Ah, the “you are to dumb to understand what we do” defense. Don’t supply the evidence that would put this to rest, just insult the study author.
HorusCat
Glen,
I used to sell ziprasidone, and we have the data breaking down the efficacy for mixed and manic episodes. So while you were unable to find the supporting data, they do exist.
HC
HorusCat
Glen,
Another note: “low weight gain” with Risperdal probably refers to patients gaining less then 7% of their body weight, since that is the cut-off for action. Most clinicians would consider less than 5 lbs to be minimal weight gain. Certainly, in the context of the atypicals, with Zyprexa causing so much weight gain, Risperdal’s claim to have minimal weight gain makes sense. (However, over the long term, many people do gain much more weight on Risperdal; however, the trials aren’t long enough to show that.)
pg
Ziprasidone / GEODON prescribed to CHILDREN. “First they came for the Communists but I was not a Communist so I did not speak out. Then they came for the Socialists and the Trade Unionists but I was not one of them, so I did not speak out. Then they came for the Jews but I was not Jewish so I did not speak out. And when they came for [the children] me, there was no one left to speak out for [them] me.”
Geodon & Children http://www.docguide.com/news/content.nsf/news/8525697700573E18852570B60065770B
pg
THIS is good for CHILDREN?
http://www.rxlist.com/cgi/generic/ziprasidone_ad.htm
Following is a list of COSTART terms that reflect treatment-emergent adverse events as defined in the introduction to the ADVERSE REACTIONS section reported by patients treated with ziprasidone in schizophrenia trials at multiple doses > 4 mg/day within the database of 3834 patients. All reported events are included except those already listed in Table 3 or elsewhere in labeling, those event terms that were so general as to be uninformative, events reported only once and that did not have a substantial probability of being acutely life-threatening, events that are part of the illness being treated or are otherwise common as background events, and events considered unlikely to be drug-related. It is important to emphasize that, although the events reported occurred during treatment with ziprasidone, they were not necessarily caused by it.
Events are further categorized by body system and listed in order of decreasing frequency according to the following definitions: frequent adverse events are those occurring in at least 1/100 patients (only those not already listed in the tabulated results from placebo-controlled trials appear in this listing); infrequent adverse events are those occurring in 1/100 to 1/1000 patients; rare events are those occurring in fewer than 1/1000 patients.
Body as a Whole: Frequent: abdominal pain, flu syndrome, fever, accidental fall, face edema, chills, photosensitivity reaction, flank pain, hypothermia, motor vehicle accident.
Cardiovascular System: Frequent: tachycardia, hypertension, postural hypotension; Infrequent: bradycardia, angina pectoris, atrial fibrillation; Rare: first degree AV block, bundle branch block, phlebitis, pulmonary embolus, cardiomegaly, cerebral infarct, cerebrovascular accident, deep thrombophlebitis, myocarditis, thrombophlebitis.
Digestive System: Frequent: anorexia, vomiting; Infrequent: rectal hemorrhage, dysphagia, tongue edema; Rare: gum hemorrhage, jaundice, fecal impaction, gamma glutamyl transpeptidase increased, hematemesis, cholestatic jaundice, hepatitis, hepatomegaly, leukoplakia of mouth, fatty liver deposit, melena.
Endocrine: Rare: hypothyroidism, hyperthyroidism, thyroiditis.
Hemic and Lymphatic System: Infrequent: anemia, ecchymosis, leukocytosis, leukopenia, eosinophilia, lymphadenopathy; Rare: thrombocytopenia, hypochromic anemia, lymphocytosis, monocytosis, basophilia, lymphedema, polycythemia, thrombocythemia.
Metabolic and Nutritional Disorders: Infrequent: thirst, transaminase increased, peripheral edema, hyperglycemia, creatine phosphokinase increased, alkaline phosphatase increased, hypercholesteremia, dehydration, lactic dehydrogenase increased, albuminuria, hypokalemia; Rare: BUN increased, creatinine increased, hyperlipemia, hypocholesteremia, hyperkalemia, hypochloremia, hypoglycemia, hyponatremia, hypoproteinemia, glucose tolerance decreased, gout, hyperchloremia, hyperuricemia, hypocalcemia, hypoglycemic reaction, hypomagnesemia, ketosis, respiratory alkalosis.
Musculoskeletal System: Frequent: myalgia; Infrequent: tenosynovitis; Rare: myopathy.
Nervous System: Frequent: agitation, extrapyramidal syndrome, tremor, dystonia, hypertonia, dyskinesia, hostility, twitching, paresthesia, confusion, vertigo, hypokinesia, hyperkinesia, abnormal gait, oculogyric crisis, hypesthesia, ataxia, amnesia, cogwheel rigidity, delirium, hypotonia, akinesia, dysarthria, withdrawal syndrome, buccoglossal syndrome, choreoathetosis, diplopia, incoordination, neuropathy; Infrequent: paralysis; Rare: myoclonus, nystagmus, torticollis, circumoral paresthesia, opisthotonos, reflexes increased, trismus.
Respiratory System: Frequent: dyspnea; Infrequent: pneumonia, epistaxis; Rare: hemoptysis, laryngismus.
Skin and Appendages: Infrequent: maculopapular rash, urticaria, alopecia, eczema, exfoliative dermatitis, contact dermatitis, vesiculobullous rash.
Special Senses: Frequent: fungal dermatitis; Infrequent: conjunctivitis, dry eyes, tinnitus, blepharitis, cataract, photophobia; Rare: eye hemorrhage, visual field defect, keratitis, keratoconjunctivitis.
Urogenital System: Infrequent: impotence, abnormal ejaculation, amenorrhea, hematuria, menorrhagia, female lactation, polyuria, urinary retention, metrorrhagia, male sexual dysfunction, anorgasmia, glycosuria; Rare: gynecomastia, vaginal hemorrhage, nocturia, oliguria, female sexual dysfunction, uterine hemorrhage.
pg
Is the above the ’supporting data’ ?
Glen Spielmans
HorusCat,
Thank you for your comments.
There may be supporting data for ziprasidone in mixed episodes. However, such data were not provided in the source cited by the advertisement. We did not search through every clinical trial ever done, just the sources cited by the ads. I think it is reasonable to expect that a source cited in an ad should provide clear support for the advertising claim.
It is an open question what exactly “low weight gain” might have been referencing. However, given that the weight gain was 1.6 kg on risperidone vs. a .25 kg decrease on placebo over a 3-week period, a difference that was statistically significant, we did not believe that the claim was supported. If they were referencing that few people gained 7% of body weight in 3 weeks, then I believe they should have made such a claim specifically.
Thank you again for your interest in the study.
HorusCat
pg,
You’re an idiot. Nowhere did I suggest using ziprasidone in children. No article appears suggesting efficacy for ziprasidone in children with bipolar disorder. You lose all credibility when you automatically shove every discussion into your “we are murdering our children!” cubbyhole.
Read the side effects on any medication. They are all the same, in varying proportions. If I take a drug in a trial and I get murdered in a bank robbery, that gets reported as a side effect. And the connection is…????
HorusCat
Glen,
Your reply is appreciated. Again, I would suggest that the marketers are using short-hand to communicate to physicians who understand the use of these drugs in a certain context–that a drug is going to be used, not placebo, and that (unless the drug is a typical), there will be weight gain with it. An outsider may think, 3.5 lbs, that’s significant…but a psychiatrist would say, well, compared to 40 lbs with Zyprexa… In some respects, it would be like an alien trying to understand our car commercials…what does a sexy woman have to do with a vehicle to transport you from point A to point B? But WE understand perfectly (well, my husband seems to!).
And I would say that 3.5 lbs isn’t significant in a 150 lb man, especially when the medically accepted definition of significant (which every psychiatrist now knows!) is gaining 7% or more of body weight.
Glen Spielmans
HorusCat,
But the 1.6 kg weight gain was over a 3-wk period. Even Zyprexa wouldn’t increase weight by 40 lbs over such a short time frame. In any case, I appreciate your point of view, though obviously we disagree somewhat.
HorusCat
Glen,
I am not sure we disagree, so much as I swim in these waters, so I read the ads differently than you do. I read that bit from Risperdal, and think, “well, yeah, try to differentiate yourself from Zyprexa, but we all know people gain weight on Risperdal, too, just not as much.” So maybe I am saying to myself what you are saying in your article.
As far as stuff like the missing ziprasidone data, I think another sort of short-hand is occurring. Doctors think in terms of class effects, and don’t always draw a bright line between manic and mixed episodes. In all the time I sold an atypical, I never had a doc ask me a distinguishing question about effects in manic vs mixed episodes. Most often, they concede efficacy in the depressed bipolar patient but question efficacy in true mania.
Anyway, I’m sort of rambling around. I think you think doctors 1) read the advertising and 2) use it as a source of information. Knowing my own approach to advertising in the periodicals of my choice…(and to commercials, and pop-ups, etc) It would be interesting to see the internal data pharma marketers have on ROI with journal ads. They are already finding that DTC may not be the best way to drive market share for an established brand. I wonder what would happen if all of pharma just quit advertising in journals? Would anyone notice?
AA
HC,
In previous threads, you seemed to not understand why people got so mad at you. Well, calling someone an idiot will not cause people to be polite. I don’t see anything that PG said that deserved that type of response.
Saying that all drugs have side effects is a cop out argument. You can’t compare for example the side effects of Allegra to an antipsychotic. Even Lipitor, which seems like it has horrific side effects doesn’t seem to compare to an antipsychotic. Of course, someone on Lipitor will shoot down my argument.
Antipsychotics are way over prescribed for kids big time. So yes, if you are prescribing an antipsychotic that someone doesn’t need, you are essentially murdering them due to the horrific side effects. I know that is blunt but let’s call it what it is.
Finally, if you don’t believe that SSRIs can cause homicidal thinking, google Jay Cohen, a psychiatrist who has prescribed these meds but has information on his website that they definitely can cause homicidal thinking. I can tell you from personal experience that if I had stayed on Celexa, there is no doubt in my mind, I would eventually become either homicidal or suicidal. I became suicidal on Prozac. But you refuse to believe personal accounts so I am wasting my time.
I just hope for your sake that the Zoloft you are on continues to work for you. For a year and a half, it was a great drug for me. Then I became severely agitated.
No, I don’t have BP in case anyone reading this post was wondering.
AA
pg
HC: “If I take a drug in a trial and I get murdered in a bank robbery, that gets reported as a side effect. And the connection is…????”
There isn’t any connection whatsoever, which is no doubt why getting murdered in a bank robbery and any other disabling or terminal non-drug related scenarios are not listed as side effects of a drug. If a child suffers injury from a prescription drug, the fact that other drugs also carry the same side effect makes some kind of difference to that child’s injury because…???
truthman30
HC..
I was not aware you were on Zoloft..
On a different note..
I worked with a girl who was prescribed Zyprexa..
She was extremely vulnerable and I witnessed first hand what effects this disgusting psychiatric drug can have on an individual..
Her tongue would dart round her mouth like a lizard, she was visibly shaking every day and her puplis were huge, she ended up in the psych ward twice not long after being prescribed this poison, her weight also ballooned within a few months..
HC..
In light of realizing that you are on medication yourself, I can kind of understand why you vehemently try to defend these meds all the time..
Maybe when you hit withdrawal or have a bad reaction you will see the truth of the devastation these meds cab cause..
Until then, I wish you luck with your journey..