How Many Adverse Events Are On The Internet?
20 CommentsBy Ed Silverman // November 10th, 2011 // 11:15 am
There is no debate that the Internet holds many potential riches for drugmakers seeking insights into patients. Consumers always have something to say about effectiveness and marketing. Yet many pharma folks dread combing through blogs, Facebook, forums and Twitter feeds, among other things, because they live in fear of having to file adverse event reports with the FDA.
But are those fears really justified? Maybe not, at least according to a new analysis by Visible Technologies, a social media monitoring and software firm, which examined more than 257,000 posts about 224 different products - 33 antacid over-the-counter meds, 38 over-the-counter decongestants, 10 prescription statins and 143 prescription drugs used to treat high blood pressure.
The upshot? Only 0.3 percent of the posts mentioned an adverse event, and 14 percent of the posts that contained an adverse event had an identifiable name and a way to contact the post that would make it possible for a drugmaker to fill out required paperwork. The analysis was conducted over a recent 30-day period.
What kind of sites were scoured? Posts were collected from millions of sources, including blogs; reader forums; message boards; message groups; social networks, notably Facebook and LinkedIn; Twitter; regular news sites; specialized health sites, such a WebMD; and video and photo sites, such as YouTube and Flickr.
And the purpose in choosing the types of meds that were reviewed was to cast as wide a net as possible, given that so many people use over-the-counter antacids and decongestants, as well as get prescriptions for treating high cholesterol and high blood pressure.
The analysis was extended to a subset of 12,530 posts that were deemed likely to contain an adverse event, based on the use of certain terminology. Among these, just 3.3 percent mentioned an adverse event - the rate was nearly identical for both OTC and prescriptions meds - and only one in seven posts with an adverse event report contained info to meet FDA reporting criteria. And 56 percent of adverse event posts had contact info that could be pursued, such as private messaging on a forum.
“There’s a big hesitation among pharmaceutical companies to go on the Internet, even though there’s so much valuable information there that every other industry is reading,” Jackie Kmetz, Visible’s director of community educational outreach, tells us. “But pharma is under a heavy cloud because people worry there may be a heavy load of (adverse event) reports to file.
“But when you look at the huge volume of comments, it’s actually not an overwhelming or intimidating amount of reporting they’re going to have to do. It’s actually very low risk. What this (finding) does is take the pressure off them, that fear of the unknown, of what’s out there and what they’ll have to do in response. It’s a ridiculously low number. The amount of follow up would be tiny.”
Of course, some brands are going to generate more adverse event reports than others, and it can still take time to sift through an analysis and do any required paperwork. Like it or not, though, drugmakers are going to eventually begin this exercise. To what extent and how fast this will occur, however, seems to largely depend on when the FDA gets around to issuing social media guidelines (here is the full report).
pic thx to birgerking on flickr
Observer
It is my observation that no one aims at a target that isn’t there. It seems likely to me that if you give those on the internet a place dedicated to a medication or firm to work with, the complaints/ AE’s / ‘kvetching / slander, ‘they will come.’
Regreatably, there is a good reason that pharma fears the unknown.
as
The biggest concern with using the internet isn’t just the “heavy load,” it’s the questionable science. If we trust the internet we’d be convinced that vaccines cause autism, Obama is a socialist, and cats play piano really well. That’s OK for marketing - which is really a study of what people are interested in - but not for medical research, which needs control groups and ideally should have blinded trials. I’m sure there are ways to do this, but pure data-mining should be, at best, idea-generating.
Greg Pawelski
The mainstay for drug safety is the spontaneous reporting of adverse events to the FDA’s MedWatch program, which suffers from underreporting, variable data quality, and the lack of a mechanism to assess confounding risk factors. These reports are a particularly poor instrument for detecting adverse events that can affect the public health caused by various drug agents. The lack of systematic collection and analysis of post-marketing data on the use of drugs and the outcomes of treatment has delayed discovery of some pretty serious problems until after millions of people have been exposed. How does this benefit patients?
Tom
There has been a lot of confusion and mis-information surrounding this topic over the past few years. The issue is not that pharma fears an increased volume of adverse events to file with FDA. As the paper shows, there are relatively few of those. It’s that every post has to be carefully evaluated, documented, tracked, and followed up if possible, to rule out an adverse event. There’s a long and permanent audit trail that has to be established for every potential adverse event in order to satisfy the requirements of FDA and other health authorities. For many companies, that’s a huge investment in resources to tease out a few actual events that are often well known and documented in the product label. BTW - Filing these events is easy once you’ve done the leg work to identify and vet them. Push a button and off they go to the FDA. It would have been useful to see how many person-hours went into identifying each “real” adverse event, and what percentage of those contributed anything to the safety profile of the drug. Drug safety resources are much better spent on scientific rather than clerical work.
SteveM
Re: “224 different products - 33 antacid over-the-counter meds, 38 over-the-counter decongestants, 10 prescription statins and 143 prescription drugs used to treat high blood pressure.”
Meh…The CNS drug category is the side-effect elephant in the living room. Plenty of dedicated web-sites collect anecdotes of miserable iatrogenic injury caused by psycho-pharmaceuticals. Where is that media study?
And the drug companies go out of their way to minimize side-effects in DTC advertising and even to consumers who vainly contact their “Help” lines after they’ve been trashed by psychotropic brain bombs.
Oh well…
john
Greg, the mainstay of drug safety is randomized, double blind, multi-center clinical trials in which two groups of patients distinguished only by treatment vs non-treatment are compared without the vagaries of selection, attribution, or reporting bias.
If spontaneous reports were reliable, they could be used as evidence of efficacy in NDA filings. The upshot of which would be that gogi berry extract would be approved as a treatment for just about everything.
original industry insider
John, you are incorrect and Greg is correct. Randomzed double blind placebo controlled trials statistically powered for efficacy, not safety. To power for safety, and to detect statistically significant treatment differences would literally require thousands of patients per trial, given the frequency of most AE’s. That’s why the mainstay for safety has been and continues to be the spontaneous reporting system, despite acknowledged limitations. I’ve been doing this stuff for almost 30 years, so I think I have a handle on clinical trial design.
john
I understand the statistical issue, OII, but looking into this further I think we’re both wrong.
Thomas Moore et. al in Arch. Int. Med. 2007, vol. 163, pg 1752 present some fascinating AERS data. They show that reported SAEs more than doubled between 1998-2001 and again between 2001-2005, and conclude that new drugs are dramatically less safe than older ones (though the mix of prescriptions written did not change that much over these short time periods, and their own data shows that much of the increase is due to increased reports for older drugs). Their data shows the rate of AERS reports for estrogens tripling in the aftermath of the WHI results reported in 2004, and insulin (which suddenly became more toxic) AERS reports rising 4 fold between 1998-2005. The toxicity of gabapentin doubled in 2005. The most fascinating example is that of Viagra, whose toxicity halved between 1998-2001, then doubled again in 2005.
So the experimental data bears me out that there is no simple relationship between AERS reports and AE incidence. But you are 100% correct that the clinical trials are insufficiently powered to detect AEs that are important enough to bear on the risk/benefit ratio in the case of drugs for less serious illnesses.
When I look at how this has been handled historically, it seems to me that the AERS is usually (and in my opinion, correctly) used as a hypothesis generating tool. Case control studies, or better yet cohort studies provide an affordable, statistically powered, and reasonably accurate way of testing these hypotheses.
Siva Nadarajah
One of our recent studies - conducted for a popular diabetes drug - showed 1.8% of the social media entries containing Adverse Events. This could be the worst case scenario. But the most important thing when it comes to social listening is how “intelligent and pharma specific” your platform is. The last thing you want is miss a potential adverse event during your listening engagement or worse, fail to integrate the findings with the current AE process of the company. It’s critical that companies take an end-to-end approach when it comes to social listening to avoid any risks.
Siva Nadarajah
CEO
Semantelli Corp.
“Compliance Ready Social Media Platform for Life Sciences”
http://www.semantelli.com/pharma
keiner
@john
“Case control studies, or better yet cohort studies provide an affordable, statistically powered, and reasonably accurate way of testing these hypotheses.”
Prospectively, yes. Retrospectively: Mostly not.
There are good statistical tools for signal detection in patient-reported garbage. And many things can be deduced from pharmacology of a compound (statins and rhabdo, for example). Or from isolated cases of AEs observed throughout clinical development (there is an FDA paper out there on the relevance of isolated cases of hepatic AEs, which should kill a compound in late clnical development). But not allergic/anaphylactic things.
keiner
btw: I calculate that in the end of the day about 100 cases of AEs would have to be reported PER MONTH from the internet, even for the very limited, well-established groups of drugs chosen for the marketing-experiement.
As outlined, cancer meds or CNS-drugs would blow up this number by several orders of magnitude. Well done, for the marketing department of this fancy web-company. Some millions-billions-trillions of $ might be out there to find from big pharma…
original industry insider
Keiner, please show me a retrospective case control study not laden with all of the biases inherent in such studies and my interest will perk up.
original industry insider
Siva, with all of the social metia sides, apps, operating systems, platforming and all of the related problems of getting connectivity of these sites with relevant databases, there is one thing that virtually all doctors have in their office: a fax machine. This to me seems to be an area where less is more.
Elmore
A friend being treated for breast cancer thought she’d lost her mind when she started experiencing a wild variety of side effects to an aromatase inhibitor. She needs the drug but there was no information anywhere about what she was experiencing–except on the web. There are whole websites devoted to nothing but this. She asked–repeatedly–why doesn’t my doctor know about this? Why doesn’t the drug company tell me, and tell me what I can do about them?
Drug companies should not be searching out of fear. They should be searching to help patients who need drugs, experience side effects, and need means to cope with them.
Jackie Kmetz
Hello. As the author of the paper I thought it might be helpful to add a few comments in response to the great dialogue started here.
To as’s comment, you are right, marketing is the biggest audience for social media monitoring at the moment and tends to drive the listening programs. The point of social programs is not medical research but an opportunity to hear what consumers say and how they talk. With this ear to the ground approach however there is the chance that AE’s will be discovered that could affect workload. So we wanted to put some numbers around what to expect.
Tom I think I can help you here. On average posts take roughly 20 seconds to review (Tweets and most thread comments are super fast and make up the majority while longer posts of course take longer). As I explained above, these programs are more driven by marketing listening and outreach programs rather than specific drug safety resources. The primary issue we were hoping to help with is to reduce or remove the concerns of the unknown volumes that are out there. Volumes will of course vary by drug and audience. We have been working with pharma clients for the past 6 years and most do not read every post that mentions their name, instead they are targeting the type of content they are looking for–efficacy discussions, sometimes side effects, campaigns, competitor conversations, etc.
SteveM we can certainly look into a follow up study of this category in the future as we’ve received some great additional questions that would make for an interesting followup piece.
Siva Nadarajah–excellent point!
keiner, as I said to SteveM, a follow up study for additional categories could help with this. We chose our categories based on the areas where we have received a significant amount of interest and where folks wanted to see “one like me”. It could very well be that we are not seeing a lot of interest from those pharma categories because of the types of discussions. I am eager to look into this further.
dzieczko
http://www.msnbc.msn.com/id/3036789/#45254744
shhh, don’t tell him that we don’t know how to collect good AE data…
1.1 billion in stimulus funding is at the Brookings Institute to troll through all HIV data…
University of Penn front and center - Zeke, Frasier - something’s going on…
Trans Parent
Example of reporting by motivated proactive patients.
http://www.askapatient.com/viewrating.asp?drug=20726&name=FEMARA
Terry Nugent
JAckie, did you break out your AE results by consumers vs. doctors/healthcare professionals? TO all, is there a difference in how AEs from HCPs are handled vs. consumers?
wright profemur hip injury attorney
So if they don’t know about any adverse events that means it never happened and the best thing to do is hide in mere ignorance just because they loathe doing some paperwork?
Jackie Kmetz
Terry–sorry for the delay was out traveling for the holidays! Great question and the answer is no. One of the challenges (for companies and marketers) and advantages (for the consumers) of social media is that you can be as anonymous as you choose. While there are a few HPC-related forums like Sermo that require verification that you are a medical professional, the rest of the online world does not require you identify let alone verify it. Consequently, in doing research such as this, it is nearly impossible to determine who is who with any degree of certainty for statistically valid reporting.
I will say, however, that due to the low reporting rates in our data I did read and research all AE mentions myself. What I can say is that as far as I could tell, via context from their posts and the 4 individuals who handily provided some form of profession and “about me” details in their post or profiles, none of them were HCP’s.
I hope that helps!