Read past blogs in this series: Part I, Animal Research | Part II, Correlations | Part IV, Expert Opinion
In this series of blogs, we used the example of the failure of Vitamin A to prevent lung cancer in smokers to illustrate why animal research and epidemiologic studies cannot be used on their own to predict whether a treatment will work in people. In this blog, we will look at anecdotes – one of the most common and popular forms of evidence. It is also, often, one of the most convincing until you recognize it for what it is—medical gossip.
The definition of an anecdote per Google’s Online Dictionary (Oxford Languages) is “a short or amusing story about a real incident or person” with a second meaning as “an account regarded as unreliable or hearsay.” Closely related to anecdotes are testimonials, anecdotes that are chosen to promote a particular individual or claim. In the health and medical context, anecdotes and testimonials come up all of the time; for example:
On Facebook a friend states that they lost ten pounds in the last month since starting a new supplement.
A website for an orthopedic surgeon has testimonials from patients who were amazed with his care.
A bestselling book on a new diet has stories in every chapter from individuals who found new hope when they started losing weight with the diet the author developed after failing dozens of other diets.
A news story on research for cannabis as a treatment for epilepsy is “substantiated” by interviewing people living with epilepsy who are either excited about cannabis or have had some success with it.
In all of these examples, if the person presenting the anecdote were to start their story by saying, “I don’t know if this really works, but let me share an anecdote with you,” you might be less likely to be convinced. Particularly if you considered how anecdotes are notoriously unreliable and prone to bias. Here are just a few common biases seen in anecdotes:
Confirmation bias: people hand-pick anecdotes that prove their point and ignore (or suppress) those that don’t
Reporting bias: people who had a great (or awful) experience are more likely to share their anecdotes than people who had no effect
Attribution bias: people may falsely connect two events that happened around the same time in a causal relationship, for example, thinking that taking vitamin C tablets cured a cold that would have gotten better on its own
Anecdotes work because we don’t see them as medical gossip, as isolated observations, but rather as stories. The power of the anecdote to convince comes from human’s love of stories and people’s ability to easily understand stories. And nothing sells like a good story. It’s your job as a bullshit buster to separate the story from the claim the story is meant to support.
While this may sound difficult, it is something that we do all the time with regular gossip, particularly when the gossip is coming from a notorious gossiper. For example, when your aunt Jane tells you that your wife was talking intently with another man at a party, and wants you to believe she is cheating on you, you will fairly automatically think: a) Jane is a trouble-maker; b) It’s possible that my wife talked to another man, but c) my wife talks to lots of people and doesn’t have affairs with any of them.
When you start looking at medical anecdotes as medical gossip, you will find that they are truly everywhere and that they rarely provide convincing support for the claims they are meant to support. You will also find that the people spreading them are often serial gossipers who, like aunt Jane in the example above, have their own agenda (attention, drama, selling products) that has nothing to do with your well-being.