People who care about reality—including scientists, philosophers, doctors, teachers, and other concerned citizens—have long been bothered by bullshit. On the one hand, it is morally and philosophically reprehensible to purposefully undermine truth and to proclaim ignorance as a virtue. But more importantly, bullshit is harmful to people who live in the real world. Bullshit can wipe out a family’s finances, lead people to take unnecessary risks, and on a larger scale, cause economic disasters and undermine democratic governance. One of the potential silver linings of COVID-19 has been making people aware of both how common and how dangerous medical bullshit and misinformation can be, and to inspire them to take action against it.
To try to combat bullshit, several authors have developed bullshit detectors—generally useful rules of thumb—to help others in recognizing and avoiding bullshit. The most famous of these may be Carl Sagan’s “Baloney Detection Kit,” described in his 1995 book The Demon Haunted World (thank you Jim and Ken for pointing me to this resource); however, this idea goes back hundreds if not thousands of years, with several of the ancient Greek skeptics seeking to create laws that might help one to more easily recognize deception.
Medical bullshit is bullshit meant to get you to either buy a worthless product or follow baseless, and potentially harmful, advice. Medical bullshit comes in many forms ranging from bestselling self-help books on the latest miracle diet, to unregulated claims on the bottles of supplements, to medical doctors offering untested treatments with stem cells for anything from arthritis to dementia.
Having seen more than my share of bullshit through questions my patients have brought me, I can assure you that sniffing out whether a medical product is bullshit is not rocket science or brain surgery. I am happy to share with you some signs that you can use to tell if a medical product or piece of advice is bullshit. In Part II of this series, I will address uncovering the unreliable expert.
How To Spot Medical Bullshit:
1. The “information” provided is clearly tied to selling products
Many websites, videos and books appear to offer information about a disease or health topic that is important to you. However, once you get past the headline, you find yourself immersed in an infomercial for a particular product, diet or expert. When evaluating healthcare products, you should always ask yourself: what does the person recommending it have to gain by me doing this therapy? A common goal of bullshit is not to cure your illness but to get your money. This may occur by selling you books, supplements, tests, procedures, medications, stem cells, news, or an expert who wants to launch their medical celebrity career.
Another very important question: are there any independent sources also promoting this product? If not, you should probably move on. One of the principles of science is that facts can and should be independently verified. People who care about truth want to see verification. People who want to sell you bullshit or have something to hide don’t want their claims tested.
2. The product is heavily advertised
When one sees a frantic amount of energy and resources being used to promote a product, it often indicates an underlying lack of confidence in that product to sell itself. Often this advertising focuses on what life will be like with this product (relaxing on a beach, going to a party with beautiful friends) and not on the product itself. This is true not just for alternative medicines and supplements, but prescription medications. Pharmaceutical companies have huge advertising budgets that target both consumers and physicians. For products that serve a unique need and clearly work, they may not need to spend much money. But for products where there are cheaper alternatives, or a need to manufacture demand there is a greater motivation to advertise.(1,2) There have been tragic examples of products that were heavily advertised that later turned out to have serious safety issues, possibly known but suppressed by the company.(3-6)
3. The claims are inconsistent with what you know from reputable sources
If something seems too good to be true it probably is. If you stop to think about it, curing Alzheimer’s disease would be a significant medical advance that would land one or more people a Nobel Prize. To cure Alzheimer’s, one would have to: a) find a therapy that cures the disease; and b) test it to make sure that it is safe and reliably works. It would also help to understand what causes Alzheimer’s, although its possible one might find a cure through sheer blind luck. Given the vast amounts of research done by teams around the world to understand and cure Alzheimer’s, it seems highly unlikely that a lone individual would stumble upon it by chance or genius.
4. The product or idea is promoted as “revolutionary,” “unbelievable,” or a “miracle”
There are certain words that should always set off your bullshit detector. These terms are unfortunately not only used to sell products; they are also used by the media to make their news stories sound exciting. News stories on medical advances are more often than not overblown hype—simply compare how often have you heard a story headline like “scientists at the University of Smartsville may have discovered a cure for cancer”? (hint – a lot) vs. how often has a cure for all cancer been discovered (hint – never)?
Granted, there are some therapies that truly are revolutionary. Some examples include penicillin, anesthesia and handwashing. But, as you might expect, truly revolutionary ideas do not come along every day.
5. The product cures everything (or almost everything)
This is an easy tip-off. Given the many different things that cause disease, ranging from bacteria to cholesterol, it is extremely unlikely that any “cure-all” could actually cure numerous illnesses. Cure-alls tend to arise in the context of new medical discoveries before they are fully or widely understood. Currently, stem cells fall into this category. Vitamins, hormones and electromagnetic energy previously filled this role. Outside of staying hydrated, fresh air, eating a healthy diet, managing stress, and moving your body, there are few things that are broadly essential to health.
6. The evidence behind a claim is based on large leaps of questionable logic
One of the biggest gaps in education today, in my opinion, is critical thinking skills. This is a topic that ties into essentially everything in life and can help you to navigate fields that may not have even existed when you went to school. Take the following made-up example of a proposed cure for Alzheimer’s disease:
We know the Pharaoh’s diet works because over thousands of years not a single pharaoh was reported to have Alzheimer’s disease.
In later blogs we will take a deeper dive into critical thinking skills, but you can start to exercise these mental muscles immediately. Does the “fact” that “none of the pharaohs had Alzheimer’s disease” really mean ancient Egyptians cured Alzheimer’s? Are there any other possible explanations for why none of the pharaohs had this diagnosis? (Take your time thinking about this before reading on.)
For one, Alzheimer’s disease wouldn’t be a recognized disease for a few thousand years. Second, people had shorter lifespans and most didn’t live long enough to get it. Third, persons writing about the pharaohs may have hidden or ignored any signs of mental illness or dementia so as not to undermine their power or to avoid being imprisoned or killed by royal censors. Fourth, the pharaohs were fairly inbred and its possible that their family had low rates of Alzheimer’s because of their genes. I could go on, but you get the point.
This example may seem silly, until you start to really read and think about what is being claimed for other products. As in this example, they most often have an observation that may be interesting (if it is even true) but doesn’t really prove anything. If the key support for a treatment has gaping logical holes, you should be wary.
7. Proof of effectiveness relies on testimonials, anecdotal evidence, and obscure publications
A testimonial is the story of a single satisfied customer in a quote: “I tried the Pharaoh brain diet and within minutes I was remembering more than I had in years!” Anecdotes are similar to testimonials and represent one or more stories used to illustrate a point. They frequently choose the most extraordinary, satisfied customers or cautionary tales and pretend that these outliers are representative. Some potential anecdotes for the Pharaoh’s diet might include “several patients added mummy droppings to their diets and had an easier time learning new languages” or “unfortunately Bob refused my initial treatments and when he called me from his deathbed it was too late even for my most powerful diet.”
Anecdotes are not inherently bad – many legitimate educators use anecdotes to illustrate points – but when they are the only evidence presented, particularly for a healthcare product, sound the bullshit alarm. We will talk about types of Bias in a later blog, but for now it is important to know that anecdotes are very prone to many types of bias. Most notably, the person choosing the anecdotes is very likely to choose only those stories that confirm his beliefs or desires.
Sometimes experts reference studies hoping to convince you that there is solid research behind their claims and also hoping that you won’t look at them too carefully. In another blog we will talk about how to evaluate these studies. However, just a quick glance at the titles or place of publication can tell you a lot. If the title of the study doesn’t clearly indicate a high-quality study in humans be skeptical. Studies in mice, cells, chemistry, history, or anything other than a large randomized trial in people should not be accepted as proof that a product is safe and effective in people. Similarly, if the publication is hard to find or your doctor has never heard of the study or even the journal it was published in, it likely indicates low quality and dubious research.
8. The idea is based on ancient wisdom, mystical knowledge OR stresses the importance of being natural
It seems that people love “ancient” wisdom and “natural” products almost as automatically as they love puppies and kittens. Why we would be more comfortable taking advice from a king who never saw a modern city in a civilization that fell apart centuries ago with an average lifespan of 40 years is beyond me—but it seems that things that are really, really old have a certain authority. You don’t need to go so far as to discount all ancient wisdom – there have been some brilliant philosophers and spiritual thinkers over the ages – but when it comes to healthcare it is worthwhile to at least test these ancient findings before you accept them as true. Some ancient treatments were bizarre, such as using crocodile dung as birth control, and others blatantly harmful, such as smoking to cure asthma.
Regarding the term natural, I would simply state that arsenic, earthquakes and great white sharks are all natural. Moreover, there have also been numerous occasions where unregulated “natural” products have been found to contain everything from lead to Viagra.
9. The credibility of the idea is based on scientific-sounding terms rather than science
One of the ways bullshit experts hope to lure you in is by sounding scientific. Very often, the “information” supporting a product uses “sciency” words in a way that superficially appears to make sense but is actually just using fancy words to distract you. My current favorite, used for everything from yoga to electric footbaths is “clearing toxins on a cellular level.” This sounds like science but what does it really mean? What “toxins” are they talking about? And since your body is made of cells, where else would the toxins be?
In contrast, many of the best scientists in the world strive to make their findings known in clear and simple language. For those scientists who do not now how to do this, Alan Alda has made it a mission to help scientists speak clearly.
The history of bullshit travels on a parallel path to science and medicine. The same snake oil might be called different things depending on trends in science, from hormonal balancing to detoxifying to antioxidants to a special blend of genetic enhancers… but the seller behind it has the same goal – to confuse you into thinking their product works.
10. The recommended products are unregulated or poorly tested
This is not just a question about good or bad science but your safety. Would you buy brakes for your car if you knew they were untested? What if you knew that other people using these brakes had serious accidents? If you wouldn’t use them, would you turn around and sell them to someone else? Why should our standards be any lower for things that we put into our body than things we put into our car?
There is a good reason that the US government formed the Food and Drug Administration (FDA); the public expects that food, medications, and medical devices meant to be put into people are safe. Unfortunately, many products either fall outside of these regulations (e.g. supplements, medicinal cannabis) or have slipped through FDA loopholes (e.g. stem cells, some medical devices – John Oliver has a sobering and scary episode on this).
One of the reasons that supplements are unregulated by the FDA is that corporations selling supplements led a large lobbying campaign to prevent their regulation whose stated purpose was to prevent the government from overly regulating supplements. The fact that many supplements are all filler (e.g. rice powder and houseplants) without active ingredients or contain illicit substances not mentioned on the label suggests that the purpose may have had more to do with the business of selling supplements than the rights of the consumer.7 While there is now some movement to limit unsubstantiated claims on supplements, the fact is that there are currently too many products to regulate and not enough regulations.
11. The promoted products are only available outside of the U.S. or established hospitals
For a while, this was a fairly safe clue that a product might not be safe or legitimate as this was an easy way for a con artist to sidestep the tougher regulations in the US and some other countries. Unfortunately, loopholes, poor regulation and brave bullshit sellers have established many of these therapies now within the US as well. There was a time where the only places you could get unregulated stem cell treatments were outside of the US. Now, billboards advertise many, but not all, of the untested treatments you could get abroad right here in the US. There are even fake “research trials” of stem cells that require participants to pay to gain access to untested products. (8)
If you have other tips to add to the list, suggestions, corrections or questions, please do not hesitate to reach out.
If you are considering a health care treatment and can’t figure out whether or not its bullshit please consider sending me a question at “Ask Dr. Benzi”
Medical Bullshit Detector Checklist Part I: Products and Ideas
References
1. Frosch DL, Krueger PM, Hornik RC, Cronholm PF, Barg FK. Creating demand for prescription drugs: a content analysis of television direct-to-consumer advertising. Ann Fam Med. 2007;5(1):6-13.
2. Podolsky SH, Herzberg D, Greene JA. Preying on Prescribers (and Their Patients) – Pharmaceutical Marketing, Iatrogenic Epidemics, and the Sackler Legacy. N Engl J Med. 2019;380(19):1785-1787.
3. Moynihan R. Court hears how drug giant Merck tried to “neutralise” and “discredit” doctors critical of Vioxx. BMJ. 2009;338:b1432.
4. Maron BJ, Hauser RG. Perspectives on the failure of pharmaceutical and medical device industries to fully protect public health interests. Am J Cardiol. 2007;100(1):147-151.
5. Van Zee A. The promotion and marketing of oxycontin: commercial triumph, public health tragedy. Am J Public Health. 2009;99(2):221-227.
6. Hadland SE, Rivera-Aguirre A, Marshall BDL, Cerda M. Association of Pharmaceutical Industry Marketing of Opioid Products With Mortality From Opioid-Related Overdoses. JAMA Netw Open. 2019;2(1):e186007.
7. December 7, 2016: GNC Enters Into Agreement with Department of Justice to Improve Its Practices and Keep Potentially Illegal Dietary Supplements Out of the Marketplace [press release]. https://www.fda.gov/inspections-compliance-enforcement-and-criminal-investigations/press-releases/december-7-2016-gnc-enters-agreement-department-justice-improve-its-practices-and-keep-potentially: US Food and Drug Administration2016.
8. Liska MG, Crowley MG, Borlongan CV. Regulated and Unregulated Clinical Trials of Stem Cell Therapies for Stroke. Transl Stroke Res. 2017;8(2):93-103.
Comments