Recently, I’ve been thinking about evidence, particularly in the context of blogs and recent health fads surrounding gluten, yeast, ‘all-natural’, etc. If you’re looking for a food blog, you’re in the wrong place, but I want to write about a trend I’ve seen when reading health blogs. I’m going to pick on health blogs because they’re what I’ve been thinking about, though this is not meant to be an indictment on health blogs (I’ve seen this problem on other categories of blogs as well). I’m also going to pick on aspartame because, well, it’s easy, and it recently came up when someone gave some well-meaning advice to my wife about how drinking diet soda is “bad for you”.
I’ve heard for years that artificial sweeteners are “bad for you” and don’t actually help you lose weight, so I’m actually sympathetic to this argument (call it great marketing from the anti-sweetener lobby). That said, what does the evidence say?
My first go-to for evaluating claims that such-and-such protein/chemical/supplement/food is good or bad for you is WebMD.com. In my experience, WebMD is a fairly conservative source of information, meaning that it takes a very large body of evidence for them to change their recommendations. There’s a trade-off here: on the one hand, this means that WebMD is not a great source to find the latest and greatest information about a new treatment (allopathic, osteopathic, homeopathic, or otherwise); on the other hand, they aren’t likely to endorse something just because it looks promising until there’s a large enough pile of evidence to suggest that it is promising. So what does WebMD say about sweeteners?
WebMD has a short, 3-page article entitled The Truth on Artificial Sweeteners (WebMD, 2002) that gives the professional opinions of a number of experts—mostly practicing and academic nutritionists. The one sentence summary is that artificial sweeteners can cause diarrhea when some people consume a lot of artificial sweeteners, but most people are fine consuming as much as they please, and although they probably don’t help that much in weight loss, “there is no credible information that aspartame—or any other artificial sweetener—causes brain tumors, or any other illness.” In other words, it’s pretty benign: it’s hard to make the argument that diet soda is good for you, but the evidence doesn’t support the claim that it’s bad for you.
As a good researcher, I didn’t want to look just at one source that I’m biased towards, so I Googled “Artificial Sweetener Health” (sans quotes in my search). I’m going to pick on Dr. Mercola for the sole reason that he showed up as the first result from Google that doesn’t appeal to what my biases say is a “credible source”. Dr. Mercola is a D.O. and sells a number of health products and claims to have “The World’s #1 Natural Health Website” (Mercola, 1997). Just the title of his article, Artificial Sweeteners—More Dangerous Than You Ever Imagined (Mercola, 2009) smacks of fear mongering, and the article itself doesn’t disappoint those of you who love a good panic attack. He spends the first three paragraphs not really saying anything except scare tactics and “[i]t’s not pleasant to learn that corporations, government-sponsored regulatory agencies, and politicians are more interested in lining their pockets than protecting your health and the health of your loved ones. But unfortunately, these are serious issues that you must consider for your and your family’s safety” (Mercola, 2009). After some scrolling, he berates aspartame for being discovered by accident—just like penicillin, which, current concerns with anti-microbial stewardship aside, has saved hundreds of millions of lives and made hospitals a place where you can get healed instead of a place where you die of infection.
I could go on, but you get the point: there’s a lot of fear mongering. Let’s get to the evidence. For the sake of time, I’m just going to pick one of his references. To his credit, Dr. Mercola did include references, though there are no citations for his allegations of fraud on the part of the FDA and the “corporations” that got aspartame approved (and since he’s not an investigative journalist, I doubt he, himself, pulled what appear to be quotes from internal FDA documents). Specifically, I want to focus on the claim about brain cancer:
“Equally alarming is evidence women of childbearing age who consumed aspartame during pregnancy were delivering babies with an increased risk of brain and spinal cord cancer” (Mercola, 2009). For this claim, he cites two sources:
- A 1992 article from the National Cancer Institute entitled Primary Central Nervous System Lymphomas: An Update (Jellinger, 1992)
- A 1997 briefing published the Journal of the National Cancer Institute entitled Aspartame Consumption in Relation to Childhood Brain Tumor Risk: Results from a Case-Control Study(Gurney, 1997)
Thanks to Google Scholar, anyone can access the abstract of the 1992 article and the whole of the 1997 briefing. The rest of the 1992 article is behind a paywall, but it appears to be an overview of prevalence of lymphoma—sites in the brain, risk factors, etc.—and of treatment success with different treatments. The word “Aspartame” doesn’t appear in the abstract, so it couldn’t have been that important to the article; it probably came up as a direction for future research or something like that (though I don’t know that for sure).
Which brings me to the second article, which is a doozie. Gurney, et al performed a Case-Control correlational study looking at incidence of brain cancer in children by looking at children born after 1981 (when the FDA approved aspartame) by interviewing their mothers about aspartame and diet soda consumption during the pregnancy, and then calculating the odds ratio for having pediatric brain cancer. Calculating causation from such a study is pretty much impossible, but it’s a useful tool to identify possibilities for future research.
For those of you not familiar with the odds ratio (OR), it’s a ratio of odds, which are a ratio of probabilities. Suppose you have a 0.01 probability (1%) of some event (getting brain cancer) occurring under one state of the world (like your mother drinking diet soda while you were in utero) and a 2% chance in another state of the world (like your mother smoking while you were in utero). The odds in the first case are 0.01/0.99 or 0.0101 and the odds in the second case are o.0202, so the odds ratio is 0.0202/0.0101 or 2 (or “two to one” in common parlance). The interpretation is slightly more nuanced than this, but it basically means (especially with such low probabilities) that you are twice as likely to have the event happen in the second state of the world. This isn’t synonymous with saying that this is a likely event—it’s still only got a probability of 0.02—but the first state of the world is more unlikely in this example. An odds ratio of 1 means that however you divided the possible states of the world has no bearing on the likelihood of the event in question. An OR less than 1 means you’re more likely to experience the even in the first state of the world, and the opposite is true for an OR greater than one. All of this is subject to your sample, so to generalize the results from a sample population, we calculate a 95% confidence interval (basically a “margin of error” to use polling terminology) to estimate a range of values that probably contains the true value for the general population (anyone not in the sample). A confidence interval that does not contain 1 implies that there probably is a difference between the two groups, even in the general populations (it is “statistically significant”).
So let’s look at Gurney’s results, which, remember, Dr. Mercola cited as showing that aspartame might be linked to pediatric cancer. It’s a little difficult to read in this image, so I encourage you to read the article for yourself here.
So what this table shows, for our purposes here, is two things:
- All the 95% confidence intervals contain 1, meaning that there’s no conclusive (“statistically significant”) evidence of an effect in either direction.
- Only consumption of diet soda during the 2nd trimester and while breastfeeding have an odds ratio greater than 1.
In other words, the children of mothers in this sample who consumed aspartame during pregnancy were, on the whole, less likely to develop brain cancer! This is the exact opposite of what Dr. Mercola said in his article! That said, the affect size and sample size was still small enough that we can’t generalize this to the larger population (i.e. the confidence intervals contain 1), but it clearly doesn’t support Dr. Mercola’s claim. In fact, the authors of the article say outright: “we found no evidence to support the hypothesis that consumption of aspartame is related to pediatric brain tumor occurrence” (Gurney, 1997). This finding is repeated with a more recent (but still before the 2007 Mercola article) 2004 meta-analysis published in the Journal of Oncology: “according to the current literature, the possible risk of artificial sweeteners to induce cancer seems to be negligible” (Weihrauch, 2004).
So what the Hell, Dr. Mercola?
Not only does the literature not support what he’s saying (because it’s not statistically significant), the authors he cites as supporting him concluded the opposite of what he’s saying! It’s intellectually dishonest and straight up unprofessional to misquote authors like that! On top of that, this blog is just full blatant and unjustified fear mongering, which needlessly guilts less skeptical readers into abstaining from all sorts of harmless (but potentially pleasurable) products (artificial sweeteners is just the tip of the iceberg for this blog).
Again, mine isn’t a health blog, and I’m not saying aspartame and other artificial sweeteners (and the diet soda that contains them) are good for you. I’m not even saying they’re not bad for you. They very well could adversely affect my wife’s health and/or mental well-being (diet coke does tend to give me a headache). But the point is that the evidence doesn’t support the claims that this natural-only/homeopathic health blog is making.
It’s certainly possible that other blogs are more intellectually honest than Dr. Mercola, but my experience with health blogs is that they aren’t (evidence available upon request, though I would prefer to see a counter-example). I don’t mean to say that gluten, aspartame, sugar, fat, salt, or whatever is good for you. I don’t know. But I’ve demonstrated that in at least this instance, a very popular (allegedly #1) health blog is just blatantly ignoring the evidence (and in fact misrepresenting the evidence and its authors; what a dick!).
Try it for yourself. Go to a random blog–I recommend health blogs as a good example, but knock yourself out. If they use scare tactics or fear mongering (or even if they don’t), then check their sources. If they don’t have any, then they probably are just making it up. If they do give citations, run a few of them through Google Scholar and see what the actual articles say. Does it line up? If not, then why are you trusting this author? They’re not only making it up, but they’re actively ignoring the evidence to the contrary! I don’t care if you want to keep reading these blogs for your own enjoyment, but don’t pretend that they’re an authoritative source.
Lastly, let me leave you with this wonderfully entertaining and useful rule of thumb from the folks at In a Nutshell. When it comes to conspiracy theories, ask yourself this simple question: Would rich and/or powerful people be affected if this conjecture is true? If so, then it’s probably false (Kurzgesagt, 2014). For example, Dr. Mercola claims that evil, aspartame-producing corporations are intentionally trying to harm you to make a buck (Mercola, 2009). Do rich and powerful people—like, say, the CEOs of Equal or NutraSweet and their loved ones—eat or drink artificial sweeteners? Yes? Okay, this conspiracy theory is probably false.
Gurney, J. P. (1997). Aspartame Consumption in Relation to Childhood Brain Tumor Risk: Results from a Case-Control Study. Journal of the National Cancer Institute, 1072-1074.
Jellinger, K. W. (1992). Primary central nervous system lymphomas—an update. Journal of Cancer Research and Clinical Oncology, 7-27.
Kurzgesagt – In a Nutshell. (2014, December 18). The Ultimate Conspiracy Debunker. Retrieved from Youtube.Com: https://www.youtube.com/watch?v=Hug0rfFC_L8
Mercola, J. D. (1997). About Dr. Mercola. Retrieved from Mercola.com: http://www.mercola.com/forms/background.htm
Mercola, J. D. (2009, October 13). Artificial Sweeteners — More Dangerous Than You Ever Imagined. Retrieved from Mercola.com: http://articles.mercola.com/sites/articles/archive/2009/10/13/artificial-sweeteners-more-dangerous-than-you-ever-imagined.aspx
WebMD. (2002). The Truth on Artificial Sweeteners. Retrieved from WebMD.com: http://www.webmd.com/food-recipes/truth-artificial-sweeteners?page=1
Weihrauch, M. R. (2004). Artificial sweeteners—do they bear a carcinogenic risk? Journal of Oncology, 1460-1465.
 The first results were from cancer.gov, Oxford Journals (which I’ll cite later), cancer.org, and Mayo Clinic. Dr. Mercola’s website was result #5.
 This is really petty, but it irritates me. He made all the sources hyperlinks to the website itself. It’s almost like he doesn’t want you to read his citations…
That’s right. I provide real links.
 A meta-analysis uses data collected in previous studies and compiles them to get a larger sample size for subsequent analysis