The dangers of selective science

As Spiderman once accepted, "with great power comes great responsibility". Such words should also be heeded by scientists pulling together data for meta-analyses, before damage is unnecessarily done to an industry based on the prevention, and not the cure, of disease. Meta-analyses are selective, and for this very reason they are also controversial - which studies do they include and exclude, and why? Looking at the details of the new analysis shows us that 67 randomised clinical trials were included, focussing on beta-carotene, vitamin A, vitamin C, vitamin E, and selenium supplements versus placebo. But amazingly, 748 trials were excluded for several reasons, including 405 trials that showed no mortality in the study groups. I attempted to contact the lead reviewer for his comments on why they had excluded 405 death-free trials, asking whether such an exclusion would skew results towards increased death statistics… I am still awaiting a reply. But is it as simple as: "you can't measure antioxidants in relation to death if you have no deaths to compare it to"? "No-one died in this trial, so it can't be included…" Selective science is all well and good when used to weed out substandard studies, but it should always be questioned when it involves excluding that appear to be perfectly good studies because they don't fit. While meta-analyses, when properly performed, can be of huge value, we now risk tarnishing the meta-analysis technique with the flawed label - we all remember the vitamin E meta-analysis of a few years ago that was subsequently dissected and rejected by many in the field, but not before the conclusions had played out across the consumer media and vitamin E sales fell. This has been followed by the antioxidant meta-analysis that, in fact, is not new. It was published last year in the Journal of the American Medical Association​, and received much criticism from both independent academics and the dietary supplements industry. Some of the criticism seemed to be taken on board by the reviewers because they published corrections earlier this year. And last week, the 'new and improved' version was published in the prestigious Cochrane Systematic Reviews​. It contained the same conclusions as the earlier JAMA​ article. It also, on first glance, contained approximately the same information. Regardless of the multiple questions that have already been raised since its publication, the analysis spawned a whole range of sensational headlines condemning antioxidant supplements, which will undoubtedly damage the health and nutrition industries. Indeed, a piece of feedback arrived in my inbox last week from a consumer. She asked whether, in light of scary accounts in the press about antioxidants, she should continue giving her child a vitamin E-rich supplement or was it dangerous. This highlights that such meta-analyses, and the headlines they generate, have clout. If caution is necessary regarding certain nutrients in supplemental form, then it is necessary to highlight this. But whittling science down to a metaphorical handful of studies that report deaths is not advancing our understanding, and could easily lead to misinterpretation and confusion. The bigger picture ​ But meta-analyses and their uses is only part of the issue. The real issue for me is the randomised clinical trials that such analyses are based on. I have commented on it before, and I will say it again - we need to be very careful about the use of such trials. Why? Because they pull nutrients out of context, and they focus on a fixed dose for a fixed period of time. The value of such trials is undeniable - I am not questioning this - but subjecting nutrients to a drug model of testing also opens the results up to poor interpretation. I mean, can a time-constrained randomised trial really capture a lifetime of consumption with respect to chronic disease? We also need to remember that nutrients often work in synergy with one another and exert effects on multiple body tissues, unlike pharmaceuticals. Additionally, many randomised clinical trials look at the effect of nutrients in diseased populations. Surely the damage of a lifetime's poor nutrition has already been done. When we obtain negative or null results from such trials, should we really be surprised? The power of nutrients is in the prevention, not cure, of disease. People seem to be losing sight of this. There is clue as to the benefits of supplements in their name: supplement. They are not intended to replace. They are not drugs that should be taken at high doses while the rest of the diet stays unchanged. Stephen Daniells is the science editor for NutraIngredients.com and FoodNavigator.com. He has a PhD in chemistry from Queen's University Belfast and has worked in research in the Netherlands and France. ​​If you would like to comment on this article, please email stephen.daniells'at'decisionnews.com​