Every day, we are bombarded with the results of new nutrition studies proclaiming the latest on everything from red wine to coconut oil. We read about the “battle” between low-fat versus low-carb diets and question whether juice cleanses really are “fat-busting”. Sure, the headlines grab our attention, but how do we know if there’s strength beyond the media hype? Last week, the Academy of Nutrition and Dietetics sponsored “Hot Off the Presses: Evaluating the Evidence Behind Today’s Nutrition Media Headlines”, a webinar mediated by IFIC’s Director of Nutrients Communications, Kris Sollid. The webinar featured speaker Andrew Brown, PhD, of the University of Alabama-Birmingham’s Office of Energetics & Nutrition Obesity Research Center (NORC). Dr. Brown offered a wealth of information on how to evaluate and communicate nutrition research. Here are 5 key takeaways:

1. Start with the basics.

Dr. Brown began with a description of four fundamentals of science:

• falsifiability (Can the assertion be proven false?),
• uncertainty (How confident are we in our conclusions?),
• reproducibility (Can the results be replicated?), and
• skepticism (Are there alternative explanations?).

Keeping these questions in mind while reading the newest nutrition study in the media can help us cast an objective, logical eye on the results.

2. Media headlines often don’t match the facts.

Many of us read the recent splashy article titled, “Lose Weight without Giving Up a Single Calorie?” and thought it was a) the best news ever; or b) too good to be true. This recent example is an excellent representation of many of the headline portrayals of nutrition research that often don’t align with the study itself. Dr. Brown highlighted a few additional examples of popular topics in nutrition, including the “obesity-fighting” effects of red wine (which was demonstrated in rats – not humans) and an opposing viewpoint – “Drinking Alcohol Sparks Eating, Leads to Weight Gain” – in which the method of administration of alcohol was very different than the typical act of drinking a beverage. These examples highlight the need to take a closer look at the research methods and results, to ensure that we aren’t taking the conclusions out of context.

3. Media coverage of nutrition can introduce biased perspectives.

Properly designed randomized controlled trials (RCTs), considered to be the “gold standard” of research designs, account for 40% of the content in high-impact research journal, with second-tier study designs (unrandomized trials, case-control or cohort epidemiologic studies) taking up just over half of journals’ content. Yet, almost 7 in 10 media reports of new research come from these second-tier study designs. So what’s the big deal?

Many of these studies report associations between a dietary component and a health outcome (for example, red meat and heart disease or green tea and weight loss) without directly proving that the food or beverage caused the outcome. One of the basic tenets of epidemiologic research, “Correlation does not equal causation”, summarizes this concept. Demonstrating that a certain food or nutrient can directly cause or prevent a certain effect can typically only be done through well-designed RCTs – preferably those with large sample sizes, which reduces variability between groups – or through compilations of randomized trial results on a given topic, called systematic reviews.

4. What we think we know isn’t always what research is telling us.

We’ve been told for years that “breakfast is the most important meal of the day” and that eating first thing in the morning can help with weight control. But what evidence do these long-standing beliefs rest on? Dr. Brown’s research team has focused on this question for several years, and he highlighted a few examples in the webinar: breakfast eating and obesity and fruit/vegetable consumption and weight loss. The relationship between breakfast and body weight is based on cross-sectional studies that found that people who skipped breakfast tended to have a higher body mass index (BMI) compared to those who ate breakfast. Yet, this association has been put to the test in five RCTs, none of which found that skipping breakfast led to weight gain, or that eating breakfast led to weight loss. In fact, one study found that people who skipped breakfast actually lost more weight than breakfast eaters. Yet, despite the evidence, media reports continue to state the merits of eating breakfast for weight loss. The same goes for the association of eating fruits and veggies with reaching a healthy weight. This can be confusing and misleading, especially when coupled with our own confirmation bias, in which we tend to seek out results that confirm what we already believe to be true – and disregard conclusions that don’t agree with our own.

5. Maintaining an evidence-based practice is possible with the right tools and a critical eye.

The webinar closed by highlighting best practices for evaluating evidence presented in nutrition research. Dr. Brown underscored the importance of prioritizing research studies involving larger sample sizes (or systematic reviews that aggregate the results of many smaller studies) and a longer duration of an intervention, and ensuring that the data adequately reflects the outcome and exposures of interest. He also emphasized that research certainly can inform – but should not substitute for – clinical judgment. The webinar closed with a quote from the philosopher Aristotle which, while stated centuries ago, still holds merit today: “It is the mark of an educated mind to be able to entertain a thought without accepting it”.    

Want to learn more? IFIC has put together a handout for evaluating evidence that describes key terms, differences between study designs, the hierarchy of sources of scientific evidence, and a study evaluation checklist with important questions to ask when deciding whether a study is well-designed. Stay in the loop on buzzworthy research by subscribing to the University of Alabama-Birmingham NORC’s Obesity and Energetics Offerings, a weekly breakdown of scientific evaluations of new and notable nutrition research.