Should you follow The Science or the Yellow Brick Road?
The path to knowledge is paved with only the best of intentions, and what more exemplifies knowledge than The Science and its handmaidens, clinical studies?
Unfortunately, not all scientists are putting out good science, even if they’re based at Harvard.
Case Study One - tennis for long life?
Want to live longer? Play tennis! According to the results obtained by researchers in the Copenhagen City Heart Study, playing tennis can extend your lifespan by almost 10 years. The study followed 8,577 people over 25 years, with the aim of examining how participation in sports affects life expectancy. The results were, according to the researchers, unexpected.
The study didn’t only look at tennis-players. People who played other sports such as badminton and soccer, or who jogged or swam or cycled, or used a gym at a health club, were also followed over 25 years but their activities didn’t seem to confer nearly as much benefit as tennis did.
Tennis was found to add 9.7 years to one’s life, compared with 6.2 years for badminton, 4.7 years for soccer, 3.7 years for cycling, 3.4 years for swimming, 3.2 years for jogging, 3.1 years for calisthenics, and just 1.5 years for “health club activities” which included using a treadmill, elliptical, or weights.
The researchers were somewhat taken aback by their findings:
Surprisingly, we found that tennis players had the longest expected lifetime among the 8 different sports.
And they suggested two possible explanations:
Belonging to a group that meets regularly promotes a sense of support, trust, and commonality, which has been shown to contribute to a sense of well-being and improved long-term health…
Tennis isn’t actually a team sport (whereas soccer is) and doesn’t necessarily involve a group (while health club activities often do). But there had to be some reason why tennis is so good for you, so the researchers suggested that perhaps,
… short repeated intervals of higher intensity exercise appear to be superior to continuous moderate intensity physical activity for improving health outcomes.
That’s assuming, of course, that the tennis-players in the study energetically bounded around the court with grim determination not to miss a single shot. No sedate doubles with plenty of time for breaks, plus strawberries and cream. But who knows? The study doesn’t say.
Undeterred by any such trivial considerations, Forbes enthusiastically reported on the study and concluded that everyone should “take up tennis!” They also referred readers to another study that purported to show the health benefits of what they call “racquet sports,” as well as swimming, aerobics, cycling, playing soccer, and running or jogging.
This study, published in the prestigious British Medical Journal, found that swimming, aerobics, and racquet sports were associated with a statistically significant reduced risk of all-cause and cardiovascular-related mortality. Cycling, by contrast, was only found to reduce CVD mortality but not all-cause mortality. Running and playing soccer came out worst, with no or very little statistical benefit found.
Here, too, the researchers were surprised by the results, especially as previous studies had found clear benefits from running, jogging, and playing soccer. They too suggested a number of reasons for the unexpected findings, such as not enough people dying over the study period to make an accurate assessment, and less-than-reliable reporting of activities by those enrolled in the study (who were asked to describe their physical activities for the past four weeks, at set points over a number of years).
The BMJ study, unlike the Copenhagen study, was quite rigorous in adjusting the data for confounders. Researchers took into account other factors that could have affected the results, such as age, sex, presence of long-standing illness, alcohol drinking frequency, psychological distress, BMI, smoking status, education level, and doctor-diagnosed CVD (cardiovascular disease).
Indeed, before accounting for these factors, running and jogging appeared far more beneficial; it was only after all these other factors that clearly affect health outcomes were taken into account that running and jogging dropped below statistical significance.
What neither study accounted for, however, is wealth (or the lack of it). Do wealthy people enjoy better health, on a population-wide level? Most certainly. Could accounting for wealth have affected the results of a study that showed that tennis, a sport mostly practiced by those with plenty of time and money to spare, improved health outcomes?
In short, where studies fail to adjust for all relevant confounders, can one trust the results at all?
Case Study Two - hire the elderly?
Are you a business owner? Do you want a competent workforce? You might want to consider hiring older people for executive positions, if what you are looking for is “valuable knowledge,” according to groundbreaking research published by the American Psychological Association (APA).
(The researchers did not express any surprise at their findings.)
In this study, researchers from a number of universities sampled thousands of individuals aged between 20 and 74, administering various cognitive tests. They found that while
... older executives bring valuable skills to the job...
they also
... performed somewhat worse on tests of general mental ability, and markedly worse on figural and inductive reasoning tests, which assess fluid intelligence.
According to the APA, such research is important because “hiring or promoting based on age-related mental abilities can be a minefield.” But now, with the study’s results in their hands, employers are well-equipped to avoid “age discrimination,” and they won’t make the unfortunate mistake of hiring or promoting younger people in the belief that wisdom accompanies youth rather than age.
Case Study Three - fasting for good health?
Intermittent fasting, also known as time-restricted eating (TRE), is a recent fad among dieters. Researchers from several universities in China and the United States (including Harvard and Wuhan) set out to discover whether such practices are actually beneficial to one’s health.
Their starting hypothesis was that:
Eight-hour TRE is associated with lower risk of all-cause and cause-specific mortality.
Around 20,000 adults over the age of 20 were enrolled in the study. Subsets had either cardiovascular disease or cancer. They were asked to write down, twice at each sampling point, everything they ate in a 24-hour period. Follow-up was between 2003 and 2019.
Trial volunteers were grouped into various categories: those who restricted their eating to an 8-hour window each day; those whose window was 8-10 hours long; those whose window was 10-12 hours long, between 12 and 16 hours long, and over 16 hours in duration.
The results were presented with great fanfare in 2024 at a session of the American Heart Association, with the striking headline: “8-hour time-restricted eating linked to a 91% higher risk of cardiovascular death.”
This was a very interesting study, not least because out of its 36 results, only 4 were statistically significant.
When the study participants were considered as a whole (i.e. not divided between those with CVD or cancer or neither), there were no statistically significant results at all. That is, no observed correlation between length of eating windows and any kind of mortality.
Within the CVD mortality group, 3 out of 12 results were statistically significant. People with CVD were more likely to die if they restricted all their eating to less than 10 hours per day (i.e. the <8-hour group and the 8-10-hour group). There were no other significant correlations.
And within the cancer mortality group, just 1 out of 12 results was statistically significant. Researchers found that people with cancer who ate throughout most of the day (over 16 hours) were less likely to die.
The researchers concluded that,
In US adults, TRE with eating duration <8 hours was significantly associated with a higher risk of cardiovascular mortality in the general population as well as in people with cardiovascular disease or cancer. These findings do not support long-term use of 16:8 TRE for preventing cardiovascular death.
The American Heart Association came to the same conclusions, and also expressed its surprise at the results:
We were surprised to find that people who followed an 8-hour, time-restricted eating schedule were more likely to die from cardiovascular disease ... our research clearly shows that, compared with a typical eating time range of 12-16 hours per day, a shorter eating duration was not associated with living longer.
Nonetheless, they accepted The Science as settled and issued recommendations based on this “latest scientific evidence”:
It’s crucial for patients, particularly those with existing heart conditions or cancer, to be aware of the association with an 8-hour eating window and increased risk of cardiovascular death. Our study’s findings encourage a more cautious, personalized approach to dietary recommendations, ensuring that they are aligned with an individual’s health status and the latest scientific evidence.
Did this study really settle The Science?
The researchers were careful to note that the results do “not mean that time-restricted eating caused cardiovascular death” in study participants.
But what do the results mean?
In any such study, if one finds that more time-restriction leads to higher mortality rates, it would be logical to assume that less restriction will mean lower mortality. But that’s not what the study found. Setting aside the fact that most of the results were not statistically significant, there was no pattern whatsoever. Restricting eating to between 8 and 12 hours per day is apparently mildly bad for one’s health — but so is spreading out one’s eating over more than 16 hours per day (unless you have cancer, that is).
Among the cancer mortality statistics, the only group where any statistical significance was found was among those with a cancer diagnosis at the start of the trial. They appeared to do a lot better if they didn’t restrict their eating to less than 16 hours per day. The only sensible conclusion to be drawn from here is that people with cancer and also a healthy appetite appear to be doing better than cancer patients with less appetite.
The most sensible conclusion of all to draw from such a “research” study, however, is that if you find that virtually all your results lack statistical significance — by a huge margin, as most results here were not anywhere near it — your study is probably not worth a press release.
The information contained in this article is for educational and information purposes only and is not intended as health, medical, financial or legal advice. Always consult a physician, lawyer or other qualified professional regarding any questions you may have about a medical condition, health objectives or legal or financial issues.
