The study was a landmark, one of the few attempts to rigorously evaluate a particular diet. And the results were striking: A Mediterranean diet, with abundant vegetables and fruit, can slash the risk of heart attacks and strokes.
But now that trial, published in the New England Journal of Medicine in 2013, has come under fire. The authors retracted their original paper on Wednesday and published an unusual “re-analysis” of their data in the same journal.
Despite serious problems in the way the study was conducted, their conclusions are the same: A Mediterranean diet can cut the risk of heart attacks and strokes by about 30 per cent in those at high risk.
Not everyone is convinced. “Nothing they have done in this re-analysed paper makes me more confident,” said Dr Barnett Kramer, director of the division of cancer prevention at the National Cancer Institute in the US.
For decades, researchers have noted that people living in some Mediterranean countries have lower rates of heart disease and cancer. Scientists have long suspected that the regional diet — rich in fruits, vegetables, nuts and olive oil, with moderate levels of fat — played a protective role.
But the idea has been hard to prove. It is very difficult to test any diet in a clinical trial. Participants may be reluctant to stick to the prescribed meal plan, for instance, and it can be difficult to monitor them over months or years.
The original study was conducted in Spain by Dr Miguel A. Martínez-González of the University of Navarra and his colleagues. The trial enrolled 7,447 participants aged 55 to 80 who were assigned one of three diets: a Mediterranean diet with at least four tablespoons a day of extra-virgin olive oil; the same diet with an ounce of mixed nuts; or a traditional low-fat diet.
The participants were followed for a median of nearly five years. Martínez-González and his colleagues reported that there were fewer cardiovascular events in the groups consuming olive oil and nuts.
But last year Martínez-González found his study on a list of clinical trials whose data seemed suspect, compiled by Dr John Carlisle of Torbay Hospital in England.
“That was the first hint that there could have been some imperfection,” Martínez-González said in an interview.
A statistician at the New England Journal of Medicine suggested the researchers look at the methods at each centre that recruited participants.
The idea of a randomised trial is to assign treatments — in this case, diets — to participants with the statistical equivalent of a coin toss. That way, the groups being compared should be equivalent, with no group healthier or sicker, or older or younger, than another on average.
The original study was a landmark, one of the few attempts to rigorously evaluate a particular diet. Source: Getty Images
If subjects are not assigned at random, the investigators cannot be sure that the effects they see result from the treatment. And attempts to correct statistically after the fact are fraught with difficulty.
On re-evaluating their data, the scientists running the Mediterranean diet study soon found what Martínez-González said were “small problems affecting 10 per cent of participants.”
Some investigators would assign one person in a household — the wife, for example — to one arm of the study — say, to the group consuming olive oil. Then they would ask other members of the household to share that diet, including them as if they had been randomly assigned to it.
“We realised we had never reported that,” Martínez-González said.
An omission like that erodes the randomised nature of the trial. Family members are likely to share more than just a diet: If a husband and wife both dodge heart disease, it’s difficult to say that their diet is the reason.
In their re-analysis, the investigators statistically adjusted data on 390 people who happened to be household members but whose diets were not randomly assigned.
Then the investigators discovered another problem.
A researcher at one of the 11 clinical centres in the trial worked in small villages. Participants there complained that some neighbours were receiving free olive oil, while they got only nuts or inexpensive gifts.
The science behind the Mediterranean diet might be flawed. Source: Pixabay
So the investigator decided to give everyone in each village the same diet. He never told the leaders of the study what he had done.
“He did not think it was important,” Martínez-González said.
But the decision meant that participants were not truly randomised and forced Martínez-González and his colleagues to make another statistical adjustment to data on 652 people in the trial.
The investigators spent a year working on the re-analysis in collaboration with Dr Miguel Hernan of the Harvard T.H. Chan School of Public Health.
In the end, they concluded that the original findings were still accurate.
“You cannot imagine what it has been like,” Martínez-González said, adding that he and his team worked through vacations and weekends — and swallowed considerable professional embarrassment.
Randomised trials are difficult, other experts agreed, and randomised diet studies so perilous they are seldom attempted.
“These people were naive,” said Donald Berry, a statistician at MD Anderson Cancer Centre in Houston. “They were sloppy and didn’t know they were being sloppy.”
Berry said he wants to believe the results: He loves nuts and has taken to cooking with extra-virgin olive oil. But he remains unconvinced: the re-analysis does not solve the study’s problems, he said.
Bradley Efron, a statistics professor at Stanford University, also was sceptical. The revamped results “wouldn’t convince me to be on a Mediterranean diet,” he said.
But Dr Steven Nissen, a cardiologist at the Cleveland Clinic, is persuaded and plans to continue advising patients to go on the Mediterranean diet.
When the initial paper was published, Nissen said, “I was thrilled to see what seemed to be an impeccable trial.”
Although it was “sobering” to learn of the errors, “I was reassured that the conclusions are correct,” he said.
Martínez-González shares the sentiment. “After all this long work, I am more convinced than ever” by the study’s data.
“Seldom has a trial undergone more scrutiny,” he added.
