Are randomized-controlled trials really the 'gold standard'?

Randomized-controlled trials (RCTs) have come to serve many purposes in the COVID era, different purposes for different people, but do they actually serve the purpose people believe they do, and do they live up to being the supposed “gold standard” of scientific research or evidence-based medicine?

Let’s take some of the assumptions people make about RCTs and see if they tally with reality.

A RCT comprehensively reviews the drug in question, testing it on its target population.

RCTs are conducted by pharma companies who are free to cherry-pick trial participants. For instance, mRNA vaccines for COVID were tested exclusively on younger, basically healthy volunteers, even though COVID only poses a significant danger to the elderly and/or infirm. The FDA then approved mRNA vaccines based on these trials, for everyone over the age of 18, including pregnant women, even though the drugs were never tested on any high-risk groups.

The results of a RCT are scientifically conclusive regarding the drug concerned; that is to say, the results can be expected to be replicated on a future occasion, with no surprises in store.

A pharma company can run as many RCTs as it likes. Suppose it ran 100, and the results of 98 showed that the drug either didn’t work or was dangerous (or both), but 2 RCTs showed that the drug has some benefit and the side-effects are moderate to low. The pharma company can take these 2 RCTs to the FDA and get the drug approved, and the FDA doesn’t need to, and in fact won’t, ask if those 2 trials were the only ones conducted. The results of 98 failed RCTs simply disappear. The public has no way of accessing that data; in fact, even the FDA has no way of accessing it, and no one other than the pharma officials knows that it even exists.

If a drug is approved based on a “successful” RCT, the public can trust that it has been shown to be safe.

Following on from the previous point, there could be any number of failed RCTs for a specific drug before the company seized on one successful trial. However, even the success of that one RCT doesn’t necessarily tell you anything about safety. Unless the trial was specifically examining the safety profile of a drug, there is no guarantee that safety issues were adequately addressed or even examined. Furthermore, it is all too easy for pharma companies to delete, relabel, or dismiss adverse effects of the drug under trial. The FDA has no way of verifying the details without a comprehensive investigation, as it is not given the raw data from the trials including the names of participants. 

A RCT focuses on the primary effect of the drug in question. All other effects can be assumed to be secondary and of minor importance.

A RCT can focus on whatever the trial managers want to focus. To use just one example of many possibilities, the anti-depressant Paxil. The FDA approved it on the basis of an RCT that showed a marginal benefit over placebo in improving mood which translates to a mood benefit for around 1 person in 10 who takes the drug. However, Paxil’s effect on sexual dysfunction has been estimated as anywhere between 1.5 and 9 people in 10 who take Paxil. Yet dysfunction is considered a side-effect, and Paxil is considered a mood enhancer.

Doctors can derive best clinical practices from RCTs that show that some drugs help and others are either useless or harmful.

Here we provide the testimony of Dr. Paul Marek on the often devastating effects of treating patients based on trials rather than on practical clinical experience. Dr. Marek was addressing a panel convened by Senator Ron Johnson.

Remdesivir, the only drug approved for COVID patients, passed its RCT with vaguely flying colors, despite the fact that even the WHO concluded, based on trial data, that it had no significant effect on mortality rates. That didn’t matter, though, because the RCT didn’t examine mortality rates, but only reduction of time to recovery. Patients who died were thus by definition excluded from the primary endpoint of the study, and deaths from or with remdesivir were merely an unfortunate side-effect.

So why are RCTs sought, esteemed, and relied upon? For doctors, they provide a safety net, absolving them of real accountability to their patients. For the FDA, they promote the appearance of genuine consideration of the risk-benefit analysis for the drugs they approve. For the patient, they provide (false) reassurance that their drug has passed rigorous scientific testing. For thinking people, their only benefit is that the limited information released to the public can be pored over with a microscope and discrepancies and warning signs sought out.

For example, a trial study that notes that “the majority of adverse events were mild or moderate” begs the question: what about the minority of adverse effects? (This, from Pfizer’s trials testing mRNA shots for young children.)

In the words of Dr. David Healy:

... we license statins on the basis of lowered cholesterol levels rather than any benefit on mortality. We license hypoglycemic agents on the basis of an ability to lower glucose, even though this commonly leads to more hospitalizations than diabetes. The drugs for osteoporosis are licensed on the basis of an ability to thicken bones rather than reduce fractures with the result that there are now more serious fractures of abnormally thickened bones in the elderly after falls than there ever were before these drugs were introduced. We have the same situation with the mRNA trials ... reducing apparent infection rates may even be a bad thing. Abolishing symptoms like cough may make infected younger people more likely to kill their elderly relatives…