Randomized controlled trials have long been held up as the “gold standard” of clinical research. There’s no doubt that well-designed trials are effective tools for testing a new drug, device, or other intervention. Yet much of modern medical care — perhaps most of it — is not based on randomized controlled trials and likely never will be. In this “dark matter” of clinical medicine, past practices and anecdotes all too often rule. We need to look beyond trials to improve medical care in these areas.
In a randomized controlled trial (RCT), participants are randomly assigned to receive either the treatment under investigation or, as a control, a placebo or the current standard treatment. The randomization process helps ensure that the various groups in the study are virtually identical in age, gender, socioeconomic status, and other variables. This minimizes the potential for bias and the influence of confounding factors.
Despite their strengths, RCTs have substantial limitations. They can be very expensive to run. They can take many years to complete, and even then may not last long enough to assess the long-term effect of an intervention such as vaccine immunity, or to detect rare or long-term adverse effects. Findings from RCTs may not be valid beyond the study population — a trial that included a high-risk population in order to maximize the possibility of detecting an effect, for example, may not be relevant to a low-risk population. RCTs may not be practical for population-wide interventions and often aren’t relevant for urgent health issues such as infectious disease outbreaks, for which public health decisions must be made quickly.
As I write this week in the New England Journal of Medicine, several other study types can generate data that are at least as effective as RCTs, or may be even more effective, at generating evidence for action, especially related to population-wide interventions.
The effectiveness of the nasal spray flu vaccine (also called the live attenuated vaccine) is a dramatic illustration of the limitations of RCTs. Trials suggested that the nasal spray vaccine was superior to flu shots, at least for some populations. In subsequent years, however, observational studies, including case-control studies, documented that, for reasons which are still unclear, the nasal spray wasn’t effective against the flu. That led the Advisory Committee on Immunization Practices to recommend, and the CDC to accept the recommendation, that the nasal spray flu vaccine not be used in the 2016-2017 flu season.