#1

A great article on how two “negative” randomized trials from the New England Journal of Medicine led to a “positive” meta-analysis of these same two trials in NEJM Evidence: Treating Patients, Not P Values 2024

It shows that two trials can be in the right direction but statistically “negative” if underpowered and combining them in a meta-analysis done properly can overcome that.

They conclude:

Paying closer attention to point estimates and 95% CIs and less attention to P values is an important lesson. This view is also shared by the American Statistical Association in their widely cited paper The ASA’s Statement on P-Values: Context, Process, and Purpose.
Solely applying a P value threshold or 95% CI to determine if a treatment works or not is an oversimplification. There are many other factors to weigh, including clinical factors (e.g., treatment side effects, costs, severity of the clinical condition), whether other treatments exist for the condition (prior to these trials there were no demonstrated treatment options for the prevention of MDR-TB), and study design (e.g., inclusion of a placebo, duration of follow-up), to name just a few. The same is also true when doctors interpret a patient’s laboratory test results. Perhaps the adage “We treat patients, not lab results” can be augmented to “We treat patients, not P values.”

2 Likes