Subgroup Analysis

The main study failed. The p-value was 0.08. The researchers are desperate. They start slicing the data.

"Did it work for men? No." "Did it work for women? No." "Did it work for smokers? No." "Did it work for smokers over 50? YES! (p < 0.05)"

They publish: "Drug X Cures Smokers Over 50!"

The ISIS-2 Trial

In 1988, a famous heart attack study (ISIS-2) showed that aspirin saved lives. To prove a point about subgroup analysis, the researchers analyzed the data by Astrological Sign.

They found that aspirin worked for everyone... except Libras and Geminis. For them, it was useless.

Was this real? No. It was random noise.

The Diagnosis

Subgroup analyses are for generating hypotheses, not proving them.

If the main result is negative, ignore the subgroups. They are usually ghosts.