What is a P-Value? A Simple Guide
What is a P-Value?
A p-value is a statistical metric that measures the probability that an observed result occurred by chance.
It is the gatekeeper of science. It decides which drugs get approved. It decides which papers get published. But most people (including many scientists) misunderstand it.
The Symptom: The "Significance" Obsession
You read a study that says: "The drug worked (p < 0.05)." You assume this means: "There is a 95% chance the drug works."
This is wrong. The p-value does not tell you the probability that the hypothesis is true.
The Mechanism: The Coin Flip Analogy
Imagine a coin. You suspect it is a "trick coin" (weighted to land on heads). To test this, you flip it 10 times. You get 10 heads in a row.
- The Null Hypothesis ($H_0$): The coin is fair. The result is just luck.
- The P-Value: The probability of getting 10 heads in a row assuming the coin is fair.
For 10 heads, the probability is 1 in 1024. So, p = 0.0009.
This is very low. It is extremely unlikely to happen by chance. Therefore, you reject the Null Hypothesis. You conclude the coin is likely tricked.
The Threshold: 0.05
In science, we use a standard cutoff: 0.05 (5%).
- p < 0.05: The result is unlikely to be noise. We call this "Statistically Significant."
- p > 0.05: The result could easily be noise. We call this "Not Significant."
This is a filter. It prevents us from getting excited about random fluctuations.
Common Misconceptions
Myth 1: "A low p-value means a large effect."
False. In a massive study (100,000 people), a tiny difference can have a tiny p-value.
- Drug A lowers blood pressure by 0.1 mmHg.
- p < 0.001 (Highly significant).
- Reality: 0.1 mmHg is clinically useless. (See Clinical vs. Statistical Significance).
Myth 2: "p > 0.05 means zero effect."
False. It means you did not find an effect. Maybe the effect is there, but your study was too small to see it. This is a "false negative" or Type II Error.
Myth 3: "p = 0.05 means there is a 95% chance the theory is true."
False. It means there is a 5% chance you would see this data if the theory were false. That is a very different thing.
The Prescription: Look Closer
Because 0.05 is the "magic number" for publication, scientists are tempted to cheat. They might:
- Test 20 variables and report the one that works.
- Stop the study as soon as p < 0.05.
- Exclude outliers to lower the p-value.
This is called P-Hacking.
When you read a study:
- Look at the exact number. p = 0.049 is suspicious. p = 0.0001 is robust.
- Look at the Effect Size. Does the drug actually do anything meaningful?
- Look at the Confidence Interval. How precise is the estimate?
At PaperScores, we analyze the p-values. We check if they support the claims. We do not let bad statistics hide behind "significance."