P-values and probability

The probability of obtaining a p-value of 0.1 or lower is exactly 0.1 (when
**H _{0}** holds).

Similarly, the probability of obtaining a p-value of 0.01 or lower is exactly
0.01, etc. (when **H _{0}** holds).

P-values are most likely to be near 0 if the alternative
hypothesis holds |

Example: Test for zero mean

Take about 50 samples. The cumulative proportions on the top right are close
to a straight line between (0,0) and (1,1), so about 50% of p-values are less
than 0.5, 20% are less than 0.2, etc. when **H _{0}** is true.

Use the slider to change the true population mean to 1.5 and repeat. Now more than 50% of p-values are less than 0.5, more than 20% are less than 0.2, etc.

Interpretation of low p-values

If the p-value for a test is 0.0023, we note that such a low p-value is unlikely
if **H _{0}** is true -- it would only occur with probability 0.0023.
We therefore conclude that the data give strong evidence that

A p-value of 0.0023 could
arise when either H or _{0}H holds. However
it is unlikely when _{A}H is true and more likely when _{0}H
is true._{A} |

General guide

p-value | Interpretation |
---|---|

over 0.1 | no evidence that the null hypothesis does not hold |

between 0.05 and 0.1 | very weak evidence that the null hypothesis does not hold |

between 0.01 and 0.05 | moderately strong evidence that the null hypothesis does not hold |

under 0.01 | strong evidence that the null hypothesis does not hold |