What Is Level of Significance?
The level of significance α is the probability of rejecting a true null hypothesis, called a Type I error. Common choices are 0.10, 0.05 and 0.01, but the value should be chosen before you look at the data, based on the cost of a false positive.
Enter α and choose a one-tailed or two-tailed test to get the critical value and the tail area in each rejection region. Add an effect size and sample size to see the power (1 − β) and the Type II error probability. The page also shows how many false positives to expect across many tests and the Bonferroni-adjusted α for 10 tests.
Below you will find the formulas, a variables table, a worked example and answers to common questions such as significance level versus p-value.
How to Use the Level of Significance Calculator
- Select α and whether the test is one- or two-tailed.
- Choose z or t; for t, enter the degrees of freedom.
- Optionally enter an effect size d and sample size n to see power and Type II error.
- Read the critical values and use them to decide a test: reject H₀ if your statistic falls in the shaded region.
Level of Significance Formula
Significance level
Two-tailed critical value
Power (two-tailed z test)
Level of Significance Variables Explained
| Symbol | Meaning | Notes |
|---|---|---|
| α | Significance level | Type I error rate |
| zα | Critical value | Cut-off for the rejection region |
| β | Type II error probability | Failing to detect a real effect |
| 1 − β | Power | Chance of detecting a real effect |
Level of Significance Worked Example
With the example values loaded in the calculator above, the steps are:
- α = 0.05 is the probability of rejecting a true null hypothesis (Type I error).
- For a two-tailed z test, the critical value is ±1.96.
- Noncentrality = d√n = 2.828; power = 0.8074, so β = 0.1926.
Result: Significance level α = 0.05; Critical values (±) = ± 1.96; Power (1 − β) = 80.74 %.
How to Read the Significance Level
A significance level of α = 0.05 on a two-tailed z-test puts 0.025 in each tail, with critical values of ±1.96. It is the chance of a false positive when the null hypothesis is true. That is why 5 out of 100 true-null tests would be expected to come up significant by luck alone.
Lowering α reduces false positives but costs power. For an effect of 0.5 standard deviations and n = 32, the power at α = 0.05 is 80.74%, so β, the chance of missing a real effect, is 19.26%. When you run several tests at once, the Bonferroni level divides α by the number of tests: 0.05 ÷ 10 = 0.005.
Type I and Type II errors
The two errors trade off for a fixed sample size: demanding stronger evidence (smaller α) makes a Type II error more likely. The only way to improve both at once is to collect more data or reduce noise.
| H₀ true | H₀ false | |
|---|---|---|
| Reject H₀ | Type I error (probability α) | Correct: power (1 − β) |
| Fail to reject H₀ | Correct (1 − α) | Type II error (probability β) |
How to choose α
Choose α based on the consequences of a false positive and decide it in advance. Picking α after seeing the p-value is a form of p-hacking.
| α | Confidence | Typical use |
|---|---|---|
| 0.10 | 90% | Exploratory work, early screening |
| 0.05 | 95% | Standard in most sciences and business |
| 0.01 | 99% | When false positives are costly, such as medical claims |
| 0.001 | 99.9% | Very high-stakes or many comparisons |
| 5 × 10⁻⁷ | 5 sigma | Particle physics discovery claims |
Multiple comparisons
If you run m independent tests at α = 0.05 each, the chance of at least one false positive is 1 − 0.95^m, which is about 40% for 10 tests. The Bonferroni correction divides α by m; the results show the corrected value for 10 tests. Less conservative procedures such as Holm or Benjamini-Hochberg are used when many tests are run.
Power and sample size
Power is the probability of correctly rejecting a false null hypothesis. Conventional targets are 80% or 90%. The chart shows how power climbs with sample size for your chosen effect size. Small effects need very large samples: detecting d = 0.2 at 80% power needs roughly 200 observations per group in a two-sample test, compared with about 25 for d = 0.8.
Where Level of Significance Is Used
Choosing α
See what a significance level means for false positives.
Critical values
Get the rejection region for z, t, χ² or F.
Power
Estimate power and the Type II error for a given effect size and n.
Teaching
Show expected false positives in 100 tests.
Common Level of Significance Mistakes to Avoid
- Set α before looking at the data.
- α = 0.05 is a convention, not a law of nature.
- Report exact p-values rather than only “significant / not significant”.
- Check power before running a study, not after.
Level of Significance Terms and Definitions
- Rejection region
- The set of test statistic values that lead to rejecting H₀.
- Critical value
- The boundary of the rejection region.
- Power
- The probability a test detects a true effect of a given size.
Level of Significance FAQs
What is the level of significance?
It is alpha, the probability of rejecting a true null hypothesis (a Type I error), chosen before the test. Alpha = 0.05 means a 5% false positive rate.
Why is 0.05 the usual significance level?
It is a long-standing convention that balances false positives against missed effects in many fields. It is not a theoretical optimum.
What is the difference between significance level and p-value?
The significance level α is a threshold you choose beforehand. The p-value is computed from your data and compared with it.
What is the critical value for 0.05?
For a two-tailed z test it is ±1.96; for a one-tailed test it is 1.645.
How does α relate to confidence level?
Confidence level = 1 − α. A 5% significance level corresponds to a 95% confidence level.
What is statistical power?
The probability of detecting an effect that really exists. It equals 1 minus the Type II error rate.
Does a smaller α always make a test better?
No. It reduces false positives but increases false negatives unless the sample size grows.
How are alpha and power related?
Lowering alpha cuts false positives but, for a fixed sample size, lowers power. The tool shows power for your effect size and sample size, 80.74% in the example.
Why use a Bonferroni alpha?
Running many tests inflates the chance of a false positive. Dividing alpha by the number of tests keeps the overall rate near your target, for example 0.05 ÷ 10 = 0.005.