HYPOTHESIS TESTING

Tags: #◇Computer Science #◇STT101A #◇Conceptual #◇Hypothesis Testing #◇Statistics #◇Probability

---

◇Overview

Hypothesis testing is the cornerstone of inferential statistics. It provides a formal, objective mathematical framework for determining whether experimental or sample data provides sufficient empirical evidence to reject a baseline assertion (the null hypothesis) in favor of a competing claim (the alternative hypothesis).

◇Key Definitions

• Null Hypothesis ($H_0$): The default presumption of no effect, no difference, or status quo: $H_0: \theta = \theta_0$.
• Alternative Hypothesis ($H_1$ or $H_a$): The research claim or hypothesis indicating that an effect, disparity, or change exists:
  • Two-tailed: $H_a: \theta \ne \theta_0$
  • Right-tailed (upper): $H_a: \theta > \theta_0$
  • Left-tailed (lower): $H_a: \theta < \theta_0$
• Test Statistic: A sample-derived standardized random variable ($Z$, $t$, $\chi^2$, or $F$) quantifying the discrepancy between the sample observation and the null hypothesis parameter.
• Significance Level ($\alpha$): The maximum permissible threshold probability of committing a Type I error: $\alpha = P(\text{Reject } H_0 \mid H_0 \text{ is true})$. Typically set to $0.05$, $0.01$, or $0.001$.
• $p$-Value (Probability Value): The exact probability, computed assuming $H_0$ is true, of obtaining a test statistic at least as extreme as the observed sample value.
  • Decision Rule: If $p \le \alpha$, reject $H_0$ (statistically significant).
  • If $p > \alpha$, fail to reject $H_0$ (insufficient evidence).
• Critical Region (Rejection Region): The set of test statistic values for which the null hypothesis is rejected.
• Statistical Power ($1 - \beta$): The probability of correctly rejecting a false null hypothesis: $P(\text{Reject } H_0 \mid H_0 \text{ is false})$.

---

◇Theoretical Framework: Error Classification Matrix

In any statistical hypothesis test, two distinct error states can occur:

Test Statistic Selection Heuristic

---

◇Stepped Numerical Walkthrough: Two-Tailed $Z$-Test

Scenario: A cloud infrastructure engineer tests the round-trip latency of a modified database cluster. Under the default configuration, latency has a mean of $\mu_0 = 250\text{ ms}$ with a known population standard deviation $\sigma = 32\text{ ms}$. A random sample of $n = 64$ query responses under heavy load yields a sample mean latency of $\bar{x} = 258\text{ ms}$. Test at the $\alpha = 0.05$ significance level whether the true mean latency has changed.

Step 1: Formulate Hypotheses

$H_0: \mu = 250\text{ ms} \quad (\text{No change in latency})$
$H_a: \mu \ne 250\text{ ms} \quad (\text{Two-tailed test: latency differs})$

Step 2: Establish Significance Level & Critical Values

Given $\alpha = 0.05$ for a two-tailed test, the significance area is split equally into both tails: $\alpha / 2 = 0.025$.
From the standard normal cumulative table ($Z \sim \mathcal{N}(0, 1)$):
$z_{\alpha/2} = z_{0.025} = 1.960$
Rejection Region: Reject $H_0$ if $Z_{\text{calc}} > 1.960$ or $Z_{\text{calc}} < -1.960$ (i.e., $|Z_{\text{calc}}| > 1.960$).

Step 3: Compute Standard Error & Test Statistic

Standard Error of the Mean:
$\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}} = \frac{32}{\sqrt{64}} = \frac{32}{8} = 4.0\text{ ms}$
Standardized Test Statistic ($Z_{\text{calc}}$):
$Z_{\text{calc}} = \frac{\bar{x} - \mu_0}{\sigma_{\bar{x}}} = \frac{258 - 250}{4.0} = \frac{8}{4.0} = \mathbf{+2.00}$

Step 4: Compute the Exact $p$-Value

For a two-tailed test:
$p = 2 \cdot P(Z \ge |Z_{\text{calc}}|) = 2 \cdot P(Z \ge 2.00)$
Using the standard normal CDF: $P(Z < 2.00) = 0.97725$, so $P(Z \ge 2.00) = 1 - 0.97725 = 0.02275$.
$p = 2 \times 0.02275 = \mathbf{0.0455}$

Step 5: Statistical & Practical Conclusion

• Comparison: $p = 0.0455 \le \alpha = 0.05$, and $|Z_{\text{calc}}| = 2.00 > 1.960$.
• Formal Decision: Reject $H_0$ at the $5\%$ significance level.
• Engineering Conclusion: There is sufficient empirical evidence to conclude that the database modification resulted in a statistically significant change in mean query response time.

---

◇References & Navigation

• Previous Topic: ◇Normal Distribution
• Next Topic: ◇Bayes's Formula
• Related Notes: ◇Discrete Probability Distribution, ◇Random Variables