Introduction

ANOVA is a statistical method used to **compare the means of three or more groups** to determine if at least one group mean differs significantly from the others.

It extends the t-test (which compares two means) to multiple groups while controlling the Type I error rate.

🔹 Intuitive Idea

Instead of comparing means pairwise (like multiple t-tests), ANOVA compares variability between groups to variability within groups.

If the between-group variability is much larger than the within-group variability, it suggests that the group means are not all equal.

🔹 Hypotheses

🔹 The ANOVA Concept

ANOVA divides the **total variation** in the data into two components:

🔹 The F-statistic

The F-statistic measures the ratio of between-group variance to within-group variance:

• \(SS\): Sum of squares
• \(df\): Degrees of freedom
• \(MS\): Mean square (average sum of squares)

If \(F\) is large, it indicates that group means differ more than expected by random chance.

🔹 One-Way ANOVA

Purpose: Compare the means of three or more groups based on one independent variable (factor).

Example: You test whether three fertilizers produce different average plant growth.

Model:

where:

• \(Y_{ij}\): observation \(j\) in group \(i\)
• \(\mu\): overall mean
• \(\tau_i\): effect of group \(i\)
• \(\varepsilon_{ij}\): random error term (assumed normally distributed)

Assumptions:

1. Independence of observations
2. Normal distribution within each group
3. Homogeneity of variances (equal variances across groups)

Decision Rule:

• Compute the F-statistic
• Compare it with the critical F-value from the F-distribution at a given significance level (e.g., α = 0.05)
• If F > F_critical, reject H₀

Post-hoc Tests (if H₀ is rejected)

If you find a significant difference, use post-hoc tests such as Tukey’s HSD or Bonferroni to identify which groups differ.

Two-Way ANOVA

Purpose

Two-way ANOVA is used to compare means across groups when you have two independent variables (factors). It also allows testing for interaction effects between the factors.

Example

You test how fertilizer type (A, B, C) and sunlight exposure (Low, High) affect plant growth.

Model

where:

• μ: overall mean
• α_i: effect of Factor A (e.g., fertilizer)
• β_j: effect of Factor B (e.g., sunlight)
• (αβ)_ij: interaction effect between A and B
• ϵ_ijk: random error term

Hypotheses

1. For Factor A:
   $$ H_0: \alpha_1 = \alpha_2 = \cdots = 0 $$
2. For Factor B:
   $$ H_0: \beta_1 = \beta_2 = \cdots = 0 $$
3. For Interaction (A × B):
   $$ H_0: (\alpha\beta)_{ij} = 0 $$

If the interaction is significant, it should be interpreted before examining the main effects.

Outputs

Two-way ANOVA produces three F-statistics:

1. Effect of Factor A
2. Effect of Factor B
3. Effect of the interaction term (A × B)

Interpreting Results

Summary

In Short

• ANOVA tests whether group means differ significantly.
• The F-ratio compares between-group and within-group variation.
• One-way ANOVA → one factor; Two-way ANOVA → two factors (with interaction).
• Post-hoc tests help identify which groups differ.

When to use t-test vs ANOVA: Choosing the Right Statistical Test

The most important difference between a t-test vs ANOVA is the number of groups you're comparing:

• T-Test: Use when comparing the means of two groups.
• ANOVA: Use when comparing the means of three or more groups.

Key Differences at a Glance

hy Not Use Multiple t-Tests Instead of ANOVA?

Example:

• Testing 3 groups using pairwise t-tests requires 3 comparisons.
• Each test at α = 0.05 increases the chance of false significance.
• ANOVA controls this by testing all groups simultaneously.

Decision Guide (Quick Rule)

Example: Sepal length of Iris species

                import seaborn as sns
                import matplotlib.pyplot as plt
                from scipy.stats import ttest_ind, f_oneway

                # Load dataset
                iris = sns.load_dataset("iris")

                # Split groups
                setosa = iris[iris["species"] == "setosa"]["sepal_length"]
                versicolor = iris[iris["species"] == "versicolor"]["sepal_length"]
                virginica = iris[iris["species"] == "virginica"]["sepal_length"]

                # ----- T-TEST -----
                t_stat, p_val = ttest_ind(setosa, versicolor)
                print("T-test (Setosa vs Versicolor)")
                print("t =", t_stat, "p =", p_val)

                # ----- ANOVA -----
                f_stat, p_val_anova = f_oneway(setosa, versicolor, virginica)
                print("\nANOVA Result")
                print("F =", f_stat, "p =", p_val_anova)
            

Code output:

                T-test (Setosa vs Versicolor)
                t = -10.52098626754911 p = 8.985235037487079e-18

                ANOVA Result
                F = 119.26450218450468 p = 1.6696691907693826e-31
            

                import seaborn as sns
                import matplotlib.pyplot as plt

                sns.boxplot(x="species", y="sepal_length", data=iris)
                plt.title("Sepal Length Comparison Across Species")
                plt.show()