Math 225

Introduction to Biostatistics

Notes from Lecture #17

1. Analysis of Variance. Analysis of variance (ANOVA) may be thought of as a generalization of hypothesis tests for differences in means from two independent samples to cases where there are more than two populations of interest. In this situation, a null hypothesis is a statement that all population means are equal. A single difference in means is not a sufficient test statistic, so we introduce the F test statistic for the more complicated situation. The main idea is that variability among the sample means is compared to the variability within samples. If the first measure is larger than can be easily explained by chance, there is evidence that the population means are not all equal.

2. A motivating problem. See the Cuckoo bird page for a description of a problem and summary data to examine it. Cuckoo birds lay eggs in the nests of other species and form subpopulations who specialize in laying eggs in nests of a particular host. A specific question is, are the sizes of cuckoo eggs laid by different subpopulations different?

3. Graphing data. Before a formal anaysis, we can examine side-by-side boxplots of sample data. In the cuckoo example, this plot shows that the distribution of lengths of cuckoo eggs laid in wren nests appears to be shifted down relative to the other distributions. ANOVA can be used to see if this difference is not well explained by chance.

4. The F statistic. We can formally test the hypothesis

   H₀: mu₁ = mu₂ = ... mu_g
   H_a: all mu_i are not equal

   by calculating an F test statistic to compare to an F distribution. The F test statistic is a ratio of a measure of variabilty among sample means to a measure of variability within samples. F statistics significantly larger than one are evidence in favor of different population means.

5. ANOVA Table. An ANOVA table is used to calculate an F statistic. Here is a summary of relevant notation.

   g = number of groups
   n_i = sample size of ith group
   N = sum of the g sample sizes (total number of observations)
   xbar_i = mean of ith sample
   xbar = grand mean (mean of all observations)
   s_i = standard deviation of ith sample
   SS_among = sum of squares among sample means
   SS_within = sum of squares within samples
   df_among = degrees of freedom among sample means
   df_within = degrees of freedom within samples
   MS_among = mean square among sample means
   MS_within = mean square within samples
   

   The sums of squares are the most tedious objects to calculate from the raw data.

   SS_among = sum ( n_i (xbar_i - xbar)² ) is a weighted measure of the squared deviations of the sample means from the grand mean with weights equal to the sample sizes.

   SS_within = sum ( (n_i-1)s_i² ) is a weighted measure of the squared deviations of individual observations from their sample means.

   SS_total = SS_among + SS_within is a measure of the sum of squared deviations from the individual observations to the grand mean.

   df_among = g-1 is the degrees of freedom among the g sampel means.

   df_within = N-g = sum ( n_i - 1 ) is the sum of the degrees of freedom within each sample.

   Each mean square is the sum of squares divided by the corresponding degrees of freedom.

   The F statistic is

   F = MS_among / MS_within

   The p-value is the area to the right of the F statistic under an F distribution with g-1 and N-g degrees of freedom.

6. ANOVA table from example. For the example, here is the ANOVA table from S-PLUS.

                Df Sum of Sq Mean Sq F Value        Pr(F) 
   birdSpecies   5  42.93965 8.58793 10.3877 3.152104e-08
     Residuals 114  94.24835 0.82674                     
   

   In the style of the textbook, it would be like this.

           SS        df MS      F       p-value
   ------------------------------------------------- 
   among   42.93965   5 8.58793 10.3877 3.152104e-08
   within  94.24835 114 0.82674                     
   -------------------------------------------------
   total  137.188   119 
   

   We can estimate the size of a typical deviation of an observation from its sample mean.

   sqrt ( MS_within ) = sqrt( 0.82674 ) = 0.91. Notice that this is in the range of the six sample standard deviations.

   The small p-value indicates very strong support against the null hypothesis. There is substantial evidence that the mean cuckoo bird egg sizes are not all the same for the different subpopulations.