Repeated Measures Design

Repeated measures design (also known as "within-subjects design") uses the same subjects with every condition of the research, including the control. For instance, repeated measures are collected in a longitudinal study in which change over time is assessed. Other studies compare the same measure under two or more different conditions. For instance, to test the effects of caffeine on cognitive function, a subject's math ability might be tested once after they consume caffeine and another time when they consume a placebo.

Repeated Measures Design

An example of a test using a repeated measures design to test the effects of caffeine on cognitive function. A subject's math ability might be tested once after they consume a caffeinated cup of coffee, and again when they consume a placebo.

Repeated measures design can be used to:

• Conduct an experiment when few participants are available: The repeated measures design reduces the variance of estimates of treatment-effects, allowing statistical inference to be made with fewer subjects.
• Conduct experiment more efficiently: Repeated measures designs allow many experiments to be completed more quickly, as only a few groups need to be trained to complete an entire experiment.
• Study changes in participants' behavior over time: Repeated measures designs allow researchers to monitor how the participants change over the passage of time, both in the case of long-term situations like longitudinal studies and in the much shorter-term case of order effects.

Advantages and Disadvantages

The primary strengths of the repeated measures design is that it makes an experiment more efficient and helps keep the variability low. This helps to keep the validity of the results higher, while still allowing for smaller than usual subject groups.

A disadvantage of the repeated measure design is that it may not be possible for each participant to be in all conditions of the experiment (due to time constraints, location of experiment, etc.). There are also several threats to the internal validity of this design, namely a regression threat (when subjects are tested several times, their scores tend to regress towards the mean), a maturation threat (subjects may change during the course of the experiment) and a history threat (events outside the experiment that may change the response of subjects between the repeated measures).

Repeated Measures ANOVA

Repeated measures analysis of variance (rANOVA) is one of the most commonly used statistical approaches to repeated measures designs.

Partitioning of Error

One of the greatest advantages to using the rANOVA, as is the case with repeated measures designs in general, is that you are able to partition out variability due to individual differences. Consider the general structure of the $F$-statistic:

 $F = \dfrac{MS_{\text{treatment}}}{MS_{\text{error}}} = \dfrac{SS_{\text{treatment}} / df_{\text{treatment}}}{SS_{\text{error}} / df_{\text{error}}}$

In a between-subjects design there is an element of variance due to individual difference that is combined in with the treatment and error terms:

$SS_{\text{total}} = SS_{\text{treatment}} + SS_{\text{error}}$ $df_{\text{total}} = n-1$

In a repeated measures design it is possible to account for these differences, and partition them out from the treatment and error terms. In such a case, the variability can be broken down into between-treatments variability (or within-subjects effects, excluding individual differences) and within-treatments variability. The within-treatments variability can be further partitioned into between-subjects variability (individual differences) and error (excluding the individual differences).

$SS_{\text{total}} = SS_{\text{treatment}} + SS_{\text{subjects}} + SS_{\text{error}}$

$\begin{aligned} df_{\text{total}} &= df_{\text{treatment}} + df_{\text{between subjects}} + df_{\text{error}}\\ &= (k-1) + (n-1) + ((n-k)-(n-1)) \end{aligned}$

In reference to the general structure of the $F$-statistic, it is clear that by partitioning out the between-subjects variability, the $F$-value will increase because the sum of squares error term will be smaller resulting in a smaller $MS_{\text{error}}$. It is noteworthy that partitioning variability pulls out degrees of freedom from the $F$-test, therefore the between-subjects variability must be significant enough to offset the loss in degrees of freedom. If between-subjects variability is small this process may actually reduce the $F$-value.

Assumptions

As with all statistical analyses, there are a number of assumptions that should be met to justify the use of this test. Violations to these assumptions can moderately to severely affect results, and often lead to an inflation of type 1 error. Univariate assumptions include:

1. Normality: For each level of the within-subjects factor, the dependent variable must have a normal distribution.
2. Sphericity: Difference scores computed between two levels of a within-subjects factor must have the same variance for the comparison of any two levels.
3. Randomness: Cases should be derived from a random sample, and the scores between participants should be independent from each other.

The rANOVA also requires that certain multivariate assumptions are met because a multivariate test is conducted on difference scores. These include:

1. Multivariate normality: The difference scores are multivariately normally distributed in the population.
2. Randomness: Individual cases should be derived from a random sample, and the difference scores for each participant are independent from those of another participant.

$F$-Test

Depending on the number of within-subjects factors and assumption violates, it is necessary to select the most appropriate of three tests:

1. Standard Univariate ANOVA $F$-test: This test is commonly used when there are only two levels of the within-subjects factor. This test is not recommended for use when there are more than 2 levels of the within-subjects factor because the assumption of sphericity is commonly violated in such cases.
2. Alternative Univariate test: These tests account for violations to the assumption of sphericity, and can be used when the within-subjects factor exceeds 2 levels. The $F$ statistic will be the same as in the Standard Univariate ANOVA F test, but is associated with a more accurate $p$-value. This correction is done by adjusting the $df$ downward for determining the critical $F$ value.
3. Multivariate Test: This test does not assume sphericity, but is also highly conservative.

While there are many advantages to repeated-measures design, the repeated measures ANOVA is not always the best statistical analyses to conduct. The rANOVA is still highly vulnerable to effects from missing values, imputation, unequivalent time points between subjects, and violations of sphericity. These issues can result in sampling bias and inflated rates of type I error.