Validity Scorecard for True Experimental Between-Subjects Designs

 

Pretest-Posttest Control Group Design

Posttest-Only Control Group Design

Construct Validity

Very troublesome

same

Reactive arrangements

- usually in a lab setting, with direct contact between participants and experimenter

? same. features normally associated with random assignment and manipulation of the IV ??

Pretest sensitization

- especially in a lab setting, where pretest is administered directly to participants

+ no pretest

Linguistic/cultural bias

? depends on nature of measurement; problematic if a verbal measure is administered directly to participants

? same

Internal validity

Well controlled

same

Extraneous effects

+ lab setting, manipulation of IV, and brief duration lessen likelihood of a coincidental event that affects outcome.

+ same

Temporal effects

+ unlikely in a brief experiment

+ same

Group composition effects

+ random assignment creates equivalent groups

+ same

Interaction of temporal and group composition effects

+ random assignment, manipulation of IV by experimenter

+ same

Selective sample attrition

+ lab setting & brief duration prevent this; if not, pretest allows evaluation of those lost

+ lab setting & brief duration make it easy to account for participants and lessen likelihood of loss

Statistical regression effects

+ even if sample is selected for extreme scores, control group is matched by random assignment

+ no pretest means no selection based on scores on M

Nonrepresentative sampling

? depends on method of sampling participants, and the universality of the relationship between T and M

? same; in lab experiments, sample is rarely random and representative

Nonrepresentative research context

- in lab experiments, task and setting are usually an artificial simulation of the natural context of behaviour.

? in rare field experiments, context can be more representative of those to which results are to be generalized

Source: Cherulnik, P. D. (2001). Methods for behavioral research : A systematic approach: Sage.