Overview of Threats to

the Validity of Research Findings

I. Threats to conceptual validity

1. Unnecessary duplication

2. Theoretical isolation

II. Threats to construct validity

A. The independent variable

a. Inadequate operationalization

3. Lack of reliability of the independent variable

4. Lack of representativeness of the independent variable

5. Lack of impact of the independent variable

b. Treatment artifacts

6. Demand characteristics in the research setting

7. Experimenter expectancy effects on the behavior being observed

8. Pretest sensitization to the treatments

B. The dependent variable

a. Inadequate operationalization

9. Lack of reliability of the dependent variable

10. Lack of representativeness of the dependent variable

11. Lack of sensitivity of the dependent variable

b. Measurement artifacts

12. Strategic responding by the research participants

13. Linguistic/cultural bias in the dependent measure

III. Threats to internal validity

14. Extraneous effects (history)

15. Temporal effects (maturation)

16. Group composition effects (selection)

17. Interaction of temporal and group composition effects

18. Selective sample attrition (mortality)

19. Statistical regression effects (regression to the mean)

IV. Threats to external validity

20. Nonrepresentative sampling

21. Nonrepresentative research context

V. Threats to statistical conclusion validity

22. Inappropriate use of statistical techniques

23. Use of a statistical test lacking sufficient power

24. Mistaking a trivial effect for support for the research hypothesis