Psychology 624 - 01

Experimental and Statistical Research Methods

Prof. Martin Packer

Spring 2004, Thur 3:05 - 5:45


This course will provide an overview of empirical-analytic research methods as these are used in psychology, specifically in clinical and developmental psychology. Readings and lecture material will address theoretical and practical issues pertaining to research design, data collection, and data analysis, as well as strategies for reviewing and critically evaluating published research reports. By the end of the course, students should be able to understand and critically evaluate the purposes and procedures of research design, including descriptive and inferential statistics.


Ray, W. J. Methods: Toward a science of behavior and experience. Wadsworth. (Available in the Duquesne bookstore. Note: Any edition of this book since the 4th can be used. Earlier editions can be found on and elsewhere.)
Additional readings will be placed on reserve in the Gumberg library, or distributed in class

Course requirements:

Final grades will be based on class participation, two exams, a final paper, short weekly homework assignments and short weekly in-class quizzes. The percentage each of these contributes to the final grade is shown below. Class participation will be evaluated in terms of students’ contributions to discussions of assigned readings and related issues. The exams will include multiple choice and brief essay questions concerning research design and statistical techniques. Homework will be short-answer questions linked to the assigned reading. Quizzes will be true/false questions linked to the assigned reading. The final paper will entail a critical analysis of a published research report (distributed in class after Spring Break). The paper will be due Friday April 19th at 4:00.
Paper 40%


Exam 1

Exam 2

Class participation

Weekly homework

Weekly quizzes


20 percent






Week 1



Steps of Empirical-Analytic Research Methodology

Problems with this Methodology



Some Useful Resources:

A GLOSSARY of statistical terms

Internet Resources in Statistics (people, journals,...)

Webpage for Cherulnik's methods book (with guides, etc.)

If you are interested...

...additional readings:

Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press.

Latour, B., & Woolgar, S. (1979/1986). Laboratory life: The construction of scientific facts: Princeton, NJ: Princeton University Press.

Bernstein, R. J. (1976). The restructuring of social and political theory: University of Pennsylvania Press.

Bernstein, R. (1983). Beyond objectivism and relativism: Science, hermeneutics, and praxis: Philadelphia: University of Pennsylvania Press.

Feyerabend, P. (1975). Against method: Verso Press.



Week 2

The logic of hypothesis testing


For this week read: Ray Ch 1, 2 & 3

Material on Electronic Reserve

Please print the 3 articles and bring them to class next week. Thanks!

Operationalizing a Hypothesis



Main Points:

  • Operationalizing a hypothesis translates theoretical concepts into independent and dependent variables

  • Operational definition of a variable specifies measurement operations


Week 3


Measurement, descriptive statistics


Reading for this week: Ch 4, Ch 13.

Homework: review questions

What chapter 4 is about

We will start to discuss validity

...and the threats to validity

...and test validity

Main Points:

  • Measurement assigns a number to a variable

  • Numbers don't remember where they came from--so you must

  • These numbers have properties that define (at least) four distinct LEVELS OF MEASUREMENT--nominal, ordinal, interval, ratio.

  • A common form of measurement in psychology is the TEST

  • We seek to make a test valid and reliable. (And this is equally true of other kinds of measure.)

Interact with a histogram (and other Applets for statistical concepts)

A more advanced account of measurement

Further Readings:

For a 1950s, Logical positive account: Woolf, H. (Ed.). (1961). Quantification: A history of the meaning of measurement in the natural and social sciences. Indianapolis: Bobbs-Merrill.

A Kuhnian alternative view: Kuhn, T. (1961/1977). The function of measurement in modern physical science. Reprinted in The essential tension (pp. 178-224). Chicago: University of Chicago Press.

A more recent argument: Rizo, F. M. (1991). The controversy about quantification in social research: An extension of Gage's "'Historical' sketch". Educational Researcher, 20(9), 9-12.

We've mentioned this several times:

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: Macmillan.


Week 4


Inferential Statistics

Reading: Ray, Chapter 5

Overview of today's topic

Zero-Order Correlations...

Concept Maps

Vassar Statistical Resources


Main Points:

  • Probability theory takes us from population to sample

  • Inferential statistics take us from sample to population

  • Hypothesis testing is a specific inferential procedure


Week 5


Testing Hypotheses

Chapter 6

Main Points:

There are 3 steps to hypothesis-testing:

  • Reject the null hypothesis?

  • Could there be confounds?

  • Is there a significant treatment effect?

The F-ratio is an important statistic:

  • it is a measure of the variation among mean scores

  • it can be used for more than 2 groups

  • = between-group variance/within-group variance

  • = treatment + error / error


Week 6



Chapter 7

Main Points:

  • Experimental design seeks to control factors, to create a meaningful comparison

  • ...and to control the various threats to validity

  • 3 kinds: True experiments; Quasi Experiments; Non Experiments

  • How confident do you want to be in rejecting the null hypothesis? Pick a value of p, and then look up the criterion value of t or F (the amount that t or F must be greater than 1).

Aspects of Research Design

Threats to Validity

True Experimental Design: Between Subjects

Portion of the Table of t


Week 7


Between-Subjects Designs


Chapter 8

ANOVA: Words and Alcohol study

We continue to explore the logic of true experiements with between-subject designs. Now we expand our focus to consider experiments with more than one independent variable.


Main Points:

  • ANOVA is just an extension of the F-test.

  • In Factorial designs, we consider the effects of 2 or more treatments

  • Again, we ask is F > 1?

  • But now we can also consider possible "interaction" of treatments.


Week 8

Exam 1


Here's a copy of the Study Guide


Week 9


Within-Subjects & Matched-Subjects Designs

Ch 9

Main Points:

  • Within-Subject designs have the same participants in different treatment conditions

  • As a consequence, they have reduced error variance

  • And so they are more sensitive to treatment effects

Paul Cherulnik, author of Methods for behavioral research : A systematic approach (Sage, 2001), has designed a set of "Validity Scorecards" for assessing the way different research designs address threats to internal and external validity. We have used these as a template for some of the discussions of validity on this page. You can use them to guide your analysis of the paper you are working on for the end of the semester.

Look here for blank Validity Scorecards for True-, Quasi-, and Pre-experimental design

You may also find Cherulnik's Study Analysis Form useful.

And Cherulnik offers Quick Guides to many of the topics we have covered in this course.

Week 10


Quasi-Experimental Design


Ch 11 & 14

How threats to validity are dealt with by:

  • Quasi-Experimental Designs

  • Pre-Experimental Designs

Main Points:

  • Quasi-Experimental designs are ones where participants cannot be assigned to treatment groups

  • They are more open to threats to internal validity than true-experimental designs


Week 11


Multiple Correlation Analysis


Readings: BioStats chapters on e-reserve

A few things about Factor Analysis

One or two things we know about Regression

Regress this! (a cute applet)

Multiple Regression with Ren & Stimpy


Main Points:

  • When IVs are continuous, multiple correlational analysis is used, rather than ANOVA

  • These approaches are based on correlation -- a description of the relationship between two variables

  • Multiple regression explores the extent to which the IVs "explain" the variance in a DV

  • More advanced techniques include factor analysis, path analysis, etc.

Week 12


Single-Subject Designs

Ch 12

Main Points:

  • N-of-1 designs may actually use more than one participant

  • They are used when there is reason to think that group averages will be misleading

Read APA on research reporting: Statistical Methods in Psychology Journals: Guidelines
and Explanations



Week 13


Parametric & Non-Parametric Statistics

Reading will be assigned

Main Points:

  • Nonparametric tests are used when data is nominal or ordinal

  • Or when distributions are not normal, or variances are not homogeneous

  • They are not as powerful as parametric tests

Common Nonparametric tests

Which nonparametric test to use? (PDF document)

Which parametric test to use?

Week 14

Study Guide for exam 2

Exam 2


New link!! Developmental Designs