Psychology 624 - 01

Experimental and Statistical Research Methods

Prof. Martin Packer

Spring 2004, Thur 3:05 - 5:45

This course will provide an overview of empirical-analytic research methods as these are used in psychology, specifically in clinical and developmental psychology. Readings and lecture material will address theoretical and practical issues pertaining to research design, data collection, and data analysis, as well as strategies for reviewing and critically evaluating published research reports. By the end of the course, students should be able to understand and critically evaluate the purposes and procedures of research design, including descriptive and inferential statistics.

Readings:

Ray, W. J. Methods: Toward a science of behavior and experience. Wadsworth. (Available in the Duquesne bookstore. Note: Any edition of this book since the 4th can be used. Earlier editions can be found on Amazon.com and elsewhere.)
Additional readings will be placed on reserve in the Gumberg library, or distributed in class

Course requirements:

Final grades will be based on class participation, two exams, a final paper, short weekly homework assignments and short weekly in-class quizzes. The percentage each of these contributes to the final grade is shown below. Class participation will be evaluated in terms of students’ contributions to discussions of assigned readings and related issues. The exams will include multiple choice and brief essay questions concerning research design and statistical techniques. Homework will be short-answer questions linked to the assigned reading. Quizzes will be true/false questions linked to the assigned reading. The final paper will entail a critical analysis of a published research report (distributed in class after Spring Break). The paper will be due Friday April 19th at 4:00.
Paper 40%

Exam 1

Exam 2

Class participation

Weekly homework

Weekly quizzes

20 percent

20

10

5

5

Week 1

Introduction

Steps of Empirical-Analytic Research Methodology

Problems with this Methodology

Some Useful Resources:

A GLOSSARY of statistical terms

Internet Resources in Statistics (people, journals,...)

Webpage for Cherulnik's methods book (with guides, etc.)

If you are interested...

...additional readings:

Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press.

Latour, B., & Woolgar, S. (1979/1986). Laboratory life: The construction of scientific facts: Princeton, NJ: Princeton University Press.

Bernstein, R. J. (1976). The restructuring of social and political theory: University of Pennsylvania Press.

Bernstein, R. (1983). Beyond objectivism and relativism: Science, hermeneutics, and praxis: Philadelphia: University of Pennsylvania Press.

Feyerabend, P. (1975). Against method: Verso Press.

Week 2

The logic of hypothesis testing

For this week read: Ray Ch 1, 2 & 3

Material on Electronic Reserve

Please print the 3 articles and bring them to class next week. Thanks!

Operationalizing a Hypothesis

Main Points:

• Operationalizing a hypothesis translates theoretical concepts into independent and dependent variables

• Operational definition of a variable specifies measurement operations

Week 3

Measurement, descriptive statistics

Reading for this week: Ch 4, Ch 13.

Homework: review questions

What chapter 4 is about

We will start to discuss validity

...and the threats to validity

...and test validity

Main Points:

• Measurement assigns a number to a variable

• Numbers don't remember where they came from--so you must

• These numbers have properties that define (at least) four distinct LEVELS OF MEASUREMENT--nominal, ordinal, interval, ratio.

• A common form of measurement in psychology is the TEST

• We seek to make a test valid and reliable. (And this is equally true of other kinds of measure.)

Interact with a histogram (and other Applets for statistical concepts)

A more advanced account of measurement

Further Readings:

For a 1950s, Logical positive account: Woolf, H. (Ed.). (1961). Quantification: A history of the meaning of measurement in the natural and social sciences. Indianapolis: Bobbs-Merrill.

A Kuhnian alternative view: Kuhn, T. (1961/1977). The function of measurement in modern physical science. Reprinted in The essential tension (pp. 178-224). Chicago: University of Chicago Press.

We've mentioned this several times:

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: Macmillan.

Week 4

Inferential Statistics

Overview of today's topic

Zero-Order Correlations...

Concept Maps

Vassar Statistical Resources

Main Points:

• Probability theory takes us from population to sample

• Inferential statistics take us from sample to population

• Hypothesis testing is a specific inferential procedure

Week 5

Testing Hypotheses

Main Points:

There are 3 steps to hypothesis-testing:

• Reject the null hypothesis?

• Could there be confounds?

• Is there a significant treatment effect?

The F-ratio is an important statistic:

• it is a measure of the variation among mean scores

• it can be used for more than 2 groups

• = between-group variance/within-group variance

• = treatment + error / error

Week 6

Experimentation

Chapter 7

Main Points:

• Experimental design seeks to control factors, to create a meaningful comparison

• ...and to control the various threats to validity

• 3 kinds: True experiments; Quasi Experiments; Non Experiments

• How confident do you want to be in rejecting the null hypothesis? Pick a value of p, and then look up the criterion value of t or F (the amount that t or F must be greater than 1).

Aspects of Research Design

Threats to Validity

True Experimental Design: Between Subjects

Portion of the Table of t

Week 7

Between-Subjects Designs

Chapter 8

ANOVA: Words and Alcohol study

We continue to explore the logic of true experiements with between-subject designs. Now we expand our focus to consider experiments with more than one independent variable.

Main Points:

• ANOVA is just an extension of the F-test.

• In Factorial designs, we consider the effects of 2 or more treatments

• Again, we ask is F > 1?

• But now we can also consider possible "interaction" of treatments.

Week 8

Exam 1

Here's a copy of the Study Guide

Week 9

Within-Subjects & Matched-Subjects Designs

Main Points:

• Within-Subject designs have the same participants in different treatment conditions

• As a consequence, they have reduced error variance

• And so they are more sensitive to treatment effects

Paul Cherulnik, author of Methods for behavioral research : A systematic approach (Sage, 2001), has designed a set of "Validity Scorecards" for assessing the way different research designs address threats to internal and external validity. We have used these as a template for some of the discussions of validity on this page. You can use them to guide your analysis of the paper you are working on for the end of the semester.

Look here for blank Validity Scorecards for True-, Quasi-, and Pre-experimental design

You may also find Cherulnik's Study Analysis Form useful.

And Cherulnik offers Quick Guides to many of the topics we have covered in this course.

Week 10

Quasi-Experimental Design

Ch 11 & 14

How threats to validity are dealt with by:

• True-Experimental Designs

• Quasi-Experimental Designs

• Pre-Experimental Designs

Main Points:

• Quasi-Experimental designs are ones where participants cannot be assigned to treatment groups

• They are more open to threats to internal validity than true-experimental designs

Week 11

Multiple Correlation Analysis

Readings: BioStats chapters on e-reserve

A few things about Factor Analysis

One or two things we know about Regression

Regress this! (a cute applet)

Multiple Regression with Ren & Stimpy

Main Points:

• When IVs are continuous, multiple correlational analysis is used, rather than ANOVA

• These approaches are based on correlation -- a description of the relationship between two variables

• Multiple regression explores the extent to which the IVs "explain" the variance in a DV

• More advanced techniques include factor analysis, path analysis, etc.

Week 12

Single-Subject Designs

Main Points:

• N-of-1 designs may actually use more than one participant

• They are used when there is reason to think that group averages will be misleading

Read APA on research reporting: Statistical Methods in Psychology Journals: Guidelines
and Explanations

Week 13

Parametric & Non-Parametric Statistics

Main Points:

• Nonparametric tests are used when data is nominal or ordinal

• Or when distributions are not normal, or variances are not homogeneous

• They are not as powerful as parametric tests

Common Nonparametric tests

Which nonparametric test to use? (PDF document)

Which parametric test to use?

Week 14

Study Guide for exam 2

Exam 2

New link!! Developmental Designs