Complex Designs

Introduction to Complex Designs

•      Researchers often investigate the effects of two or more independent variables simultaneously; this type of experiment is referred to as a complex design.

•      The simplest complex design has two independent variables (IVs) and one dependent variable (DV).

–  The simplest independent variable has two levels (conditions).

Complex Designs

•      Factorial Combination refers to how we combine independent variables in an experiment to describe their effects on the dependent variable(s).

–  We factorially combine independent variables by pairing each level of one independent variable with each level of the other independent variable.

Complex Designs (continued)

•      Example: We can extend the emotional writing research of Pennebaker and Francis (1996).

        IV: Type of Writing

                '              (

   Emotional            Superficial

Complex Designs (continued)

–  We can study the effect of a second independent variable that manipulates participants’ focus in their writing using two levels of instructions.          

              IV: Type of Instructions

                        '                 (

               Insight               No-insight

        (identify causes)       (details only)

Complex Designs (continued)

Factorial Combination of Two Independent Variables:

Complex Designs (continued)

•      This factorial design is called a 2 x 2 design (read “2 by 2"). It has four conditions.

•      Factorial Combination allows us to examine the overall effect of Type of Writing, the overall effect of Type of Instructions, and the combined effects of both independent variables.

Complex Designs (continued)

•      The overall effect of an independent variable in a complex design is called a main effect.

–  The main effect is the effect of the independent variable on the dependent variable as if only that variable was manipulated in the experiment.

Complex Designs (continued)

•      The combined effect of independent variables in a complex design is called an interaction effect.

–  An interaction effect occurs when the effect of an independent variable differs depending on the level of the second independent variable.

Guidelines for Identifying an Experimental Design

•      Complex designs have at least two independent variables.

•      The independent variables can be manipulated using an independent groups design or a repeated measures design, or both.

•      When different types of independent variables are used, the complex design is called a mixed design.

Research Example: Bazzini and Shaffer’s (1999) Experiment (see pp. 297-301 of text)

•      Research Questions:

–  Do individuals in committed relationships (exclusive daters) derogate (put down) potential dating partners to maintain their relationship?

–  Do individuals not in committed relationships (nonexclusive daters) enhance potential dating partners, because they are seeking a relationship?

Research Example (continued)

•      Research Design

–  To answer these questions, Bazzini and Shaffer conducted a complex design experiment in which participants read a hypothetical scenario describing a potential dating partner who was:

•   interested in the participant, or

•   interested in the participant’s friend.

Research Example (continued)

•     Their complex design had two independent variables:

–  Dating Status had two levels: Exclusive Daters, Nonexclusive Daters.

–  What type of independent variable is this?

•   a natural groups variable.

–  Type of Scenario had two levels: Stranger Attracted to Participant, Stranger Attracted to Friend.

–  What type of independent variable is this?

•   a random groups variable because participants were randomly assigned to read one of the scenarios.

Main Effects

Means for Type of Scenario:           9.88                                10.17

•      Stranger Attracted to Friend: M = 9.88

•      Stranger Attracted to Participant: M = 10.17

•      Because these means are similar (using a test of statistical significance), we conclude that:

–  participants, regardless of dating status, rated their romantic interest similarly in the two scenario conditions.

Main Effects (continued)

–  Next, we can assess the main effect of the Dating Status independent variable.

–  To do this, we compare the two levels of Dating Status:

•    exclusive daters, and

•    nonexclusive daters.

•      Exclusive Daters: M = 9.52

•      Nonexclusive Daters: M = 10.54

•      Because these means are statistically different (using a test of statistical significance), we conclude that:

–  nonexclusive daters rated their romantic interest in the stranger higher than did exclusive daters (collapsed across Type of Scenario).

Main Effects (continued)

•      Does this mean that the Type of Scenario independent variable,

–  stranger attracted to friend vs. stranger attracted to participant,

  had no effect on participants’ romantic interest ratings?

–  The main effect of Type of Scenario was not statistically significant — it did not produce an “overall” effect on participants’ ratings.

Main Effects (continued)

•      Nonexclusive daters, relative to exclusive daters, rated their romantic interest higher for both scenario conditions.

•      Even though the Type of Scenario didn’t produce a statistically significant main effect, this variable was important.

•      Type of Scenario interacted with Dating Status to influence participants’ romantic interest.

•      Thus, type of scenario is a relevant independent variable, because it interacted with the dating status independent variable.

Interaction Effects

•      Interaction effects represent how independent variables work together to influence behavior.

•      An interaction effect occurs when the effect of one independent variable differs depending on the level of the second independent variable.

Interaction Effects:
Bazzini & Shaffer’s (1999) experiment

•      To look for an interaction effect in Bazzini and Shaffer’s experiment, we can look at

   the effect of dating status at each level of the scenario independent variable.

•      When we look for interaction effects between independent variables, we often use the subtraction method.

Difference Between Means:            -0.23                           -1.88

–  The difference between means for exclusive daters and nonexclusive daters in the Stranger-Attracted-to-Friend condition is -0.23 (9.77 - 10.00).

–  The difference between means for exclusive daters and nonexclusive daters in the Stranger-Attracted-to-Participant condition is -1.88 (9.25 - 11.13).

Interaction Effects (continued)

•      Because the outcome of the subtraction method yielded different values (-.23 and -1.88), an interaction effect between the independent variables is likely,

–  but a test of statistical significance would be needed to confirm this.

•      We need to examine the means to understand the interaction effect.

•      The interaction effect tells us that exclusive daters (M = 9.77) and nonexclusive daters (M = 10.00) rated their romantic interest similarly when the stranger was attracted to the friend.

–   A t-test comparing these two means reveals that the difference between 9.77 and 10.00 is not statistically significant.

•      This condition — stranger attracted to the friend — was Bazzini and Shaffer’s objective standard for comparison.

•      However, when the stranger was attracted to the participant, nonexclusive daters (M = 11.13) rated their romantic interest higher than did exclusive daters (M = 9.25).

–   A t-test comparing these two means reveals that the difference between 11.13 and 9.25 is statistically significant.

•      Thus, exclusive and nonexclusive daters differed in their romantic interest ratings depending on the scenario condition — an  interaction effect between Dating Status and Type of Scenario independent variables.

Interaction Effects (continued)

•      Another way to say this is that the effect of one independent variable, Dating Status, differed depending on the level of the second independent variable, Type of Scenario.

•      Recall that this is our definition of an interaction effect.

The diverging lines indicate an interaction effect is likely present in the data;

    however, a statistical test is used to determine whether the interaction effect is statistically significant.

Analysis

•      Simple main effects:

the effect of one independent variable at one level of the second independent variable.

For example, the effect of the Dating Status independent variable in the

•   stranger-attracted-to-friend condition, or

•   stranger-attracted-to-participant condition.

•      One definition of interaction effects is that the simple main effects of one independent variable are different across the levels of the second independent variable.

•      In Bazzini and Shaffer’s experiment, the simple main effect of Dating Status was:

–  not statistically significant in the friend-scenario condition, but

–  was statistically significant in the participant-scenario condition.

Interaction Effects and Ceiling/Floor Effects

•      Floor and Ceiling Effects

Sometimes an interaction effect can be statistically significant “by mistake.”

This occurs when the means for one or more condition reach the highest possible score (ceiling effect) or the lowest possible score (floor effect).

When floor or ceiling effects occur, an interaction effect is uninterpretable.

This graphs shows an interaction effect between Test Difficulty (easy, hard) and Study Hours (10, 15).

Hours of study had an effect only in the hard-test condition, not in the easy-test condition.

How do we interpret this interaction effect when we know the highest possible score on the tests is 50?

If we have enough “room” in our dependent variable to assess the effect of the independent variables, the interaction effect disappears.

This graph shows two main effects: A main effect of Study Hours and a main effect of Test Difficulty.

Interaction Effects and Natural Groups Designs

•      Using complex designs, researchers can test causal inferences for natural groups variables.

•      Recall that we can’t make causal inferences with natural groups variables.

–  Natural groups variables are correlational.

•      So, how can we make causal inferences using a complex design?

•      We can make causal inferences about natural groups when we test a theory for why the natural groups differ.

–  For example, we can theorize that the reason exclusive daters are in a committed relationship is because they derogate potential dating partners who say they are attracted to them (to maintain their relationship).

•      Steps for making causal inferences about natural groups variables in a complex design:

–  State your theory. Why are the groups different? What is the theoretical process?

–  Identify a relevant independent variable. This IV should influence the likelihood that the theorized process will occur (e.g., relationship maintenance).

–  Look for an interaction effect. In order to make a causal inference, the natural groups variable and manipulated variable should produce a statistically significant interaction effect in the predicted direction.

   This interaction effect allows us to make causal inferences about why individuals differ — that is, we begin to understand why people differ.