Statistics for CSAI 2

by Albert Major

1. Motivation and basic idea of interaction

• Goal: explain an outcome variable Y (for example exam performance) from several predictors.
• Earlier regression models used additive effects only.
  • Example: exam grade = b0 + b1·study_hours + b2·work_hours + error.
• Interaction idea: sometimes the effect of one predictor depends on the level of another.
  • Example ideas from class:
    • Poverty and inequality together related to crime rate.
    • Coffee and hours of study together related to exam grade.
    • Diet and sleep together related to BMI or health.
• Students’ informal definitions of “interaction”:
  • Together the two predictors might have a stronger effect than either one on its own.
  • The effect might appear only when both predictors are present or high.
  • If one predictor is zero or absent, the other one shows little or no effect.
• So, in an interaction, the relationship between a predictor and Y depends on another predictor.

1. Shapes in regression: line vs plane

• A question in class: we try to estimate the best fitting … what?
  • For simple linear regression with one predictor: a line in 2D.
  • For multiple linear regression with two predictors: a plane in 3D.
• The “least squares” idea still holds.
  • We pick the line or plane that gives the smallest sum of squared residuals.
• This reminder links the geometry to the model form and prepares for interactions.

1. From additive regression to interaction regression

• Simple linear regression:
  • Y = β0 + β1·X + ε
• Multiple regression with two predictors, still additive:
  • Y = β0 + β1·X + β2·Z + ε
• Regression with interaction between X and Z:
  • Y = β0 + β1·X + β2·Z + β3·X·Z + ε
• Interpretation of coefficients in this model:
  • β0: predicted value of Y when X = 0 and Z = 0.
  • β1: effect of X on Y when Z = 0.
  • β2: effect of Z on Y when X = 0.
  • β3 (interaction term): change in the slope of X for a one-unit change in Z
    • Equivalently, change in the slope of Z for a one-unit change in X.
  • So β3 describes how the effect of one predictor depends on the level of the other.

1. Visual examples of interactions

4.1 Categorical moderator (ANOVA style)