This repository contains a number of (very ugly) R scripts that I've written to explore the interpretation of the intercept term in multiple linear regression models that contain different categorical predictors.
The code directory contains several R scripts that each contain a series of test cases. Each test case follows the same recipe:
- Simulate some regression data
- Estimate the (true) regression model
- Compare the estimated intercept to its theoretical equivalent computed from the raw data
- Compare the estimated intercept to its theoretical equivalent computed via the emmeans package
The test cases represent a nearly complete crossing of the following design factors:
- Type of Code: {Dummy Code, Unweighted Effects Code, Weighted Effects Code}
- Correlation between Predictors: {Independent Predictors, Correlated Predictors}
- Error Variance in the Outcome: {Deterministic Outcome, Noisy Outcome}
- Balance of Group Sizes: {Balanced Groups, Unbalanced Groups}
Regardless of the setup, the estimated intercept always matches the respective marginal mean computed via the emmeans package. The estimated intercepts sometime match, and sometimes do not match, their respective marginal means computed from the raw data. A summary of the concordance between the intercept estimates and the raw data-based mean estimates is available in this Google Sheet.