Linear Models

Regression and analysis of variance (anova or ANOVA) are the basic work horses of statistical inference. Regression measures how response changes with predictor along a straight line. Anova is used to compare means for two or more groups. Regression and anova are often lumped together as aspects of linear models. In fact, analysis of covariance (ancova or ANCOVA) is a simple way to have separate (possibly parallel–same slope) regression lines for each group.

aov() & lm() commands both fit linear models, with aov focused on comparing “factors” (categorical predictors) and lm focused on “regressors” (continuous predictors). Actually, lm can be used for both, but be careful how you set up factors. The anova() and drop1() commands are used to get tabular summaries of fits with aov or lm. anova gives sequential (Type I) tests, while drop1 gives adjusted (Type III) tests.

• Analysis of variance

Several useful packages have been developed for linear models:

• car package (companion to Fox & Weisberg book ⚠️)
• lsmeans package
• regress package

Regression with more than one predictor gets complicated, and there is no one best way to proceed. However, there are guidelines and useful tools.

• multiple regression from Quick-R
• step: stepwise model fits
• stepwise package
• variable selection ⚠️

Models that are not linear

There are two types of models that are not linear. One involves models with non-normal errors, such as arises with counts. Counts might be of yes/no (binomial and logistic) situations or counts of some event (Poisson). There are more general approaches, hence the complicated name of generalized linear models. Underlying these models is a linear relationship. For instance, Poisson counts assume the log of the mean depends linearly on predictors, while binomial counts assume the logit of the proportion (logit(p) = log(p)/log(1-p)) depends linearly on predictors.

The other type are models in which predictors have a non-linear relation with response, such as y ~ x/(1+w).

• generalized linear models
  • Poisson test (poisson.test)
  • binomial test (binom.test)
  • Poisson regression
  • logistic regression (glm)
• non-linear models
  • nls package

Mixed models

Some models mix so-called fixed and random effects. Fixed are as above. Random effects account for the fact that the observed levels of components of a model might be uncertain or be sampled from some broader class of possible levels. Sometimes these random components are interpreted as having a distribution of interest, hence Bayesian.

• lmer in lme4
• lmmlite
• gemma
• glmmnet package
• Andrew Gelman ⚠️’s arm package ⚠️ & stan Platform
• Mixed Models in R: lme4, nlme, or both?
• Statistics with R: Mixed Effects Models (Carpentries Style)
• Coding Club: Introduction to Linear Mixed Models
• Coursera: Introduction to Mixed Effects Models
• R BGU Course: 8 Linear Mixed Models (JD Rosenblatt)
• R Course: 6.1 Linear Mixed Models (P Piccinini)
• PhD Training Workshop: Statistics in R: 15 Linear Model and Mixed Methods (A Ushakova, M Valasek)
• Generalized Linear Mixed-Effects Modeling in R (S Anderson)
• Statistical Modeling and Mixed Models with R (Singmann)