8.2 The differences between logistic and linear regression
The application examples listed above should provide hints about the purpose of logistic regression (classification) as well as the outcome type (categorical). These stand in contrast to the purpose and outcome type of logistic regression’s better-known cousin, linear regression, but their differences do not end there. The table below lists other ways in which both types of regression vary:
| Logistic regression | Linear regression | |
| Purpose | Classification | Prediction |
| Outcome type | Categorical | Numerical |
| Sample size | Sufficiently large sample size needed for model to have statistical power | There is no clear rule for a minimum sample size3, with some believing that the minimum sample size depends on variance4 and the number of predictor variables.5 |
| Basis | Probability | Ordinary least squares |
| Other things to be mindful of | Outliers Multicollinearity between independent variables Overfitting Observations must be independent of each other | Outliers Multicollinearity between independent variables Observations must be independent of each other Model residuals are normally distributed Variance of residuals should be approximately the same (homoscedasticity) |
3 University of Illinois (n.d.). ‘How many data points are enough?’ http://www.phy.ilstu.edu/pte/302content/student_lab_hdbk/How_Many_Data_Points.pdf
4 Jenkins, David G. and Quintana-Ascencio, Pedro F. (2020) ‘A solution to minimum sample size for regressions’. PLoS One; 15(2): e0229345. doi: 10.1371/journal.pone.0229345
5 Van Voorhis, Carmen R. W. and Morgan, B. (2007). ‘Understanding power and rules of thumb for determining sample sizes’. Tutorials for Quantitative Methods in Psychology. 2007, vol. 3 (2), p. 43‐50. doi: 10.20982/tqmp.03.2.p043