What is Linear Regression?
Linear regression is a statistical method that fits a straight line, ŷ = a + bx, to best describe the relationship between two variables. It uses an independent variable (x) to predict a dependent variable (y), minimizing the total squared prediction error. It's one of the most widely used tools in statistics and machine learning.
Linear regression models the relationship between x and y with the equation ŷ = a + bx, finding the line of best fit using the least-squares method to minimize prediction errors.
Try it: interactive calculator
Step-by-step worked examples
A regression equation is ŷ = 2 + 3x. Predict y when x = 5.
ŷ = 2 + 3(5) ŷ = 2 + 15 = 17
Data points (1,3), (2,5), (3,7). Find the slope (b) and intercept (a).
x̄ = 2, ȳ = 5 Deviations: (-1,-2), (0,0), (1,2) Σ(x-x̄)(y-ȳ) = 2 + 0 + 2 = 4 Σ(x-x̄)² = 1 + 0 + 1 = 2 b = 4/2 = 2 a = ȳ - b·x̄ = 5 - 2(2) = 1 Result: ŷ = 1 + 2x
Using ŷ = 1 + 2x from the example above, predict y when x = 4.
ŷ = 1 + 2(4) ŷ = 1 + 8 = 9
Flashcards
Quick quiz
Q1.In ŷ = a + bx, what does b represent?
Q2.A regression line is ŷ = 3 + 2x. What is ŷ when x = 4?
Q3.What method finds the best-fit regression line?
Q4.What does a residual measure?
The full card deck, worked steps and AI-tutor support for “What is Linear Regression?” are in Notek — study by hand before your exam.
Common mistakes
Assuming the regression line means x causes y. — Correct: Regression shows association, not causation.
Using the line to predict far outside the data range. — Correct: Extrapolation beyond observed x-values is unreliable.
Confusing the slope with the correlation coefficient. — Correct: Slope depends on units; correlation r is unitless (-1 to 1).
Ignoring outliers when fitting the line. — Correct: Outliers can heavily skew the least-squares fit — check residual plots.
FAQ
What is linear regression?
Linear regression is a statistical method that uses an independent variable (x) to predict a dependent variable (y) by fitting the best straight line, ŷ = a + bx.
What is the formula for linear regression?
The formula is ŷ = a + bx, where a is the intercept and b is the slope, both calculated using the least-squares method.
How do you calculate the slope and intercept in linear regression?
b = Σ(x-x̄)(y-ȳ)/Σ(x-x̄)² and a = ȳ - b·x̄, using the means of x and y from the data.
What are examples of linear regression in real life?
Predicting house price from square footage, forecasting sales from ad spend, or relating height to weight are all examples of linear regression.