MathematicsStatistics and Regression AnalysisUniversity

Simple Linear Regression Line

This equation defines the line of best fit that minimizes the sum of squared residuals between observed and predicted values for a linear relationship between two variables.

Understand the formulaSee the free derivationOpen the full walkthrough

\overset{y}{^} = b_{0} + b_{1} x where b_{1} = \frac{n \sum x y - ( \sum x ) ( \sum y )}{n \sum x ^{2} - ( \sum x ) ^{2}} and b_{0} = \overset{y}{ˉ} - b_{1} \overset{x}{ˉ}

Open Full Walkthrough Try Calculator

This public page keeps the free explanation visible and leaves premium worked solving, advanced walkthroughs, and saved study tools inside the app.

Core idea

Overview

The regression line is calculated using the Ordinary Least Squares (OLS) method, which seeks to minimize the variance of the errors. The slope, b1, represents the expected change in y per unit change in x, while the intercept, b0, indicates the predicted value of y when x is zero. Together, these parameters characterize the linear trend within a dataset.

When to use: Use this when you need to model the relationship between two continuous variables and predict future outcomes based on linear trends.

Why it matters: It is the foundational tool for predictive analytics, enabling researchers and businesses to forecast trends and quantify the strength of relationships between variables.

Symbols

Variables

y^ = Predicted Value, $b_{1}$ = Slope, $b_{0}$ = Y-Intercept, x = Independent Variable, n = Sample Size

Predicted Value

Variable

b_{1}

Slope

Variable

b_{0}

Y-Intercept

Variable

x

Independent Variable

Variable

n

Sample Size

Variable

\overset{y}{^}

\hat{y}

Variable

Walkthrough

Derivation

Derivation of Simple Linear Regression Line

This derivation uses the Method of Least Squares to minimize the sum of squared residuals between observed data points and the linear regression model.

The relationship between variables x and y is linear.
The errors are independent and identically distributed with zero mean.

Define the Sum of Squared Residuals (SSR)

We define the objective function S as the sum of the squares of the vertical distances between each observed data point $y_{i}$ and the predicted value on the regression line.

S (b_{0}, b_{1}) = i = 1 \sum n (y_{i} - (b_{0} + b_{1} x_{i}))^{2}

Note: Minimizing the squared residuals ensures that positive and negative deviations do not cancel each other out.

Partial Differentiation with respect to b_0

To minimize S, we take the partial derivative with respect to $b_{0}$ and set it to zero, which leads to the normal equation for the intercept.

\frac{\partial S}{\partial b _{0}} = - 2 i = 1 \sum n (y_{i} - b_{0} - b_{1} x_{i}) = 0

Note: Simplifying this results in the equation $b_{0}$ = $\overset{y}{ˉ}$ - $b_{1}$ $\overset{x}{ˉ}$ .

Partial Differentiation with respect to b_1

We take the partial derivative with respect to $b_{1}$ and set it to zero to find the slope that minimizes the error.

\frac{\partial S}{\partial b _{1}} = - 2 i = 1 \sum n x_{i} (y_{i} - b_{0} - b_{1} x_{i}) = 0

Note: Substitute the expression for $b_{0}$ from the previous step into this equation to isolate $b_{1}$ .

Solve the System for b_1

By substituting $b_{0} = \overset{y}{ˉ} - b_{1} \overset{x}{ˉ}$ into the second normal equation and solving algebraically, we derive the computational formula for the slope coefficient.

b_{1} = \frac{n \sum x _{i} y _{i} - ( \sum x _{i} ) ( \sum y _{i} )}{n \sum x _{i}^{2} - ( \sum x _{i} ) ^{2}}

Note: This is equivalent to $b_{1} = \frac{Cov ( x , y )}{Var ( x )}$ .

Result

b_{1} = \frac{n \sum x _{i} y _{i} - ( \sum x _{i} ) ( \sum y _{i} )}{n \sum x _{i}^{2} - ( \sum x _{i} ) ^{2}}

Source: Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to Linear Regression Analysis.

Visual intuition

Graph

Graph unavailable for this formula.

Contains advanced operator notation (integrals/sums/limits)

Why it behaves this way

Intuition

Imagine a scatter plot of data points as a cloud of floating particles. The regression line acts like a rigid, weighted stick passing through the center of the cloud. The formula acts as a 'gravity' mechanism that rotates and shifts this stick until the sum of the vertical distances (squared) between the stick and every point in the cloud is at an absolute minimum.

\overset{y}{^}

Predicted dependent variable

The 'target' coordinate on the line of best fit for a given input, acting as the model's 'best guess' for where a data point should fall.

b_{1}

Slope (Regression Coefficient)

The 'rate of change' or sensitivity; it tells you how much the output is expected to increase or decrease for every one-unit increase in the input.

b_{0}

Intercept

The 'baseline' value; the expected value of the output when the input is zero, anchoring the line to the vertical axis.

n

Sample size

The weight of the evidence; it tells the equation how many data points are contributing to the determination of the trend.

Signs and relationships

b_1: The sign of $b_{1}$ indicates the direction of the relationship: positive means both variables move in the same direction, while negative indicates an inverse relationship.
b_0: This is an additive constant that shifts the entire line vertically, ensuring the line passes through the centroid (mean) of the data.

One free problem

Practice Problem

Given the data points (1, 2), (2, 3), and (3, 5), calculate the slope b1 of the regression line.

Sample Size3

sum_xy23

sum_x6

sum_y10

sum_x214

Solve for: $b 1$

Hint: Calculate the numerator n*sum(xy) - sum(x)*sum(y) and the denominator n*sum( $x^{2}$ ) - (sum(x))^2 separately.

The full worked solution stays in the interactive walkthrough.

Where it shows up

Real-World Context

An economist uses this equation to model the relationship between marketing spend and total sales revenue to predict how much revenue a specific budget will generate.

Study smarter

Tips

Always create a scatter plot first to ensure the relationship is actually linear.
Check for outliers, as they can disproportionately influence the slope of the regression line.
Calculate the correlation coefficient (r) to quantify the strength and direction of the linear relationship.

Avoid these traps

Common Mistakes

Assuming that a strong correlation implies causation.
Extrapolating the regression line far beyond the range of the observed x data.

Common questions

Frequently Asked Questions

This derivation uses the Method of Least Squares to minimize the sum of squared residuals between observed data points and the linear regression model.

Use this when you need to model the relationship between two continuous variables and predict future outcomes based on linear trends.

It is the foundational tool for predictive analytics, enabling researchers and businesses to forecast trends and quantify the strength of relationships between variables.

Assuming that a strong correlation implies causation. Extrapolating the regression line far beyond the range of the observed x data.

An economist uses this equation to model the relationship between marketing spend and total sales revenue to predict how much revenue a specific budget will generate.

Always create a scatter plot first to ensure the relationship is actually linear. Check for outliers, as they can disproportionately influence the slope of the regression line. Calculate the correlation coefficient (r) to quantify the strength and direction of the linear relationship.

References

Sources

Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to Linear Regression Analysis.
Freedman, D., Pisani, R., & Purves, R. (2007). Statistics.