Logistic Function

Core idea

Overview

The logistic function, commonly known as the sigmoid function, maps any real-valued input into a constrained range between 0 and 1. In machine learning, it serves as the fundamental activation function for binary classification and neural networks, transforming linear combinations into probabilities.

When to use: Use this function when performing binary classification to predict the probability of a specific class. It is particularly effective when the relationship between the features and the target outcome follows an S-shaped curve rather than a linear trend.

Why it matters: It allows models to make probabilistic interpretations of continuous data, which is essential for risk assessment and decision-making systems. Its differentiable nature also makes it vital for the gradient descent optimization used in training complex neural networks.

Symbols

Variables

$σ$ (x) = Output (0-1), x = Input Value

σ (x)

Output (0-1)

Variable

x

Input Value

Variable

Walkthrough

Derivation

Formula: Logistic (Sigmoid) Function

The logistic function maps any real input to a value strictly between 0 and 1, so it can be interpreted as a probability in binary classification.

Input x is any real number.
Output is interpreted as a probability of the positive class.

1

State the sigmoid function:

Exponentials ensure the denominator is always positive, keeping the output in (0,1).

σ (x) = \frac{1}{1 + e ^{- x}}

2

Check the limiting behaviour:

Large positive x makes $e^{- x}$ tiny, while large negative x makes $e^{- x}$ huge, pushing the fraction towards 0.

x \to \infty lim σ (x) = 1, x \to - \infty lim σ (x) = 0

Note: At x=0, $σ$ (0)=1/2.

Result

x \to \infty lim σ (x) = 1, x \to - \infty lim σ (x) = 0

Source: Standard curriculum — A-Level Data Science & Machine Learning

Free formulas

Rearrangements

Solve for $x$

Make x the subject

x = ln (\frac{S}{1 - S})

Rearranges the logistic function formula to solve for the input value x.

Difficulty: 4/5

The static page shows the finished rearrangements. The app keeps the full worked algebra walkthrough.

Visual intuition

Graph

The logistic function produces an S-shaped curve that begins with exponential growth before transitioning into a period of deceleration as it approaches a horizontal carrying capacity. It features two horizontal asymptotes, typically at y=0 and y=1, and an inflection point at the center where the rate of growth is maximized. This shape effectively models population growth or resource diffusion in a system where growth is initially rapid but eventually constrained by external limits.

Graph type: sigmoid

Why it behaves this way

Intuition

A smooth, S-shaped curve that maps any real input to an output between 0 and 1, representing a gradual transition from one state to another.

σ (x)

The output of the logistic function, representing a probability or activation level.

It quantifies the likelihood of a specific event (e.g., belonging to the positive class), always scaled between 0 and 1.

x

The input to the function, often a linear combination of features in a machine learning model.

Represents the 'evidence' or 'score' for the positive outcome. A higher 'x' indicates stronger evidence, pushing the probability closer to 1.

Signs and relationships

-x: The negative sign in the exponent ' $e^{- x}$ ' is crucial for the S-shape. As the input 'x' increases, '-x' decreases, causing ' $e^{- x}$ ' to approach zero.
1 + e^{-x}: The denominator ensures that the output ' $σ$ (x)' is always bounded between 0 and 1. Since ' $e^{- x}$ ' is always positive, '1 + $e^{- x}$ ' is always greater than 1, which guarantees that the fraction '1 / (1 + $e^{- x}$ )' is well defined.

Free study cues

Insight

Canonical usage

The logistic function takes a dimensionless input and produces a dimensionless output, typically interpreted as a probability or a value between 0 and 1.

Common confusion

A common mistake is attempting to assign physical units to the input 'x'. However, 'x' (often representing a log-odds or a linear combination of features)

Dimension note

Both the input 'x' and the output ' $σ$ (x)' of the logistic function are dimensionless. The exponent of 'e' must always be dimensionless, and the function's output is a probability, which is a ratio without physical

Unit systems

$x$ dimensionless - The input 'x' (often a linear combination of features or log-odds) must be dimensionless for the exponential function e^(-x) to be mathematically valid.

$σ (x)$ dimensionless - The output of the logistic function represents a probability or a normalized value, which is inherently dimensionless and constrained between 0 and 1.

One free problem

Practice Problem

A neuron in a deep learning model receives a weighted sum (logit) of 0. Calculate the output activation S using the logistic function.

Input Value0

Solve for: $S$

Hint: Any non-zero base raised to the power of 0 is 1.

The full worked solution stays in the interactive walkthrough.

Where it shows up

Real-World Context

When predicting the probability of a positive class, Logistic Function is used to calculate Output from Input Value. The result matters because it helps estimate likelihood and make a risk or decision statement rather than treating the number as certainty.

Study smarter

Tips

The output S is exactly 0.5 when the input x is 0.
Inputs far from zero lead to 'vanishing gradients' where the function becomes very flat.
Always normalize input features to prevent the function from saturating at 0 or 1 too quickly.

Avoid these traps

Common Mistakes

Forgetting the negative sign in e^-x.
Treating output as unbounded.

Keep going

Related Formulas

Common questions

Frequently Asked Questions

The logistic function maps any real input to a value strictly between 0 and 1, so it can be interpreted as a probability in binary classification.

Use this function when performing binary classification to predict the probability of a specific class. It is particularly effective when the relationship between the features and the target outcome follows an S-shaped curve rather than a linear trend.

It allows models to make probabilistic interpretations of continuous data, which is essential for risk assessment and decision-making systems. Its differentiable nature also makes it vital for the gradient descent optimization used in training complex neural networks.

Forgetting the negative sign in e^-x. Treating output as unbounded.

When predicting the probability of a positive class, Logistic Function is used to calculate Output from Input Value. The result matters because it helps estimate likelihood and make a risk or decision statement rather than treating the number as certainty.

The output S is exactly 0.5 when the input x is 0. Inputs far from zero lead to 'vanishing gradients' where the function becomes very flat. Always normalize input features to prevent the function from saturating at 0 or 1 too quickly.

References

Sources

Wikipedia: Logistic function
Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville
Wikipedia: Sigmoid function
Ian Goodfellow, Yoshua Bengio, Aaron Courville Deep Learning
Christopher M. Bishop Pattern Recognition and Machine Learning
Trevor Hastie, Robert Tibshirani, Jerome Friedman The Elements of Statistical Learning
Standard curriculum — A-Level Data Science & Machine Learning

Overview

Variables

Derivation

State the sigmoid function:

Check the limiting behaviour:

Rearrangements

Graph

Intuition

Insight

Practice Problem

Real-World Context

Tips

Common Mistakes

Related Formulas

Binary Cross-Entropy

Frequently Asked Questions

Sources