Activation Function Visualizer

Compare ReLU, GELU, sigmoid, tanh, Swish and more — with derivatives and a movable probe point

x = 1.0

ReLUf(1.0) = 1.000 · f′ = 1.000

max(0, x) — the default hidden-layer activation. Cheap, sparse, but can 'die' for negative inputs.

Tanhf(1.0) = 0.762 · f′ = 0.420

Squashes to (−1, 1), zero-centered — often better than sigmoid for hidden layers.

Choosing an activation function

Activation functions give neural networks their non-linearity. Toggle the derivative view to understand training behavior: where f′(x) ≈ 0 the gradient vanishes and learning stalls (see sigmoid's flat tails), which is why ReLU-family functions dominate deep networks and why transformers use GELU. Drag the probe to compare exact values across functions at any input.

Your data stays private. All processing happens locally in your browser. No data is collected, stored, or sent to any server.