Watch SGD, Momentum, and Adam race across loss surfaces — click anywhere to drop the optimizers
Darker = lower loss. Click the surface to restart from that point.
Vanilla SGD follows the raw gradient and zigzags in narrow ravines. Momentum accumulates velocity, damping the zigzag and powering through flat regions — but it can overshoot. Adam adapts the step size per dimension using running gradient statistics, which is why it handles badly-scaled surfaces so well. Try the saddle-point surface to see SGD stall where adaptive methods escape, and raise the learning rate until momentum starts orbiting the minimum.
Your data stays private. All processing happens locally in your browser. No data is collected, stored, or sent to any server.