Learn · Demo
Gradient descent, visualised
Optimisation is how models learn: nudge the parameters in the direction that lowers the loss, over and over. Here a point rolls across a 2-D loss surface using the real analytic gradient. Change the learning rate and watch it converge smoothly, oscillate, or diverge.
Brighter = lower loss. The dot starts at the marked spot and steps downhill along the gradient.
Small = slow but stable. Large = zig-zag, then diverge.
The surface is f(x, y) = ½(5x² + 0.5y²) over x, y ∈ [−3, 3] — an elliptical bowl, steep in x and shallow in y.
What the learning rate does
Each step moves the point by −(learning rate) × gradient. Too small and it crawls toward the minimum; just right and it converges quickly while still correcting over its overshoots; too large and it does not settle in the valley, bouncing side to side (oscillation) or flying off entirely (divergence).
Caveat: this is a hand-picked 2-D surface for intuition. Real models optimise millions of parameters over noisy mini-batches with momentum and learning-rate schedules — but the core trade-off is exactly this. More on the Learn shelf.