Walk downhill on a loss curve or surface: a learning rate too small crawls, one too large diverges. Momentum builds up speed along a valley; Adam scales each step by the size of its recent gradients.
This interactive visualization needs JavaScript.