The Algorithms Lab

The world's oldest known numerical algorithm. Computes square roots by iteratively averaging a guess with its complement, demonstrating quadratic convergence.

A powerful numerical method that iteratively finds roots using the first derivative (tangent lines). Exhibits rapid quadratic convergence for well-behaved functions.

A root-finding algorithm that uses a succession of roots of secant lines to better approximate a root of a function f.

A fundamental numerical method for finding the roots of continuous functions. The conceptual ancestor of binary search and gradient descent.

A method that rewrites f(x)=0 as x=g(x). By applying g repeatedly, a sequence is generated that can converge to a fixed point, visualized through cobweb plots.

The workhorse of machine learning. A first-order optimization algorithm that takes steps proportional to the negative of the gradient of a function at the current point.

An iterative method for optimizing an objective function with suitable smoothness properties (e.g. differentiable or subdifferentiable).

An optimization algorithm with parameter-specific learning rates, adapting to the geometry of the data. Ideal for sparse features, but suffers from premature freezing.

An optimization algorithm that combines the best properties of the AdaGrad and RMSProp algorithms to provide an optimization heuristic for noisy, sparse gradients.