Zahri-Edu › Math-AI › AI Lab

AI Lab

Six interactive demos of the mathematics behind machine learning: optimisation, regression, classification, clustering and neural networks.

Demo 1

Gradient Descent

Training a model means minimising a loss L(θ). Gradient descent repeatedly steps downhill: θ ← θ − η ∇L(θ), where η is the learning rate.

Click the map to choose a starting point.

Too small η: slow. Too large η: the iterates overshoot and can diverge. Momentum (β) remembers previous steps and rolls through narrow valleys.

Loss shown: the elongated bowl L(x, y) = x² + 10y² or the Rosenbrock "banana" valley, a classic test for optimisers.

Demo 2

Least-Squares Regression

Click to add data points. The line y = a x + b minimising the mean squared error Σ(yᵢ − a xᵢ − b)²/n is computed from the normal equations.

a = Cov(x, y)/Var(x), b = ȳ − a x̄. R² measures how much of the variation the line explains.

Demo 3

The Perceptron

Rosenblatt's perceptron (1958) learns a straight decision boundary w·x + b = 0. Each misclassified point nudges the line: w ← w + η y x.

Left-click adds a blue point, Shift + click (or the toggle) a red one. If the classes are linearly separable, training always converges.

XOR-like data cannot be separated by a single line. That limitation led to multi-layer networks (Demo 5).

Demo 4

k-Means Clustering

Unsupervised learning: group points into k clusters. Alternate two steps until nothing changes:

1. assign each point to its nearest centre; 2. move each centre to the mean of its points.

Each step lowers the within-cluster sum of squares, so the algorithm stops, but possibly in a local minimum: try new random starts.

Demo 5

A Small Neural Network

A network with 2 inputs, one hidden layer of tanh neurons and a sigmoid output learns a curved decision boundary by back-propagation and gradient descent.

Background colour = predicted probability of class "red". With more hidden neurons the network can draw more complex boundaries (universal approximation).

Demo 6

Activation Functions & Softmax

Non-linear activations give networks their power. Softmax turns scores z₁, …, zₖ into probabilities pᵢ = e^{zᵢ/T} / Σ e^{zⱼ/T}.

sigmoid 1/(1+e⁻ˣ)tanhReLU max(0,x)

Temperature T: small T makes the choice sharp (almost argmax), large T makes it uniform. Language models use it to control randomness.