Matrices are not tables of numbers. They are transformations of space. Eigenvectors are the directions that survive transformation. The dot product is the attention mechanism. This is where the transformer architecture lives.
A 2×2 matrix doesn't store data. It defines a function that takes every point in the plane and moves it somewhere else. The identity matrix leaves everything in place. A rotation matrix spins the plane. A scaling matrix stretches or compresses. Every matrix is a verb, not a noun.
Most vectors change direction when you apply a matrix transformation. But some vectors only get stretched or compressed — they keep pointing the same way. These are eigenvectors. The amount they get scaled is the eigenvalue.
In machine learning, eigenvectors reveal the principal directions of variation in data. Principal Component Analysis (PCA) is eigendecomposition applied to covariance matrices. The most important features are the eigenvectors with the largest eigenvalues.
The canvas below lets you define a 2×2 transformation matrix and watch it act on the plane. Preset matrices demonstrate rotation, reflection, shearing, and projection. Eigenvectors are shown in real time.
Interactive canvas coming soon. This lab is under active development.
2×2 matrix transformation visualizer
The transformer architecture that powers GPT, Claude, and Apple's Foundation Models computes attention using scaled dot products. Two vectors' dot product measures alignment — how much one vector points in the direction of another. High dot product means high attention.
Q is the query. K is the key. V is the value. The dot product QKT produces the attention scores. Softmax normalizes them into a probability distribution. This is linear algebra at the heart of every large language model.
Below is where the attention heatmap visualization will live — showing how tokens in a sequence attend to each other. Each cell represents the dot-product similarity between a query token and a key token.
Interactive visualization coming soon.
The same linear algebra that powers the attention mechanism also governs the state space of SwiftVector. Constraint evaluation is a linear map. The codex composes constraints the way matrices compose transformations. Understanding linear algebra is understanding the machinery underneath.
— From the curriculum thesis