Skip to content
Computational Lab.

Naive Bayes Classifier

Bayes' rule, applied one feature at a time.

Beginner / classification / probability / generative

Each class gets an axis-aligned Gaussian over the features; Bayes' rule with the class frequencies as priors turns any point into a posterior probability. The shaded field is P(class A | x) - recomputed from scratch every time you add a labeled point, so you can watch one click move the decision boundary.

Tagged: Gaussian NB, posterior, decision boundary, Bayes

Live simulation

Click to add labeled points; hover anywhere to read the posterior P(A | x).

Speed

Click to add labeled points; hover anywhere to read the posterior P(A | x).

Mathematics

Bayes' rule turns a generative model into a classifier: model each class's features as an axis-aligned Gaussian, combine with the class frequencies as priors, and the posterior P(class | x) follows in closed form. 'Naive' is the independence assumption between features - which is exactly what the elliptical contours show.

P(A | x) = π_A N(x; μ_A, Σ_A) / (π_A N_A + π_B N_B)

log-posterior: log π − ½ (x − μ)ᵀ Σ⁻¹ (x − μ) − ½ log |Σ|

decision boundary: P(A | x) = 0.5

Everything is recomputed from scratch whenever the data changes: means, variances (with a variance floor so single points cannot claim zero width), priors, the full posterior field, and the training accuracy. Adding one point on the wrong side of the boundary drags its class mean and visibly bends the 0.5 contour - generative classifiers answer to their data directly.

How it works

The posterior field renders on a 96×48 grid through a token-built diverging colormap (class A toward accent, class B toward sky) at 55% opacity so points stay dominant; the 1σ ellipses are drawn per class from the live statistics. The pointer's own posterior P(A | x) is a live metric - hover anywhere and read the classifier's mind.

The tool selector adds class A, adds class B, or erases the nearest point; class counts and priors update with every click.

Implementation

src/engine/experiments/naive-bayes.ts · 253 lines · the module running above

TypeScript · fixed-timestep engine

1import { BaseExperiment } from "../core/base-experiment";
2import { sampleInto } from "../render/colormaps";
3import type { ParameterDef, PointerState, ThemeColors } from "../core/types";
4import { divergingFor } from "../render/diverging-cache";
5
6const CAPACITY = 240;
7const FIELD_W = 96;
8
9/**
10 * Gaussian Naive Bayes: per class, an axis-aligned Gaussian over the two
11 * features; priors are the class frequencies. The posterior field P(A | x)
12 * is recomputed from Bayes' rule every time the data changes - adding a
13 * single labeled point moves the decision boundary on the next tick.
14 */
15export class NaiveBayesExperiment extends BaseExperiment {
16 readonly id = "naive-bayes";
17
18 private xs = new Float32Array(CAPACITY);
19 private ys = new Float32Array(CAPACITY);
20 private labels = new Float32Array(CAPACITY);
21 private count = 0;
22 private stats = {
23 meanAx: 0.5, meanAy: 0.5, varAx: 1e-4, varAy: 1e-4,
24 meanBx: 0.5, meanBy: 0.5, varBx: 1e-4, varBy: 1e-4,
25 priorA: 0.5,
26 };
27 private accuracy = 0;
28 private pointerPosterior = 0.5;
29 private pointer = { x: 0, y: 0, down: false, inside: false };
30 private lastCell = { x: -1, y: -1 };
31 private image: ImageData | null = null;
32 private scratch: HTMLCanvasElement | null = null;
33
34 protected params(): ParameterDef[] {
35 return [
36 {
37 key: "tool",
38 label: "Pointer tool",
39 options: [
40 { value: "classA", label: "Add class A (+)" },
41 { value: "classB", label: "Add class B (-)" },
42 { value: "erase", label: "Erase nearest" },
43 ],
44 defaultValue: "classA",
45 },
46 ];
47 }
48
49 protected onReset(): void {
50 this.count = 0;
51 // Two seeded, clearly separated blobs to start from.
52 for (let i = 0; i < 20; i++) {
53 this.addPoint(0.3 + (this.rng() - 0.5) * 0.22, 0.35 + (this.rng() - 0.5) * 0.22, 1, true);
54 }
55 for (let i = 0; i < 20; i++) {
56 this.addPoint(0.7 + (this.rng() - 0.5) * 0.22, 0.65 + (this.rng() - 0.5) * 0.22, -1, true);
57 }
58 this.updateModel();
59 }
60
61 private addPoint(x: number, y: number, label: number, skipUpdate = false): void {
62 if (this.count >= CAPACITY) return;
63 this.xs[this.count] = Math.min(1, Math.max(0, x));
64 this.ys[this.count] = Math.min(1, Math.max(0, y));
65 this.labels[this.count] = label;
66 this.count += 1;
67 if (!skipUpdate) this.updateModel();
68 }
69
70 /** Static field: recomputes on data and parameter changes. */
71 protected onUpdate(): void {}
72
73 resize(width: number, height: number): void {
74 this.width = width;
75 this.height = height;
76 }
77
78 onPointer(state: PointerState): void {
79 this.pointer = state;
80 if (state.inside) {
81 this.pointerPosterior = this.posteriorA(state.x / Math.max(1, this.width), state.y / Math.max(1, this.height));
82 }
83 if (!state.inside || !state.down) {
84 this.lastCell = { x: -1, y: -1 };
85 return;
86 }
87 const x = state.x / Math.max(1, this.width);
88 const y = state.y / Math.max(1, this.height);
89 const cell = { x: Math.round(x * 50), y: Math.round(y * 50) };
90 if (cell.x === this.lastCell.x && cell.y === this.lastCell.y) return;
91 this.lastCell = cell;
92 const tool = this.str("tool");
93 if (tool === "erase") {
94 let best = -1;
95 let bestDist = 0.06;
96 for (let i = 0; i < this.count; i++) {
97 const d = Math.hypot(this.xs[i] - x, this.ys[i] - y);
98 if (d < bestDist) {
99 bestDist = d;
100 best = i;
101 }
102 }
103 if (best >= 0) {
104 this.count -= 1;
105 this.xs[best] = this.xs[this.count];
106 this.ys[best] = this.ys[this.count];
107 this.labels[best] = this.labels[this.count];
108 this.updateModel();
109 }
110 } else {
111 this.addPoint(x, y, tool === "classA" ? 1 : -1);
112 }
113 }
114
115 /** Bayes' rule with axis-aligned Gaussian class-conditionals. */
116 private posteriorA(x: number, y: number): number {
117 const s = this.stats;
118 const logA =
119 Math.log(Math.max(1e-9, s.priorA)) -
120 (0.5 * (x - s.meanAx) ** 2) / s.varAx -
121 (0.5 * (y - s.meanAy) ** 2) / s.varAy -
122 0.5 * Math.log(s.varAx * s.varAy);
123 const logB =
124 Math.log(Math.max(1e-9, 1 - s.priorA)) -
125 (0.5 * (x - s.meanBx) ** 2) / s.varBx -
126 (0.5 * (y - s.meanBy) ** 2) / s.varBy -
127 0.5 * Math.log(s.varBx * s.varBy);
128 // Softmax of the two log posteriors.
129 const max = Math.max(logA, logB);
130 const expA = Math.exp(logA - max);
131 const expB = Math.exp(logB - max);
132 return expA / (expA + expB);
133 }
134
135 private updateModel(): void {
136 let ax = 0;
137 let ay = 0;
138 let bx = 0;
139 let by = 0;
140 let nA = 0;
141 let nB = 0;
142 for (let i = 0; i < this.count; i++) {
143 if (this.labels[i] > 0) {
144 ax += this.xs[i];
145 ay += this.ys[i];
146 nA += 1;
147 } else {
148 bx += this.xs[i];
149 by += this.ys[i];
150 nB += 1;
151 }
152 }
153 const varFloor = 1e-4;
154 const stats = {
155 meanAx: nA > 0 ? ax / nA : 0.5,
156 meanAy: nA > 0 ? ay / nA : 0.5,
157 meanBx: nB > 0 ? bx / nB : 0.5,
158 meanBy: nB > 0 ? by / nB : 0.5,
159 priorA: this.count > 0 ? nA / this.count : 0.5,
160 varAx: varFloor,
161 varAy: varFloor,
162 varBx: varFloor,
163 varBy: varFloor,
164 };
165 for (let i = 0; i < this.count; i++) {
166 if (this.labels[i] > 0) {
167 stats.varAx = Math.max(varFloor, stats.varAx + (this.xs[i] - stats.meanAx) ** 2 / Math.max(1, nA));
168 stats.varAy = Math.max(varFloor, stats.varAy + (this.ys[i] - stats.meanAy) ** 2 / Math.max(1, nA));
169 } else {
170 stats.varBx = Math.max(varFloor, stats.varBx + (this.xs[i] - stats.meanBx) ** 2 / Math.max(1, nB));
171 stats.varBy = Math.max(varFloor, stats.varBy + (this.ys[i] - stats.meanBy) ** 2 / Math.max(1, nB));
172 }
173 }
174 this.stats = stats;
175 // Training accuracy
176 let correct = 0;
177 for (let i = 0; i < this.count; i++) {
178 const p = this.posteriorA(this.xs[i], this.ys[i]);
179 const prediction = p >= 0.5 ? 1 : -1;
180 if (prediction === this.labels[i]) correct += 1;
181 }
182 this.accuracy = this.count > 0 ? correct / this.count : 0;
183 }
184
185 render(ctx: CanvasRenderingContext2D, theme: ThemeColors): void {
186 ctx.fillStyle = theme.bg;
187 ctx.fillRect(0, 0, this.width, this.height);
188 const diverging = divergingFor(theme);
189 const fieldH = Math.floor(FIELD_W / 2);
190 if (!this.image || this.image.width !== FIELD_W) {
191 this.image = new ImageData(FIELD_W, fieldH);
192 this.scratch = document.createElement("canvas");
193 this.scratch.width = FIELD_W;
194 this.scratch.height = fieldH;
195 }
196 const data = this.image.data;
197 for (let gy = 0; gy < fieldH; gy++) {
198 for (let gx = 0; gx < FIELD_W; gx++) {
199 const p = this.posteriorA((gx + 0.5) / FIELD_W, (gy + 0.5) / fieldH);
200 const t = Math.min(1, Math.max(0, 0.5 + (p - 0.5) * 1.1));
201 sampleInto(diverging, t, data, (gy * FIELD_W + gx) * 4, 140);
202 }
203 }
204 this.scratch!.getContext("2d")!.putImageData(this.image, 0, 0);
205 ctx.imageSmoothingEnabled = true;
206 ctx.drawImage(this.scratch!, 0, 0, this.width, this.height);
207 // 1-sigma ellipses per class
208 const s = this.stats;
209 const scale = Math.min(this.width, this.height);
210 ctx.strokeStyle = theme.fgSecondary;
211 ctx.lineWidth = 1;
212 for (const cls of [
213 { cx: s.meanAx, cy: s.meanAy, sx: Math.sqrt(s.varAx), sy: Math.sqrt(s.varAy) },
214 { cx: s.meanBx, cy: s.meanBy, sx: Math.sqrt(s.varBx), sy: Math.sqrt(s.varBy) },
215 ]) {
216 ctx.beginPath();
217 ctx.ellipse(cls.cx * this.width, cls.cy * this.height, cls.sx * scale, cls.sy * scale, 0, 0, Math.PI * 2);
218 ctx.stroke();
219 }
220 // Points
221 for (let i = 0; i < this.count; i++) {
222 ctx.fillStyle = this.labels[i] > 0 ? theme.accent : theme.viz[1];
223 ctx.beginPath();
224 ctx.arc(this.xs[i] * this.width, this.ys[i] * this.height, 3.4, 0, Math.PI * 2);
225 ctx.fill();
226 }
227 }
228
229 getMetrics() {
230 return {
231 classA: this.count > 0 ? this.countClass(1) : 0,
232 classB: this.count > 0 ? this.countClass(-1) : 0,
233 priorA: `${(this.stats.priorA * 100).toFixed(0)} %`,
234 accuracy: `${(this.accuracy * 100).toFixed(1)} %`,
235 pointerPA: this.pointer.inside ? this.pointerPosterior.toFixed(3) : "n/a",
236 };
237 }
238
239 private countClass(label: number): number {
240 let sum = 0;
241 for (let i = 0; i < this.count; i++) if (this.labels[i] === label) sum += 1;
242 return sum;
243 }
244
245 describe(): string {
246 return `Gaussian Naive Bayes trained on ${this.count} labeled points (${this.countClass(1)} class A, ${this.countClass(-1)} class B), ${(this.accuracy * 100).toFixed(0)}% training accuracy. Click to add points and watch the boundary move.`;
247 }
248
249 entities(): number {
250 return this.count;
251 }
252}
253

Performance

Time per tick
O(points) per model update; O(grid) per field render
Space
Eight numbers of class statistics
Notes
Compare with the SVM: generative model versus discriminative margin, Gaussian boundary versus kernelized hinge - same data, two philosophies.

Live numbers (FPS, frame time, steps per second, microseconds per step, entity count) are in the performance overlay on the simulation above: the pulse icon in its top-right corner.