# Methodology

> Build settings, browser and OS versions, hardware, iterations, median and p95, fixtures, and the command that reproduces each number.

Page: https://svelte-lean.vercel.app/performance/methodology · Markdown: https://svelte-lean.vercel.app/performance/methodology.md · Index: https://svelte-lean.vercel.app/llms.txt

How every number in this repository is produced; the site reads the files named here and shows not measured when one is missing.

## Quantities and files

One script and one output file per quantity. Bench and stress runs write to the untracked `test-results/`; `yarn bench:publish` copies them to `apps/playground/results/` with the environment block and a publication date.

| Quantity | Script | Output | Committed |
| --- | --- | --- | --- |
| Kernel entry and one consumer behavior | `packages/core/scripts/size.mjs` | `packages/core/artifacts/size.json` | yes |
| Tier 1 registrations (tabs, menu) including the kernel | `packages/primitives/scripts/size.mjs` | `packages/primitives/artifacts/size.json` | yes |
| CSS per file and the resolved index.css | `packages/styles/scripts/size.mjs` | `packages/styles/artifacts/size.json` | yes |
| Table entries (renderer, headless, each module) | `packages/table/scripts/size.mjs` | `packages/table/artifacts/size.json` | yes |
| What a consumer application ships | `fixtures/run.mjs` | `fixtures/results.json` | yes |
| Listener counts at one and a thousand roots | `apps/playground/tests/primitives/listeners.spec.ts` | `Playwright report, annotation "listeners" (an exact assertion, not a file)` | no |
| Startup and interaction timings | `apps/playground/tests/bench, published by yarn bench:publish` | `apps/playground/results/bench.json` | yes |
| Heap, nodes and listeners after stress cycles | `apps/playground/tests/stress, published by yarn bench:publish` | `apps/playground/results/stress.json` | yes |

## Compression

Raw, gzip level 9 and brotli quality 11 from `node:zlib`, through one helper shared by the package scripts and the fixtures (`fixtures/_shared/measure.mjs`).

## Production builds only

- **Packages:** `dist/` bundled with Vite in production mode (the budgeted, quoted figure, what applications ship) and with esbuild under the `production` condition for comparison; esbuild keeps helpers behind folded development gates.
- **Fixtures:** one `vite build` per fixture, Vite defaults, packages through `node_modules` links; Svelte Lean modules and Svelte's runtime go into separate chunks.
- **Browser tests and benchmarks:** the playground built with `adapter-static`, served by `vite preview`; development-server runs are never quoted.

## Environment

Read from the measurement files of this build.

- **Fixtures (`fixtures/results.json`)**: node v22.23.3vite 7.3.6svelte 5.57.1@sveltejs/vite-plugin-svelte 6.2.4
- **Benchmarks (`apps/playground/results/bench.json`)**: chromium 153.0.8010.12 linux 6.12.107+deb13-cloud-amd64 x64 AMD EPYC 9555P 64-Core Processor 16 cores · 31.3 GiB node v22.23.3 · production · headless 2026-10-05T07:11:03.593Z

The method note of each size file is on [Budgets](https://svelte-lean.vercel.app/performance/budgets.md); toolchain versions are in docs/compatibility.md.

## Sampling and statistics

- Sizes are deterministic for a toolchain: one build, one result.
- Timings: median of five fresh page loads (`RUNS`), recorded with every sample, min, max, nearest-rank p95 and a one-sentence `method`.
- Startup and interaction are separate entries, never one score.
- Benchmarks assert only generous sanity thresholds, never a machine-specific value.

## Listener counts

Two independent accounts: the runtime's `listeners()` and a wrapper around `EventTarget.prototype.addEventListener` installed before any page script. The whole per-type count on `window`, `document`, `<html>` and `<body>` must be identical for one root and a thousand, so SvelteKit's listeners cannot mask a regression. [Proof](https://svelte-lean.vercel.app/proof.md) runs both on this site.

## What is and is not compared

- Svelte Lean against itself: one root against a thousand, native-only against tabs, a table with and without a module.
- Against Bits UI: the same UI per case, production builds, source and script in benchmarks/compare, results on [Why not Bits UI](https://svelte-lean.vercel.app/docs/why-not-bits-ui.md#measured).
- Never: development against production builds, unstyled against styled pages, one primitive against another library's full stack.

## Losing results

A worse number or a lost comparison is published with the same method as any other; it is never removed from a report.

## Reproducing each number

From the repository root, Node from `.nvmrc`, Yarn 4. Playwright uses the installed Google Chrome when present, else Chromium; `bench.json` records which.

`Terminal`

```bash
yarn install --immutable
yarn build:packages                       # dist/ for every package; the fixtures consume it

yarn size                                 # every packages/<name>/artifacts/size.json
yarn workspace @svelte-lean/primitives size   # one package

yarn test:fixtures                        # every fixture → fixtures/results.json
node fixtures/run.mjs tabs-only           # one fixture; results.json is left alone
node fixtures/_shared/assert.mjs fixtures/tabs-only   # re-assert an existing build

yarn playwright install --with-deps chromium          # CI; locally the installed Chrome is used
yarn workspace @svelte-lean/playground test:e2e --project=chromium tests/primitives/listeners.spec.ts
yarn workspace @svelte-lean/playground bench          # → apps/playground/test-results/bench.json (not tracked)
yarn workspace @svelte-lean/playground stress         # → apps/playground/test-results/stress.json (not tracked)
yarn bench:publish                        # bench + stress in one run → apps/playground/results/ (tracked)
```

## Publication rule for timings

- Run `yarn bench:publish` on an idle machine, then commit `apps/playground/results/`; a run alone publishes nothing.
- The nightly workflow runs the same command and uploads the result without committing it.
- A published timing describes the recorded environment, not every machine.

## Regression thresholds

| Quantity | Gate |
| --- | --- |
| Sizes | Budgets fail `yarn size` and `yarn verify` ([Budgets](https://svelte-lean.vercel.app/performance/budgets.md)) |
| Fixtures | `expect.json` (module graph, code substrings, manifest) fails `yarn test:fixtures`; sizes are recorded, not gated |
| Listeners | Exact per-type equality for one and a thousand roots fails the browser suite |
| Timings | Sanity thresholds only; regressions found by comparing `bench.json` with a new run or the nightly artifacts |

Canonical text: docs/proof/methodology.md; definition of done for a performance claim: CONTRIBUTING.md.