go-bananas/docs/adr/0001-embedded-sqlite.md

# ADR 0001: Embedded SQLite rather than a client/server database

- **Status:** accepted
- **Date:** 2026-10-04
- **Deciders:** platform team
- **Applies to:** all deployments of the current major version

## Context

go-bananas stores themes. A theme is a slug, some metadata and a JSON document of
design tokens, typically 10–60 kB. A useful instance has tens to a few thousand of
them.

The workload shape is:

- **Reads:** constant and bursty. Every page view in every consuming application
  requests CSS. A service with ten consumer applications and a few hundred users
  can easily see thousands of stylesheet requests a minute, and 95%+ of them are
  the same handful of bytes served from a render cache.
- **Writes:** rare. A theme is edited by a human. A busy week might be fifty writes.
- **Total size:** a few megabytes. Even ten thousand large themes would fit in
  fifty.

The conventional answer for a service with this shape is PostgreSQL. That answer is
right for almost every service and wrong for this one, for reasons worth writing
down rather than rediscovering in a year.

## Decision

Use SQLite through `modernc.org/sqlite`, a pure-Go translation, on a local volume.

## Rationale

### What a client/server database would add

For this dataset, Postgres buys nothing and costs:

- **A network hop per query.** A local SQLite read is microseconds; a query over a
  pod network is a millisecond and a tail-latency distribution with a long right
  tail. For a stylesheet endpoint on the critical path of every page view, that
  tail is the difference between a fast site and a janky one.
- **A connection pool to get wrong.** Every deployment needs its own tuning, and the
  failure mode of under-tuning is a saturated pool under load.
- **Credentials to create, distribute, rotate and revoke.** Another secret in
  Kubernetes, another thing to get wrong.
- **A schema migration process.** For a dataset that fits in a few megabytes, this
  is a disproportionate amount of ceremony.
- **A second thing to operate.** Backups, upgrades, monitoring, incident response,
  for a component whose whole job is to hold some JSON.
- **A second failure mode.** The database becoming unavailable is now the most
  likely way the theme service goes down.

### What SQLite gives up

Honesty requires stating the costs:

- **One writer at a time.** Correct at this write rate. A theme edit is a
  millisecond-scale transaction; even a hundred concurrent editors would not
  saturate it.
- **Single-node.** This is the real constraint. See Scaling below.
- **No network access.** A consuming application cannot query the database directly.
  It does not need to; the runtime API is the interface.

### Why this particular configuration is fast

The default SQLite configuration is not fast. These settings are what make it fast
enough that the question does not arise:

| Setting | Effect |
| --- | --- |
| `journal_mode=WAL` | Readers never block the writer and the writer never blocks readers. This is the single most important setting. |
| `synchronous=NORMAL` | In WAL mode this is durable across an application crash and risks only the last transactions on an operating-system crash. A lost publish is re-doable; an outage is not. |
| `cache_size=-16000` | A 16 MiB page cache, which holds the entire dataset. |
| `mmap_size=268435456` | The file is read through the page cache directly, removing a copy from the read path. |
| `temp_store=MEMORY` | Sort and group-by spill stays in memory. |
| Max 8 connections | SQLite serialises writers. A large pool adds contention, not throughput. This is counter-intuitive and correct. |
| In-memory render cache | The steady-state runtime request performs **no database access at all**. |

The render cache is the decisive optimisation. Rendering a theme traverses its whole
token document; serving the cached string is one map lookup. Because the cache key
includes the theme version, publishing invalidates exactly one theme's entries and
disturbs nothing else. The database is consulted only on a cache miss, which is a
publish rather than a read.

### Why `modernc.org/sqlite` rather than `mattn/go-sqlite3`

`mattn/go-sqlite3` is a cgo binding to the C library. It is faster, and it means:

- `CGO_ENABLED=1`, so the release image needs a C toolchain at build time and libc
  at run time, and cross-compilation needs a cross-C toolchain;
- a statically linked binary is impossible, so the image carries a C library whose
  CVEs need tracking;
- `go install` from source fails on a machine without a compiler.

`modernc.org/sqlite` is a transpiled Go implementation: no cgo, a fully static
binary, and cross-compilation is `GOOS=linux GOARCH=arm64` and nothing else. The
performance difference is irrelevant here because the hot path is the in-memory
render cache, not SQLite.

## Alternatives considered

**PostgreSQL.** The conventional choice. Rejected on the grounds above. It remains
the right answer if the product changes shape — see Scaling.

**Redis.** Excellent for this read-heavy workload, and a good fit for a render
cache. But it is another service to operate for a cache that is already
process-local and loses nothing on restart, because the source of truth is the
database. Rejected as unnecessary operational surface. Worth revisiting if a
many-node deployment needs a shared cache; the render cache is an interface, so
that change is contained.

**bbolt.** Faster than SQLite for pure key/value workloads and simpler. Rejected
because the schema needs indexes for listing, filtering and sorting, and because
SQLite's query planner handles the "a few hundred rows filtered by status and
ordered by date" shape without the ceremony of managing bbolt buckets by hand.

**A JSON file per theme on disk.** No dependency at all. Rejected because concurrent
writes need locking, and because every listing requires reading and parsing every
file. SQLite is a better version of this idea that is already written.

**An in-memory store with write-through to object storage.** Attractive for scale.
Rejected for v1: it removes the transactional guarantee that makes version history
and rollback correct, and it adds an object-storage dependency. The write path
here is not hot enough to justify losing the invariant.

## Scaling

SQLite on a local volume is single-node. Two shapes are supported:

1. **One replica, one volume.** The recommended production shape. Horizontally
   scalable only in the sense that the service is so cheap that one replica serves
   a very large number of consumers.

2. **Many replicas, a shared filesystem.** Works when the storage class supports
   concurrent mounts — NFS, CephFS, or a ReadWriteMany block device. WAL mode
   requires the filesystem to support shared memory mapping, which these do. The
   Helm chart warns when a ReadWriteOnce volume is combined with more than one
   replica, because that combination cannot work.

If the workload ever outgrows both — genuinely concurrent multi-writer access across
nodes — the migration is to Postgres. It is contained: `internal/store` is the only
package that writes SQL, and its interface is defined by the methods it exposes
rather than by SQLite's features.

## Consequences

**Good**

- One less thing to operate, back up, monitor and secure.
- No credential to leak.
- A static binary and a small image with no libc.
- A backup is `cp` of one file. There is no consistent-snapshot dance.
- The render cache can be process-local because there is only one process.

**Bad**

- Cannot scale horizontally past one node without a shared filesystem.
- A shared-filesystem deployment is slower than local disk, and WAL over NFS is
  sensitive to latency. Measure before choosing it.
- Single-writer means a pathological write loop would queue. Nothing in the
  application can produce one.

**Neutral**

- The `store` package has a wider interface than a repository might otherwise need,
  because it must express listing, filtering and sorting. Moving to another engine
  means reimplementing that surface.

## Revisit when

- A single instance cannot serve the load, and profiling shows the database rather
  than the render cache is the bottleneck.
- Concurrent writers become a real requirement, for example if themes can be edited
  by an automated pipeline from several deployments.
- The dataset outgrows a few hundred megabytes, at which point the time to back up
  by copying a file becomes its own problem.