Daniel Noulet 8d8970d5a7 test: drive the whole application over HTTP, and serialise migrations

The behaviour suite in internal/behaviour installs the application into an
empty database through the web installer, signs in, walks every flow, and then
replaces the process to confirm the state was persisted rather than held in
memory. It found four defects, each fixed here or in the preceding commit:

- Migrate was not safe to run from two instances at once. CREATE TABLE IF NOT
  EXISTS is not race free in PostgreSQL and collided on the pg_type unique
  index, and two replicas could apply the same file. The run now holds a
  session-level advisory lock, so a rolling deployment waits instead of failing.

- Bootstrap refused ownership whenever a tenant held any member, so installing
  with the demonstration seed and then setting a local administrator password
  produced an account that could never reach the console. It now grants
  ownership only while the tenant has no owner, which still never takes a tenant
  over from one that does.

Three of the fixes are covered by store-level regression tests in the same
commit: the ownership rule, the duplicate tenant slug conflict, and concurrent
migration runs.

The suite is skipped by go test ./... because it needs a scratch database and
the Google credentials for the installer. Run it with make behaviour-test, which
also resets the database afterwards. make test no longer inherits the database
variables, so unit runs need nothing.

Completes the earlier correction to the docs: the role matrix lives in the
database rather than in internal/role, and a local administrator claims the
system administrator role when no Google account has.

2026-10-05 19:37:09 +02:00

7.5 KiB

Raw Permalink Blame History

Go Loose

Go Loose is a multi-tenant API-key control plane for Go HTTP services. It gives operators a web console to register tenant applications, issue and revoke keys, import OpenAPI contracts, and inspect API metadata. Applications use the public client package as fail-closed middleware: requests without a valid key never reach the protected handler.

Start locally

docker compose up --build

Start the shared TLS proxy using /home/dnoulet/go/infrasctruture, add 127.0.0.1 auth.dev nmbs.auth.dev ypto.auth.dev to /etc/hosts, then open https://auth.dev. The Compose setup deliberately enables database login and private OpenAPI URL scanning for local development. Never use unrestricted private URL scanning in production.

On a fresh database, open https://auth.dev/install, enter the Google OAuth client secret from Google Cloud Console, and optionally seed the NMBS and YPTO Guess demonstration data. The CMS Demo client ID and callback URL are prefilled. See the installation guide for Google setup, first-admin behavior, waiting-room approval, and all demo credentials.

The bare auth.dev host is reserved for system-administrator SSO. Tenant administrators use database credentials on their tenant host, such as https://nmbs.auth.dev.

If PostgreSQL port 5433 is occupied, set GO_LOOSE_POSTGRES_PORT=15432.

The first Google SSO user becomes system administrator. Later SSO users remain in a waiting room until the system administrator assigns a tenant, role, and application access. Tenant administrators can manage only their own tenants.

Request flow

Caller
  └─ X-API-Key: gl_...
       └─ Your Go API + go-loose/client middleware
            └─ POST /api/v1/authorize → Go Loose
                 ├─ hash key, evaluate status/expiry/tenant/app/host
                 ├─ write authorization audit event
                 └─ allow with tenant/key principal OR deny

Only a SHA-256 digest and a short display prefix are stored. A plaintext key is shown once at creation. The client strips the credential before calling the protected handler.

Configuration

Copy .env.example for direct local execution. Important settings:

Variable	Purpose
`GO_LOOSE_DATABASE_URL`	PostgreSQL connection string
`GO_LOOSE_SESSION_SECRET`	At least 32 random characters; rotate through a planned session invalidation
`GO_LOOSE_BASE_URL`	Public origin, used for secure cookies and same-origin checks
`GO_LOOSE_OIDC_*`	OIDC issuer, client ID, secret, and callback URL
`GO_LOOSE_ALLOWED_EMAIL_DOMAINS`	Optional comma-separated SSO email-domain allowlist
`GO_LOOSE_LOCAL_LOGIN`	Enables database-backed login for seeded and invited users
`GO_LOOSE_LOCAL_ADMIN_PASSWORD`	Optional compatibility bootstrap; omit for installer-first deployment
`GO_LOOSE_DEV_LOGIN`	Local-only login; keep `false` in production
`GO_LOOSE_ALLOW_PRIVATE_CONTRACT_URLS`	Allows scanning internal/loopback OpenAPI URLs; local-only unless egress is isolated

Production must place Go Loose behind TLS, use a secret manager, set GO_LOOSE_BASE_URL to the HTTPS origin, disable development login, and configure OIDC. Swagger UI is available at /docs only after login.

Documentation

Profile

Every management session can open /profile to set a display name and a profile picture. Pictures are stored in PostgreSQL and served from a public, unguessable URL that client applications can embed directly. Display names stop being overwritten by the identity provider once the user customizes them. See roles and capabilities.

Development

go mod download
make fmt
make test
make vet
make build

Development requires Go 1.27.1 or newer.

Migrations run transactionally at startup and are tracked in schema_migrations. See the operations guide before creating one.

make test runs the unit tests and needs nothing. make integration-db creates the scratch databases the database-backed tests use, and make integration-test runs them. Those tests drop every table they touch, so they refuse to run against a database whose name does not contain test. That guard exists because pointing them at a development database destroys it.

Behaviour suite

make behaviour-test drives the whole application over real HTTP against a real PostgreSQL database. It resets the database, installs the application through the web installer using the Google client credentials in google_secrets.json, signs in, walks every flow, then replaces the process and checks that the installation, tenants, applications, contracts, roles, sessions, profile, and picture are all still there. It resets the database again afterwards.

The suite is skipped by go test ./... because it needs that credentials file and a scratch database. Set GO_LOOSE_BEHAVIOUR_CLIENT_ID and GO_LOOSE_BEHAVIOUR_CLIENT_SECRET instead of the file if you prefer. The one step it cannot automate is Google's interactive consent screen: it verifies the redirect to Google carries the installed client id and redirect URI, but completing the round trip needs a real Google account.

Phases live in internal/behaviour, one file per area, and share a single ordered journey so each step can use what the previous one created.

Hot reload

make compose-dev-up starts a development stack that runs Go Loose under Air. Air watches cmd, internal, client, and web, rebuilds on every change, and writes the binary to ./dist. Changes to web/ are embedded at build time, so editing a template, stylesheet, or script triggers a rebuild too. The server restarts in place and /healthz confirms it came back.

make compose-dev-up      # build the image, start postgres and Air
make compose-dev-logs    # follow the rebuild and request logs
make compose-dev-down    # stop and remove the stack

Variable	Default	Purpose
`GO_LOOSE_DEV_PORT`	`8080`	Host port for the dev server
`GO_LOOSE_POSTGRES_PORT`	`5433`	Host port for PostgreSQL

The dev stack binds mounts the source tree, so Air writes dist/go-loose as root inside the container. On Linux, delete it with sudo rm dist/go-loose if the file ownership gets in the way.

docker-compose.yaml remains the stack that matches production: a distroless image, no dev login, and the local-dev-edge network. docker-compose.dev.yaml and Dockerfile.dev are development-only.

The database is not bind mounted into the source tree: PostgreSQL stores its files as 0700 owned by its own user, which would make go build ./... fail to walk the directory. docker-compose.yaml uses the db-data volume and docker-compose.dev.yaml uses a separate db-data-dev, so docker compose -f docker-compose.dev.yaml down -v cannot reach the real database. Reach a database through the published 5433 port or docker compose exec postgres psql -U goloose -d goloose.

The dev database starts empty and migrations rebuild it on boot. make compose-dev-down also empties ./dist, because Air writes that binary as root inside the container.

7.5 KiB Raw Permalink Blame History