Ring Promoter is a small, production-grade control plane that promotes versions of multiple applications through an ordered set of deployment rings: int (Integration) → test (Test) → acc (Acceptance) → prod (Production). It is a single Go binary — web UI, JSON REST API and promoter in one process — typically run on k3s/Kubernetes, with current/previous version and full history stored per (application, ring). The product tagline is direct: Every release earns production.
This article is for platform engineers, SREs and DevOps engineers who need to operate or evaluate the control plane. It walks the protocol in enough detail to run it day-to-day: rings, health and version verification, concurrency, deployers, promotion gates, production password and auto-promote, the CI-facing REST API, onboarding, and a k3s deploy outline. Claims below track the product README and ringpromoter.com — no invented metrics, customers or pricing.
If you want the shorter, benefits-led overview first, start with the companion blog post. Otherwise, watch the full promotion, then read how each step works.
Watch the full Ring Promoter demo on YouTube — one application promoted through all four rings in a single session.
The ring model: why “never skip a ring” matters
Rings are shared and ordered. Each managed application declares, per ring, where it lives and how to reach it — namespace, Deployment, container, image repository, health URL, or for VM/CI apps a target_env and GitHub workflow mapping. That registry lives in configuration, not code.
The rule is structural: promote {from_ring} always targets the next ring in the pipeline. You cannot ask the API to jump from int to prod. That removes a whole class of “just this once” failures that policy documents never catch under pressure.
Why four rings rather than two? Integration catches wiring and smoke. Test absorbs automated and QA validation. Acceptance is the business/staging gate. Production is the only ring that serves customers. Auto-promote can compress early hops when they are safe; humans (and the production password) keep the last hop deliberate. For a benefits-led overview, see the companion blog post.
The promotion protocol, step by step
- Acquire the per-app lock — operations on the same application are serialised (Postgres session advisory lock in production).
- Source health — the source ring must be healthy on a live check before promotion proceeds.
- Deploy — the configured deployer moves the source version onto the target ring (kubectl image set, GitHub workflow dispatch, or k8s Job).
- Update stored state — state is written as soon as the deploy lands, so it never lags the cluster even if later health or rollback fails.
- Target health with retries — configurable retry count and delay after the first check.
- Optional version verification — if configured, the health response must report the exact version just deployed.
- Auto-rollback on failure — if the target stays unhealthy after all retries, roll back to the previous version.
- History — every seed, promote and rollback is recorded, success or failure.
- Auto-promote chain (optional) — if the target ring has auto-promote enabled, continue onward inside the same operation and lock until a ring with the flag off or any failure.
Seed sets an initial version for a ring. Promote moves the current version of from_ring to the next ring. Rollback restores a ring’s previous version. Rollbacks are exempt from the production password so incident response is never blocked.
Version-verified health checks
A plain URL check can be fooled: the deploy “succeeds” but the old version is still answering 200 OK. If a ring sets health_version_field (a JSON field in the health response, e.g. version — dotted paths like build.version work too) or health_version_header (e.g. X-App-Version), every post-deploy check also requires the endpoint to report the exact version that was just deployed. Otherwise the check fails and the usual auto-rollback kicks in. The same mechanism verifies the source ring really runs the version about to be promoted. The application must expose the deployed version string (image tag / branch) on its health endpoint.
On a ref-pinned ring (ref: release) the expected version is not knowable up front — the pipeline decides what the ref ships — so the field is used the other way round: after a healthy deploy the ring records the version the endpoint reports (for example v1.0.36) instead of the ref name.
Separately, health_expect_status on a ring can treat a non-2xx status as healthy when that is the real signal (for example a registry edge that returns 401 on a healthy /v2/).
Concurrency and failure safety
- Advisory lock — the Postgres store uses a session advisory lock so serialisation holds across replicas. An accidental scale-up cannot run two concurrent promotions on the same app. The in-memory store’s lock is process-local (fine for single-process local development).
- Detached context — seed/promote/rollback run under a context detached from the HTTP request and bounded by
operation_timeout. A client disconnect or load-balancer idle timeout therefore cannot abort an in-flight deploy or, critically, its automatic rollback. - State that never lags — stored state updates as soon as a deploy lands.
replicas: 1 remains a sensible default for simplest reasoning, but correctness no longer silently depends on it when using Postgres.
Deployers: kubectl, GitHub Actions, Kubernetes Jobs
Interfaces are swappable:
| Concern | Interface | Production impls | Local/dev |
|---|---|---|---|
| Deploy | deployer.Deployer | KubectlDeployer, GitHubActionsDeployer, k8sjob | LogDeployer (no-op) |
| Execution | executor.Executor | GitHub Actions, Kubernetes Jobs | scripted fakes |
| Health check | health.Checker | HTTPChecker | AlwaysHealthy |
| Persistence | store.Store | Postgres | Memory |
KubectlDeployer
Shells out to kubectl (set image + rollout status), authenticating in-cluster via the pod’s ServiceAccount. Keeps the binary and dependency tree small while using battle-tested rollout semantics.
GitHubActionsDeployer (VM / existing CI)
Targets applications that run on VMs but already have CI/CD. It triggers the pipeline via the GitHub Actions workflow-dispatch API, waits for the run, and returns an error unless it succeeds — so the same health-check and auto-rollback logic applies. The real example is wslproxy (OpenResty on VMs).
A version is a git branch, tag or commit SHA. Each ring’s target_env becomes the workflow’s environment input. For GitHub-deployed apps, seed verifies the requested version actually resolves in the repository before dispatching — a typo’d ref is rejected with 400 instead of starting a doomed run.
apps:
- name: wslproxy
deployer: github
github:
owner: bwalia
repo: wslproxy
workflow: deploy-single-environment.yml
ref: main
deploy_mode: full
token_env: RP_GITHUB_TOKEN
poll_interval: 20s
run_lookup_timeout: 90s
rings:
int: { target_env: int, health_url: "https://int-our.wslproxy.com/healthz" }
test: { target_env: test, health_url: "https://test.wslproxy.com/healthz" }
acc: { target_env: acc, health_url: "https://prod-our-v1.wslproxy.com/healthz" }
prod: { target_env: prod, health_url: "https://lon1.pop0.uk/healthz" }
Use a single-environment workflow, not a cumulative delivery cascade: pointing the deployer at a pipeline that always starts from int and chains to prod would make a single-ring deploy run the whole chain.
k8sjob
Each seed/promote/rollback runs as a Kubernetes Job (typically in a ring-exec namespace). A configurable runner image receives the runner contract as environment variables: RP_APP, RP_RING, RP_VERSION, RP_TARGET_ENV, RP_EXECUTION_ID. Stdout streams live into the job’s step log; the exit code decides success. Both GitHub and k8sjob sit on the shared execution abstraction (Start/Status/Logs/Cancel/Cleanup).
The deployer is selected per application via an optional deployer: field; apps without it use the global default.
Driving wslproxy from the API
Once configured, the mechanism is transparent to callers — seed a ref into int, then promote hop by hop:
RP=https://ring-promoter.example.com; APP=wslproxy; TOKEN=$RING_PROMOTER_TOKEN
curl --fail -sS -X POST "$RP/api/apps/$APP/seed" \
-H "Authorization: Bearer $TOKEN" -d '{"ring":"int","version":"release-1.0.10"}'
curl --fail -sS -X POST "$RP/api/apps/$APP/promote" \
-H "Authorization: Bearer $TOKEN" -d '{"from_ring":"int"}' # -> test
curl --fail -sS -X POST "$RP/api/apps/$APP/promote" \
-H "Authorization: Bearer $TOKEN" -d '{"from_ring":"test"}' # -> next ring
Matching the dispatched run relies on it being the newest workflow_dispatch for that workflow after Ring Promoter fired it. Per-app serialisation means the control plane never races itself; a human manually dispatching the same workflow at the same instant is the edge case to avoid. The matched run’s URL is logged for verification.
Promotion gates
An app can require extra checks before a version enters a sensitive ring (default target rings: acc + prod). Each gate is independent, opt-in per app, and enforced before any deploy, so a failed gate leaves all state untouched.
Maintenance windows
Promotion into a guarded ring is allowed only while a window is open. Windows are a union of permanent recurring windows from config (for example Sat 02:00–04:00 Europe/London) and ad-hoc windows opened at runtime. Closed → 409.
QA / release Go-No-Go
A release engineer records a GO sign-off for the exact version. A missing or NO-GO sign-off blocks the promotion (409). A GO for v1 does not authorise v2.
Change-request code
The promotion must carry a valid cr_code. Provider test accepts only the demo code; provider jira validates against a JIRA issue (token from RP_JIRA_TOKEN). The demo code test is always accepted for demos, whatever the provider.
Grafana go/no-go
Dashboard queries become GO / CHECK / NO-GO. One red check blocks. max_age turns stale results into NO DATA so a suite that silently stopped running cannot show its last GO forever. An unreachable Grafana is advisory, never blocking — an observability outage must not become a release outage — but the UI shows it so it is never mistaken for a GO.
This is the only overridable gate. Override needs override_grafana: true with a non-empty override_reason that is logged. The override applies to that gate only — a closed window, missing sign-off or bad CR code still block.
Auto-promote into a change-request-gated ring fails closed (there is no interactive CR code). Auto-promote into a Grafana-gated ring stops at a no-go, since an override needs a human reason.
The GET …/rings response carries a per-ring gates object (which gates guard the ring, the CR provider, whether a window is open now, and the live Grafana verdict) so the UI can prompt appropriately. Seed/Promote dialogs let a release engineer open a window, record a sign-off, enter a CR code and state an override reason inline. Treat gates as preconditions, not as post-deploy hope: if a gate fails, nothing was deployed.
Production password and auto-promote semantics
When RP_PROD_PASSWORD is set, any request that deploys to the last ring must carry it in the JSON body as "password": promoting into production, seeding production directly, and enabling auto-promote into production (403 otherwise). Rollbacks are exempt.
Auto-promote can be runtime state (toggled from UI/API) or declared in config:
rings:
int: { namespace: int, ... } # unset: operator-controlled
test: { namespace: test, ..., auto_promote: true } # config-owned: on
acc: { namespace: acc, ..., auto_promote: false } # config-owned: off
Absent, true and false are three different things. Absent keeps historical behaviour — the API toggle works. Either explicit value makes the ring config-owned: applied at start-up and re-applied on every restart; PUT …/auto-promote returns 409 for that ring.
Config may not set auto_promote: true on the ring before production — refused at start-up — because that would bypass the password check that enabling the hands-free path into production requires at the API. Declare false freely; turn production auto-promote on through the API if you really want it.
Typical operational pattern: enable auto-promote on test so int → test carries on to acc, leave acc off so nothing reaches prod without a human.
Driving it from CI
All /api routes require Authorization: Bearer <token>. /healthz and the UI are unauthenticated.
| Method & path | Body | Description |
|---|---|---|
POST /api/apps/{app}/seed | {"ring","version","cr_code?"} | Set an initial version for a ring |
POST /api/apps/{app}/promote | {"from_ring","cr_code?"} | Promote to the next ring |
POST /api/apps/{app}/rollback | {"ring"} | Roll a ring back to its previous version |
GET /api/apps/{app}/jobs/{id} | – | Live job status (steps, logs, result) |
Sync vs async. Seed/promote/rollback run synchronously by default and return the final Result (200 success / 422 ran-but-failed) — ideal for CI (curl --fail). Add ?async=1 to run in the background: the call returns 202 with {"job_id":"..."}, and you poll the jobs endpoint for live progress. The web UI uses the async path.
Status codes for mutating operations:
| Code | Meaning |
|---|---|
| 200 | Succeeded (deployed and healthy) |
| 422 | Ran but failed (e.g. health failed and target rolled back) |
| 400 | Empty version / past last ring / version does not exist in source repo |
| 401 | Bad token |
| 404 | Unknown app/ring |
| 409 | Nothing to promote/roll back (or gate closed) |
Every mutating call returns a Result object:
{
"app": "web-frontend",
"action": "promote",
"ring": "test",
"from_ring": "int",
"version": "1.4.2",
"success": true,
"rolled_back": false,
"message": "promoted 1.4.2 from int to test and healthy",
"state": { "current_version": "1.4.2", "previous_version": "", "healthy": true }
}
Example CI seed + promote:
RP=https://ring-promoter.example.com
APP=web-frontend
TOKEN=${RING_PROMOTER_TOKEN}
VERSION=$GITHUB_SHA
curl --fail -sS -X POST "$RP/api/apps/$APP/seed" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "{\"ring\":\"int\",\"version\":\"$VERSION\"}"
curl --fail -sS -X POST "$RP/api/apps/$APP/promote" \
-H "Authorization: Bearer $TOKEN" \
-d '{"from_ring":"int"}'
CI builds and tests; Ring Promoter moves what CI produced through environments. Higher rings are typically promoted deliberately from the UI or a separate approved job.
Onboarding an app and adding a ring
No code change, no rebuild. Add an entry under apps: — locally in config.yaml, in production in the ring-promoter-config ConfigMap:
apps:
- name: billing-worker
location: { lat: 51.5074, lng: -0.1278, city: London, region: GB }
rings:
int:
namespace: int
deployment: billing-worker
container: worker
image: registry.example.com/billing-worker
health_url: http://billing-worker.int.svc.cluster.local/health
test:
namespace: test
deployment: billing-worker
container: worker
image: registry.example.com/billing-worker
health_url: http://billing-worker.test.svc.cluster.local/health
# ...acc, prod
kubectl apply -f deploy/k8s/configmap.yaml
kubectl rollout restart deploy/ring-promoter -n ring-system
An app does not need to define every ring — only the ones it lives in.
To add a ring, change the ordered pipeline in one place (internal/ring/ring.go):
var ordered = []Ring{
{Name: "int", Label: "Integration"},
{Name: "test", Label: "Test"},
{Name: "acc", Label: "Acceptance"},
{Name: "canary", Label: "Canary"}, // <-- new ring
{Name: "prod", Label: "Production"},
}
Promotion order, API responses and the UI update from that list. Operationally: create the ring’s namespace and a matching RoleBinding, and add the ring to each app’s config where it applies.
The web UI and live jobs
The UI is embedded in the same binary as the API (a static export of a Next.js front end). Operators paste a bearer token, pick an application, and use Seed / Promote / Rollback. Ring cards show current and previous versions, live health, auto-promote switches (disabled when config-owned), and gate status where configured.
Mutating operations surface as live jobs: the console polls roughly every second for per-step status, logs and durations — acquire lock, source health, deploy, target health retries, history. Failed jobs can request AI-assisted diagnosis grounded in those persisted step logs (configured via optional Ollama settings such as RP_OLLAMA_URL and related secrets). Describe that feature briefly in runbooks; do not treat it as a substitute for reading the step evidence.
Application groups (server-side, shared by all users) let operators organise large app catalogues without changing promotion rules.
Configuration knobs that matter in production
Config comes from YAML; environment variables win when set. Secrets should always come from the environment or a Kubernetes Secret, never from a committed ConfigMap.
| Env var | Role |
|---|---|
RP_API_TOKEN | Bearer token for /api (required) |
RP_PROD_PASSWORD | Extra password for last-ring deploys and enabling prod auto-promote |
RP_DEPLOYER | Global default: kubectl, log or github |
RP_HEALTH | http or always |
RP_DB_DRIVER / RP_DB_DSN | postgres (with DSN) or memory |
RP_RETRY_COUNT / RP_RETRY_DELAY | Health retries after the first check (default 3 / 5s) |
RP_OP_TIMEOUT | Max time for one seed/promote/rollback (default 10m) |
RP_GITHUB_TOKEN | Token for apps using the github deployer |
RP_JIRA_TOKEN | Token for the jira change-request provider |
RP_GRAFANA_TOKEN | Token for Grafana go/no-go queries |
The application registry (apps:) lives in the file only. Locally you can run with zero dependencies; production should use Postgres and real health checks.
What “version” means in practice
For Kubernetes apps, a version is typically an image tag that kubectl set image applies. For GitHub-deployed apps, a version is a branch, tag or commit SHA passed into the workflow (for example as DEPLOY_BRANCH). The UI’s Seed dialog can call GET /api/apps/{app}/versions to offer only real branches and tags when the github deployer is in use. That distinction matters in incident reviews: “we promoted 1.4.2” must mean the same thing in history that it meant in the deploy backend.
Seed validates GitHub refs before dispatch. Promote always moves the version currently recorded on the source ring — you do not re-type a different tag mid-pipeline unless you seed again. That is intentional: the protocol promotes what already earned the source ring, not a new candidate invented at promote time.
Operational failure modes to plan for
- Health that stays green on the wrong build — enable version-verified health; require apps to expose version on the health endpoint.
- Observability outage during a gate — Grafana unreachable is advisory; do not confuse NO DATA with GO; decide policy for overrides in advance.
- Auto-promote left on too high — keep
acc(or the ring before prod) off; refuse config that setsauto_promote: trueon the ring before prod. - Client timeouts during long deploys — prefer async jobs from UIs and long-running CI, or rely on the detached context so sync callers can disconnect safely while the operation continues.
- VM workflow cascade — point the github deployer at a single-environment workflow, never a cumulative delivery pipeline that always starts at int.
Deploying on k3s
- Build and push the image.
- Provide Postgres reachable from the cluster; put its DSN and a strong
RP_API_TOKENin the Secret. - Edit manifests for the real image and app registry.
- Apply in order: namespace, RBAC, secret, configmap, deployment, service, then ingress if needed.
- Reach it via Ingress or
kubectl -n ring-system port-forward svc/ring-promoter 8080:80.
Locally, zero dependencies:
go run ./cmd/ringpromoter --config config.yaml
# -> http://localhost:8080 (UI + API), token: local-dev-token
Defaults use the no-op deployer, an always-healthy checker and the in-memory store. Drive it from the CLI with the same REST contract you will use in CI:
TOKEN=local-dev-token
BASE=http://localhost:8080
curl -s -H "Authorization: Bearer $TOKEN" $BASE/api/apps | jq
curl -s -H "Authorization: Bearer $TOKEN" -X POST \
-d '{"ring":"int","version":"1.4.2"}' $BASE/api/apps/web-frontend/seed | jq
curl -s -H "Authorization: Bearer $TOKEN" -X POST \
-d '{"from_ring":"int"}' $BASE/api/apps/web-frontend/promote | jq
curl -s -H "Authorization: Bearer $TOKEN" $BASE/api/apps/web-frontend/rings | jq
curl -s -H "Authorization: Bearer $TOKEN" $BASE/api/apps/web-frontend/history | jq
To exercise real backends locally, set deployer: kubectl, health: http and database.driver: postgres (with a DSN) in the config or via environment variables. The KubectlDeployer uses the ring-promoter ServiceAccount; RBAC grants patch/update on Deployments (and read on ReplicaSets/Pods for rollout status) in each ring namespace. The Deployment runs as a single non-root replica with a read-only root filesystem. State and history live in Postgres, so restarts are safe.
Putting the pieces together
A realistic path looks like this. CI builds an image (or a VM artefact), seeds int, and optionally promotes into test when auto-promote is configured. Acceptance stays gated: maintenance window open, Go-No-Go recorded for that exact version, CR code present if required, Grafana not NO-GO (or overridden with a written reason). A human supplies the production password for the last hop. If health fails — especially version-verified health — the target rolls back and history shows rolled_back=true. The next engineer can read the job steps instead of reconstructing the night from chat.
That is the whole point of a promotion protocol: the same five rules, the same deployers, the same history shape, whether the app lives in Kubernetes or on VMs.
When you adopt it, start narrow. Onboard one Kubernetes app with HTTP health and version verification. Add auto-promote only on early rings. Turn on the production password before the first real prod hop. Add gates where your organisation already has process — maintenance windows and Go-No-Go usually come first; Grafana when you have a release dashboard worth trusting. Expand to a GitHub-deployed VM app only after the single-environment workflow contract is clear. The control plane will not invent release discipline for you, but it will stop the most common shortcuts from becoming silent defaults.
Testing the promotion rules themselves is covered in the project’s promoter unit tests (source health gating, the retry loop, automatic rollback, never-skip-a-ring, and concurrency safety). Prefer proving a change with those tests and a local go run session before promoting the control plane’s own image through your rings — yes, Ring Promoter can promote Ring Promoter, subject to the same protocol. Run go test ./... (and go test -race ./... when changing lock or concurrency paths) before you trust a new build in a shared cluster. Treat those tests as part of the release checklist for the control plane itself, not as optional developer hygiene.
Closing
Ring Promoter makes promotion boring on purpose: one hop at a time, health-gated, optionally version-verified, rolled back automatically, and written to history. Kubernetes and VM apps share the same protocol. Humans keep the gate to production via password and auto-promote policy; optional gates add maintenance windows, Go-No-Go, change requests and Grafana signals without inventing another ad-hoc script.
Watch the full demo, then continue at ringpromoter.com. The shorter benefits overview is the companion blog post.
