bunker-admin
3f6102cf6d
- DEV_WORKFLOW.md: reflect that build-and-push.sh now produces 5 images (api, admin, media-api, nginx, ccp-agent), not 4. - CampaignsListPage: move card title below cover photo instead of overlaying it, so titles remain legible when no cover image is set. Bunker Admin
334 lines
14 KiB
Markdown
334 lines
14 KiB
Markdown
# Development & Release Workflow
|
|
|
|
How code changes move from development to production deployments across all installation methods.
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
There are **three ways** Changemaker Lite gets deployed:
|
|
|
|
| Method | Who uses it | Images from | Compose file |
|
|
|--------|------------|-------------|--------------|
|
|
| **Source install** | Developers, contributors | Built locally from source | `docker-compose.yml` |
|
|
| **Release install** | Production servers, evaluators | Gitea registry (pre-built) | `docker-compose.prod.yml` (ships as `docker-compose.yml` in tarball) |
|
|
| **CCP provisioned** | Fleet operators (Control Panel) | Gitea registry (pre-built) | Rendered from `templates/docker-compose.yml.hbs` |
|
|
|
|
All three methods share the same Gitea container registry at `gitea.bnkops.com/admin`.
|
|
|
|
---
|
|
|
|
## The Pipeline
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────────┐
|
|
│ DEVELOPMENT (your machine) │
|
|
│ │
|
|
│ Edit code → docker compose up -d → test locally │
|
|
│ Uses: docker-compose.yml (build: blocks + ./api:/app mounts) │
|
|
└──────────────────┬───────────────────────────────────────────────┘
|
|
│ git push
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────┐
|
|
│ BUILD & PUBLISH │
|
|
│ │
|
|
│ Step 1: ./scripts/build-and-push.sh │
|
|
│ Builds 5 production images, pushes to Gitea registry │
|
|
│ (api, admin, media-api, nginx, ccp-agent) │
|
|
│ tagged :SHA + :latest │
|
|
│ │
|
|
│ Step 2: ./scripts/mirror-images.sh (run once/rarely) │
|
|
│ Mirrors 36 third-party images to Gitea registry │
|
|
│ (postgres, redis, nocodb, jitsi, grafana, etc.) │
|
|
│ │
|
|
│ Step 3: ./scripts/build-release.sh --tag vX.Y.Z --upload │
|
|
│ Packages runtime files into ~9MB tarball, uploads to │
|
|
│ Gitea Releases │
|
|
└──────────────────┬─────────────────100.90.78.47──────────────────────────────┘
|
|
│
|
|
┌───────────┴───────────┐
|
|
▼ ▼
|
|
┌─────────────────┐ ┌──────────────────┐
|
|
│ RELEASE INSTALL │ │ CCP PROVISIONED │
|
|
│ │ │ │
|
|
│ curl installer │ │ Control Panel │
|
|
│ or manual tarball│ │ creates instance │
|
|
│ → config.sh │ │ via web UI │
|
|
│ → docker compose │ │ → renders config │
|
|
│ up -d │ │ → docker compose │
|
|
│ │ │ up -d │
|
|
└─────────────────┘ └──────────────────┘
|
|
│ │
|
|
└───────────┬───────────┘
|
|
▼
|
|
All images pulled from
|
|
gitea.bnkops.com/admin
|
|
(zero external dependencies)
|
|
```
|
|
|
|
---
|
|
|
|
## Step-by-Step
|
|
|
|
### 1. Local Development
|
|
|
|
Standard Docker Compose workflow with hot-reload:
|
|
|
|
```bash
|
|
# Start core services
|
|
docker compose up -d v2-postgres redis api admin
|
|
|
|
# API logs (watch for errors)
|
|
docker compose logs -f api
|
|
|
|
# Run with media API
|
|
docker compose up -d media-api
|
|
|
|
# Run with monitoring stack
|
|
docker compose --profile monitoring up -d
|
|
```
|
|
|
|
**Key:** `docker-compose.yml` uses `build:` blocks to compile TypeScript from source and mounts `./api:/app` for live code changes. This is the only compose file that builds from source.
|
|
|
|
### 2. Build & Push Production Images
|
|
|
|
After code changes are tested locally:
|
|
|
|
```bash
|
|
# Build production images and push to Gitea registry
|
|
./scripts/build-and-push.sh
|
|
```
|
|
|
|
This builds **5 services** with multi-stage Dockerfiles (production target, no dev dependencies), tags each image with `:SHA` and `:latest`, and pushes to `gitea.bnkops.com/admin/changemaker-{service}`:
|
|
|
|
| Service | Dockerfile | What it produces |
|
|
|---------|-----------|-----------------|
|
|
| `api` | `api/Dockerfile` | Express + Prisma (compiled JS, no TS) |
|
|
| `admin` | `admin/Dockerfile` | Nginx serving React build output |
|
|
| `media-api` | `api/Dockerfile.media` | Fastify + FFmpeg (compiled JS) |
|
|
| `nginx` | `nginx/Dockerfile` | Nginx with `envsubst` domain templating |
|
|
| `ccp-agent` | `../changemaker-control-panel/agent/Dockerfile` | Remote management agent (sibling repo) |
|
|
|
|
```bash
|
|
# Build specific services only
|
|
./scripts/build-and-push.sh --services api,admin
|
|
|
|
# Build without pushing (verify first)
|
|
./scripts/build-and-push.sh --no-push
|
|
|
|
# Include code-server (~9GB, only when Dockerfile changes)
|
|
./scripts/build-and-push.sh --include-code-server
|
|
```
|
|
|
|
### 3. Mirror Third-Party Images (Run Once / On Version Bumps)
|
|
|
|
Copies all third-party Docker images used by the platform to the Gitea registry, so deployments never depend on Docker Hub, GHCR, LSCR, or GCR:
|
|
|
|
```bash
|
|
# Mirror all 36 images (core + platform + comms + monitoring)
|
|
./scripts/mirror-images.sh
|
|
|
|
# Mirror only essential infrastructure (postgres, redis, alpine)
|
|
./scripts/mirror-images.sh --core-only
|
|
|
|
# Preview without executing
|
|
./scripts/mirror-images.sh --dry-run
|
|
```
|
|
|
|
**When to re-run:** Only when upgrading a third-party image version. The script has explicit version pins — update the version in `mirror-images.sh`, then re-run.
|
|
|
|
Images are organized into 4 groups:
|
|
|
|
| Group | Count | Examples |
|
|
|-------|-------|---------|
|
|
| Core Infrastructure | 5 | postgres:16-alpine, redis:7-alpine, alpine:3 |
|
|
| Platform Services | 16 | nocodb, listmonk, gitea, n8n, vaultwarden, nginx, code-server |
|
|
| Communication | 8 | rocket.chat, mongo, nats, gancio, jitsi (4 containers) |
|
|
| Monitoring | 7 | prometheus, grafana, alertmanager, cadvisor, exporters, gotify |
|
|
|
|
### 4. Build Release Tarball
|
|
|
|
Packages only runtime files (~9 MB) — no source code, no node_modules:
|
|
|
|
```bash
|
|
# Build tarball
|
|
./scripts/build-release.sh --tag v2.2.0
|
|
|
|
# Build and upload to Gitea Releases
|
|
./scripts/build-release.sh --tag v2.2.0 --upload
|
|
|
|
# Preview contents without creating tarball
|
|
./scripts/build-release.sh --dry-run
|
|
|
|
# --upload refuses to overwrite an existing tag. To deliberately replace
|
|
# a release (destructive — users on that tag see no upgrade signal):
|
|
./scripts/build-release.sh --tag v2.2.0 --upload --replace
|
|
```
|
|
|
|
**Version hygiene:** bump the tag when changing release contents. Overwriting
|
|
an existing release silently breaks upgrade checks for users already on that
|
|
version — they see "no update available" even though the tarball they'd
|
|
download differs.
|
|
|
|
The tarball contains:
|
|
- `docker-compose.yml` (copy of `docker-compose.prod.yml` — image-only, no build blocks)
|
|
- `.env.example`, `config.sh` (configuration wizard)
|
|
- `scripts/` (init scripts, backup, upgrade, systemd units)
|
|
- `configs/` (prometheus, grafana, alertmanager, homepage, pangolin)
|
|
- `nginx/conf.d/` (templates for reference)
|
|
- `mkdocs/` (starter documentation)
|
|
- Empty data directories
|
|
|
|
### 5. Deploying
|
|
|
|
#### New Release Install (End Users)
|
|
|
|
```bash
|
|
# One-liner
|
|
curl -fsSL https://gitea.bnkops.com/admin/changemaker.lite/raw/branch/main/scripts/install.sh | bash
|
|
|
|
# Or manual
|
|
curl -LO https://gitea.bnkops.com/admin/changemaker.lite/releases/latest/download/changemaker-lite-latest.tar.gz
|
|
tar xzf changemaker-lite-latest.tar.gz
|
|
cd changemaker-lite
|
|
bash config.sh
|
|
docker compose up -d
|
|
```
|
|
|
|
All images (custom + third-party) pull from `gitea.bnkops.com/admin`. No external registry access needed.
|
|
|
|
#### New CCP Instance (Fleet Operators)
|
|
|
|
The Control Panel provisions instances via its web UI:
|
|
|
|
1. Operator fills in the Create Instance wizard (domain, features, email, tunnel)
|
|
2. CCP copies source files, renders templates (Handlebars), generates secrets
|
|
3. With `USE_REGISTRY_IMAGES=true` (default): pulls pre-built images from Gitea (~2 min)
|
|
4. With `USE_REGISTRY_IMAGES=false`: builds from source (~10+ min)
|
|
5. Starts infrastructure → runs migrations → starts all services
|
|
|
|
CCP registry settings (in `changemaker-control-panel/.env`):
|
|
```bash
|
|
GITEA_REGISTRY=gitea.bnkops.com/admin # Registry URL for all images
|
|
USE_REGISTRY_IMAGES=true # true = pull pre-built, false = build from source
|
|
IMAGE_TAG=latest # Tag for custom images (api, admin, media-api)
|
|
```
|
|
|
|
### 6. Upgrading Existing Installations
|
|
|
|
#### Source Installs
|
|
|
|
```bash
|
|
./scripts/upgrade.sh # Standard: git pull + rebuild from source
|
|
./scripts/upgrade.sh --use-registry # Fast: pull pre-built images instead of rebuilding
|
|
./scripts/upgrade.sh --dry-run # Preview changes
|
|
```
|
|
|
|
#### Release Installs
|
|
|
|
```bash
|
|
./scripts/upgrade.sh # Auto-detects release mode, downloads latest tarball
|
|
```
|
|
|
|
Release installs are detected by the presence of a `VERSION` file and absence of `.git/`. The upgrade script automatically downloads the latest tarball from Gitea instead of running `git pull`.
|
|
|
|
---
|
|
|
|
## Image Naming Conventions
|
|
|
|
All images live under `gitea.bnkops.com/admin/`:
|
|
|
|
| Type | Naming Pattern | Example |
|
|
|------|---------------|---------|
|
|
| Custom services | `changemaker-{service}:{sha\|latest}` | `changemaker-api:latest` |
|
|
| Simple names | Same as upstream | `postgres:16-alpine`, `redis:7-alpine` |
|
|
| Namespaced → short | Org removed | `nocodb/nocodb` → `nocodb:0.301.3` |
|
|
| Conflict resolution | Explicit short name | `gotify/server` → `gotify`, `vaultwarden/server` → `vaultwarden` |
|
|
| Jitsi suite | `jitsi-{component}` | `jitsi-web:stable-9823`, `jitsi-prosody:stable-9823` |
|
|
| LinuxServer nginx | `ls-nginx` (avoids nginx conflict) | `ls-nginx:1.28.2` |
|
|
|
|
---
|
|
|
|
## Two Compose Files
|
|
|
|
| File | Purpose | Build? | Source mounts? | Image source |
|
|
|------|---------|--------|---------------|-------------|
|
|
| `docker-compose.yml` | Development | Yes (`build:` blocks) | Yes (`./api:/app`) | Built locally |
|
|
| `docker-compose.prod.yml` | Production | No | No | `${GITEA_REGISTRY:-gitea.bnkops.com/admin}/...` |
|
|
|
|
Release tarballs ship `docker-compose.prod.yml` renamed as `docker-compose.yml`.
|
|
|
|
The CCP template (`templates/docker-compose.yml.hbs`) generates a compose file that works like `docker-compose.prod.yml` when `USE_REGISTRY_IMAGES=true`, or like `docker-compose.yml` when `false`.
|
|
|
|
---
|
|
|
|
## Quick Reference
|
|
|
|
```bash
|
|
# ── Development ──
|
|
docker compose up -d v2-postgres redis api admin # Start dev stack
|
|
docker compose logs -f api # Watch API logs
|
|
docker compose exec api npx prisma migrate dev # Create migration
|
|
|
|
# ── Build & Publish ──
|
|
./scripts/build-and-push.sh # Build + push 5 images
|
|
./scripts/mirror-images.sh # Mirror 36 third-party images
|
|
git tag --sort=-v:refname | head -3 # Check latest version tags
|
|
./scripts/build-release.sh --tag vX.Y.Z --upload # Package + upload release
|
|
|
|
# ── Deploy ──
|
|
curl -fsSL .../install.sh | bash # New install (release)
|
|
./scripts/upgrade.sh # Upgrade existing install
|
|
./scripts/upgrade.sh --use-registry # Fast upgrade (registry images)
|
|
|
|
# ── Verify ──
|
|
curl -s http://localhost:4000/api/health # API health check
|
|
docker compose ps # Container status
|
|
```
|
|
|
|
---
|
|
|
|
## Gitea API Tokens
|
|
|
|
There are **two separate Gitea tokens** with different purposes. Using the wrong one is a common mistake:
|
|
|
|
| Variable | Target | Used by | Create at |
|
|
|----------|--------|---------|-----------|
|
|
| `GITEA_REGISTRY_API_TOKEN` | Remote registry (`gitea.bnkops.com`) | `build-release.sh --upload`, release API calls | `https://gitea.bnkops.com/user/settings/applications` |
|
|
| `GITEA_API_TOKEN` | Local Gitea instance | Docs comments, user provisioning, SSO | `http://localhost:3030/user/settings/applications` |
|
|
|
|
**Key:** Release uploads and the Gitea Releases API require `GITEA_REGISTRY_API_TOKEN`. If you get `"user does not exist"` from the API, you're using the wrong token.
|
|
|
|
---
|
|
|
|
## Checklist: Cutting a New Release
|
|
|
|
1. [ ] All code changes committed and pushed to `main` branch
|
|
2. [ ] `docker compose up -d` works locally (smoke test)
|
|
3. [ ] **Determine version tag:**
|
|
```bash
|
|
# Check the latest existing tag to pick the next version
|
|
git tag --sort=-v:refname | head -5
|
|
# Check commits since the last tag
|
|
git log $(git tag --sort=-v:refname | head -1)..HEAD --oneline
|
|
```
|
|
4. [ ] `./scripts/build-and-push.sh` — builds and pushes 5 production images
|
|
5. [ ] `./scripts/mirror-images.sh` — only if third-party versions changed
|
|
6. [ ] `./scripts/build-release.sh --tag vX.Y.Z --upload` — packages and uploads tarball
|
|
7. [ ] **Add release notes** (via Gitea web UI or API):
|
|
```bash
|
|
# Update release body via API (use GITEA_REGISTRY_API_TOKEN, not GITEA_API_TOKEN)
|
|
GITEA_TOKEN=$(grep -oP 'GITEA_REGISTRY_API_TOKEN=\K.*' .env)
|
|
# Find release ID
|
|
curl -s "https://gitea.bnkops.com/api/v1/repos/admin/changemaker.lite/releases?limit=1" \
|
|
-H "Authorization: token $GITEA_TOKEN" | python3 -c "import sys,json; r=json.load(sys.stdin)[0]; print(f'ID: {r[\"id\"]}, Tag: {r[\"tag_name\"]}')"
|
|
# Update with release notes (write JSON body to /tmp/release-notes.json first)
|
|
curl -s -X PATCH "https://gitea.bnkops.com/api/v1/repos/admin/changemaker.lite/releases/RELEASE_ID" \
|
|
-H "Authorization: token $GITEA_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d @/tmp/release-notes.json
|
|
```
|
|
8. [ ] Test clean install: `tar xzf ... && cd changemaker-lite && bash config.sh && docker compose up -d`
|
|
9. [ ] Test upgrade: `./scripts/upgrade.sh` on an existing installation
|
|
10. [ ] Verify: `curl http://localhost:4000/api/health` returns `{"status":"ok"}`
|