changemaker.lite/DEV_WORKFLOW.md

Development & Release Workflow

How code changes move from development to production deployments across all installation methods.

---

Overview

There are three ways Changemaker Lite gets deployed:

Method	Who uses it	Images from	Compose file
Source install	Developers, contributors	Built locally from source	docker-compose.yml
Release install	Production servers, evaluators	Gitea registry (pre-built)	docker-compose.prod.yml (ships as docker-compose.yml in tarball)
CCP provisioned	Fleet operators (Control Panel)	Gitea registry (pre-built)	Rendered from templates/docker-compose.yml.hbs

All three methods share the same Gitea container registry at gitea.bnkops.com/admin.

---

The Pipeline

 ┌──────────────────────────────────────────────────────────────────┐
 │  DEVELOPMENT (your machine)                                     │
 │                                                                  │
 │  Edit code → docker compose up -d → test locally                │
 │  Uses: docker-compose.yml (build: blocks + ./api:/app mounts)   │
 └──────────────────┬───────────────────────────────────────────────┘
                    │ git push
                    ▼
 ┌──────────────────────────────────────────────────────────────────┐
 │  BUILD & PUBLISH                                                │
 │                                                                  │
 │  Step 1: ./scripts/build-and-push.sh                            │
 │          Builds 4 production images, pushes to Gitea registry   │
 │          (api, admin, media-api, nginx) tagged :SHA + :latest   │
 │                                                                  │
 │  Step 2: ./scripts/mirror-images.sh          (run once/rarely)  │
 │          Mirrors 36 third-party images to Gitea registry        │
 │          (postgres, redis, nocodb, jitsi, grafana, etc.)        │
 │                                                                  │
 │  Step 3: ./scripts/build-release.sh --tag vX.Y.Z --upload      │
 │          Packages runtime files into ~9MB tarball, uploads to   │
 │          Gitea Releases                                         │
 └──────────────────┬─────────────────100.90.78.47──────────────────────────────┘
                    │
        ┌───────────┴───────────┐
        ▼                       ▼
 ┌─────────────────┐   ┌──────────────────┐
 │ RELEASE INSTALL  │   │ CCP PROVISIONED  │
 │                  │   │                  │
 │ curl installer   │   │ Control Panel    │
 │ or manual tarball│   │ creates instance │
 │ → config.sh      │   │ via web UI       │
 │ → docker compose │   │ → renders config │
 │   up -d          │   │ → docker compose │
 │                  │   │   up -d          │
 └─────────────────┘   └──────────────────┘
        │                       │
        └───────────┬───────────┘
                    ▼
         All images pulled from
         gitea.bnkops.com/admin
         (zero external dependencies)

---

Step-by-Step

1. Local Development

Standard Docker Compose workflow with hot-reload:

# Start core services
docker compose up -d v2-postgres redis api admin

# API logs (watch for errors)
docker compose logs -f api

# Run with media API
docker compose up -d media-api

# Run with monitoring stack
docker compose --profile monitoring up -d

Key: docker-compose.yml uses build: blocks to compile TypeScript from source and mounts ./api:/app for live code changes. This is the only compose file that builds from source.

2. Build & Push Production Images

After code changes are tested locally:

# Build production images and push to Gitea registry
./scripts/build-and-push.sh

This builds 4 services with multi-stage Dockerfiles (production target, no dev dependencies), tags each image with :SHA and :latest, and pushes to gitea.bnkops.com/admin/changemaker-{service}:

Service	Dockerfile	What it produces
api	api/Dockerfile	Express + Prisma (compiled JS, no TS)
admin	admin/Dockerfile	Nginx serving React build output
media-api	api/Dockerfile.media	Fastify + FFmpeg (compiled JS)
nginx	nginx/Dockerfile	Nginx with envsubst domain templating

# Build specific services only
./scripts/build-and-push.sh --services api,admin

# Build without pushing (verify first)
./scripts/build-and-push.sh --no-push

# Include code-server (~9GB, only when Dockerfile changes)
./scripts/build-and-push.sh --include-code-server

3. Mirror Third-Party Images (Run Once / On Version Bumps)

Copies all third-party Docker images used by the platform to the Gitea registry, so deployments never depend on Docker Hub, GHCR, LSCR, or GCR:

# Mirror all 36 images (core + platform + comms + monitoring)
./scripts/mirror-images.sh

# Mirror only essential infrastructure (postgres, redis, alpine)
./scripts/mirror-images.sh --core-only

# Preview without executing
./scripts/mirror-images.sh --dry-run

When to re-run: Only when upgrading a third-party image version. The script has explicit version pins — update the version in mirror-images.sh, then re-run.

Images are organized into 4 groups:

Group	Count	Examples
Core Infrastructure	5	postgres:16-alpine, redis:7-alpine, alpine:3
Platform Services	16	nocodb, listmonk, gitea, n8n, vaultwarden, nginx, code-server
Communication	8	rocket.chat, mongo, nats, gancio, jitsi (4 containers)
Monitoring	7	prometheus, grafana, alertmanager, cadvisor, exporters, gotify

4. Build Release Tarball

Packages only runtime files (~9 MB) — no source code, no node_modules:

# Build tarball
./scripts/build-release.sh --tag v2.2.0

# Build and upload to Gitea Releases
./scripts/build-release.sh --tag v2.2.0 --upload

# Preview contents without creating tarball
./scripts/build-release.sh --dry-run

# --upload refuses to overwrite an existing tag. To deliberately replace
# a release (destructive — users on that tag see no upgrade signal):
./scripts/build-release.sh --tag v2.2.0 --upload --replace

Version hygiene: bump the tag when changing release contents. Overwriting an existing release silently breaks upgrade checks for users already on that version — they see "no update available" even though the tarball they'd download differs.

The tarball contains:

• docker-compose.yml (copy of docker-compose.prod.yml — image-only, no build blocks)
• .env.example, config.sh (configuration wizard)
• scripts/ (init scripts, backup, upgrade, systemd units)
• configs/ (prometheus, grafana, alertmanager, homepage, pangolin)
• nginx/conf.d/ (templates for reference)
• mkdocs/ (starter documentation)
• Empty data directories

5. Deploying

New Release Install (End Users)

# One-liner
curl -fsSL https://gitea.bnkops.com/admin/changemaker.lite/raw/branch/main/scripts/install.sh | bash

# Or manual
curl -LO https://gitea.bnkops.com/admin/changemaker.lite/releases/latest/download/changemaker-lite-latest.tar.gz
tar xzf changemaker-lite-latest.tar.gz
cd changemaker-lite
bash config.sh
docker compose up -d

All images (custom + third-party) pull from gitea.bnkops.com/admin. No external registry access needed.

New CCP Instance (Fleet Operators)

The Control Panel provisions instances via its web UI:

1. Operator fills in the Create Instance wizard (domain, features, email, tunnel)
2. CCP copies source files, renders templates (Handlebars), generates secrets
3. With USE_REGISTRY_IMAGES=true (default): pulls pre-built images from Gitea (~2 min)
4. With USE_REGISTRY_IMAGES=false: builds from source (~10+ min)
5. Starts infrastructure → runs migrations → starts all services

CCP registry settings (in changemaker-control-panel/.env):

GITEA_REGISTRY=gitea.bnkops.com/admin    # Registry URL for all images
USE_REGISTRY_IMAGES=true                   # true = pull pre-built, false = build from source
IMAGE_TAG=latest                           # Tag for custom images (api, admin, media-api)

6. Upgrading Existing Installations

Source Installs

./scripts/upgrade.sh                    # Standard: git pull + rebuild from source
./scripts/upgrade.sh --use-registry     # Fast: pull pre-built images instead of rebuilding
./scripts/upgrade.sh --dry-run          # Preview changes

Release Installs

./scripts/upgrade.sh                    # Auto-detects release mode, downloads latest tarball

Release installs are detected by the presence of a VERSION file and absence of .git/. The upgrade script automatically downloads the latest tarball from Gitea instead of running git pull.

---

Image Naming Conventions

All images live under gitea.bnkops.com/admin/:

Type	Naming Pattern	Example
Custom services	changemaker-{service}:{sha|latest}	changemaker-api:latest
Simple names	Same as upstream	postgres:16-alpine, redis:7-alpine
Namespaced → short	Org removed	nocodb/nocodb → nocodb:0.301.3
Conflict resolution	Explicit short name	gotify/server → gotify, vaultwarden/server → vaultwarden
Jitsi suite	jitsi-{component}	jitsi-web:stable-9823, jitsi-prosody:stable-9823
LinuxServer nginx	ls-nginx (avoids nginx conflict)	ls-nginx:1.28.2

---

Two Compose Files

File	Purpose	Build?	Source mounts?	Image source
docker-compose.yml	Development	Yes (build: blocks)	Yes (./api:/app)	Built locally
docker-compose.prod.yml	Production	No	No	${GITEA_REGISTRY:-gitea.bnkops.com/admin}/...

Release tarballs ship docker-compose.prod.yml renamed as docker-compose.yml.

The CCP template (templates/docker-compose.yml.hbs) generates a compose file that works like docker-compose.prod.yml when USE_REGISTRY_IMAGES=true, or like docker-compose.yml when false.

---

Quick Reference

# ── Development ──
docker compose up -d v2-postgres redis api admin     # Start dev stack
docker compose logs -f api                            # Watch API logs
docker compose exec api npx prisma migrate dev        # Create migration

# ── Build & Publish ──
./scripts/build-and-push.sh                           # Build + push 4 images
./scripts/mirror-images.sh                            # Mirror 36 third-party images
git tag --sort=-v:refname | head -3                   # Check latest version tags
./scripts/build-release.sh --tag vX.Y.Z --upload      # Package + upload release

# ── Deploy ──
curl -fsSL .../install.sh | bash                      # New install (release)
./scripts/upgrade.sh                                  # Upgrade existing install
./scripts/upgrade.sh --use-registry                   # Fast upgrade (registry images)

# ── Verify ──
curl -s http://localhost:4000/api/health              # API health check
docker compose ps                                     # Container status

---

Gitea API Tokens

There are two separate Gitea tokens with different purposes. Using the wrong one is a common mistake:

Variable	Target	Used by	Create at
GITEA_REGISTRY_API_TOKEN	Remote registry (gitea.bnkops.com)	build-release.sh --upload, release API calls	https://gitea.bnkops.com/user/settings/applications
GITEA_API_TOKEN	Local Gitea instance	Docs comments, user provisioning, SSO	http://localhost:3030/user/settings/applications

Key: Release uploads and the Gitea Releases API require GITEA_REGISTRY_API_TOKEN. If you get "user does not exist" from the API, you're using the wrong token.

---

Checklist: Cutting a New Release

1. All code changes committed and pushed to main branch
2. docker compose up -d works locally (smoke test)
3. Determine version tag:

   # Check the latest existing tag to pick the next version
   git tag --sort=-v:refname | head -5
   # Check commits since the last tag
   git log $(git tag --sort=-v:refname | head -1)..HEAD --oneline
   

4. ./scripts/build-and-push.sh — builds and pushes 4 production images
5. ./scripts/mirror-images.sh — only if third-party versions changed
6. ./scripts/build-release.sh --tag vX.Y.Z --upload — packages and uploads tarball
7. Add release notes (via Gitea web UI or API):

   # Update release body via API (use GITEA_REGISTRY_API_TOKEN, not GITEA_API_TOKEN)
   GITEA_TOKEN=$(grep -oP 'GITEA_REGISTRY_API_TOKEN=\K.*' .env)
   # Find release ID
   curl -s "https://gitea.bnkops.com/api/v1/repos/admin/changemaker.lite/releases?limit=1" \
     -H "Authorization: token $GITEA_TOKEN" | python3 -c "import sys,json; r=json.load(sys.stdin)[0]; print(f'ID: {r[\"id\"]}, Tag: {r[\"tag_name\"]}')"
   # Update with release notes (write JSON body to /tmp/release-notes.json first)
   curl -s -X PATCH "https://gitea.bnkops.com/api/v1/repos/admin/changemaker.lite/releases/RELEASE_ID" \
     -H "Authorization: token $GITEA_TOKEN" \
     -H "Content-Type: application/json" \
     -d @/tmp/release-notes.json
   

8. Test clean install: tar xzf ... && cd changemaker-lite && bash config.sh && docker compose up -d
9. Test upgrade: ./scripts/upgrade.sh on an existing installation
10. Verify: curl http://localhost:4000/api/health returns {"status":"ok"}