admin/changemaker.lite
1
0
Fork 0
Code Issues Packages Projects Releases 31 Wiki Activity
changemaker.lite/DEV_WORKFLOW.md
bunker-admin e55bc07eb6 Security hardening: red-team remediation + CCP/WIP updates 
## Security (red-team audit 2026-04-12)

Public data exposure (P0):
- Public map converted to server-side heatmap, 2-decimal (~1.1km) bucketing,
  no addresses/support-levels/sign-info returned
- Petition signers endpoint strips displayName/signerComment/geoCity/geoCountry
- Petition public-stats drops recentSigners entirely
- Response wall strips userComment + submittedByName
- Campaign createdByUserEmail + moderation fields gated to SUPER_ADMIN

Access control (P1):
- Campaign findById/update/delete/email-stats enforce owner === req.user.id
  (SUPER_ADMIN bypasses), return 404 to avoid enumeration
- GPS tracking session route restricted to session owner or SUPER_ADMIN
- Canvass volunteer stats restricted to self or SUPER_ADMIN
- People household endpoints restricted to INFLUENCE + MAP roles (was ADMIN*)
- CCP upgrade.service.ts + certificate.service.ts gate user-controlled
  shell inputs (branch, path, slug, SAN hostname) behind regex validators

Token security (P2):
- Query-param JWT auth replaced with HMAC-signed short-lived URLs
  (utils/signed-url.ts + /api/media/sign endpoint); legacy ?token= removed
  from media streaming, photos, chat-notifications, and social SSE
- GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT now REQUIRED (min 32 chars);
  JWT_ACCESS_SECRET fallback removed — BREAKING for existing deployments
- Refresh tokens bound to device fingerprint (UA + /24 IP) via `df` JWT
  claim; mismatch revokes all user sessions
- Refresh expiry reduced 7d → 24h
- Refresh/logout via request body removed — httpOnly cookie only
- Password-reset + verification-resend rate limits now keyed on (IP, email)
  composite to prevent both IP rotation and email enumeration

Defense-in-depth (P3):
- DOMPurify sanitization applied to GrapesJS landing page HTML/CSS
- /api/health?detailed=true disk-space leak removed
- Password-reset/verification token log lines no longer include userId

## Deployment

- docker-compose.yml + docker-compose.prod.yml: media-api now receives
  GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT; empty fallbacks removed
- CCP templates/env.hbs adds both new secrets; refresh expiry → 24h
- CCP secret-generator.ts generates giteaSsoSecret + servicePasswordSalt
- leaflet.heat added to admin/package.json for heatmap rendering

## Operator action required on existing installs

Run `./config.sh` once (idempotent — only fills empty values) or manually
add GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT to .env via
`openssl rand -hex 32`. Startup fails with a clear Zod error otherwise.

See SECURITY_REDTEAM_2026-04-12.md for full audit and verification matrix.

## Other

Includes in-flight CCP work: instance schema tweaks, agent server updates,
health service, tunnel service, DEV_WORKFLOW doc updates, and new migration
dropping composeProject uniqueness.

Bunker Admin
2026-04-12 15:17:00 -06:00

323 lines
14 KiB
Markdown
Raw Blame History

# Development & Release Workflow
How code changes move from development to production deployments across all installation methods.
---
## Overview
There are **three ways** Changemaker Lite gets deployed:
| Method | Who uses it | Images from | Compose file |
|--------|------------|-------------|--------------|
| **Source install** | Developers, contributors | Built locally from source | `docker-compose.yml` |
| **Release install** | Production servers, evaluators | Gitea registry (pre-built) | `docker-compose.prod.yml` (ships as `docker-compose.yml` in tarball) |
| **CCP provisioned** | Fleet operators (Control Panel) | Gitea registry (pre-built) | Rendered from `templates/docker-compose.yml.hbs` |
All three methods share the same Gitea container registry at `gitea.bnkops.com/admin`.
---
## The Pipeline
```
┌──────────────────────────────────────────────────────────────────┐
│ DEVELOPMENT (your machine) │
│ │
│ Edit code → docker compose up -d → test locally │
│ Uses: docker-compose.yml (build: blocks + ./api:/app mounts) │
└──────────────────┬───────────────────────────────────────────────┘
│ git push
▼
┌──────────────────────────────────────────────────────────────────┐
│ BUILD & PUBLISH │
│ │
│ Step 1: ./scripts/build-and-push.sh │
│ Builds 4 production images, pushes to Gitea registry │
│ (api, admin, media-api, nginx) tagged :SHA + :latest │
│ │
│ Step 2: ./scripts/mirror-images.sh (run once/rarely) │
│ Mirrors 36 third-party images to Gitea registry │
│ (postgres, redis, nocodb, jitsi, grafana, etc.) │
│ │
│ Step 3: ./scripts/build-release.sh --tag vX.Y.Z --upload │
│ Packages runtime files into ~9MB tarball, uploads to │
│ Gitea Releases │
└──────────────────┬─────────────────100.90.78.47──────────────────────────────┘
│
┌───────────┴───────────┐
▼ ▼
┌─────────────────┐ ┌──────────────────┐
│ RELEASE INSTALL │ │ CCP PROVISIONED │
│ │ │ │
│ curl installer │ │ Control Panel │
│ or manual tarball│ │ creates instance │
│ → config.sh │ │ via web UI │
│ → docker compose │ │ → renders config │
│ up -d │ │ → docker compose │
│ │ │ up -d │
└─────────────────┘ └──────────────────┘
│ │
└───────────┬───────────┘
▼
All images pulled from
gitea.bnkops.com/admin
(zero external dependencies)
```
---
## Step-by-Step
### 1. Local Development
Standard Docker Compose workflow with hot-reload:
```bash
# Start core services
docker compose up -d v2-postgres redis api admin
# API logs (watch for errors)
docker compose logs -f api
# Run with media API
docker compose up -d media-api
# Run with monitoring stack
docker compose --profile monitoring up -d
```
**Key:** `docker-compose.yml` uses `build:` blocks to compile TypeScript from source and mounts `./api:/app` for live code changes. This is the only compose file that builds from source.
### 2. Build & Push Production Images
After code changes are tested locally:
```bash
# Build production images and push to Gitea registry
./scripts/build-and-push.sh
```
This builds **4 services** with multi-stage Dockerfiles (production target, no dev dependencies), tags each image with `:SHA` and `:latest`, and pushes to `gitea.bnkops.com/admin/changemaker-{service}`:
| Service | Dockerfile | What it produces |
|---------|-----------|-----------------|
| `api` | `api/Dockerfile` | Express + Prisma (compiled JS, no TS) |
| `admin` | `admin/Dockerfile` | Nginx serving React build output |
| `media-api` | `api/Dockerfile.media` | Fastify + FFmpeg (compiled JS) |
| `nginx` | `nginx/Dockerfile` | Nginx with `envsubst` domain templating |
```bash
# Build specific services only
./scripts/build-and-push.sh --services api,admin
# Build without pushing (verify first)
./scripts/build-and-push.sh --no-push
# Include code-server (~9GB, only when Dockerfile changes)
./scripts/build-and-push.sh --include-code-server
```
### 3. Mirror Third-Party Images (Run Once / On Version Bumps)
Copies all third-party Docker images used by the platform to the Gitea registry, so deployments never depend on Docker Hub, GHCR, LSCR, or GCR:
```bash
# Mirror all 36 images (core + platform + comms + monitoring)
./scripts/mirror-images.sh
# Mirror only essential infrastructure (postgres, redis, alpine)
./scripts/mirror-images.sh --core-only
# Preview without executing
./scripts/mirror-images.sh --dry-run
```
**When to re-run:** Only when upgrading a third-party image version. The script has explicit version pins — update the version in `mirror-images.sh`, then re-run.
Images are organized into 4 groups:
| Group | Count | Examples |
|-------|-------|---------|
| Core Infrastructure | 5 | postgres:16-alpine, redis:7-alpine, alpine:3 |
| Platform Services | 16 | nocodb, listmonk, gitea, n8n, vaultwarden, nginx, code-server |
| Communication | 8 | rocket.chat, mongo, nats, gancio, jitsi (4 containers) |
| Monitoring | 7 | prometheus, grafana, alertmanager, cadvisor, exporters, gotify |
### 4. Build Release Tarball
Packages only runtime files (~9 MB) — no source code, no node_modules:
```bash
# Build tarball
./scripts/build-release.sh --tag v2.2.0
# Build and upload to Gitea Releases
./scripts/build-release.sh --tag v2.2.0 --upload
# Preview contents without creating tarball
./scripts/build-release.sh --dry-run
```
The tarball contains:
- `docker-compose.yml` (copy of `docker-compose.prod.yml` — image-only, no build blocks)
- `.env.example`, `config.sh` (configuration wizard)
- `scripts/` (init scripts, backup, upgrade, systemd units)
- `configs/` (prometheus, grafana, alertmanager, homepage, pangolin)
- `nginx/conf.d/` (templates for reference)
- `mkdocs/` (starter documentation)
- Empty data directories
### 5. Deploying
#### New Release Install (End Users)
```bash
# One-liner
curl -fsSL https://gitea.bnkops.com/admin/changemaker.lite/raw/branch/main/scripts/install.sh | bash
# Or manual
curl -LO https://gitea.bnkops.com/admin/changemaker.lite/releases/latest/download/changemaker-lite-latest.tar.gz
tar xzf changemaker-lite-latest.tar.gz
cd changemaker-lite
bash config.sh
docker compose up -d
```
All images (custom + third-party) pull from `gitea.bnkops.com/admin`. No external registry access needed.
#### New CCP Instance (Fleet Operators)
The Control Panel provisions instances via its web UI:
1. Operator fills in the Create Instance wizard (domain, features, email, tunnel)
2. CCP copies source files, renders templates (Handlebars), generates secrets
3. With `USE_REGISTRY_IMAGES=true` (default): pulls pre-built images from Gitea (~2 min)
4. With `USE_REGISTRY_IMAGES=false`: builds from source (~10+ min)
5. Starts infrastructure → runs migrations → starts all services
CCP registry settings (in `changemaker-control-panel/.env`):
```bash
GITEA_REGISTRY=gitea.bnkops.com/admin # Registry URL for all images
USE_REGISTRY_IMAGES=true # true = pull pre-built, false = build from source
IMAGE_TAG=latest # Tag for custom images (api, admin, media-api)
```
### 6. Upgrading Existing Installations
#### Source Installs
```bash
./scripts/upgrade.sh # Standard: git pull + rebuild from source
./scripts/upgrade.sh --use-registry # Fast: pull pre-built images instead of rebuilding
./scripts/upgrade.sh --dry-run # Preview changes
```
#### Release Installs
```bash
./scripts/upgrade.sh # Auto-detects release mode, downloads latest tarball
```
Release installs are detected by the presence of a `VERSION` file and absence of `.git/`. The upgrade script automatically downloads the latest tarball from Gitea instead of running `git pull`.
---
## Image Naming Conventions
All images live under `gitea.bnkops.com/admin/`:
| Type | Naming Pattern | Example |
|------|---------------|---------|
| Custom services | `changemaker-{service}:{sha\|latest}` | `changemaker-api:latest` |
| Simple names | Same as upstream | `postgres:16-alpine`, `redis:7-alpine` |
| Namespaced → short | Org removed | `nocodb/nocodb` → `nocodb:0.301.3` |
| Conflict resolution | Explicit short name | `gotify/server` → `gotify`, `vaultwarden/server` → `vaultwarden` |
| Jitsi suite | `jitsi-{component}` | `jitsi-web:stable-9823`, `jitsi-prosody:stable-9823` |
| LinuxServer nginx | `ls-nginx` (avoids nginx conflict) | `ls-nginx:1.28.2` |
---
## Two Compose Files
| File | Purpose | Build? | Source mounts? | Image source |
|------|---------|--------|---------------|-------------|
| `docker-compose.yml` | Development | Yes (`build:` blocks) | Yes (`./api:/app`) | Built locally |
| `docker-compose.prod.yml` | Production | No | No | `${GITEA_REGISTRY:-gitea.bnkops.com/admin}/...` |
Release tarballs ship `docker-compose.prod.yml` renamed as `docker-compose.yml`.
The CCP template (`templates/docker-compose.yml.hbs`) generates a compose file that works like `docker-compose.prod.yml` when `USE_REGISTRY_IMAGES=true`, or like `docker-compose.yml` when `false`.
---
## Quick Reference
```bash
# ── Development ──
docker compose up -d v2-postgres redis api admin # Start dev stack
docker compose logs -f api # Watch API logs
docker compose exec api npx prisma migrate dev # Create migration
# ── Build & Publish ──
./scripts/build-and-push.sh # Build + push 4 images
./scripts/mirror-images.sh # Mirror 36 third-party images
git tag --sort=-v:refname | head -3 # Check latest version tags
./scripts/build-release.sh --tag vX.Y.Z --upload # Package + upload release
# ── Deploy ──
curl -fsSL .../install.sh | bash # New install (release)
./scripts/upgrade.sh # Upgrade existing install
./scripts/upgrade.sh --use-registry # Fast upgrade (registry images)
# ── Verify ──
curl -s http://localhost:4000/api/health # API health check
docker compose ps # Container status
```
---
## Gitea API Tokens
There are **two separate Gitea tokens** with different purposes. Using the wrong one is a common mistake:
| Variable | Target | Used by | Create at |
|----------|--------|---------|-----------|
| `GITEA_REGISTRY_API_TOKEN` | Remote registry (`gitea.bnkops.com`) | `build-release.sh --upload`, release API calls | `https://gitea.bnkops.com/user/settings/applications` |
| `GITEA_API_TOKEN` | Local Gitea instance | Docs comments, user provisioning, SSO | `http://localhost:3030/user/settings/applications` |
**Key:** Release uploads and the Gitea Releases API require `GITEA_REGISTRY_API_TOKEN`. If you get `"user does not exist"` from the API, you're using the wrong token.
---
## Checklist: Cutting a New Release
1. [ ] All code changes committed and pushed to `main` branch
2. [ ] `docker compose up -d` works locally (smoke test)
3. [ ] **Determine version tag:**
```bash
# Check the latest existing tag to pick the next version
git tag --sort=-v:refname | head -5
# Check commits since the last tag
git log $(git tag --sort=-v:refname | head -1)..HEAD --oneline
```
4. [ ] `./scripts/build-and-push.sh` — builds and pushes 4 production images
5. [ ] `./scripts/mirror-images.sh` — only if third-party versions changed
6. [ ] `./scripts/build-release.sh --tag vX.Y.Z --upload` — packages and uploads tarball
7. [ ] **Add release notes** (via Gitea web UI or API):
```bash
# Update release body via API (use GITEA_REGISTRY_API_TOKEN, not GITEA_API_TOKEN)
GITEA_TOKEN=$(grep -oP 'GITEA_REGISTRY_API_TOKEN=\K.*' .env)
# Find release ID
curl -s "https://gitea.bnkops.com/api/v1/repos/admin/changemaker.lite/releases?limit=1" \
-H "Authorization: token $GITEA_TOKEN" | python3 -c "import sys,json; r=json.load(sys.stdin)[0]; print(f'ID: {r[\"id\"]}, Tag: {r[\"tag_name\"]}')"
# Update with release notes (write JSON body to /tmp/release-notes.json first)
curl -s -X PATCH "https://gitea.bnkops.com/api/v1/repos/admin/changemaker.lite/releases/RELEASE_ID" \
-H "Authorization: token $GITEA_TOKEN" \
-H "Content-Type: application/json" \
-d @/tmp/release-notes.json
```
8. [ ] Test clean install: `tar xzf ... && cd changemaker-lite && bash config.sh && docker compose up -d`
9. [ ] Test upgrade: `./scripts/upgrade.sh` on an existing installation
10. [ ] Verify: `curl http://localhost:4000/api/health` returns `{"status":"ok"}`
Reference in New Issue View Git Blame Copy Permalink