bunker-admin
91db29402c
Gitea SSO: cookie-based single sign-on via nginx auth_request — sets cml_session cookie on login/refresh, validates via /api/auth/gitea-sso-validate, injects X-WEBAUTH-USER header for reverse proxy auth. Dedicated GITEA_SSO_SECRET and SERVICE_PASSWORD_SALT env vars isolate secret rotation. Security fixes from March 30 audit: IDOR on ticketed events (requireEventOwnership middleware), IDOR on action items (admin/assignee/creator check), path traversal on photos (resolve-based validation), CSV upload size limit (5MB), shared calendar email exposure removed. Gitea provisioner: auto-sync docs repo collaborator access based on role (CONTENT_ROLES get write, SUPER_ADMIN gets admin). Gitea client extended with collaborator management API methods. Production hardening: NODE_ENV defaults to production in docker-compose.prod.yml, Grafana anonymous auth disabled, install.sh branch ref updated to main. Admin UI: moved docs reset from toolbar to MkDocs Settings danger zone, improved collab Ctrl+S to explicitly save + cache-bust preview. MkDocs site rebuild with updated repo data, upgrade screenshots, and content. Bunker Admin
747 lines
34 KiB
Markdown
747 lines
34 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Project Overview
|
|
|
|
Changemaker Lite is a self-hosted political campaign platform built with Docker Compose. It consolidates advocacy email campaigns, geographic mapping, volunteer management, and administration into a single TypeScript stack. The primary domain is `cmlite.org`.
|
|
|
|
**Current state:** V2 rebuild substantially complete (merged to `main`). Core platform operational with Phases 1-14 complete. See `V2_PLAN.md` for the full roadmap.
|
|
|
|
**Status Summary:**
|
|
- ✅ Phases 1-14 Complete (Foundation through Monitoring + DevOps)
|
|
- ✅ Security Audit Complete (13 findings addressed, Feb 2026)
|
|
- ✅ NAR 2025 Server Import (Canadian electoral data)
|
|
- ✅ Media Manager Integration (dual API architecture)
|
|
- ✅ Email Templates System
|
|
- ✅ Data Quality Dashboard
|
|
- ✅ Observability Dashboard
|
|
- ✅ **Drizzle to Prisma Migration Complete** (Media API consolidated to single-ORM, Feb 2026)
|
|
- ✅ **Automated Pangolin Setup** (One-command tunnel deployment, Feb 2026)
|
|
- ✅ **Migration Drift Fixed** (Baseline catch-up migration, 14 migrations cover full schema, Feb 2026)
|
|
- 🚧 Phase 15 (Testing + Polish) - Next
|
|
|
|
---
|
|
|
|
## V2 Architecture
|
|
|
|
### Stack
|
|
|
|
- **Dual API Architecture**
|
|
- **Express.js API** (TypeScript, port 4000) — Main V2 features with Prisma ORM + PostgreSQL 16
|
|
- **Fastify Media API** (TypeScript, port 4100) — Video library with Prisma ORM (shared DB) ✅ **Migrated from Drizzle (Feb 2026)**
|
|
- **React Admin GUI** — Vite + Ant Design + Zustand, port 3000
|
|
- **Nginx reverse proxy** — subdomain routing (`*.cmlite.org`)
|
|
- **NocoDB v2** — read-only data browser on port 8091
|
|
- **Redis** — caching, rate limiting, BullMQ backend, geocoding queue (authenticated)
|
|
- **Monitoring Stack** (Docker profile: `monitoring`) — Prometheus, Grafana, Alertmanager, cAdvisor, exporters
|
|
|
|
### Authentication & Security
|
|
|
|
- **JWT-based auth:** access tokens (15min) + refresh tokens (7 days, stored in DB)
|
|
- **Password policy:** 12+ characters, uppercase, lowercase, digit (enforced at schema level)
|
|
- **Initial admin:** Configured via `INITIAL_ADMIN_EMAIL` and `INITIAL_ADMIN_PASSWORD` env vars (auto-created during database seeding)
|
|
- **Roles:** `SUPER_ADMIN`, `INFLUENCE_ADMIN`, `MAP_ADMIN`, `BROADCAST_ADMIN`, `CONTENT_ADMIN`, `MEDIA_ADMIN`, `PAYMENTS_ADMIN`, `EVENTS_ADMIN`, `SOCIAL_ADMIN`, `USER`, `TEMP`
|
|
- **RBAC:** `requireRole(...roles)`, `requireNonTemp`, `authenticate` middleware; `SUPER_ADMIN` implicitly bypasses all role checks
|
|
- **Module-specific role groups** (defined in `api/src/utils/roles.ts`): `INFLUENCE_ROLES`, `MAP_ROLES`, `BROADCAST_ROLES`, `CONTENT_ROLES`, `MEDIA_ROLES`, `PAYMENTS_ROLES`, `EVENTS_ROLES`, `SOCIAL_ROLES`, `SYSTEM_ROLES`, `SCHEDULING_ROLES`
|
|
- **User management:** `SUPER_ADMIN` always; other admins need `permissions.canManageUsers: true` for write operations
|
|
- **Security:** See Security & Configuration section below + `SECURITY_AUDIT_2025-02-11.md`
|
|
|
|
### Email Systems
|
|
|
|
- **BullMQ** — async advocacy email job queue with SMTP
|
|
- **Listmonk** — newsletter/marketing campaigns (opt-in sync via `LISTMONK_SYNC_ENABLED`)
|
|
- **MailHog** — dev email capture (`EMAIL_TEST_MODE=true`)
|
|
|
|
### Directory Structure (Annotated)
|
|
|
|
```
|
|
changemaker.lite/
|
|
├── api/ # Dual API servers (Express + Fastify)
|
|
│ ├── prisma/
|
|
│ │ ├── schema.prisma # 30+ models: User, Campaign, Location, Shift, etc.
|
|
│ │ ├── migrations/ # Prisma migration history
|
|
│ │ └── seed.ts # Admin user, settings, page blocks
|
|
│ ├── drizzle/ # Media tables (Drizzle ORM)
|
|
│ ├── Dockerfile.media # Fastify media server container
|
|
│ └── src/
|
|
│ ├── server.ts # Express API entry point (port 4000)
|
|
│ ├── media-server.ts # Fastify media API entry point (port 4100)
|
|
│ ├── config/
|
|
│ │ └── env.ts # Zod-validated environment config (100+ vars)
|
|
│ ├── middleware/ # auth, rbac, rate-limit, validate, error-handler
|
|
│ ├── modules/
|
|
│ │ ├── auth/ # JWT login, register, refresh, logout
|
|
│ │ ├── users/ # User CRUD + pagination + search
|
|
│ │ ├── settings/ # Site settings singleton
|
|
│ │ ├── services/ # Service health checks
|
|
│ │ ├── influence/
|
|
│ │ │ ├── campaigns/ # Campaign CRUD + public routes
|
|
│ │ │ ├── representatives/ # Represent API integration + cache
|
|
│ │ │ ├── responses/ # Response wall + moderation + upvoting
|
|
│ │ │ ├── postal-codes/ # Postal code cache service
|
|
│ │ │ ├── campaign-emails/ # Email tracking + stats
|
|
│ │ │ └── email-queue/ # BullMQ queue admin
|
|
│ │ ├── map/
|
|
│ │ │ ├── locations/ # Location CRUD + geocoding + NAR import
|
|
│ │ │ ├── geocoding/ # Multi-provider geocoding (6 providers)
|
|
│ │ │ ├── cuts/ # Polygon CRUD + spatial queries
|
|
│ │ │ ├── shifts/ # Shift CRUD + signups
|
|
│ │ │ ├── canvass/ # Canvassing sessions + visits + routes
|
|
│ │ │ ├── tracking/ # GPS tracking sessions (volunteer + admin routes)
|
|
│ │ │ └── settings/ # Map settings singleton
|
|
│ │ ├── pages/
|
|
│ │ │ ├── pages-admin.routes.ts # Landing page CRUD
|
|
│ │ │ ├── pages-public.routes.ts # Public page renderer
|
|
│ │ │ └── blocks.routes.ts # Block library API
|
|
│ │ ├── email-templates/ # Email template CRUD + rendering
|
|
│ │ ├── media/ # Fastify media API (videos, reactions, jobs)
|
|
│ │ ├── listmonk/ # Newsletter sync admin routes
|
|
│ │ ├── pangolin/ # Tunnel management (Newt integration)
|
|
│ │ ├── docs/ # MkDocs + Code Server health checks
|
|
│ │ ├── qr/ # QR code PNG generation (public)
|
|
│ │ └── observability/ # Prometheus/Grafana/Alertmanager integration
|
|
│ ├── services/ # email, email-queue, geocode-queue, listmonk, pangolin, docker
|
|
│ ├── types/ # express.d.ts (Request augmentation)
|
|
│ └── utils/ # logger (Winston), metrics (prom-client), spatial
|
|
│
|
|
├── admin/ # React Admin (Vite + Ant Design + Zustand)
|
|
│ └── src/
|
|
│ ├── App.tsx # Main router + route definitions
|
|
│ ├── components/
|
|
│ │ ├── AppLayout.tsx # Admin sidebar layout
|
|
│ │ ├── PublicLayout.tsx # Public dark theme layout
|
|
│ │ ├── VolunteerLayout.tsx # Volunteer portal layout
|
|
│ │ ├── MediaPublicLayout.tsx # Public media gallery layout
|
|
│ │ ├── GrapesJSEditor.tsx # Landing page editor wrapper (forwardRef, Ctrl+S)
|
|
│ │ ├── map/ # Leaflet map components + controls + drawing modes
|
|
│ │ ├── canvass/ # GPS tracking, markers, route, visit recording
|
|
│ │ ├── media/ # VideoCard, BulkActions, gallery components
|
|
│ │ ├── email-templates/ # Email template components
|
|
│ │ └── observability/ # Monitoring components
|
|
│ ├── pages/
|
|
│ │ ├── auth/ # LoginPage
|
|
│ │ ├── influence/ # CampaignsPage, ResponsesPage, RepresentativesPage, EmailQueuePage
|
|
│ │ ├── map/ # LocationsPage, CutsPage, ShiftsPage, MapSettingsPage, DataQualityDashboardPage
|
|
│ │ ├── volunteer/ # VolunteerMapPage, VolunteerShiftsPage, MyActivityPage, MyRoutesPage
|
|
│ │ ├── public/ # CampaignsListPage, CampaignPage, ResponseWallPage, MapPage, ShiftsPage, LandingPage, MediaGalleryPage, MediaViewerPage
|
|
│ │ ├── media/ # LibraryPage, SharedMediaPage, MediaJobsPage, AnalyticsDashboardPage
|
|
│ │ ├── services/ # MiniQRPage, MailHogPage, CodeEditorPage, N8nPage, GiteaPage, NocoDBPage
|
|
│ │ └── (root) # DashboardPage, UsersPage, SettingsPage, CanvassDashboardPage, WalkSheetPage, CutExportPage, LandingPagesPage, PageEditorPage, EmailTemplatesPage, ListmonkPage, PangolinPage, ObservabilityPage
|
|
│ ├── stores/ # auth.store.ts, canvass.store.ts (Zustand)
|
|
│ ├── lib/ # api.ts, media-api.ts, media-public-api.ts (axios)
|
|
│ ├── hooks/ # useDebounce, useLocalStorage
|
|
│ └── types/ # api.ts, canvass.ts, media.ts (TypeScript interfaces)
|
|
│
|
|
├── media-manager/ # Legacy media manager (reference)
|
|
├── nginx/ # Reverse proxy config (subdomain routing + CSP)
|
|
├── configs/ # Prometheus, Grafana, Alertmanager configs
|
|
├── scripts/ # Deployment, backup, upgrade, registry scripts
|
|
│ ├── install.sh # Curl-friendly installer (downloads tarball + runs config.sh)
|
|
│ ├── build-and-push.sh # Build production images → push to Gitea registry
|
|
│ ├── build-release.sh # Package runtime files into release tarball
|
|
│ ├── mirror-images.sh # Mirror third-party images to Gitea
|
|
│ ├── upgrade.sh # 6-phase upgrade (git or release-tarball mode)
|
|
│ ├── upgrade-check.sh # Check for updates (git or Gitea API)
|
|
│ ├── upgrade-watcher.sh # Systemd bridge for admin GUI upgrades
|
|
│ └── backup.sh # PostgreSQL + Listmonk + uploads backup
|
|
├── docker-compose.yml # V2 orchestration (20+ services)
|
|
├── docker-compose.v1.yml # V1 backup (reference)
|
|
├── .env.example # All required environment variables
|
|
└── V2_PLAN.md # Full 14-phase roadmap
|
|
```
|
|
|
|
---
|
|
|
|
## Quick Start Guide
|
|
|
|
### Pre-built Install (Production — Recommended)
|
|
|
|
The fastest way to deploy. No source code, no compilation:
|
|
|
|
```bash
|
|
curl -fsSL https://gitea.bnkops.com/admin/changemaker.lite/raw/branch/main/scripts/install.sh | bash
|
|
```
|
|
|
|
This downloads a ~9MB release tarball, runs the config wizard, and sets `IMAGE_TAG=latest`. Then:
|
|
|
|
```bash
|
|
cd ~/changemaker.lite && docker compose up -d
|
|
```
|
|
|
|
Pre-built images are pulled from `gitea.bnkops.com/admin` (~2 min). Database migrations and seeding run automatically via the API entrypoint. Access the admin GUI at http://localhost:3000.
|
|
|
|
### Source Install (Development)
|
|
|
|
1. **Clone repository:**
|
|
```bash
|
|
git clone <repo-url> changemaker.lite
|
|
cd changemaker.lite
|
|
```
|
|
|
|
2. **Create environment file:**
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env and set:
|
|
# - V2_POSTGRES_PASSWORD (strong password)
|
|
# - REDIS_PASSWORD (strong password)
|
|
# - JWT_ACCESS_SECRET (openssl rand -hex 32)
|
|
# - JWT_REFRESH_SECRET (openssl rand -hex 32)
|
|
# - ENCRYPTION_KEY (openssl rand -hex 32, must differ from JWT secrets)
|
|
```
|
|
|
|
3. **Start core services:**
|
|
```bash
|
|
docker compose up -d v2-postgres redis api admin
|
|
```
|
|
|
|
4. **Run database migrations:**
|
|
```bash
|
|
docker compose exec api npx prisma migrate deploy
|
|
docker compose exec api npx prisma db seed
|
|
```
|
|
|
|
5. **Access the application:**
|
|
- Admin GUI: http://localhost:3000 (see INITIAL_ADMIN_EMAIL/INITIAL_ADMIN_PASSWORD in .env)
|
|
- API: http://localhost:4000
|
|
- **Change default password immediately**
|
|
|
|
### Development Workflow
|
|
|
|
**Starting services:**
|
|
```bash
|
|
# Core services
|
|
docker compose up -d v2-postgres redis api admin
|
|
|
|
# Include monitoring stack
|
|
docker compose --profile monitoring up -d
|
|
|
|
# Include media API
|
|
docker compose up -d media-api
|
|
```
|
|
|
|
**Local development (without Docker):**
|
|
```bash
|
|
# Terminal 1: API
|
|
cd api && npm install && npm run dev
|
|
|
|
# Terminal 2: Admin
|
|
cd admin && npm install && npm run dev
|
|
|
|
# Terminal 3 (optional): Media API
|
|
cd api && npm run dev:media
|
|
```
|
|
|
|
### Accessing Services
|
|
|
|
| Service | URL | Default Credentials |
|
|
|---------|-----|---------------------|
|
|
| Admin GUI | http://localhost:3000 | See INITIAL_ADMIN_EMAIL/INITIAL_ADMIN_PASSWORD in .env |
|
|
| API | http://localhost:4000 | - |
|
|
| NocoDB | http://localhost:8091 | See `NC_ADMIN_EMAIL`/`NC_ADMIN_PASSWORD` in .env |
|
|
| MailHog | http://localhost:8025 | - |
|
|
| Grafana | http://localhost:3001 | admin / admin |
|
|
| Prometheus | http://localhost:9090 | - |
|
|
| Listmonk | http://localhost:9001 | See `LISTMONK_WEB_ADMIN_USER`/`PASSWORD` in .env |
|
|
|
|
### Feature Flags
|
|
|
|
Enable optional features in `.env`:
|
|
|
|
```bash
|
|
# Media Manager
|
|
ENABLE_MEDIA_FEATURES=true
|
|
|
|
# Listmonk Newsletter Sync
|
|
LISTMONK_SYNC_ENABLED=true
|
|
|
|
# Email Test Mode (sends to MailHog instead of SMTP)
|
|
EMAIL_TEST_MODE=true
|
|
```
|
|
|
|
---
|
|
|
|
## Development Commands
|
|
|
|
The user likes to use Docker - recereating services as if in production.
|
|
|
|
### API Development
|
|
```bash
|
|
cd api && npm run dev # Express dev server (port 4000)
|
|
cd api && npm run dev:media # Fastify media dev server (port 4100)
|
|
cd api && npx tsc --noEmit # Type-check
|
|
cd api && npx prisma migrate dev # Run/create Prisma migrations
|
|
cd api && npx prisma studio # Browse database
|
|
cd api && npx drizzle-kit push # Push Drizzle schema changes (media)
|
|
```
|
|
|
|
### Admin Development
|
|
```bash
|
|
cd admin && npm run dev # Vite dev server (port 3000)
|
|
cd admin && npx tsc --noEmit # Type-check
|
|
cd admin && npm run build # Production build
|
|
```
|
|
|
|
### Docker Operations
|
|
```bash
|
|
# Start services
|
|
docker compose up -d v2-postgres redis api admin
|
|
docker compose up -d media-api
|
|
docker compose --profile monitoring up -d
|
|
|
|
# View logs
|
|
docker compose logs -f api
|
|
docker compose logs -f media-api
|
|
|
|
# Database operations
|
|
docker compose exec api npx prisma migrate dev
|
|
docker compose exec api npx drizzle-kit push
|
|
|
|
# Stop services
|
|
docker compose down
|
|
```
|
|
|
|
### Registry & Release Operations
|
|
```bash
|
|
# Build production images and push to Gitea registry
|
|
./scripts/build-and-push.sh --services api,admin,media-api,nginx
|
|
./scripts/build-and-push.sh --no-push # Build only, no push (verify)
|
|
|
|
# Mirror third-party images to Gitea
|
|
./scripts/mirror-images.sh # Core images (postgres, redis, etc.)
|
|
./scripts/mirror-images.sh --all # Include heavy images (RC, Jitsi, n8n)
|
|
|
|
# Build release tarball (for pre-built installs — run AFTER build-and-push)
|
|
./scripts/build-release.sh --tag v2.1.0 # Creates releases/changemaker-lite-v2.1.0.tar.gz
|
|
./scripts/build-release.sh --tag v2.1.0 --upload # Also upload to Gitea Releases API
|
|
./scripts/build-release.sh --dry-run # Preview tarball contents
|
|
|
|
# Use registry images in upgrade (source installs)
|
|
./scripts/upgrade.sh --use-registry --force --skip-backup
|
|
|
|
# Install from tarball (end-user one-liner)
|
|
curl -fsSL https://gitea.bnkops.com/admin/changemaker.lite/raw/branch/main/scripts/install.sh | bash
|
|
```
|
|
|
|
**Two compose files:**
|
|
- `docker-compose.yml` — Development: includes `build:` blocks and `./api:/app` source mounts
|
|
- `docker-compose.prod.yml` — Production: `image:` only, no source mounts, `IMAGE_TAG:-latest`
|
|
|
|
Release tarballs ship `docker-compose.prod.yml` as the compose file. Source installs use `docker-compose.yml`.
|
|
|
|
**Note:** gitea.bnkops.com must use Pangolin tunnel (not Cloudflare proxy) for large image layers (>100MB). See `docs/REGISTRY_GUIDE.md`.
|
|
|
|
### Testing & Backup
|
|
```bash
|
|
# Media API tests
|
|
cd api && ./test-media-api.sh
|
|
|
|
# Backup (PostgreSQL + Listmonk + uploads)
|
|
./scripts/backup.sh
|
|
|
|
# Type-check all projects
|
|
cd api && npx tsc --noEmit && cd ../admin && npx tsc --noEmit
|
|
```
|
|
|
|
### API Testing Credentials & Login
|
|
|
|
**Test admin account:** `admin@bnkops.ca` / `ChangeMe2025!` (SUPER_ADMIN role)
|
|
|
|
**Reliable login method (avoids shell `!` escaping issues):**
|
|
|
|
1. Write the JSON body to a file using the **Write tool** (NOT echo/printf — the `!` gets backslash-escaped by bash):
|
|
```
|
|
Write /tmp/login.json → {"email":"admin@bnkops.ca","password":"ChangeMe2025!"}
|
|
```
|
|
2. Use `curl -d @/tmp/login.json`:
|
|
```bash
|
|
curl -s -X POST http://localhost:4002/api/auth/login \
|
|
-H "Content-Type: application/json" -d @/tmp/login.json
|
|
```
|
|
3. Extract token and use for authenticated requests:
|
|
```bash
|
|
TOKEN=$(curl -s -X POST http://localhost:4002/api/auth/login \
|
|
-H "Content-Type: application/json" -d @/tmp/login.json \
|
|
| python3 -c "import sys,json; print(json.load(sys.stdin)['accessToken'])")
|
|
curl -s http://localhost:4002/api/some-endpoint -H "Authorization: Bearer $TOKEN"
|
|
```
|
|
|
|
**Port mapping:** API container port 4000 → host port **4002**, Admin port 3000 → host port **3002**
|
|
|
|
**Important:** The `!` character in `ChangeMe2025!` triggers bash history expansion. NEVER pass this password directly in bash command strings. Always use the Write-tool-to-file approach above.
|
|
|
|
---
|
|
|
|
## Core Modules Reference
|
|
|
|
### Auth & Users
|
|
|
|
**Files:**
|
|
- `api/src/modules/auth/` — JWT login, register, refresh, logout
|
|
- `api/src/modules/users/` — User CRUD + pagination + search
|
|
- `api/src/middleware/auth.ts` — JWT verification + RBAC
|
|
- `admin/src/stores/auth.store.ts` — Zustand auth state + token persistence
|
|
- `admin/src/lib/api.ts` — Axios with 401 refresh interceptor
|
|
|
|
**Features:** JWT access/refresh tokens, bcrypt passwords (12+ chars), role-based access control, user enumeration prevention, rate limiting
|
|
|
|
### Influence Module (Advocacy Campaigns)
|
|
|
|
**Files:**
|
|
- `api/src/modules/influence/campaigns/` — Campaign CRUD + public routes
|
|
- `api/src/modules/influence/representatives/` — Represent API client + cache
|
|
- `api/src/modules/influence/responses/` — Response wall + moderation + upvoting
|
|
- `api/src/services/email-queue.service.ts` — BullMQ queue + worker
|
|
- `admin/src/pages/CampaignsPage.tsx` — Campaign management
|
|
- `admin/src/pages/public/CampaignPage.tsx` — Public campaign page
|
|
|
|
**Features:** Postal code → representative lookup, email campaigns, response wall with moderation, BullMQ async email queue
|
|
|
|
**Routes:**
|
|
- Admin: `/app/influence/campaigns`, `/app/influence/responses`, `/app/influence/email-queue`
|
|
- Public: `/campaigns`, `/campaigns/:id`, `/responses/:campaignId`
|
|
|
|
### Map Module (Locations & Canvassing)
|
|
|
|
**Files:**
|
|
- `api/src/modules/map/locations/` — Location CRUD + geocoding + NAR import
|
|
- `api/src/modules/map/geocoding/geocoding.service.ts` — Multi-provider geocoding (6 providers)
|
|
- `api/src/modules/map/cuts/` — Polygon CRUD + spatial queries
|
|
- `api/src/modules/map/shifts/` — Shift CRUD + signups
|
|
- `api/src/modules/map/canvass/` — Canvassing sessions + visits + routes
|
|
- `api/src/modules/map/tracking/` — GPS tracking sessions (volunteer + admin routes)
|
|
- `api/src/utils/spatial.ts` — Point-in-polygon, haversine, bounds, centroids
|
|
- `admin/src/pages/LocationsPage.tsx` — Location CRUD + CSV + geocoding
|
|
- `admin/src/pages/CutsPage.tsx` — Cut table + map drawing editor
|
|
- `admin/src/pages/CanvassDashboardPage.tsx` — Admin canvass overview
|
|
- `admin/src/pages/volunteer/VolunteerMapPage.tsx` — Full-screen GPS canvass map
|
|
|
|
**Features:** Multi-provider geocoding, NAR 2025 import (Canadian electoral data), polygon cuts, volunteer shifts, canvassing system with GPS tracking, walking route algorithm, printable walk sheets
|
|
|
|
**Routes:**
|
|
- Admin: `/app/map/locations`, `/app/map/cuts`, `/app/map/shifts`, `/app/canvass/dashboard`
|
|
- Public: `/map`, `/shifts`
|
|
- Volunteer: `/volunteer/canvass/:cutId`, `/volunteer/assignments`, `/volunteer/activity`
|
|
|
|
### Landing Pages & Email Templates
|
|
|
|
**Files:**
|
|
- `api/src/modules/pages/` — Landing page CRUD + block library (3 route files)
|
|
- `api/src/modules/email-templates/` — Email template CRUD + rendering
|
|
- `admin/src/components/GrapesJSEditor.tsx` — GrapesJS wrapper (forwardRef, Ctrl+S)
|
|
- `admin/src/pages/PageEditorPage.tsx` — Full-screen page editor
|
|
- `admin/src/pages/EmailTemplateEditorPage.tsx` — Email template editor
|
|
|
|
**Features:** GrapesJS WYSIWYG editor, page/template CRUD, MkDocs export (Jinja2 Material overrides), public renderer, desktop-only editor warning
|
|
|
|
**Routes:**
|
|
- Admin: `/app/pages`, `/app/pages/:id/edit`, `/app/email-templates`
|
|
- Public: `/p/:slug`
|
|
|
|
### Media Manager (Dual API)
|
|
|
|
**Files:**
|
|
- `api/src/modules/media/` — Fastify media API (videos, reactions, jobs, analytics)
|
|
- `api/src/modules/media/services/` — FFprobe, video analytics service
|
|
- `api/src/modules/media/routes/` — Video CRUD, actions, schedule, analytics, tracking, upload
|
|
- `api/src/services/video-schedule-queue.service.ts` — BullMQ queue for scheduled publishing
|
|
- `admin/src/lib/media-api.ts` — Dedicated axios instance for Media API
|
|
- `admin/src/pages/media/LibraryPage.tsx` — Video library with quick actions + calendar
|
|
- `admin/src/pages/media/AnalyticsDashboardPage.tsx` — Global analytics dashboard
|
|
- `admin/src/pages/media/SharedMediaPage.tsx` — Public gallery admin
|
|
- `admin/src/pages/public/MediaGalleryPage.tsx` — Public video gallery
|
|
- `admin/src/components/media/` — VideoCard, VideoActions, modals, charts
|
|
|
|
**Features:** Video CRUD with FFprobe metadata, quick actions, scheduled publishing (BullMQ + timezones), analytics (GDPR-compliant), public tracking endpoints, keyboard shortcuts
|
|
|
|
**Routes:**
|
|
- Admin: `/app/media/library`, `/app/media/analytics`, `/app/media/shared`, `/app/media/jobs`
|
|
- Public: `/gallery`, `/gallery/watch/:id`, `/media/:id` (legacy)
|
|
- Public gallery uses `MediaPublicLayout` (purple theme, optional auth)
|
|
|
|
### Services & Integrations
|
|
|
|
**Listmonk Newsletter Sync:**
|
|
- `api/src/services/listmonk.client.ts` — Listmonk REST API client (native fetch)
|
|
- `api/src/services/listmonk-sync.service.ts` — Sync participants/locations → lists
|
|
- `admin/src/pages/ListmonkPage.tsx` — Newsletter sync management
|
|
- Opt-in sync: `LISTMONK_SYNC_ENABLED=true`
|
|
|
|
**Pangolin Tunnel Management:**
|
|
- `api/src/services/pangolin.client.ts` — Pangolin Integration API client
|
|
- `api/src/modules/pangolin/pangolin.routes.ts` — Tunnel management routes (includes `/setup-automated`)
|
|
- `admin/src/pages/PangolinPage.tsx` — Setup wizard + status dashboard + automated setup button
|
|
- `scripts/pangolin-setup.sh` — CLI wrapper for automated setup
|
|
- `configs/pangolin/resources.yml` — Central resource definitions (12 services)
|
|
- Newt container integration (Cloudflare alternative)
|
|
- **Automated setup:** One-command deployment (creates site, updates .env, restarts Newt)
|
|
- **Continuous sync:** Hourly resource sync via nginx cron job
|
|
|
|
**MkDocs + Code Server:**
|
|
- `api/src/modules/docs/docs.routes.ts` — Health checks + export routes
|
|
- `admin/src/pages/DocsPage.tsx` — MkDocs export management
|
|
- `admin/src/pages/CodeEditorPage.tsx` — Code Server management
|
|
- Embedded iframes in admin (CSP `frame-ancestors` for embedding)
|
|
|
|
**Mini QR Service:**
|
|
- `api/src/modules/qr/qr.routes.ts` — QR code PNG generation (public, no auth)
|
|
- `admin/src/pages/MiniQRPage.tsx` — Mini QR iframe
|
|
- Used by walk sheets + cut exports
|
|
|
|
### Observability & Monitoring
|
|
|
|
**Files:**
|
|
- `api/src/modules/observability/observability.routes.ts` — Prometheus/Grafana/Alertmanager integration
|
|
- `api/src/utils/metrics.ts` — 12 custom `cm_*` Prometheus metrics
|
|
- `admin/src/pages/ObservabilityPage.tsx` — Monitoring dashboard (3 tabs)
|
|
- `admin/src/pages/DataQualityDashboardPage.tsx` — Geocoding quality metrics
|
|
- `configs/prometheus/` — Scrape targets, alert rules
|
|
- `configs/grafana/` — 3 pre-configured dashboards
|
|
|
|
**Features:** 12 custom `cm_*` metrics (API uptime, queue size, sessions, etc.), HTTP request metrics, external service health gauges, 3 Grafana dashboards, alert rules, auto-start banner
|
|
|
|
**Routes:**
|
|
- Admin: `/app/observability`, `/app/map/data-quality`
|
|
- Direct: `localhost:9090` (Prometheus), `localhost:3001` (Grafana)
|
|
|
|
---
|
|
|
|
## Port Reference
|
|
|
|
| Port | Service | Notes |
|
|
|------|---------|-------|
|
|
| **Core Services** | | |
|
|
| 3000 | Admin GUI | Vite dev / React production |
|
|
| 4000 | Express API | Main V2 API (Prisma) |
|
|
| 4100 | Fastify Media API | Video library (Drizzle) |
|
|
| 5433 | V2 PostgreSQL | Localhost (container: 5432) |
|
|
| 6379 | Redis | Cache, rate limit, BullMQ |
|
|
| **Supporting Services** | | |
|
|
| 3001 | Grafana | Metrics visualization |
|
|
| 3010 | Homepage | Service dashboard |
|
|
| 3030 | Gitea | Git hosting |
|
|
| 4001 | MkDocs Site | Served docs |
|
|
| 4003 | MkDocs Dev | Live preview |
|
|
| 5432 | Listmonk PostgreSQL | Listmonk DB |
|
|
| 5678 | n8n | Workflow automation |
|
|
| 8025 | MailHog | Email capture (dev) |
|
|
| 8089 | Mini QR | QR generator |
|
|
| 8091 | NocoDB | Data browser |
|
|
| 8885 | Mini QR Proxy | Iframe-friendly |
|
|
| 8888 | Code Server | Web IDE |
|
|
| 9001 | Listmonk | Newsletter platform |
|
|
| **Monitoring** (profile: `monitoring`) | | |
|
|
| 8080 | cAdvisor | Container metrics |
|
|
| 8889 | Gotify | Notifications |
|
|
| 9090 | Prometheus | Metrics collection |
|
|
| 9093 | Alertmanager | Alert routing |
|
|
| 9100 | Node Exporter | Host metrics |
|
|
| 9121 | Redis Exporter | Redis metrics |
|
|
|
|
---
|
|
|
|
## Nginx Routing
|
|
|
|
| Subdomain | Target | Purpose |
|
|
|-----------|--------|---------|
|
|
| `app.cmlite.org` | Admin (3000) | **All application routes** (admin + public pages, campaigns, map, shifts, media) |
|
|
| `api.cmlite.org` | Express (4000) | Main API |
|
|
| `media.cmlite.org` | Fastify (4100) | Media API |
|
|
| `db.cmlite.org` | NocoDB (8091) | Data browser |
|
|
| `docs.cmlite.org` | MkDocs (4003) | Docs site |
|
|
| `code.cmlite.org` | Code Server (8888) | Web IDE |
|
|
| `n8n.cmlite.org` | n8n (5678) | Workflow automation |
|
|
| `git.cmlite.org` | Gitea (3030) | Git hosting |
|
|
| `home.cmlite.org` | Homepage (3010) | Dashboard |
|
|
| `grafana.cmlite.org` | Grafana (3001) | Metrics viz |
|
|
| `listmonk.cmlite.org` | Listmonk (9001) | Newsletters |
|
|
| `qr.cmlite.org` | Mini QR (8089) | QR generator |
|
|
| `cmlite.org` | MkDocs Static (4004) | **Documentation/marketing site only** |
|
|
|
|
**Clean separation:** Root domain (`${DOMAIN}`) serves MkDocs documentation site. All application functionality (admin GUI, public campaigns, map, shifts, media gallery) is accessible via `app.${DOMAIN}` subdomain. This provides clear separation between public documentation and the application.
|
|
|
|
---
|
|
|
|
## Common Patterns
|
|
|
|
**Note:** See `MEMORY.md` for comprehensive development patterns, gotchas, and lessons learned. Below are V2-specific patterns only.
|
|
|
|
### API Router Structure
|
|
- Service layer (`*.service.ts`) — business logic, database queries
|
|
- Routes (`*.routes.ts`) — Express router, middleware, validation
|
|
- Schemas (`*.schemas.ts`) — Zod validation schemas
|
|
- Split admin/public routes when needed (e.g., `campaigns.routes.ts` + `campaigns-public.routes.ts`)
|
|
|
|
### Authentication Middleware
|
|
- `authenticate` — requires any logged-in user
|
|
- `requireRole(...roles)` — requires specific role(s)
|
|
- `requireNonTemp` — blocks TEMP users
|
|
- Login redirects: ADMIN_ROLES → `/app`, USER/TEMP → `/volunteer`
|
|
|
|
### Frontend Architecture
|
|
- Admin pages: `admin/src/pages/` (AppLayout)
|
|
- Public pages: `admin/src/pages/public/` (PublicLayout, dark theme)
|
|
- Volunteer pages: `admin/src/pages/volunteer/` (VolunteerLayout)
|
|
- Zustand stores: `auth.store.ts`, `canvass.store.ts`
|
|
- API clients: `{ api }` from `lib/api.ts`, `mediaApi` from `lib/media-api.ts`
|
|
|
|
### Database ORMs
|
|
- **Prisma** (main API): Use `UncheckedCreateInput`/`UncheckedUpdateInput` for foreign keys, `Prisma.InputJsonValue` for JSON arrays
|
|
- **Drizzle** (media API): Separate schema file, push with `npx drizzle-kit push`, no migrations generated
|
|
|
|
### Prisma Migration Workflow
|
|
- **Always use `prisma migrate dev`** for schema changes (not `prisma db push`) — `db push` applies changes directly but doesn't create migration files, causing drift
|
|
- **Migration history:** 14 migrations in `api/prisma/migrations/` fully cover the schema (baseline catch-up applied Feb 2026)
|
|
- **Fixing drift:** Use `prisma migrate diff --from-migrations ... --to-schema-datamodel ... --script` with a shadow DB to generate catch-up SQL, then `prisma migrate resolve --applied`. See MEMORY.md for detailed steps
|
|
- **Production deploys:** Use `prisma migrate deploy` (not `migrate dev`)
|
|
|
|
### V2-Specific Gotchas
|
|
- **Prisma migrations:** Never use `db push` — always `migrate dev` to keep history in sync
|
|
- Nginx media API block must come BEFORE general API block
|
|
- `IMAGE_TAG=local` (default) never pulls from registry; set to SHA or `latest` for pre-built images
|
|
- **Release vs source installs:** Detected by `VERSION` file + absence of `.git/`; release uses `docker-compose.prod.yml`, source uses `docker-compose.yml`
|
|
- **`api/dist/` is gitignored** — never commit; if root-owned from container builds, fix with `chown`
|
|
- See MEMORY.md "Common Gotchas" for additional gotchas (ports, volumes, media upload, registry, etc.)
|
|
|
|
---
|
|
|
|
## Security & Configuration
|
|
|
|
### Security Audit
|
|
Comprehensive security audit completed 2025-02-11, addressing 13 findings. See `SECURITY_AUDIT_2025-02-11.md` for full report.
|
|
|
|
**Key improvements:**
|
|
- Password policy: 12+ chars, uppercase, lowercase, digit (schema-enforced)
|
|
- Rate limits on auth endpoints (10/min per IP)
|
|
- Refresh token rotation (atomic transaction)
|
|
- User enumeration prevention (401 not 404)
|
|
- Redis authentication required
|
|
- XSS/injection prevention (HTML escaping)
|
|
- Path traversal protection
|
|
- Encryption key for DB secrets (`ENCRYPTION_KEY` required in all environments)
|
|
- Nginx security headers (HSTS, Permissions-Policy, CSP)
|
|
|
|
### Required Environment Variables
|
|
See `.env.example` for all 100+ variables. Critical ones:
|
|
- `V2_POSTGRES_PASSWORD`, `REDIS_PASSWORD`
|
|
- `JWT_ACCESS_SECRET`, `JWT_REFRESH_SECRET`
|
|
- `ENCRYPTION_KEY` (must differ from JWT secrets)
|
|
- `LISTMONK_SYNC_ENABLED` (opt-in newsletter sync)
|
|
- `EMAIL_TEST_MODE` (MailHog vs SMTP)
|
|
- `ENABLE_MEDIA_FEATURES` (media manager)
|
|
|
|
### Production Deployment
|
|
- **Tunneling:** Pangolin with Newt container (Cloudflare alternative)
|
|
- **SSL/TLS:** Handled by tunnel provider (Pangolin/Cloudflare)
|
|
- **Docker Networking:** All containers share `changemaker-lite` bridge network, reference by container name
|
|
- **Monitoring:** Enable with `docker compose --profile monitoring up -d`
|
|
- **Backups:** Run `./scripts/backup.sh` (PostgreSQL + Listmonk + uploads, optional S3 upload)
|
|
|
|
#### Production CORS Configuration
|
|
|
|
When deploying to a production domain via Pangolin tunnel, you MUST update the `.env` file to include the production domain in `CORS_ORIGINS`:
|
|
|
|
```bash
|
|
# Example for betteredmonton.org
|
|
CORS_ORIGINS=http://app.betteredmonton.org,https://app.betteredmonton.org,http://localhost:3000,http://localhost
|
|
|
|
# Also set production mode
|
|
NODE_ENV=production
|
|
```
|
|
|
|
Without this, API requests from the production domain will fail CORS validation. After updating `.env`, restart the API container:
|
|
|
|
```bash
|
|
docker compose restart api
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Production 403/302 Errors - Pangolin Resources
|
|
|
|
**Symptom:** All API endpoints return 302 redirects to Pangolin authentication page, or 403 Forbidden errors.
|
|
|
|
**Root Cause:** Pangolin tunnel resources are configured with authentication enabled (default behavior).
|
|
|
|
**Fix:** Log in to your Pangolin dashboard and edit each resource:
|
|
1. Navigate to **Resources** → **Public**
|
|
2. For each resource (app, api, media, docs, etc.), click **Edit**
|
|
3. Change **Authentication** setting to **"Not Protected"** (or "Public Access"/"No Authentication")
|
|
4. Save changes
|
|
|
|
**Critical resources to fix first:**
|
|
- `api.betteredmonton.org` - Main API (all endpoints fail without this)
|
|
- `app.betteredmonton.org` - Admin GUI + public pages
|
|
- `media.betteredmonton.org` - Media API
|
|
|
|
**Verification:**
|
|
```bash
|
|
# Should return JSON, NOT a 302 redirect
|
|
curl https://api.betteredmonton.org/api/health
|
|
```
|
|
|
|
**See Also:** `PRODUCTION_403_FIX.md` for detailed step-by-step instructions.
|
|
|
|
### CORS Errors in Production
|
|
|
|
**Symptom:** Browser console shows CORS errors when accessing production domain.
|
|
|
|
**Fix:** Add production domain to `CORS_ORIGINS` in `.env` file (see Production CORS Configuration above).
|
|
|
|
### API Works Locally But Not Via Tunnel
|
|
|
|
Check in order:
|
|
1. **Newt container running:** `docker compose ps newt`
|
|
2. **Newt connected:** `docker compose logs newt --tail 50` (should show successful connection)
|
|
3. **Environment variables set:** `PANGOLIN_SITE_ID`, `PANGOLIN_NEWT_ID`, `PANGOLIN_NEWT_SECRET` in `.env`
|
|
4. **Pangolin resources configured:** All resources set to "Not Protected"
|
|
5. **Nginx running:** `docker compose ps nginx`
|
|
|
|
### Database/Redis Connection Failures
|
|
Check container status (`docker compose ps`), verify credentials in `.env`, check logs (`docker compose logs <service> --tail 50`). Test DB: `docker compose exec api npx prisma db execute --stdin <<< "SELECT 1"`. Test Redis: `docker compose exec redis-changemaker redis-cli -a $REDIS_PASSWORD ping`.
|
|
|
|
---
|
|
|
|
## V1 Reference (Legacy)
|
|
|
|
V1 code archived in `influence/`, `map/`, and `docker-compose.v1.yml`. Two independent Express apps using NocoDB REST API. See individual README files for V1 documentation:
|
|
- `influence/README.MD` — Features, config, campaign management
|
|
- `map/README.md` — Features, config, setup instructions
|
|
- Both use session-based auth, bcryptjs passwords, Bull job queues
|
|
|
|
---
|
|
|
|
## Key Configuration Files
|
|
|
|
### Infrastructure
|
|
- `docker-compose.yml` — Development orchestration (build blocks + source mounts, 20+ services)
|
|
- `docker-compose.prod.yml` — Production orchestration (image-only, no source mounts, `IMAGE_TAG:-latest`)
|
|
- `.env` / `.env.example` — Environment variables (100+ vars)
|
|
- `config.sh` — Interactive setup wizard (14 steps, release-mode aware)
|
|
|
|
### Database
|
|
- `api/prisma/schema.prisma` — Main schema (30+ Prisma models)
|
|
- `api/prisma/migrations/` — 14 migration files (fully cover schema as of Feb 2026)
|
|
- `api/drizzle.config.ts` — Drizzle config for media tables
|
|
- `api/prisma/seed.ts` — Database seeding
|
|
|
|
### Nginx
|
|
- `nginx/nginx.conf` — Global config + security headers
|
|
- `nginx/conf.d/default.conf` — Subdomain routing (12+ subdomains)
|
|
- `nginx/conf.d/api.conf` — API reverse proxy (Express + Fastify)
|
|
- `nginx/conf.d/services.conf` — Service proxies
|
|
|
|
### Monitoring
|
|
- `configs/prometheus/prometheus.yml` — Scrape targets + global config
|
|
- `configs/prometheus/alerts.yml` — Alert rules (12 rules)
|
|
- `configs/grafana/` — 3 pre-configured dashboards
|
|
- `configs/alertmanager/alertmanager.yml` — Alert routing
|
|
|
|
### Documentation
|
|
- `CLAUDE.md` — Project-wide instructions (this file)
|
|
- `V2_PLAN.md` — Full 14-phase roadmap
|
|
- `SECURITY_AUDIT_2025-02-11.md` — Security audit report
|
|
- `MEMORY.md` — Development patterns and gotchas
|