767 lines
35 KiB
Markdown
767 lines
35 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Project Overview
|
|
|
|
Changemaker Lite is a self-hosted political campaign platform built with Docker Compose. It consolidates advocacy email campaigns, geographic mapping, volunteer management, and administration into a single TypeScript stack. The primary domain is `cmlite.org`.
|
|
|
|
**Current state:** V2 rebuild substantially complete on the `v2` branch. Core platform operational with Phases 1-14 complete. See `V2_PLAN.md` for the full roadmap.
|
|
|
|
**Status Summary:**
|
|
- ✅ Phases 1-14 Complete (Foundation through Monitoring + DevOps)
|
|
- ✅ Security Audit Complete (13 findings addressed, Feb 2026)
|
|
- ✅ NAR 2025 Server Import (Canadian electoral data)
|
|
- ✅ Media Manager Integration (dual API architecture)
|
|
- ✅ Email Templates System
|
|
- ✅ Data Quality Dashboard
|
|
- ✅ Observability Dashboard
|
|
- ✅ **Drizzle to Prisma Migration Complete** (Media API consolidated to single-ORM, Feb 2026)
|
|
- ✅ **Automated Pangolin Setup** (One-command tunnel deployment, Feb 2026)
|
|
- 🚧 Phase 15 (Testing + Polish) - Next
|
|
|
|
---
|
|
|
|
## V2 Architecture
|
|
|
|
### Stack
|
|
|
|
- **Dual API Architecture**
|
|
- **Express.js API** (TypeScript, port 4000) — Main V2 features with Prisma ORM + PostgreSQL 16
|
|
- **Fastify Media API** (TypeScript, port 4100) — Video library with Prisma ORM (shared DB) ✅ **Migrated from Drizzle (Feb 2026)**
|
|
- **React Admin GUI** — Vite + Ant Design + Zustand, port 3000
|
|
- **Nginx reverse proxy** — subdomain routing (`*.cmlite.org`)
|
|
- **NocoDB v2** — read-only data browser on port 8091
|
|
- **Redis** — caching, rate limiting, BullMQ backend, geocoding queue (authenticated)
|
|
- **Monitoring Stack** (Docker profile: `monitoring`) — Prometheus, Grafana, Alertmanager, cAdvisor, exporters
|
|
|
|
### Authentication & Security
|
|
|
|
- **JWT-based auth:** access tokens (15min) + refresh tokens (7 days, stored in DB)
|
|
- **Password policy:** 12+ characters, uppercase, lowercase, digit (enforced at schema level)
|
|
- **Initial admin:** Configured via `INITIAL_ADMIN_EMAIL` and `INITIAL_ADMIN_PASSWORD` env vars (auto-created during database seeding)
|
|
- **Roles:** `SUPER_ADMIN`, `INFLUENCE_ADMIN`, `MAP_ADMIN`, `USER`, `TEMP`
|
|
- **RBAC:** `requireRole(...roles)`, `requireNonTemp`, `authenticate` middleware
|
|
- **Security features:**
|
|
- Refresh token rotation (atomic transaction)
|
|
- User enumeration prevention (401 not 404)
|
|
- Rate limiting on auth endpoints (10/min)
|
|
- Redis authentication required
|
|
- XSS/injection prevention (HTML escaping)
|
|
- Path traversal protection
|
|
- Encryption key for DB secrets (ENCRYPTION_KEY env var)
|
|
- Security audit complete (13 findings addressed, see `SECURITY_AUDIT_2025-02-11.md`)
|
|
|
|
### Email Systems
|
|
|
|
- **BullMQ** — async advocacy email job queue with SMTP
|
|
- **Listmonk** — newsletter/marketing campaigns (opt-in sync via `LISTMONK_SYNC_ENABLED`)
|
|
- **MailHog** — dev email capture (`EMAIL_TEST_MODE=true`)
|
|
|
|
### Directory Structure (Annotated)
|
|
|
|
```
|
|
changemaker.lite/
|
|
├── api/ # Dual API servers (Express + Fastify)
|
|
│ ├── prisma/
|
|
│ │ ├── schema.prisma # 30+ models: User, Campaign, Location, Shift, etc.
|
|
│ │ ├── migrations/ # Prisma migration history
|
|
│ │ └── seed.ts # Admin user, settings, page blocks
|
|
│ ├── drizzle/ # Media tables (Drizzle ORM)
|
|
│ ├── Dockerfile.media # Fastify media server container
|
|
│ └── src/
|
|
│ ├── server.ts # Express API entry point (port 4000)
|
|
│ ├── media-server.ts # Fastify media API entry point (port 4100)
|
|
│ ├── config/
|
|
│ │ └── env.ts # Zod-validated environment config (100+ vars)
|
|
│ ├── middleware/ # auth, rbac, rate-limit, validate, error-handler
|
|
│ ├── modules/
|
|
│ │ ├── auth/ # JWT login, register, refresh, logout
|
|
│ │ ├── users/ # User CRUD + pagination + search
|
|
│ │ ├── settings/ # Site settings singleton
|
|
│ │ ├── services/ # Service health checks
|
|
│ │ ├── influence/
|
|
│ │ │ ├── campaigns/ # Campaign CRUD + public routes
|
|
│ │ │ ├── representatives/ # Represent API integration + cache
|
|
│ │ │ ├── responses/ # Response wall + moderation + upvoting
|
|
│ │ │ ├── postal-codes/ # Postal code cache service
|
|
│ │ │ ├── campaign-emails/ # Email tracking + stats
|
|
│ │ │ └── email-queue/ # BullMQ queue admin
|
|
│ │ ├── map/
|
|
│ │ │ ├── locations/ # Location CRUD + geocoding + NAR import
|
|
│ │ │ ├── geocoding/ # Multi-provider geocoding (6 providers)
|
|
│ │ │ ├── cuts/ # Polygon CRUD + spatial queries
|
|
│ │ │ ├── shifts/ # Shift CRUD + signups
|
|
│ │ │ ├── canvass/ # Canvassing sessions + visits + routes
|
|
│ │ │ ├── tracking/ # GPS tracking sessions (volunteer + admin routes)
|
|
│ │ │ └── settings/ # Map settings singleton
|
|
│ │ ├── pages/
|
|
│ │ │ ├── pages-admin.routes.ts # Landing page CRUD
|
|
│ │ │ ├── pages-public.routes.ts # Public page renderer
|
|
│ │ │ └── blocks.routes.ts # Block library API
|
|
│ │ ├── email-templates/ # Email template CRUD + rendering
|
|
│ │ ├── media/ # Fastify media API (videos, reactions, jobs)
|
|
│ │ ├── listmonk/ # Newsletter sync admin routes
|
|
│ │ ├── pangolin/ # Tunnel management (Newt integration)
|
|
│ │ ├── docs/ # MkDocs + Code Server health checks
|
|
│ │ ├── qr/ # QR code PNG generation (public)
|
|
│ │ └── observability/ # Prometheus/Grafana/Alertmanager integration
|
|
│ ├── services/ # email, email-queue, geocode-queue, listmonk, pangolin, docker
|
|
│ ├── types/ # express.d.ts (Request augmentation)
|
|
│ └── utils/ # logger (Winston), metrics (prom-client), spatial
|
|
│
|
|
├── admin/ # React Admin (Vite + Ant Design + Zustand)
|
|
│ └── src/
|
|
│ ├── App.tsx # Main router + route definitions
|
|
│ ├── components/
|
|
│ │ ├── AppLayout.tsx # Admin sidebar layout
|
|
│ │ ├── PublicLayout.tsx # Public dark theme layout
|
|
│ │ ├── VolunteerLayout.tsx # Volunteer portal layout
|
|
│ │ ├── MediaPublicLayout.tsx # Public media gallery layout
|
|
│ │ ├── GrapesJSEditor.tsx # Landing page editor wrapper (forwardRef, Ctrl+S)
|
|
│ │ ├── map/ # Leaflet map components + controls + drawing modes
|
|
│ │ ├── canvass/ # GPS tracking, markers, route, visit recording
|
|
│ │ ├── media/ # VideoCard, BulkActions, gallery components
|
|
│ │ ├── email-templates/ # Email template components
|
|
│ │ └── observability/ # Monitoring components
|
|
│ ├── pages/
|
|
│ │ ├── auth/ # LoginPage
|
|
│ │ ├── DashboardPage.tsx # Admin dashboard
|
|
│ │ ├── UsersPage.tsx # User CRUD
|
|
│ │ ├── SettingsPage.tsx # Global site settings
|
|
│ │ ├── influence/
|
|
│ │ │ ├── CampaignsPage.tsx # Campaign management
|
|
│ │ │ ├── ResponsesPage.tsx # Response moderation
|
|
│ │ │ ├── RepresentativesPage.tsx # Rep cache admin
|
|
│ │ │ └── EmailQueuePage.tsx # Queue monitoring
|
|
│ │ ├── map/
|
|
│ │ │ ├── LocationsPage.tsx # Location CRUD + CSV + geocoding
|
|
│ │ │ ├── CutsPage.tsx # Cut table + map drawing editor
|
|
│ │ │ ├── ShiftsPage.tsx # Shift CRUD + signups drawer
|
|
│ │ │ ├── MapSettingsPage.tsx # Map settings
|
|
│ │ │ └── DataQualityDashboardPage.tsx # Geocoding quality metrics
|
|
│ │ ├── CanvassDashboardPage.tsx # Admin canvass overview
|
|
│ │ ├── WalkSheetPage.tsx # Printable walk sheet
|
|
│ │ ├── CutExportPage.tsx # Printable location report
|
|
│ │ ├── volunteer/
|
|
│ │ │ ├── VolunteerMapPage.tsx # Full-screen GPS canvass map
|
|
│ │ │ ├── VolunteerShiftsPage.tsx # Assigned shifts
|
|
│ │ │ ├── MyActivityPage.tsx # Visit history + outcomes
|
|
│ │ │ └── MyRoutesPage.tsx # Route history
|
|
│ │ ├── public/
|
|
│ │ │ ├── CampaignsListPage.tsx # Public campaign listing
|
|
│ │ │ ├── CampaignPage.tsx # Campaign detail + email form
|
|
│ │ │ ├── ResponseWallPage.tsx # Public response wall
|
|
│ │ │ ├── MapPage.tsx # Public Leaflet map
|
|
│ │ │ ├── ShiftsPage.tsx # Public shift signup
|
|
│ │ │ ├── LandingPage.tsx # Rendered landing page (/p/:slug)
|
|
│ │ │ ├── MediaGalleryPage.tsx # Public video gallery
|
|
│ │ │ └── MediaViewerPage.tsx # Video detail page
|
|
│ │ ├── media/
|
|
│ │ │ ├── LibraryPage.tsx # Video library management
|
|
│ │ │ ├── SharedMediaPage.tsx # Public gallery admin
|
|
│ │ │ └── MediaJobsPage.tsx # Job queue monitoring
|
|
│ │ ├── LandingPagesPage.tsx # Landing page manager
|
|
│ │ ├── PageEditorPage.tsx # Full-screen GrapesJS editor
|
|
│ │ ├── EmailTemplatesPage.tsx # Email template CRUD
|
|
│ │ ├── EmailTemplateEditorPage.tsx # Email template editor
|
|
│ │ ├── ListmonkPage.tsx # Newsletter sync management
|
|
│ │ ├── PangolinPage.tsx # Tunnel setup wizard
|
|
│ │ ├── DocsPage.tsx # MkDocs export management
|
|
│ │ ├── MkDocsSettingsPage.tsx # Documentation config
|
|
│ │ ├── ObservabilityPage.tsx # Monitoring dashboard
|
|
│ │ └── services/
|
|
│ │ ├── MiniQRPage.tsx # Mini QR iframe
|
|
│ │ ├── MailHogPage.tsx # Email capture UI
|
|
│ │ ├── CodeEditorPage.tsx # Code Server management
|
|
│ │ ├── N8nPage.tsx # Workflow automation
|
|
│ │ ├── GiteaPage.tsx # Git repository hosting
|
|
│ │ └── NocoDBPage.tsx # Data browser management
|
|
│ ├── stores/ # auth.store.ts, canvass.store.ts (Zustand)
|
|
│ ├── lib/ # api.ts, media-api.ts, media-public-api.ts (axios)
|
|
│ ├── hooks/ # useDebounce, useLocalStorage
|
|
│ └── types/ # api.ts, canvass.ts, media.ts (TypeScript interfaces)
|
|
│
|
|
├── media-manager/ # Legacy media manager (reference)
|
|
├── nginx/ # Reverse proxy config (subdomain routing + CSP)
|
|
├── configs/ # Prometheus, Grafana, Alertmanager configs
|
|
├── scripts/ # backup.sh, legacy Cloudflare scripts
|
|
├── docker-compose.yml # V2 orchestration (20+ services)
|
|
├── docker-compose.v1.yml # V1 backup (reference)
|
|
├── .env.example # All required environment variables
|
|
└── V2_PLAN.md # Full 14-phase roadmap
|
|
```
|
|
|
|
---
|
|
|
|
## Quick Start Guide
|
|
|
|
### Initial Setup (First Time)
|
|
|
|
1. **Clone repository and checkout v2 branch:**
|
|
```bash
|
|
git clone <repo-url> changemaker.lite
|
|
cd changemaker.lite
|
|
git checkout v2
|
|
```
|
|
|
|
2. **Create environment file:**
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env and set:
|
|
# - V2_POSTGRES_PASSWORD (strong password)
|
|
# - REDIS_PASSWORD (strong password)
|
|
# - JWT_ACCESS_SECRET (openssl rand -hex 32)
|
|
# - JWT_REFRESH_SECRET (openssl rand -hex 32)
|
|
# - ENCRYPTION_KEY (openssl rand -hex 32, must differ from JWT secrets)
|
|
```
|
|
|
|
3. **Start core services:**
|
|
```bash
|
|
docker compose up -d v2-postgres redis api admin
|
|
```
|
|
|
|
4. **Run database migrations:**
|
|
```bash
|
|
docker compose exec api npx prisma migrate deploy
|
|
docker compose exec api npx prisma db seed
|
|
```
|
|
|
|
5. **Access the application:**
|
|
- Admin GUI: http://localhost:3000 (see INITIAL_ADMIN_EMAIL/INITIAL_ADMIN_PASSWORD in .env)
|
|
- API: http://localhost:4000
|
|
- **Change default password immediately**
|
|
|
|
### Development Workflow
|
|
|
|
**Starting services:**
|
|
```bash
|
|
# Core services
|
|
docker compose up -d v2-postgres redis api admin
|
|
|
|
# Include monitoring stack
|
|
docker compose --profile monitoring up -d
|
|
|
|
# Include media API
|
|
docker compose up -d media-api
|
|
```
|
|
|
|
**Local development (without Docker):**
|
|
```bash
|
|
# Terminal 1: API
|
|
cd api && npm install && npm run dev
|
|
|
|
# Terminal 2: Admin
|
|
cd admin && npm install && npm run dev
|
|
|
|
# Terminal 3 (optional): Media API
|
|
cd api && npm run dev:media
|
|
```
|
|
|
|
### Accessing Services
|
|
|
|
| Service | URL | Default Credentials |
|
|
|---------|-----|---------------------|
|
|
| Admin GUI | http://localhost:3000 | See INITIAL_ADMIN_EMAIL/INITIAL_ADMIN_PASSWORD in .env |
|
|
| API | http://localhost:4000 | - |
|
|
| NocoDB | http://localhost:8091 | See `NC_ADMIN_EMAIL`/`NC_ADMIN_PASSWORD` in .env |
|
|
| MailHog | http://localhost:8025 | - |
|
|
| Grafana | http://localhost:3001 | admin / admin |
|
|
| Prometheus | http://localhost:9090 | - |
|
|
| Listmonk | http://localhost:9001 | See `LISTMONK_WEB_ADMIN_USER`/`PASSWORD` in .env |
|
|
|
|
### Feature Flags
|
|
|
|
Enable optional features in `.env`:
|
|
|
|
```bash
|
|
# Media Manager
|
|
ENABLE_MEDIA_FEATURES=true
|
|
|
|
# Listmonk Newsletter Sync
|
|
LISTMONK_SYNC_ENABLED=true
|
|
|
|
# Email Test Mode (sends to MailHog instead of SMTP)
|
|
EMAIL_TEST_MODE=true
|
|
```
|
|
|
|
---
|
|
|
|
## Development Commands
|
|
|
|
The user likes to use Docker - recereating services as if in production.
|
|
|
|
### API Development
|
|
```bash
|
|
cd api && npm run dev # Express dev server (port 4000)
|
|
cd api && npm run dev:media # Fastify media dev server (port 4100)
|
|
cd api && npx tsc --noEmit # Type-check
|
|
cd api && npx prisma migrate dev # Run/create Prisma migrations
|
|
cd api && npx prisma studio # Browse database
|
|
cd api && npx drizzle-kit push # Push Drizzle schema changes (media)
|
|
```
|
|
|
|
### Admin Development
|
|
```bash
|
|
cd admin && npm run dev # Vite dev server (port 3000)
|
|
cd admin && npx tsc --noEmit # Type-check
|
|
cd admin && npm run build # Production build
|
|
```
|
|
|
|
### Docker Operations
|
|
```bash
|
|
# Start services
|
|
docker compose up -d v2-postgres redis api admin
|
|
docker compose up -d media-api
|
|
docker compose --profile monitoring up -d
|
|
|
|
# View logs
|
|
docker compose logs -f api
|
|
docker compose logs -f media-api
|
|
|
|
# Database operations
|
|
docker compose exec api npx prisma migrate dev
|
|
docker compose exec api npx drizzle-kit push
|
|
|
|
# Stop services
|
|
docker compose down
|
|
```
|
|
|
|
### Testing & Backup
|
|
```bash
|
|
# Media API tests
|
|
cd api && ./test-media-api.sh
|
|
|
|
# Backup (PostgreSQL + Listmonk + uploads)
|
|
./scripts/backup.sh
|
|
|
|
# Type-check all projects
|
|
cd api && npx tsc --noEmit && cd ../admin && npx tsc --noEmit
|
|
```
|
|
|
|
### API Testing Credentials & Login
|
|
|
|
**Test admin account:** `admin@bnkops.ca` / `ChangeMe2025!` (SUPER_ADMIN role)
|
|
|
|
**Reliable login method (avoids shell `!` escaping issues):**
|
|
|
|
1. Write the JSON body to a file using the **Write tool** (NOT echo/printf — the `!` gets backslash-escaped by bash):
|
|
```
|
|
Write /tmp/login.json → {"email":"admin@bnkops.ca","password":"ChangeMe2025!"}
|
|
```
|
|
2. Use `curl -d @/tmp/login.json`:
|
|
```bash
|
|
curl -s -X POST http://localhost:4002/api/auth/login \
|
|
-H "Content-Type: application/json" -d @/tmp/login.json
|
|
```
|
|
3. Extract token and use for authenticated requests:
|
|
```bash
|
|
TOKEN=$(curl -s -X POST http://localhost:4002/api/auth/login \
|
|
-H "Content-Type: application/json" -d @/tmp/login.json \
|
|
| python3 -c "import sys,json; print(json.load(sys.stdin)['accessToken'])")
|
|
curl -s http://localhost:4002/api/some-endpoint -H "Authorization: Bearer $TOKEN"
|
|
```
|
|
|
|
**Port mapping:** API container port 4000 → host port **4002**, Admin port 3000 → host port **3002**
|
|
|
|
**Important:** The `!` character in `ChangeMe2025!` triggers bash history expansion. NEVER pass this password directly in bash command strings. Always use the Write-tool-to-file approach above.
|
|
|
|
---
|
|
|
|
## Core Modules Reference
|
|
|
|
### Auth & Users
|
|
|
|
**Files:**
|
|
- `api/src/modules/auth/` — JWT login, register, refresh, logout
|
|
- `api/src/modules/users/` — User CRUD + pagination + search
|
|
- `api/src/middleware/auth.ts` — JWT verification + RBAC
|
|
- `admin/src/stores/auth.store.ts` — Zustand auth state + token persistence
|
|
- `admin/src/lib/api.ts` — Axios with 401 refresh interceptor
|
|
|
|
**Features:** JWT access/refresh tokens, bcrypt passwords (12+ chars), role-based access control, user enumeration prevention, rate limiting
|
|
|
|
### Influence Module (Advocacy Campaigns)
|
|
|
|
**Files:**
|
|
- `api/src/modules/influence/campaigns/` — Campaign CRUD + public routes
|
|
- `api/src/modules/influence/representatives/` — Represent API client + cache
|
|
- `api/src/modules/influence/responses/` — Response wall + moderation + upvoting
|
|
- `api/src/services/email-queue.service.ts` — BullMQ queue + worker
|
|
- `admin/src/pages/CampaignsPage.tsx` — Campaign management
|
|
- `admin/src/pages/public/CampaignPage.tsx` — Public campaign page
|
|
|
|
**Features:** Postal code → representative lookup, email campaigns, response wall with moderation, BullMQ async email queue
|
|
|
|
**Routes:**
|
|
- Admin: `/app/influence/campaigns`, `/app/influence/responses`, `/app/influence/email-queue`
|
|
- Public: `/campaigns`, `/campaigns/:id`, `/responses/:campaignId`
|
|
|
|
### Map Module (Locations & Canvassing)
|
|
|
|
**Files:**
|
|
- `api/src/modules/map/locations/` — Location CRUD + geocoding + NAR import
|
|
- `api/src/modules/map/geocoding/geocoding.service.ts` — Multi-provider geocoding (6 providers)
|
|
- `api/src/modules/map/cuts/` — Polygon CRUD + spatial queries
|
|
- `api/src/modules/map/shifts/` — Shift CRUD + signups
|
|
- `api/src/modules/map/canvass/` — Canvassing sessions + visits + routes
|
|
- `api/src/modules/map/tracking/` — GPS tracking sessions (volunteer + admin routes)
|
|
- `api/src/utils/spatial.ts` — Point-in-polygon, haversine, bounds, centroids
|
|
- `admin/src/pages/LocationsPage.tsx` — Location CRUD + CSV + geocoding
|
|
- `admin/src/pages/CutsPage.tsx` — Cut table + map drawing editor
|
|
- `admin/src/pages/CanvassDashboardPage.tsx` — Admin canvass overview
|
|
- `admin/src/pages/volunteer/VolunteerMapPage.tsx` — Full-screen GPS canvass map
|
|
|
|
**Features:** Multi-provider geocoding, NAR 2025 import (Canadian electoral data), polygon cuts, volunteer shifts, canvassing system with GPS tracking, walking route algorithm, printable walk sheets
|
|
|
|
**Routes:**
|
|
- Admin: `/app/map/locations`, `/app/map/cuts`, `/app/map/shifts`, `/app/canvass/dashboard`
|
|
- Public: `/map`, `/shifts`
|
|
- Volunteer: `/volunteer/canvass/:cutId`, `/volunteer/assignments`, `/volunteer/activity`
|
|
|
|
### Landing Pages & Email Templates
|
|
|
|
**Files:**
|
|
- `api/src/modules/pages/` — Landing page CRUD + block library (3 route files)
|
|
- `api/src/modules/email-templates/` — Email template CRUD + rendering
|
|
- `admin/src/components/GrapesJSEditor.tsx` — GrapesJS wrapper (forwardRef, Ctrl+S)
|
|
- `admin/src/pages/PageEditorPage.tsx` — Full-screen page editor
|
|
- `admin/src/pages/EmailTemplateEditorPage.tsx` — Email template editor
|
|
|
|
**Features:** GrapesJS WYSIWYG editor, page/template CRUD, MkDocs export (Jinja2 Material overrides), public renderer, desktop-only editor warning
|
|
|
|
**Routes:**
|
|
- Admin: `/app/pages`, `/app/pages/:id/edit`, `/app/email-templates`
|
|
- Public: `/p/:slug`
|
|
|
|
### Media Manager (Dual API)
|
|
|
|
**Files:**
|
|
- `api/src/modules/media/` — Fastify media API (videos, reactions, jobs, analytics)
|
|
- `api/src/modules/media/services/` — FFprobe, video analytics service
|
|
- `api/src/modules/media/routes/` — Video CRUD, actions, schedule, analytics, tracking, upload
|
|
- `api/src/services/video-schedule-queue.service.ts` — BullMQ queue for scheduled publishing
|
|
- `admin/src/lib/media-api.ts` — Dedicated axios instance for Media API
|
|
- `admin/src/pages/media/LibraryPage.tsx` — Video library with quick actions + calendar
|
|
- `admin/src/pages/media/AnalyticsDashboardPage.tsx` — Global analytics dashboard
|
|
- `admin/src/pages/media/SharedMediaPage.tsx` — Public gallery admin
|
|
- `admin/src/pages/public/MediaGalleryPage.tsx` — Public video gallery
|
|
- `admin/src/components/media/` — VideoCard, VideoActions, modals, charts
|
|
|
|
**Features:**
|
|
- **Video CRUD:** Upload with FFprobe metadata extraction (duration, dimensions, orientation, quality), bulk operations
|
|
- **Quick Actions** (Feb 2026): Edit, preview, analytics, schedule, duplicate, preview links (24h JWT), reset analytics
|
|
- **Scheduled Publishing** (Feb 2026): BullMQ job queue, timezone support (11 zones), calendar view, publish/unpublish automation
|
|
- **Analytics** (Feb 2026): Views, watch time, completion rate, traffic sources, registered viewers, GDPR-compliant (IP hashing, 90-day retention)
|
|
- **Tracking:** Public endpoints for view/event recording, 10s heartbeat, navigator.sendBeacon for reliability
|
|
- **UI Features:** Keyboard shortcuts (E/P/A/S), hover overlays, skeleton loading, error handling, mobile responsive
|
|
|
|
**Routes:**
|
|
- Admin: `/app/media/library`, `/app/media/analytics`, `/app/media/shared`, `/app/media/jobs`
|
|
- Public: `/gallery` (public video gallery), `/gallery/watch/:id` (video viewer), `/media/:id` (backwards compatible viewer route)
|
|
- Tracking (public): `/track/view`, `/track/event`, `/track/heartbeat`
|
|
|
|
**Note:** The public gallery is served at `/gallery` via the admin app using `MediaPublicLayout`. This provides a unified purple interface for both authenticated and unauthenticated users. The gallery supports optional authentication (session-based upvoting/commenting for anonymous users).
|
|
|
|
**Documentation:**
|
|
- [Media Admin Features Guide](./docs/MEDIA_ADMIN_FEATURES.md) — Complete feature documentation
|
|
- [Video Analytics Guide](./docs/VIDEO_ANALYTICS_GUIDE.md) — Analytics setup and interpretation
|
|
- [Media API README](./api/src/modules/media/README.md) — Architecture overview
|
|
|
|
### Services & Integrations
|
|
|
|
**Listmonk Newsletter Sync:**
|
|
- `api/src/services/listmonk.client.ts` — Listmonk REST API client (native fetch)
|
|
- `api/src/services/listmonk-sync.service.ts` — Sync participants/locations → lists
|
|
- `admin/src/pages/ListmonkPage.tsx` — Newsletter sync management
|
|
- Opt-in sync: `LISTMONK_SYNC_ENABLED=true`
|
|
|
|
**Pangolin Tunnel Management:**
|
|
- `api/src/services/pangolin.client.ts` — Pangolin Integration API client
|
|
- `api/src/modules/pangolin/pangolin.routes.ts` — Tunnel management routes (includes `/setup-automated`)
|
|
- `admin/src/pages/PangolinPage.tsx` — Setup wizard + status dashboard + automated setup button
|
|
- `scripts/pangolin-setup.sh` — CLI wrapper for automated setup
|
|
- `configs/pangolin/resources.yml` — Central resource definitions (12 services)
|
|
- Newt container integration (Cloudflare alternative)
|
|
- **Automated setup:** One-command deployment (creates site, updates .env, restarts Newt)
|
|
- **Continuous sync:** Hourly resource sync via nginx cron job
|
|
|
|
**MkDocs + Code Server:**
|
|
- `api/src/modules/docs/docs.routes.ts` — Health checks + export routes
|
|
- `admin/src/pages/DocsPage.tsx` — MkDocs export management
|
|
- `admin/src/pages/CodeEditorPage.tsx` — Code Server management
|
|
- Embedded iframes in admin (CSP `frame-ancestors` for embedding)
|
|
|
|
**Mini QR Service:**
|
|
- `api/src/modules/qr/qr.routes.ts` — QR code PNG generation (public, no auth)
|
|
- `admin/src/pages/MiniQRPage.tsx` — Mini QR iframe
|
|
- Used by walk sheets + cut exports
|
|
|
|
### Observability & Monitoring
|
|
|
|
**Files:**
|
|
- `api/src/modules/observability/observability.routes.ts` — Prometheus/Grafana/Alertmanager integration
|
|
- `api/src/utils/metrics.ts` — 12 custom `cm_*` Prometheus metrics
|
|
- `admin/src/pages/ObservabilityPage.tsx` — Monitoring dashboard (3 tabs)
|
|
- `admin/src/pages/DataQualityDashboardPage.tsx` — Geocoding quality metrics
|
|
- `configs/prometheus/` — Scrape targets, alert rules
|
|
- `configs/grafana/` — 3 pre-configured dashboards
|
|
|
|
**Features:** 12 custom `cm_*` metrics (API uptime, queue size, sessions, etc.), HTTP request metrics, external service health gauges, 3 Grafana dashboards, alert rules, auto-start banner
|
|
|
|
**Routes:**
|
|
- Admin: `/app/observability`, `/app/map/data-quality`
|
|
- Direct: `localhost:9090` (Prometheus), `localhost:3001` (Grafana)
|
|
|
|
---
|
|
|
|
## Port Reference
|
|
|
|
| Port | Service | Notes |
|
|
|------|---------|-------|
|
|
| **Core Services** | | |
|
|
| 3000 | Admin GUI | Vite dev / React production |
|
|
| 4000 | Express API | Main V2 API (Prisma) |
|
|
| 4100 | Fastify Media API | Video library (Drizzle) |
|
|
| 5433 | V2 PostgreSQL | Localhost (container: 5432) |
|
|
| 6379 | Redis | Cache, rate limit, BullMQ |
|
|
| **Supporting Services** | | |
|
|
| 3001 | Grafana | Metrics visualization |
|
|
| 3010 | Homepage | Service dashboard |
|
|
| 3030 | Gitea | Git hosting |
|
|
| 4001 | MkDocs Site | Served docs |
|
|
| 4003 | MkDocs Dev | Live preview |
|
|
| 5432 | Listmonk PostgreSQL | Listmonk DB |
|
|
| 5678 | n8n | Workflow automation |
|
|
| 8025 | MailHog | Email capture (dev) |
|
|
| 8089 | Mini QR | QR generator |
|
|
| 8091 | NocoDB | Data browser |
|
|
| 8885 | Mini QR Proxy | Iframe-friendly |
|
|
| 8888 | Code Server | Web IDE |
|
|
| 9001 | Listmonk | Newsletter platform |
|
|
| **Monitoring** (profile: `monitoring`) | | |
|
|
| 8080 | cAdvisor | Container metrics |
|
|
| 8889 | Gotify | Notifications |
|
|
| 9090 | Prometheus | Metrics collection |
|
|
| 9093 | Alertmanager | Alert routing |
|
|
| 9100 | Node Exporter | Host metrics |
|
|
| 9121 | Redis Exporter | Redis metrics |
|
|
|
|
---
|
|
|
|
## Nginx Routing
|
|
|
|
| Subdomain | Target | Purpose |
|
|
|-----------|--------|---------|
|
|
| `app.cmlite.org` | Admin (3000) | **All application routes** (admin + public pages, campaigns, map, shifts, media) |
|
|
| `api.cmlite.org` | Express (4000) | Main API |
|
|
| `media.cmlite.org` | Fastify (4100) | Media API |
|
|
| `db.cmlite.org` | NocoDB (8091) | Data browser |
|
|
| `docs.cmlite.org` | MkDocs (4003) | Docs site |
|
|
| `code.cmlite.org` | Code Server (8888) | Web IDE |
|
|
| `n8n.cmlite.org` | n8n (5678) | Workflow automation |
|
|
| `git.cmlite.org` | Gitea (3030) | Git hosting |
|
|
| `home.cmlite.org` | Homepage (3010) | Dashboard |
|
|
| `grafana.cmlite.org` | Grafana (3001) | Metrics viz |
|
|
| `listmonk.cmlite.org` | Listmonk (9001) | Newsletters |
|
|
| `qr.cmlite.org` | Mini QR (8089) | QR generator |
|
|
| `cmlite.org` | MkDocs Static (4004) | **Documentation/marketing site only** |
|
|
|
|
**Clean separation:** Root domain (`${DOMAIN}`) serves MkDocs documentation site. All application functionality (admin GUI, public campaigns, map, shifts, media gallery) is accessible via `app.${DOMAIN}` subdomain. This provides clear separation between public documentation and the application.
|
|
|
|
---
|
|
|
|
## Common Patterns
|
|
|
|
**Note:** See `MEMORY.md` for comprehensive development patterns, gotchas, and lessons learned. Below are V2-specific patterns only.
|
|
|
|
### API Router Structure
|
|
- Service layer (`*.service.ts`) — business logic, database queries
|
|
- Routes (`*.routes.ts`) — Express router, middleware, validation
|
|
- Schemas (`*.schemas.ts`) — Zod validation schemas
|
|
- Split admin/public routes when needed (e.g., `campaigns.routes.ts` + `campaigns-public.routes.ts`)
|
|
|
|
### Authentication Middleware
|
|
- `authenticate` — requires any logged-in user
|
|
- `requireRole(...roles)` — requires specific role(s)
|
|
- `requireNonTemp` — blocks TEMP users
|
|
- Login redirects: ADMIN_ROLES → `/app`, USER/TEMP → `/volunteer`
|
|
|
|
### Frontend Architecture
|
|
- Admin pages: `admin/src/pages/` (AppLayout)
|
|
- Public pages: `admin/src/pages/public/` (PublicLayout, dark theme)
|
|
- Volunteer pages: `admin/src/pages/volunteer/` (VolunteerLayout)
|
|
- Zustand stores: `auth.store.ts`, `canvass.store.ts`
|
|
- API clients: `{ api }` from `lib/api.ts`, `mediaApi` from `lib/media-api.ts`
|
|
|
|
### Database ORMs
|
|
- **Prisma** (main API): Use `UncheckedCreateInput`/`UncheckedUpdateInput` for foreign keys, `Prisma.InputJsonValue` for JSON arrays
|
|
- **Drizzle** (media API): Separate schema file, push with `npx drizzle-kit push`, no migrations generated
|
|
|
|
### V2-Specific Gotchas
|
|
- Fastify media API on port 4100, separate from Express on 4000 (same DB, different ORM)
|
|
- Volunteer page naming: `VolunteerShiftsPage.tsx` (not "MyAssignmentsPage")
|
|
- Tracking module: `api/src/modules/map/tracking/` (volunteer + admin routes)
|
|
- Pages module: 3 route files (pages-admin, pages-public, blocks)
|
|
- Vite proxy: `VITE_API_URL`, `VITE_MKDOCS_URL` env vars (Docker sets to container hostnames)
|
|
- Nginx media API block must come BEFORE general API block
|
|
- MkDocs port 4003 (was 4000, conflicted with API)
|
|
- Media upload: requires separate RW volume mount for inbox directory (`:rw` on `/media/local/inbox`), library remains read-only
|
|
- FFmpeg/FFprobe: installed in media-api container (Alpine `apk add --no-cache ffmpeg`), used for metadata extraction
|
|
|
|
---
|
|
|
|
## Security & Configuration
|
|
|
|
### Security Audit
|
|
Comprehensive security audit completed 2025-02-11, addressing 13 findings. See `SECURITY_AUDIT_2025-02-11.md` for full report.
|
|
|
|
**Key improvements:**
|
|
- Password policy: 12+ chars, uppercase, lowercase, digit (schema-enforced)
|
|
- Rate limits on auth endpoints (10/min per IP)
|
|
- Refresh token rotation (atomic transaction)
|
|
- User enumeration prevention (401 not 404)
|
|
- Redis authentication required
|
|
- XSS/injection prevention (HTML escaping)
|
|
- Path traversal protection
|
|
- Encryption key for DB secrets (`ENCRYPTION_KEY` required in production)
|
|
- Nginx security headers (HSTS, Permissions-Policy, CSP)
|
|
|
|
### Required Environment Variables
|
|
See `.env.example` for all 100+ variables. Critical ones:
|
|
- `V2_POSTGRES_PASSWORD`, `REDIS_PASSWORD`
|
|
- `JWT_ACCESS_SECRET`, `JWT_REFRESH_SECRET`
|
|
- `ENCRYPTION_KEY` (must differ from JWT secrets)
|
|
- `LISTMONK_SYNC_ENABLED` (opt-in newsletter sync)
|
|
- `EMAIL_TEST_MODE` (MailHog vs SMTP)
|
|
- `ENABLE_MEDIA_FEATURES` (media manager)
|
|
|
|
### Production Deployment
|
|
- **Tunneling:** Pangolin with Newt container (Cloudflare alternative)
|
|
- **SSL/TLS:** Handled by tunnel provider (Pangolin/Cloudflare)
|
|
- **Docker Networking:** All containers share `changemaker-lite` bridge network, reference by container name
|
|
- **Monitoring:** Enable with `docker compose --profile monitoring up -d`
|
|
- **Backups:** Run `./scripts/backup.sh` (PostgreSQL + Listmonk + uploads, optional S3 upload)
|
|
|
|
#### Production CORS Configuration
|
|
|
|
When deploying to a production domain via Pangolin tunnel, you MUST update the `.env` file to include the production domain in `CORS_ORIGINS`:
|
|
|
|
```bash
|
|
# Example for betteredmonton.org
|
|
CORS_ORIGINS=http://app.betteredmonton.org,https://app.betteredmonton.org,http://localhost:3000,http://localhost
|
|
|
|
# Also set production mode
|
|
NODE_ENV=production
|
|
```
|
|
|
|
Without this, API requests from the production domain will fail CORS validation. After updating `.env`, restart the API container:
|
|
|
|
```bash
|
|
docker compose restart api
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Production 403/302 Errors - Pangolin Resources
|
|
|
|
**Symptom:** All API endpoints return 302 redirects to Pangolin authentication page, or 403 Forbidden errors.
|
|
|
|
**Root Cause:** Pangolin tunnel resources are configured with authentication enabled (default behavior).
|
|
|
|
**Fix:** Log in to your Pangolin dashboard and edit each resource:
|
|
1. Navigate to **Resources** → **Public**
|
|
2. For each resource (app, api, media, docs, etc.), click **Edit**
|
|
3. Change **Authentication** setting to **"Not Protected"** (or "Public Access"/"No Authentication")
|
|
4. Save changes
|
|
|
|
**Critical resources to fix first:**
|
|
- `api.betteredmonton.org` - Main API (all endpoints fail without this)
|
|
- `app.betteredmonton.org` - Admin GUI + public pages
|
|
- `media.betteredmonton.org` - Media API
|
|
|
|
**Verification:**
|
|
```bash
|
|
# Should return JSON, NOT a 302 redirect
|
|
curl https://api.betteredmonton.org/api/health
|
|
```
|
|
|
|
**See Also:** `PRODUCTION_403_FIX.md` for detailed step-by-step instructions.
|
|
|
|
### CORS Errors in Production
|
|
|
|
**Symptom:** Browser console shows CORS errors when accessing production domain.
|
|
|
|
**Fix:** Add production domain to `CORS_ORIGINS` in `.env` file (see Production CORS Configuration above).
|
|
|
|
### API Works Locally But Not Via Tunnel
|
|
|
|
Check in order:
|
|
1. **Newt container running:** `docker compose ps newt`
|
|
2. **Newt connected:** `docker compose logs newt --tail 50` (should show successful connection)
|
|
3. **Environment variables set:** `PANGOLIN_SITE_ID`, `PANGOLIN_NEWT_ID`, `PANGOLIN_NEWT_SECRET` in `.env`
|
|
4. **Pangolin resources configured:** All resources set to "Not Protected"
|
|
5. **Nginx running:** `docker compose ps nginx`
|
|
|
|
### Database Connection Failures
|
|
|
|
**Symptom:** API logs show database connection errors.
|
|
|
|
**Fix:**
|
|
1. Check PostgreSQL container: `docker compose ps v2-postgres`
|
|
2. Verify `DATABASE_URL` in `.env` matches container name and port
|
|
3. Check PostgreSQL logs: `docker compose logs v2-postgres --tail 50`
|
|
4. Test connection: `docker compose exec api npx prisma db execute --stdin <<< "SELECT 1"`
|
|
|
|
### Redis Connection Failures
|
|
|
|
**Symptom:** API logs show Redis connection errors, rate limiting doesn't work.
|
|
|
|
**Fix:**
|
|
1. Check Redis container: `docker compose ps redis-changemaker`
|
|
2. Verify `REDIS_PASSWORD` matches in `.env` and `REDIS_URL` format
|
|
3. Check Redis logs: `docker compose logs redis-changemaker --tail 50`
|
|
4. Test connection: `docker compose exec redis-changemaker redis-cli -a $REDIS_PASSWORD ping`
|
|
|
|
---
|
|
|
|
## V1 Reference (Legacy)
|
|
|
|
V1 code archived in `influence/`, `map/`, and `docker-compose.v1.yml`. Two independent Express apps using NocoDB REST API. See individual README files for V1 documentation:
|
|
- `influence/README.MD` — Features, config, campaign management
|
|
- `map/README.md` — Features, config, setup instructions
|
|
- Both use session-based auth, bcryptjs passwords, Bull job queues
|
|
|
|
---
|
|
|
|
## Key Configuration Files
|
|
|
|
### Infrastructure
|
|
- `docker-compose.yml` — V2 orchestration (20+ services, monitoring profile)
|
|
- `.env` / `.env.example` — Environment variables (100+ vars)
|
|
|
|
### Database
|
|
- `api/prisma/schema.prisma` — Main schema (30+ Prisma models)
|
|
- `api/drizzle.config.ts` — Drizzle config for media tables
|
|
- `api/prisma/seed.ts` — Database seeding
|
|
|
|
### Nginx
|
|
- `nginx/nginx.conf` — Global config + security headers
|
|
- `nginx/conf.d/default.conf` — Subdomain routing (12+ subdomains)
|
|
- `nginx/conf.d/api.conf` — API reverse proxy (Express + Fastify)
|
|
- `nginx/conf.d/services.conf` — Service proxies
|
|
|
|
### Monitoring
|
|
- `configs/prometheus/prometheus.yml` — Scrape targets + global config
|
|
- `configs/prometheus/alerts.yml` — Alert rules (12 rules)
|
|
- `configs/grafana/` — 3 pre-configured dashboards
|
|
- `configs/alertmanager/alertmanager.yml` — Alert routing
|
|
|
|
### Documentation
|
|
- `CLAUDE.md` — Project-wide instructions (this file)
|
|
- `V2_PLAN.md` — Full 14-phase roadmap
|
|
- `SECURITY_AUDIT_2025-02-11.md` — Security audit report
|
|
- `MEMORY.md` — Development patterns and gotchas
|