changemaker.lite/V2_PLAN.md

V2 Roadmap — Changemaker Lite

This document is the full roadmap for the v2 rebuild of Changemaker Lite, a self-hosted political campaign platform. V1 consisted of two separate Express apps (Influence and Map) using NocoDB as a data layer. V2 consolidates everything into a single unified TypeScript API with a React admin interface.

---

Architecture Overview

• Single unified Express.js API (TypeScript, port 4000) with Prisma ORM + PostgreSQL 16
• React Admin GUI (Vite + Ant Design + Zustand, port 3000)
• Nginx reverse proxy (subdomain routing: *.cmlite.org)
• NocoDB v2 as read-only data browser on port 8091
• JWT auth (access + refresh tokens), bcrypt passwords
• BullMQ for Influence advocacy emails, Listmonk for newsletters
• Redis for caching, rate limiting, BullMQ

---

Directory Structure

changemaker.lite/
├── api/                  # Unified Express.js API (TypeScript)
│   ├── prisma/           # Schema, migrations, seed
│   └── src/
│       ├── config/       # env.ts, database.ts, redis.ts
│       ├── middleware/    # error-handler, validate, rate-limit, auth, rbac
│       ├── modules/
│       │   ├── auth/     # auth.service, auth.routes, auth.schemas
│       │   ├── users/    # users.service, users.routes, users.schemas
│       │   ├── influence/
│       │   │   ├── campaigns/
│       │   │   ├── representatives/
│       │   │   ├── responses/
│       │   │   └── postal-codes/
│       │   └── map/
│       │       ├── locations/
│       │       ├── shifts/
│       │       └── cuts/
│       ├── types/        # express.d.ts
│       └── utils/        # logger.ts, metrics.ts
├── admin/                # React Admin (Vite + Ant Design)
│   └── src/
│       ├── components/
│       ├── pages/
│       ├── stores/
│       └── services/
├── nginx/                # Reverse proxy config
├── public-web/           # Public landing pages
└── docker-compose.yml    # V2 orchestration

---

Schema Summary

27+ models organized into the following groups:

• Auth & Users — User, RefreshToken
• Influence — Campaign, CampaignEmail, Representative, RepresentativeResponse, ResponseUpvote, CustomRecipient, PostalCodeCache, EmailLog, EmailVerification, Call
• Map — Location, Shift, ShiftSignup, Cut, MapSettings
• Canvassing — CanvassSession, CanvassVisit, TrackingSession, TrackPoint
• Landing Pages — LandingPage, PageBlock
• Settings — SiteSettings

---

Auth Flow

• JWT-based with access tokens (15min) and refresh tokens (7 days, stored in DB)
• Login: verify bcrypt hash, generate token pair, return tokens + user
• Refresh: validate refresh token, rotate (invalidate old, issue new), return new pair
• Roles: SUPER_ADMIN, INFLUENCE_ADMIN, MAP_ADMIN, USER, TEMP

---

Email Architecture

• BullMQ job queue for Influence advocacy emails (async send via SMTP)
• Listmonk for newsletter/marketing emails
• MailHog for dev email capture (EMAIL_TEST_MODE=true)

---

Nginx Routing

Subdomain	Target
app.cmlite.org	Admin React app (port 3000)
api.cmlite.org	Express API (port 4000)
db.cmlite.org	NocoDB read-only (port 8091)
docs.cmlite.org	MkDocs (port 4003)
code.cmlite.org	Code Server (port 8888)
n8n.cmlite.org	n8n Workflows (port 5678)
git.cmlite.org	Gitea (port 3030)
home.cmlite.org	Homepage (port 3010)
grafana.cmlite.org	Grafana (port 3001)
listmonk.cmlite.org	Listmonk (port 9001)
cmlite.org	Public static site (MkDocs)

---

Phase Checklist

Phase 1: Foundation [x] COMPLETE

• Initialize api/ with TypeScript, Express, Prisma
• Create prisma/schema.prisma with all models
• Set up config (env.ts, database.ts, redis.ts)
• Create middleware (error-handler.ts, validate.ts, rate-limit.ts)
• Set up utils (logger.ts, metrics.ts)
• Create server.ts with health check and metrics
• Initialize admin/ with Vite + React + Ant Design
• Create docker-compose.yml for v2
• Create .env.example
• Backup v1 to docker-compose.v1.yml

Note: DB migrations are pending until docker compose up runs PostgreSQL

---

Phase 2: Auth + User Management [x] COMPLETE

• Express Request type augmentation (express.d.ts)
• Auth schemas (Zod validation)
• Auth service (login, register, refresh, logout)
• Auth middleware (JWT verification)
• RBAC middleware (role-based access)
• Auth routes (login, register, refresh, logout, me)
• User schemas (Zod validation)
• User service (CRUD + pagination)
• User routes (list, get, create, update, delete)
• Wire routes into server.ts

---

Phase 3: Admin GUI Foundation [x] COMPLETE

• Zustand auth store with token management
• Login page
• Protected route wrapper
• App layout with sidebar navigation
• User management page (CRUD table)
• API client with interceptors (auto-refresh)

---

Phase 4: Influence — Campaigns [x] COMPLETE

• Campaign schemas (Zod validation)
• Campaign service (CRUD, slug generation, highlight toggling)
• Campaign routes with admin protection
• Admin campaign management page (table, filters, create/edit/delete)
• Public campaign view page (deferred to later phase)

---

Phase 5: Influence — Representatives + Postal Codes [x] COMPLETE

• Postal code schemas (normalize, validate all Canadian postal codes)
• Postal code cache service (upsert, find, paginate, delete)
• Represent API client (typed HTTP client, in-memory rate limiter 55/min)
• Representative schemas (list/filter validation)
• Representative service (cache-first lookup, fire-and-forget cache write, admin CRUD, cache stats)
• Representative routes (public: lookup + health check; admin: list, stats, detail, delete)
• Admin representatives page (lookup, stats cards, table with filters, detail modal)
• Public postal code to representative lookup UI (deferred to later phase)

---

Phase 6: Influence — Email Sending [x] COMPLETE

• BullMQ email queue setup (email-queue.service.ts)
• Email worker (SMTP send via nodemailer, email.service.ts)
• Campaign email service (compose, queue, track)
• Campaign email routes (public: send-email, track-mailto; admin: list, stats)
• Email queue admin routes (stats, pause, resume, clean)
• Admin email queue page (stats cards, pause/resume, clean)
• Admin campaign emails drawer (stats + email list from CampaignsPage)
• Email rate limiting (30 req/hour per IP)
• Public email sending UI (SMTP + mailto fallback) — deferred to later phase

---

Phase 7: Influence — Response Wall + Public Campaign View [x]

• Response service (submit, moderate, verify)
• Response routes (3 routers: campaign-public, response-public, admin)
• Email verification for responses (HTML templates, verify/report endpoints)
• Admin moderation interface (ResponsesPage with filters, approve/reject/delete)
• Public response wall display (ResponseWallPage with sort, filter, submit modal)
• Upvoting system (IP + user dedup, optimistic UI)
• Public campaign page (CampaignPage with postal code lookup, email sending)
• PublicLayout with light theme for public pages
• Campaign public details endpoint (findBySlugPublic)

---

Phase 8: Map — Locations [x]

• Multi-provider geocoding service (Nominatim, ArcGIS, Photon, Mapbox)
• Location service (CRUD, geocoding, stats, bulk operations)
• Location routes (admin + public)
• Map settings service + routes (singleton config)
• Admin LocationsPage (table, stats, create/edit, geocode button)
• Admin MapSettingsPage (center/zoom, walk sheet config)
• Public Leaflet.js map (circle markers, color-coded support levels, multi-unit grouping)
• CSV import/export with flexible column mapping

---

Phase 9: Map — Shifts [x] COMPLETE

• Shift service (CRUD, signup management)
• Shift routes (admin + public)
• Admin shift management (ShiftsPage with signups drawer)
• Public shift calendar + signup
• Temp user creation on public signup
• Confirmation emails for signups

---

Phase 10: Walk Sheets & QR Codes [x] COMPLETE

• QR code generation endpoint (GET /api/qr)
• Walk sheet page (printable form with QR codes)
• Cut export page (printable location report)
• Sidebar navigation + route wiring

---

Phase 11: Listmonk Integration [x] COMPLETE

• Listmonk API client service (typed HTTP, basic auth)
• Sync service (campaign participants, locations, users → subscriber lists)
• Admin routes (status, stats, sync triggers, test connection, reinitialize)
• Admin ListmonkPage (status, sync buttons, list stats)
• Sidebar navigation + route wiring
• Proton Mail SMTP configuration (listmonk-init auto-configures via SQL)

---

Phase 12: Landing Page Builder [x] COMPLETE

• Landing page service (CRUD, slug generation, MkDocs export)
• Page block service (seed blocks, CRUD, library API)
• GrapesJS editor integration (custom blocks, Ctrl+S save, error boundary)
• Admin page builder UI (LandingPagesPage, PageEditorPage full-screen)
• Public page rendering (/p/:slug)
• MkDocs export (Jinja2 Material override template, themed + standalone modes)

---

MkDocs + Code Server + Docs Editor [x] COMPLETE

• Docker port fix (MkDocs → 4003, avoiding API conflict)
• API mkdocs volume mount for override file sync
• Nginx code.cmlite.org + X-Frame-Options per-server-block
• Docs API routes (status, config health check for MkDocs/Code Server)
• DocsEditorPage (split-view with Code Server + MkDocs iframes, draggable divider)
• DocsPage (management — status cards, MkDocs export table)

---

Phase 13: Settings + Branding [x] COMPLETE

• SiteSettings Prisma model (singleton, organization/theme/feature toggles)
• Settings API module (public GET, SUPER_ADMIN PUT)
• Zustand settings store (fetch on app startup, public endpoint)
• Admin SettingsPage with tabs (Organization, Theme, Email, Features)
• Dynamic admin theme (colorPrimary, colorBgBase from settings)
• Dynamic public theme (colors, gradient, footer from settings)
• Dynamic branding (organization name, short name, login subtitle)
• Feature toggle navigation (hide sidebar sections when disabled)

---

Code Editor (Standalone) [x] COMPLETE

• Docker volume mount for entire project directory
• Admin CodeEditorPage with full-bleed Code Server iframe
• Health status check (reuses /api/docs/status endpoint)
• Sidebar navigation under Web submenu
• SUPER_ADMIN access restriction

---

Volunteer Canvassing System [x] COMPLETE

• CanvassSession + CanvassVisit Prisma models (session lifecycle, visit outcomes)
• TrackingSession + TrackPoint Prisma models (GPS trail recording)
• Canvass API — volunteer routes (start/end session, record visits, walking route)
• Canvass API — admin routes (dashboard stats, activity feed, cut progress, leaderboard)
• Walking route algorithm (nearest-neighbor with haversine distance)
• Canvass visit rate limiter (30/min per IP)
• Abandoned session cleanup (startup + hourly interval, ACTIVE > 12h → ABANDONED)
• GPS tracking routes (volunteer + admin)
• Old tracking data cleanup (startup + daily, 30-day retention)
• Stale tracking session cleanup (no data for 2h, hourly check)
• VolunteerLayout (top-nav, dark theme, mobile hamburger menu)
• Volunteer portal — full-screen canvass map (Leaflet, GPS, markers, route, bottom sheet visit recording)
• Volunteer portal — activity page (visit history + outcome breakdown)
• Admin CanvassDashboardPage (stats, activity feed, cut progress, leaderboard)
• Admin WalkSheetPage enhancements
• ShiftsPage cutId dropdown (link shifts to cuts)
• Role-aware login redirect (ADMIN_ROLES → /app, USER/TEMP → /volunteer)
• Shift.cutId optional relation — shifts without a cut don't appear in volunteer assignments

---

Platform Service Integration [x] COMPLETE

• Services API module (health check NocoDB, n8n, Gitea + config endpoint)
• Admin NocoDBPage (iframe to db.cmlite.org, status badge, open in new tab)
• Admin N8nPage (iframe to n8n.cmlite.org, status badge, open in new tab)
• Admin GiteaPage (iframe to git.cmlite.org, status badge, open in new tab)
• Sidebar "Services" submenu (Database, Workflows, Git)
• Nginx CSP frame-ancestors for NocoDB, n8n, Gitea (iframe embedding from admin)
• Embed proxy ports (8881-8883) for X-Frame-Options stripping
• buildServiceUrl() helper for dynamic iframe URLs

---

Phase 14: Monitoring + DevOps [x] COMPLETE

Pangolin Tunnel (replaced Cloudflare Tunnel):

• Pangolin Integration API client (api/src/services/pangolin.client.ts)
• Admin pangolin routes — status, config, sites, resources, setup, sync, delete
• Admin PangolinPage — setup wizard + resource status dashboard
• Newt container in docker-compose.yml
• Env vars: PANGOLIN_API_URL, API_KEY, ORG_ID, SITE_ID, ENDPOINT, NEWT_ID, NEWT_SECRET
• Retired Cloudflare scripts → scripts/legacy/
• Sidebar "Tunnel" nav item under Services

Prometheus Metrics:

• 12 domain-specific cm_* metrics in api/src/utils/metrics.ts
• Email: cm_emails_sent_total, cm_emails_failed_total, cm_email_queue_size, cm_email_send_duration_seconds
• Auth: cm_login_attempts_total, cm_active_sessions
• Campaigns: cm_campaign_emails_total, cm_response_submissions_total
• Canvass: cm_canvass_visits_total, cm_active_canvass_sessions, cm_shift_signups_total
• Services: cm_external_service_up
• Instrumented: email-queue, auth, campaign-emails, responses, canvass, shifts, services

Monitoring Configs:

• Prometheus: V2 API scrape job (changemaker-v2-api:4000), removed V1 influence-app
• Alerts: rewritten for V2 cm_* / http_* metric names
• Alertmanager: Gotify webhook receiver (commented, ready to enable)
• Grafana dashboards: system-health (updated), application-overview (new), api-performance (new)

Docker Healthchecks:

• API (wget /api/health, 15s)
• Admin (wget /, 30s)
• Nginx (wget /, 30s)
• NocoDB (wget /api/v1/health, 30s)
• n8n (wget /healthz, 30s)
• Gitea (curl /, 30s)
• Listmonk (wget /, 30s)

Backup:

• scripts/backup.sh — V2 PostgreSQL dump + Listmonk dump + uploads archive
• Manifest with timestamps, sizes, sha256 checksums
• Configurable retention (default 30 days)
• Optional S3 upload (--s3 flag)

---

Phase 15: Testing + Polish [ ]

• API integration tests (Jest/Vitest)
• Admin E2E tests
• Performance optimization
• Security audit
• Documentation updates

PHASE 1: Extras

• Add apache answers
• Add geo-tracking and blocking
• Add in the video platform
• Add in excalidraw
• Add in chats - integrate into canvass application