389 lines
16 KiB
Markdown
389 lines
16 KiB
Markdown
# V2 Roadmap — Changemaker Lite
|
|
|
|
This document is the full roadmap for the v2 rebuild of Changemaker Lite, a self-hosted political campaign platform. V1 consisted of two separate Express apps (Influence and Map) using NocoDB as a data layer. V2 consolidates everything into a single unified TypeScript API with a React admin interface.
|
|
|
|
---
|
|
|
|
## Architecture Overview
|
|
|
|
- **Single unified Express.js API** (TypeScript, port 4000) with Prisma ORM + PostgreSQL 16
|
|
- **React Admin GUI** (Vite + Ant Design + Zustand, port 3000)
|
|
- **Nginx reverse proxy** (subdomain routing: *.cmlite.org)
|
|
- **NocoDB v2** as read-only data browser on port 8091
|
|
- **JWT auth** (access + refresh tokens), bcrypt passwords
|
|
- **BullMQ** for Influence advocacy emails, Listmonk for newsletters
|
|
- **Redis** for caching, rate limiting, BullMQ
|
|
|
|
---
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
changemaker.lite/
|
|
├── api/ # Unified Express.js API (TypeScript)
|
|
│ ├── prisma/ # Schema, migrations, seed
|
|
│ └── src/
|
|
│ ├── config/ # env.ts, database.ts, redis.ts
|
|
│ ├── middleware/ # error-handler, validate, rate-limit, auth, rbac
|
|
│ ├── modules/
|
|
│ │ ├── auth/ # auth.service, auth.routes, auth.schemas
|
|
│ │ ├── users/ # users.service, users.routes, users.schemas
|
|
│ │ ├── influence/
|
|
│ │ │ ├── campaigns/
|
|
│ │ │ ├── representatives/
|
|
│ │ │ ├── responses/
|
|
│ │ │ └── postal-codes/
|
|
│ │ └── map/
|
|
│ │ ├── locations/
|
|
│ │ ├── shifts/
|
|
│ │ └── cuts/
|
|
│ ├── types/ # express.d.ts
|
|
│ └── utils/ # logger.ts, metrics.ts
|
|
├── admin/ # React Admin (Vite + Ant Design)
|
|
│ └── src/
|
|
│ ├── components/
|
|
│ ├── pages/
|
|
│ ├── stores/
|
|
│ └── services/
|
|
├── nginx/ # Reverse proxy config
|
|
├── public-web/ # Public landing pages
|
|
└── docker-compose.yml # V2 orchestration
|
|
```
|
|
|
|
---
|
|
|
|
## Schema Summary
|
|
|
|
27+ models organized into the following groups:
|
|
|
|
- **Auth & Users** — User, RefreshToken
|
|
- **Influence** — Campaign, CampaignEmail, Representative, RepresentativeResponse, ResponseUpvote, CustomRecipient, PostalCodeCache, EmailLog, EmailVerification, Call
|
|
- **Map** — Location, Shift, ShiftSignup, Cut, MapSettings
|
|
- **Canvassing** — CanvassSession, CanvassVisit, TrackingSession, TrackPoint
|
|
- **Landing Pages** — LandingPage, PageBlock
|
|
- **Settings** — SiteSettings
|
|
|
|
---
|
|
|
|
## Auth Flow
|
|
|
|
- JWT-based with access tokens (15min) and refresh tokens (7 days, stored in DB)
|
|
- **Login:** verify bcrypt hash, generate token pair, return tokens + user
|
|
- **Refresh:** validate refresh token, rotate (invalidate old, issue new), return new pair
|
|
- **Roles:** SUPER_ADMIN, INFLUENCE_ADMIN, MAP_ADMIN, USER, TEMP
|
|
|
|
---
|
|
|
|
## Email Architecture
|
|
|
|
- **BullMQ job queue** for Influence advocacy emails (async send via SMTP)
|
|
- **Listmonk** for newsletter/marketing emails
|
|
- **MailHog** for dev email capture (EMAIL_TEST_MODE=true)
|
|
|
|
---
|
|
|
|
## Nginx Routing
|
|
|
|
| Subdomain | Target |
|
|
|-----------------------|--------------------------------|
|
|
| app.cmlite.org | Admin React app (port 3000) |
|
|
| api.cmlite.org | Express API (port 4000) |
|
|
| db.cmlite.org | NocoDB read-only (port 8091) |
|
|
| docs.cmlite.org | MkDocs (port 4003) |
|
|
| code.cmlite.org | Code Server (port 8888) |
|
|
| n8n.cmlite.org | n8n Workflows (port 5678) |
|
|
| git.cmlite.org | Gitea (port 3030) |
|
|
| home.cmlite.org | Homepage (port 3010) |
|
|
| grafana.cmlite.org | Grafana (port 3001) |
|
|
| listmonk.cmlite.org | Listmonk (port 9001) |
|
|
| cmlite.org | Public static site (MkDocs) |
|
|
|
|
---
|
|
|
|
## Phase Checklist
|
|
|
|
### Phase 1: Foundation [x] COMPLETE
|
|
|
|
- [x] Initialize api/ with TypeScript, Express, Prisma
|
|
- [x] Create prisma/schema.prisma with all models
|
|
- [x] Set up config (env.ts, database.ts, redis.ts)
|
|
- [x] Create middleware (error-handler.ts, validate.ts, rate-limit.ts)
|
|
- [x] Set up utils (logger.ts, metrics.ts)
|
|
- [x] Create server.ts with health check and metrics
|
|
- [x] Initialize admin/ with Vite + React + Ant Design
|
|
- [x] Create docker-compose.yml for v2
|
|
- [x] Create .env.example
|
|
- [x] Backup v1 to docker-compose.v1.yml
|
|
|
|
> Note: DB migrations are pending until `docker compose up` runs PostgreSQL
|
|
|
|
---
|
|
|
|
### Phase 2: Auth + User Management [x] COMPLETE
|
|
|
|
- [x] Express Request type augmentation (express.d.ts)
|
|
- [x] Auth schemas (Zod validation)
|
|
- [x] Auth service (login, register, refresh, logout)
|
|
- [x] Auth middleware (JWT verification)
|
|
- [x] RBAC middleware (role-based access)
|
|
- [x] Auth routes (login, register, refresh, logout, me)
|
|
- [x] User schemas (Zod validation)
|
|
- [x] User service (CRUD + pagination)
|
|
- [x] User routes (list, get, create, update, delete)
|
|
- [x] Wire routes into server.ts
|
|
|
|
---
|
|
|
|
### Phase 3: Admin GUI Foundation [x] COMPLETE
|
|
|
|
- [x] Zustand auth store with token management
|
|
- [x] Login page
|
|
- [x] Protected route wrapper
|
|
- [x] App layout with sidebar navigation
|
|
- [x] User management page (CRUD table)
|
|
- [x] API client with interceptors (auto-refresh)
|
|
|
|
---
|
|
|
|
### Phase 4: Influence — Campaigns [x] COMPLETE
|
|
|
|
- [x] Campaign schemas (Zod validation)
|
|
- [x] Campaign service (CRUD, slug generation, highlight toggling)
|
|
- [x] Campaign routes with admin protection
|
|
- [x] Admin campaign management page (table, filters, create/edit/delete)
|
|
- [ ] Public campaign view page (deferred to later phase)
|
|
|
|
---
|
|
|
|
### Phase 5: Influence — Representatives + Postal Codes [x] COMPLETE
|
|
|
|
- [x] Postal code schemas (normalize, validate all Canadian postal codes)
|
|
- [x] Postal code cache service (upsert, find, paginate, delete)
|
|
- [x] Represent API client (typed HTTP client, in-memory rate limiter 55/min)
|
|
- [x] Representative schemas (list/filter validation)
|
|
- [x] Representative service (cache-first lookup, fire-and-forget cache write, admin CRUD, cache stats)
|
|
- [x] Representative routes (public: lookup + health check; admin: list, stats, detail, delete)
|
|
- [x] Admin representatives page (lookup, stats cards, table with filters, detail modal)
|
|
- [ ] Public postal code to representative lookup UI (deferred to later phase)
|
|
|
|
---
|
|
|
|
### Phase 6: Influence — Email Sending [x] COMPLETE
|
|
|
|
- [x] BullMQ email queue setup (email-queue.service.ts)
|
|
- [x] Email worker (SMTP send via nodemailer, email.service.ts)
|
|
- [x] Campaign email service (compose, queue, track)
|
|
- [x] Campaign email routes (public: send-email, track-mailto; admin: list, stats)
|
|
- [x] Email queue admin routes (stats, pause, resume, clean)
|
|
- [x] Admin email queue page (stats cards, pause/resume, clean)
|
|
- [x] Admin campaign emails drawer (stats + email list from CampaignsPage)
|
|
- [x] Email rate limiting (30 req/hour per IP)
|
|
- [ ] Public email sending UI (SMTP + mailto fallback) — deferred to later phase
|
|
|
|
---
|
|
|
|
### Phase 7: Influence — Response Wall + Public Campaign View [x]
|
|
|
|
- [x] Response service (submit, moderate, verify)
|
|
- [x] Response routes (3 routers: campaign-public, response-public, admin)
|
|
- [x] Email verification for responses (HTML templates, verify/report endpoints)
|
|
- [x] Admin moderation interface (ResponsesPage with filters, approve/reject/delete)
|
|
- [x] Public response wall display (ResponseWallPage with sort, filter, submit modal)
|
|
- [x] Upvoting system (IP + user dedup, optimistic UI)
|
|
- [x] Public campaign page (CampaignPage with postal code lookup, email sending)
|
|
- [x] PublicLayout with light theme for public pages
|
|
- [x] Campaign public details endpoint (findBySlugPublic)
|
|
|
|
---
|
|
|
|
### Phase 8: Map — Locations [x]
|
|
|
|
- [x] Multi-provider geocoding service (Nominatim, ArcGIS, Photon, Mapbox)
|
|
- [x] Location service (CRUD, geocoding, stats, bulk operations)
|
|
- [x] Location routes (admin + public)
|
|
- [x] Map settings service + routes (singleton config)
|
|
- [x] Admin LocationsPage (table, stats, create/edit, geocode button)
|
|
- [x] Admin MapSettingsPage (center/zoom, walk sheet config)
|
|
- [x] Public Leaflet.js map (circle markers, color-coded support levels, multi-unit grouping)
|
|
- [x] CSV import/export with flexible column mapping
|
|
|
|
---
|
|
|
|
### Phase 9: Map — Shifts [x] COMPLETE
|
|
|
|
- [x] Shift service (CRUD, signup management)
|
|
- [x] Shift routes (admin + public)
|
|
- [x] Admin shift management (ShiftsPage with signups drawer)
|
|
- [x] Public shift calendar + signup
|
|
- [x] Temp user creation on public signup
|
|
- [x] Confirmation emails for signups
|
|
|
|
---
|
|
|
|
### Phase 10: Walk Sheets & QR Codes [x] COMPLETE
|
|
|
|
- [x] QR code generation endpoint (GET /api/qr)
|
|
- [x] Walk sheet page (printable form with QR codes)
|
|
- [x] Cut export page (printable location report)
|
|
- [x] Sidebar navigation + route wiring
|
|
|
|
---
|
|
|
|
### Phase 11: Listmonk Integration [x] COMPLETE
|
|
|
|
- [x] Listmonk API client service (typed HTTP, basic auth)
|
|
- [x] Sync service (campaign participants, locations, users → subscriber lists)
|
|
- [x] Admin routes (status, stats, sync triggers, test connection, reinitialize)
|
|
- [x] Admin ListmonkPage (status, sync buttons, list stats)
|
|
- [x] Sidebar navigation + route wiring
|
|
- [x] Proton Mail SMTP configuration (listmonk-init auto-configures via SQL)
|
|
|
|
---
|
|
|
|
### Phase 12: Landing Page Builder [x] COMPLETE
|
|
|
|
- [x] Landing page service (CRUD, slug generation, MkDocs export)
|
|
- [x] Page block service (seed blocks, CRUD, library API)
|
|
- [x] GrapesJS editor integration (custom blocks, Ctrl+S save, error boundary)
|
|
- [x] Admin page builder UI (LandingPagesPage, PageEditorPage full-screen)
|
|
- [x] Public page rendering (/p/:slug)
|
|
- [x] MkDocs export (Jinja2 Material override template, themed + standalone modes)
|
|
|
|
---
|
|
|
|
### MkDocs + Code Server + Docs Editor [x] COMPLETE
|
|
|
|
- [x] Docker port fix (MkDocs → 4003, avoiding API conflict)
|
|
- [x] API mkdocs volume mount for override file sync
|
|
- [x] Nginx code.cmlite.org + X-Frame-Options per-server-block
|
|
- [x] Docs API routes (status, config health check for MkDocs/Code Server)
|
|
- [x] DocsEditorPage (split-view with Code Server + MkDocs iframes, draggable divider)
|
|
- [x] DocsPage (management — status cards, MkDocs export table)
|
|
|
|
---
|
|
|
|
### Phase 13: Settings + Branding [x] COMPLETE
|
|
|
|
- [x] SiteSettings Prisma model (singleton, organization/theme/feature toggles)
|
|
- [x] Settings API module (public GET, SUPER_ADMIN PUT)
|
|
- [x] Zustand settings store (fetch on app startup, public endpoint)
|
|
- [x] Admin SettingsPage with tabs (Organization, Theme, Email, Features)
|
|
- [x] Dynamic admin theme (colorPrimary, colorBgBase from settings)
|
|
- [x] Dynamic public theme (colors, gradient, footer from settings)
|
|
- [x] Dynamic branding (organization name, short name, login subtitle)
|
|
- [x] Feature toggle navigation (hide sidebar sections when disabled)
|
|
|
|
---
|
|
|
|
### Code Editor (Standalone) [x] COMPLETE
|
|
|
|
- [x] Docker volume mount for entire project directory
|
|
- [x] Admin CodeEditorPage with full-bleed Code Server iframe
|
|
- [x] Health status check (reuses /api/docs/status endpoint)
|
|
- [x] Sidebar navigation under Web submenu
|
|
- [x] SUPER_ADMIN access restriction
|
|
|
|
---
|
|
|
|
### Volunteer Canvassing System [x] COMPLETE
|
|
|
|
- [x] CanvassSession + CanvassVisit Prisma models (session lifecycle, visit outcomes)
|
|
- [x] TrackingSession + TrackPoint Prisma models (GPS trail recording)
|
|
- [x] Canvass API — volunteer routes (start/end session, record visits, walking route)
|
|
- [x] Canvass API — admin routes (dashboard stats, activity feed, cut progress, leaderboard)
|
|
- [x] Walking route algorithm (nearest-neighbor with haversine distance)
|
|
- [x] Canvass visit rate limiter (30/min per IP)
|
|
- [x] Abandoned session cleanup (startup + hourly interval, ACTIVE > 12h → ABANDONED)
|
|
- [x] GPS tracking routes (volunteer + admin)
|
|
- [x] Old tracking data cleanup (startup + daily, 30-day retention)
|
|
- [x] Stale tracking session cleanup (no data for 2h, hourly check)
|
|
- [x] VolunteerLayout (top-nav, dark theme, mobile hamburger menu)
|
|
- [x] Volunteer portal — full-screen canvass map (Leaflet, GPS, markers, route, bottom sheet visit recording)
|
|
- [x] Volunteer portal — activity page (visit history + outcome breakdown)
|
|
- [x] Admin CanvassDashboardPage (stats, activity feed, cut progress, leaderboard)
|
|
- [x] Admin WalkSheetPage enhancements
|
|
- [x] ShiftsPage cutId dropdown (link shifts to cuts)
|
|
- [x] Role-aware login redirect (ADMIN_ROLES → /app, USER/TEMP → /volunteer)
|
|
- [x] Shift.cutId optional relation — shifts without a cut don't appear in volunteer assignments
|
|
|
|
---
|
|
|
|
### Platform Service Integration [x] COMPLETE
|
|
|
|
- [x] Services API module (health check NocoDB, n8n, Gitea + config endpoint)
|
|
- [x] Admin NocoDBPage (iframe to db.cmlite.org, status badge, open in new tab)
|
|
- [x] Admin N8nPage (iframe to n8n.cmlite.org, status badge, open in new tab)
|
|
- [x] Admin GiteaPage (iframe to git.cmlite.org, status badge, open in new tab)
|
|
- [x] Sidebar "Services" submenu (Database, Workflows, Git)
|
|
- [x] Nginx CSP frame-ancestors for NocoDB, n8n, Gitea (iframe embedding from admin)
|
|
- [x] Embed proxy ports (8881-8883) for X-Frame-Options stripping
|
|
- [x] `buildServiceUrl()` helper for dynamic iframe URLs
|
|
|
|
---
|
|
|
|
### Phase 14: Monitoring + DevOps [x] COMPLETE
|
|
|
|
**Pangolin Tunnel (replaced Cloudflare Tunnel):**
|
|
- [x] Pangolin Integration API client (`api/src/services/pangolin.client.ts`)
|
|
- [x] Admin pangolin routes — status, config, sites, resources, setup, sync, delete
|
|
- [x] Admin PangolinPage — setup wizard + resource status dashboard
|
|
- [x] Newt container in docker-compose.yml
|
|
- [x] Env vars: PANGOLIN_API_URL, API_KEY, ORG_ID, SITE_ID, ENDPOINT, NEWT_ID, NEWT_SECRET
|
|
- [x] Retired Cloudflare scripts → `scripts/legacy/`
|
|
- [x] Sidebar "Tunnel" nav item under Services
|
|
|
|
**Prometheus Metrics:**
|
|
- [x] 12 domain-specific `cm_*` metrics in `api/src/utils/metrics.ts`
|
|
- [x] Email: cm_emails_sent_total, cm_emails_failed_total, cm_email_queue_size, cm_email_send_duration_seconds
|
|
- [x] Auth: cm_login_attempts_total, cm_active_sessions
|
|
- [x] Campaigns: cm_campaign_emails_total, cm_response_submissions_total
|
|
- [x] Canvass: cm_canvass_visits_total, cm_active_canvass_sessions, cm_shift_signups_total
|
|
- [x] Services: cm_external_service_up
|
|
- [x] Instrumented: email-queue, auth, campaign-emails, responses, canvass, shifts, services
|
|
|
|
**Monitoring Configs:**
|
|
- [x] Prometheus: V2 API scrape job (`changemaker-v2-api:4000`), removed V1 influence-app
|
|
- [x] Alerts: rewritten for V2 `cm_*` / `http_*` metric names
|
|
- [x] Alertmanager: Gotify webhook receiver (commented, ready to enable)
|
|
- [x] Grafana dashboards: system-health (updated), application-overview (new), api-performance (new)
|
|
|
|
**Docker Healthchecks:**
|
|
- [x] API (wget /api/health, 15s)
|
|
- [x] Admin (wget /, 30s)
|
|
- [x] Nginx (wget /, 30s)
|
|
- [x] NocoDB (wget /api/v1/health, 30s)
|
|
- [x] n8n (wget /healthz, 30s)
|
|
- [x] Gitea (curl /, 30s)
|
|
- [x] Listmonk (wget /, 30s)
|
|
|
|
**Backup:**
|
|
- [x] `scripts/backup.sh` — V2 PostgreSQL dump + Listmonk dump + uploads archive
|
|
- [x] Manifest with timestamps, sizes, sha256 checksums
|
|
- [x] Configurable retention (default 30 days)
|
|
- [x] Optional S3 upload (--s3 flag)
|
|
|
|
---
|
|
|
|
### Phase 15: Testing + Polish [IN PROGRESS]
|
|
|
|
**Media Admin Features (Feb 2026) [COMPLETE]:**
|
|
- [x] Quick Action Buttons — Edit, preview, analytics, schedule, duplicate, preview links (24h JWT), reset analytics
|
|
- [x] Scheduled Publishing — BullMQ job queue, timezone support (11 zones), calendar view, publish/unpublish automation
|
|
- [x] Video Analytics — Views, watch time, completion rate, traffic sources, registered viewers tracking
|
|
- [x] Privacy & Compliance — IP hashing (SHA-256), user agent truncation, 90-day retention, GDPR-compliant
|
|
- [x] UI/UX Polish — Keyboard shortcuts (E/P/A/S), hover overlays, skeleton loading, error handling, mobile responsive
|
|
- [x] Documentation — MEDIA_ADMIN_FEATURES.md, VIDEO_ANALYTICS_GUIDE.md, api/src/modules/media/README.md
|
|
|
|
**Remaining Testing + Polish:**
|
|
- [ ] API integration tests (Jest/Vitest)
|
|
- [ ] Admin E2E tests
|
|
- [ ] Performance optimization
|
|
- [ ] Security audit (auth-security-reviewer for media features)
|
|
- [ ] UI design review (ui-design-critic for media components)
|
|
|
|
### PHASE 1: Extras
|
|
- [ ] Add apache answers
|
|
- [ ] Add geo-tracking and blocking
|
|
- [ ] Add in the video platform
|
|
- [ ] Add in excalidraw
|
|
- [ ] Add in chats - integrate into canvass application |