bunker-admin 7895ce683e Tonne of debugging - getting ready for the production builds

2026-02-16 10:44:18 -07:00

16 KiB

Raw Blame History

V2 Architecture Overview

Changemaker Lite V2 is built on a modern microservices architecture with a dual API design, React admin interface, and comprehensive observability.

System Architecture

graph TB
    subgraph "User Access"
        Browser[Web Browser]
        VolunteerApp[Volunteer Mobile]
    end

    subgraph "Nginx Reverse Proxy"
        Nginx[Nginx<br/>Subdomain Router]
    end

    subgraph "Frontend Layer"
        AdminGUI[Admin GUI<br/>React + Vite + Ant Design<br/>Port 3000]
        PublicPages[Public Pages<br/>Dark Theme]
        VolunteerPortal[Volunteer Portal<br/>GPS Canvassing]
    end

    subgraph "Backend Layer - Dual API"
        ExpressAPI[Express API<br/>Main Features<br/>Port 4000<br/>Prisma ORM]
        FastifyAPI[Fastify API<br/>Media Library<br/>Port 4100<br/>Drizzle ORM]
    end

    subgraph "Data Layer"
        Postgres[(PostgreSQL 16<br/>27+ Models)]
        Redis[(Redis<br/>Cache + Queues)]
    end

    subgraph "Job Processing"
        EmailQueue[BullMQ<br/>Email Queue]
        GeocodeQueue[BullMQ<br/>Geocoding Queue]
    end

    subgraph "External Services"
        SMTP[SMTP Server<br/>Email Delivery]
        Represent[Represent API<br/>Canadian Reps]
        Geocoding[Geocoding Providers<br/>6 Services]
        Listmonk[Listmonk<br/>Newsletter Platform]
    end

    subgraph "Observability"
        Prometheus[Prometheus<br/>Metrics]
        Grafana[Grafana<br/>Dashboards]
        Alertmanager[Alertmanager<br/>Notifications]
    end

    Browser --> Nginx
    VolunteerApp --> Nginx

    Nginx --> AdminGUI
    Nginx --> PublicPages
    Nginx --> VolunteerPortal

    AdminGUI --> ExpressAPI
    AdminGUI --> FastifyAPI
    PublicPages --> ExpressAPI
    VolunteerPortal --> ExpressAPI

    ExpressAPI --> Postgres
    ExpressAPI --> Redis
    ExpressAPI --> EmailQueue
    ExpressAPI --> GeocodeQueue
    ExpressAPI --> Represent
    ExpressAPI --> Geocoding
    ExpressAPI --> Listmonk
    ExpressAPI --> Prometheus

    FastifyAPI --> Postgres
    FastifyAPI --> Redis
    FastifyAPI --> Prometheus

    EmailQueue --> Redis
    EmailQueue --> SMTP
    GeocodeQueue --> Redis
    GeocodeQueue --> Geocoding

    Prometheus --> Grafana
    Prometheus --> Alertmanager

Core Components

1. Nginx Reverse Proxy

Purpose: Routes HTTP requests to appropriate services based on subdomain

Subdomains:

app.cmlite.org → Admin GUI (React)
api.cmlite.org → Express API (main features)
media.cmlite.org → Fastify API (video library)
db.cmlite.org → NocoDB (data browser)
docs.cmlite.org → MkDocs (documentation)
listmonk.cmlite.org → Listmonk (newsletter)
grafana.cmlite.org → Grafana (monitoring)
And 8 more service subdomains...

Configuration: /nginx/conf.d/

Learn more →

2. Frontend Layer

Admin GUI (Port 3000)

Framework: React 19 with Vite build tool
UI Library: Ant Design 5 (Table, Form, Modal, Drawer components)
State Management: Zustand stores (auth, canvass)
Routing: React Router v6
HTTP Client: Axios with 401 refresh interceptor

Structure:

32 admin pages (campaigns, locations, users, settings, etc.)
6 public pages (campaign view, response wall, map, shifts)
4 volunteer portal pages (canvassing, assignments, activity)
Shared components (map, canvass, GrapesJS editor)

Learn more →

Public Pages

Dark blue/teal theme (consistent with V1 branding)
No authentication required
Mobile-responsive layouts
Public campaign submission
Response wall with upvoting
Public map with location markers
Shift signup forms

Volunteer Portal

Top navigation layout
Mobile-optimized (hamburger menu)
GPS-tracked canvassing
Full-screen map interface
Visit recording forms
Activity tracking

3. Backend Layer - Dual API Design

Express API (Port 4000)

Main application server handling core features:

14 Feature Modules:

auth - JWT login, register, refresh, logout
users - User CRUD with pagination
settings - Site settings singleton
campaigns - Campaign CRUD + public routes
representatives - Represent API integration
responses - Response wall + moderation
email-queue - BullMQ queue admin
campaign-emails - Email tracking + stats
postal-codes - Postal code cache
locations - Location CRUD + geocoding + NAR import
cuts - Cut (polygon) CRUD + spatial queries
shifts - Shift CRUD + signups
canvass - Volunteer canvassing (sessions, visits, routes)
pages - Landing page builder (GrapesJS)

Plus: email-templates, listmonk, pangolin, docs, qr, services, observability

ORM: Prisma (27+ models)

Architecture:

Layered structure (routes → services → database)
Zod schema validation
Role-based access control (RBAC)
Error handling middleware
Winston logging

Learn more →

Fastify API (Port 4100)

Specialized microservice for media library:

Features:

Video CRUD (title, duration, orientation, producer)
Shared media (public gallery categories)
Lock/unlock system (public visibility control)
Reaction system (6 standard emojis)
Job queue monitoring
Bulk operations

ORM: Drizzle (lightweight schema-first)

Why Separate?:

Performance isolation (video ops don't slow main API)
Different ORM evaluation (Drizzle vs Prisma)
Independent scaling
Clear service boundaries

Shared Resources:

Same PostgreSQL database (different schemas)
Same Redis instance
Reuses JWT_ACCESS_SECRET for auth

Learn more →

4. Data Layer

PostgreSQL 16

Primary database with two ORM schemas:

Prisma Schema (27+ models):

User, RefreshToken (auth)
Campaign, Representative, Response, CampaignEmail (influence)
Location, Cut, Shift, ShiftSignup (map)
CanvassSession, CanvassVisit, TrackingSession, TrackPoint (canvass)
LandingPage, PageBlock, EmailTemplate (content)
SiteSettings, MapSettings (config)

Drizzle Schema (media tables):

videos
shared_media
reactions
jobs

Indexes: Optimized for common queries (userId, campaignId, cutId, etc.)

Learn more →

Redis

Multi-purpose cache and queue backend:

Caching: Postal codes (7-day TTL), representatives
Rate Limiting: Per-endpoint limits (Redis-backed)
BullMQ Queues: Email sending, bulk geocoding
Sessions: Future session storage (if needed)

Authentication: Required (REDIS_PASSWORD env var)

5. Job Processing

BullMQ Queues

Async job processing for long-running operations:

Email Queue:

Campaign email sending (SMTP)
Email verification (double opt-in)
Confirmation emails (shift signups)
Retry logic (exponential backoff)
Rate limiting (avoid spam flagging)

Geocoding Queue:

Bulk address geocoding
Multi-provider fallback (6 services)
Rate limit compliance (500 jobs/min)
Result caching

Queue Management:

Admin routes for pause/resume
Job status monitoring
Failed job inspection
Queue metrics (Prometheus)

6. External Services

SMTP Server

Email delivery for:

Campaign advocacy emails
Email verification
Password reset
Shift confirmation
Admin notifications

Dev Mode: MailHog captures emails (EMAIL_TEST_MODE=true)

Represent API

Canadian elected representative lookup:

Postal code → MPs, MPPs, councillors
Caching (7-day TTL per postal code)
Fallback to cached data on API errors

Geocoding Providers

Multi-provider geocoding with fallback:

Nominatim (OpenStreetMap, free)
Mapbox (requires API key, best accuracy)
ArcGIS (free tier available)
Photon (OSM-based, no key required)
Google (requires API key, high cost)
LocationIQ (requires API key, generous free tier)

Strategy: Try each provider in order until success

Email marketing integration:

Sync participants/locations/users → subscriber lists
Newsletter campaigns (separate from advocacy emails)
API integration (basic auth)
Health monitoring

7. Observability Stack

Prometheus

Metrics collection with custom instrumentation:

12 Custom Metrics (cm_* prefix):

cm_api_uptime_seconds - API availability
cm_email_queue_size - Queue depth
cm_email_sent_total - Email delivery count
cm_geocode_success_rate - Geocoding quality
cm_active_canvass_sessions - Live canvassing
And 7 more domain-specific metrics...

HTTP Metrics:

http_request_total - Total requests
http_request_duration_seconds - Latency histogram
http_request_errors_total - Error count

Scrape Targets:

Express API (:4000/metrics)
Fastify API (:4100/metrics)
Redis Exporter
Node Exporter (host metrics)
cAdvisor (container metrics)

Learn more →

Grafana

Visualization dashboards:

Application Overview - API metrics, queue stats, sessions
Infrastructure - Container metrics, host resources, Redis
Alerts & SLOs - Error budgets, SLI tracking

Auto-provisioned: Dashboards in /configs/grafana/

Alertmanager

Alert routing and notifications:

12 Alert Rules:

High error rate (>5% for 5 minutes)
Email queue stuck (no jobs processed in 10 minutes)
Service down (health check fails)
Database connection pool exhausted
Redis unavailable
And 7 more critical conditions...

Notification Channels:

Gotify (self-hosted push notifications)
Email (SMTP)
Webhook (custom integrations)

Request Lifecycle

Example: Public Campaign Email Submission

sequenceDiagram
    participant User as User Browser
    participant Nginx
    participant Admin as Admin GUI
    participant Express as Express API
    participant DB as PostgreSQL
    participant Redis
    participant Queue as BullMQ
    participant SMTP as SMTP Server
    participant Rep as Represent API

    User->>Nginx: Visit /campaigns/123
    Nginx->>Admin: Route to React app
    Admin->>Express: GET /api/campaigns/123 (public)
    Express->>DB: Query campaign
    DB-->>Express: Campaign data
    Express-->>Admin: Campaign JSON
    Admin-->>User: Render campaign page

    User->>Admin: Enter postal code + submit
    Admin->>Express: POST /api/postal-codes (lookup)
    Express->>Redis: Check cache
    Redis-->>Express: Cache miss
    Express->>Rep: GET /representatives/postal-code
    Rep-->>Express: Representative list
    Express->>Redis: Cache for 7 days
    Express-->>Admin: Representatives JSON
    Admin-->>User: Show rep selection

    User->>Admin: Select rep + write email + submit
    Admin->>Express: POST /api/responses (create)
    Express->>DB: Insert response
    Express->>Queue: Enqueue verification email
    Express->>DB: Insert campaign email record
    DB-->>Express: Response created
    Express-->>Admin: Success response
    Admin-->>User: Show success message

    Queue->>SMTP: Send verification email
    SMTP-->>Queue: Delivery confirmed

    User->>User: Click verification link (email)
    User->>Nginx: GET /verify-response/:token
    Nginx->>Admin: Route to React app
    Admin->>Express: POST /api/responses/:id/verify
    Express->>DB: Update response (verified=true)
    Express->>Queue: Enqueue campaign email to rep
    DB-->>Express: Response verified
    Express-->>Admin: Success
    Admin-->>User: Email sent confirmation

    Queue->>SMTP: Send campaign email to rep
    SMTP-->>Queue: Delivery confirmed

Technology Decisions

Why TypeScript?

Type safety reduces runtime errors
Better IDE support and autocomplete
Easier refactoring
Self-documenting code

Why Prisma + Drizzle?

Prisma: Great for complex models, migrations, auto-generated types
Drizzle: Lightweight, perfect for simple media tables
Evaluate both ORMs in production

Why Dual API?

Separation of concerns: Media ops isolated from core features
Performance: Video processing doesn't block main API
Scalability: Independent horizontal scaling
Technology evaluation: Compare Express vs Fastify

Why JWT over Sessions?

Stateless (scales horizontally)
No session storage overhead
Works across multiple API servers
Standard claims (iat, exp, sub)

Why BullMQ over Bull?

Better TypeScript support
Improved performance
Active maintenance
Better documentation

Why PostgreSQL over NoSQL?

Complex relational data (campaigns, locations, users)
ACID transactions (critical for email queue)
Full-text search
Spatial queries (PostGIS for future geo features)

Deployment Architecture

Docker Compose

All services orchestrated in docker-compose.yml:

Profiles:

default: Core services (postgres, redis, api, admin, nginx)
monitoring: Prometheus, Grafana, Alertmanager, exporters

Networks:

changemaker-lite bridge network
Service discovery via container names

Volumes:

PostgreSQL data persistence
Redis data persistence
Uploads directory
Logs directory

Learn more →

Nginx Routing

Subdomain-based routing:

# Admin GUI
server {
    server_name app.cmlite.org;
    location / {
        proxy_pass http://admin:3000;
    }
}

# Express API
server {
    server_name api.cmlite.org;
    location / {
        proxy_pass http://api:4000;
    }
}

# Fastify Media API
server {
    server_name media.cmlite.org;
    location / {
        proxy_pass http://media-api:4100;
    }
}

Learn more →

Security Architecture

Authentication Flow

sequenceDiagram
    participant Client
    participant API as Express API
    participant DB as PostgreSQL
    participant Redis

    Client->>API: POST /api/auth/login
    API->>DB: Verify credentials
    DB-->>API: User record
    API->>DB: Create refresh token (expires 7d)
    API->>Redis: Rate limit check
    API-->>Client: Access token (15min) + Refresh token (7d)

    Note over Client: Access token expires

    Client->>API: POST /api/auth/refresh
    API->>DB: Validate refresh token
    DB-->>API: Token valid
    API->>DB: Rotate refresh token (transaction)
    API-->>Client: New access token + New refresh token

Features:

bcrypt password hashing (12+ chars, complexity requirements)
JWT access tokens (15min expiry)
Refresh tokens (7 days, stored in DB, rotated on use)
Rate limiting (10 requests/min on auth endpoints)
User enumeration prevention (401 not 404)
RBAC middleware (requireRole, requireNonTemp)

Learn more →

Security Layers

Network: Nginx rate limiting, fail2ban
Application: Input validation (Zod schemas), RBAC
Data: Encrypted fields (ENCRYPTION_KEY), SQL injection prevention (Prisma)
Transport: HTTPS only (production), HSTS headers

Learn more →

Scalability Considerations

Horizontal Scaling

Stateless APIs: JWT auth allows multiple API instances
Redis-backed queues: Share job queues across workers
Database connection pooling: Prisma manages connections
Nginx load balancing: Distribute requests across API instances

Vertical Scaling

Increase container resources (CPU, memory)
Optimize database queries (indexes, query planning)
Redis memory limits (LRU eviction policy)

Bottlenecks

PostgreSQL: Single primary (future: read replicas)
Redis: Single instance (future: Redis Cluster)
File uploads: Local disk (future: S3-compatible storage)

Monitoring & Observability

Golden Signals

Latency: Request duration histograms
Traffic: Request rate by endpoint
Errors: Error rate (5xx responses)
Saturation: Database connections, Redis memory, queue depth

SLOs (Service Level Objectives)

Availability: 99.9% uptime (8.76 hours downtime/year)
Latency: p95 < 500ms, p99 < 1000ms
Error Rate: < 0.1% (1 error per 1000 requests)

Alerting Strategy

Critical: Page on-call (service down, database unavailable)
Warning: Create ticket (queue growing, elevated errors)
Info: Log only (slow query, cache miss)

Learn more →

16 KiB Raw Blame History

V2 Architecture Overview

System Architecture

Core Components

1. Nginx Reverse Proxy

2. Frontend Layer

Admin GUI (Port 3000)

Public Pages

Volunteer Portal

3. Backend Layer - Dual API Design

Express API (Port 4000)

Fastify API (Port 4100)

4. Data Layer

PostgreSQL 16

Redis

5. Job Processing

BullMQ Queues

6. External Services

SMTP Server

Represent API

Geocoding Providers

Listmonk Newsletter Platform

7. Observability Stack

Prometheus

Grafana

Alertmanager

Request Lifecycle

Example: Public Campaign Email Submission

Technology Decisions

Why TypeScript?

Why Prisma + Drizzle?

Why Dual API?

Why JWT over Sessions?

Why BullMQ over Bull?

Why PostgreSQL over NoSQL?

Deployment Architecture

Docker Compose

Nginx Routing

Security Architecture

Authentication Flow

Security Layers

Scalability Considerations

Horizontal Scaling

Vertical Scaling

Bottlenecks

Monitoring & Observability

Golden Signals

SLOs (Service Level Objectives)

Alerting Strategy

Further Reading

16 KiB

Raw Blame History