# Nginx Reverse Proxy Configuration

## Overview

Nginx serves as the central reverse proxy for Changemaker Lite V2, routing traffic to 15+ backend services via subdomain-based routing. It handles SSL termination, security headers, static file serving, and WebSocket upgrades.

**Key Responsibilities:**

- **Subdomain Routing**: `api.cmlite.org`, `app.cmlite.org`, `db.cmlite.org`, etc.
- **SSL/TLS Termination**: Handles HTTPS certificates (Let's Encrypt, Cloudflare, or Pangolin)
- **Security Headers**: CSP, HSTS, X-Frame-Options, Permissions-Policy
- **Proxy Pass**: Forwards requests to backend Docker containers
- **Static File Serving**: Serves admin GUI production builds + MkDocs site
- **WebSocket Support**: Upgrades connections for n8n, MailHog, MkDocs live reload
- **Iframe Embedding**: CSP policies allow admin to embed services (NocoDB, Gitea, etc.)

**Architecture:**

```
Internet → Nginx (:80, :443) → [Docker Internal Network]
                                  ├─ api:4000 (Express)
                                  ├─ media-api:4100 (Fastify)
                                  ├─ admin:3000 (Vite / static)
                                  ├─ nocodb:8080
                                  ├─ listmonk:9000
                                  ├─ gitea:3000
                                  ├─ n8n:5678
                                  ├─ mkdocs:8000
                                  ├─ code-server:8080
                                  ├─ mailhog:8025
                                  ├─ mini-qr:8080
                                  ├─ homepage:3000
                                  ├─ grafana:3000
                                  └─ public-media:80
```

---

## Architecture

```mermaid
graph LR
    subgraph "External Access"
        USER[User Browser]
        TUNNEL[Pangolin Tunnel]
    end

    subgraph "Nginx Proxy :80, :443"
        NGINX{Nginx<br/>Subdomain Router}
    end

    subgraph "Backend Services (Docker Network)"
        API[api:4000<br/>Express]
        MEDIA[media-api:4100<br/>Fastify]
        ADMIN[admin:3000<br/>Vite]
        NOCODB[nocodb:8080]
        LISTMONK[listmonk:9000]
        GITEA[gitea:3000]
        N8N[n8n:5678]
        MKDOCS[mkdocs:8000]
        CODE[code-server:8080]
    end

    USER -->|HTTP/HTTPS| NGINX
    TUNNEL -->|HTTP| NGINX

    NGINX -->|api.cmlite.org| API
    NGINX -->|api.cmlite.org/media| MEDIA
    NGINX -->|app.cmlite.org| ADMIN
    NGINX -->|db.cmlite.org| NOCODB
    NGINX -->|listmonk.cmlite.org| LISTMONK
    NGINX -->|git.cmlite.org| GITEA
    NGINX -->|n8n.cmlite.org| N8N
    NGINX -->|docs.cmlite.org| MKDOCS
    NGINX -->|code.cmlite.org| CODE
```

---

## Configuration Files

Nginx configuration split across multiple files:

| File | Purpose | Type |
|------|---------|------|
| `nginx/nginx.conf` | Global settings, gzip, security headers | Main config |
| `nginx/conf.d/default.conf` | Localhost fallback, path-based routing | Server block |
| `nginx/conf.d/api.conf` | API subdomain routing (Express + Fastify) | Server block |
| `nginx/conf.d/services.conf` | Supporting service subdomains | Server blocks (12+) |

**Configuration hierarchy**:
```
nginx.conf
├─ Global: worker_processes, events, http
├─ Security headers (applied to all)
├─ Gzip compression
├─ Docker DNS resolver (127.0.0.11)
└─ Include conf.d/*.conf
    ├─ default.conf (localhost)
    ├─ api.conf (api.cmlite.org)
    └─ services.conf (all other subdomains)
```

---

## Global Configuration (nginx.conf)

**File**: `nginx/nginx.conf`

### Worker Configuration

```nginx
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 1024;
}
```

**Explanation**:
- `worker_processes auto`: Detects CPU cores (1 worker per core)
- `worker_connections 1024`: Max 1024 concurrent connections per worker
- Total capacity: `auto × 1024` (e.g., 4 cores = 4096 connections)

---

### HTTP Block

```nginx
http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

    access_log /var/log/nginx/access.log main;

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    client_max_body_size 50m;  # Default max upload size

    # Include server blocks
    include /etc/nginx/conf.d/*.conf;
}
```

**Key Settings**:
- `sendfile on`: Optimized file serving (kernel-level copy)
- `tcp_nopush on`: Sends HTTP headers in single packet
- `client_max_body_size 50m`: Default upload limit (overridden per location)

---

### Gzip Compression

```nginx
# Gzip compression
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css application/json application/javascript
           text/xml application/xml application/xml+rss text/javascript
           image/svg+xml;
```

**Performance Impact**:
- **CPU usage**: Level 6 provides 80% compression with moderate CPU cost
- **Bandwidth savings**: ~60-80% reduction for text/JSON responses
- **Excluded**: Images, video (already compressed)

---

### Security Headers

```nginx
# Security headers (applied globally)
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header Permissions-Policy "geolocation=(self), microphone=(), camera=()" always;
```

**Header Explanation**:
- **X-Content-Type-Options**: Prevents MIME sniffing attacks
- **X-XSS-Protection**: Enables browser XSS filter (legacy browsers)
- **Referrer-Policy**: Controls referer header sent to external sites
- **HSTS**: Forces HTTPS for 1 year (31536000 seconds)
- **Permissions-Policy**: Restricts geolocation/media access

**Note**: `X-Frame-Options` set per server block (not global).

---

### Docker DNS Resolver

```nginx
# Docker internal DNS — enables runtime resolution
resolver 127.0.0.11 valid=30s;
```

**Purpose**: Docker's embedded DNS server at `127.0.0.11` resolves container names.

**Why needed**: Allows Nginx to start even when optional services are down. Without this, Nginx fails to start if any upstream is missing.

**Usage pattern**:
```nginx
location / {
    set $upstream_api http://changemaker-v2-api:4000;
    proxy_pass $upstream_api;  # Resolves at request time, not config parse
}
```

**Alternative** (fails if container missing):
```nginx
proxy_pass http://changemaker-v2-api:4000;  # Resolved at config parse — fails if down
```

---

## Subdomain Routing

### Default Server (localhost)

**File**: `nginx/conf.d/default.conf`

```nginx
server {
    listen 80 default_server;
    server_name localhost _;
    add_header X-Frame-Options "SAMEORIGIN" always;

    # Admin GUI (default)
    location / {
        set $upstream_admin http://changemaker-v2-admin:3000;
        proxy_pass $upstream_admin;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }

    # Media API (must come BEFORE /api/ for longest prefix match)
    location /api/media/ {
        set $upstream_media http://changemaker-media-api:4100;
        proxy_pass $upstream_media;
        # ... (proxy headers)

        # Large upload support
        client_max_body_size 10G;
        proxy_read_timeout 3600s;
        proxy_connect_timeout 75s;
        proxy_request_buffering off;
    }

    # API (Express)
    location /api/ {
        set $upstream_api http://changemaker-v2-api:4000;
        proxy_pass $upstream_api;
        # ... (proxy headers)
    }

    # Public Media Gallery
    location /gallery/ {
        proxy_pass http://changemaker-public-media:80/;
        # ... (proxy headers)
    }
}
```

**Routing Logic**:
1. Request to `http://localhost/api/media/videos` → media-api:4100
2. Request to `http://localhost/api/campaigns` → api:4000
3. Request to `http://localhost/` → admin:3000
4. Request to `http://localhost/gallery/` → public-media:80

**Important**: `/api/media/` location **must** come before `/api/` in config file (longest prefix match).

---

### API Subdomain (api.cmlite.org)

**File**: `nginx/conf.d/api.conf`

```nginx
server {
    listen 80;
    server_name api.cmlite.org;
    add_header X-Frame-Options "SAMEORIGIN" always;

    # Media API endpoints (must come BEFORE / for longest prefix match)
    location /media/ {
        set $upstream_media http://changemaker-media-api:4100/api/;
        proxy_pass $upstream_media;
        # ... (proxy headers)

        # Large upload support
        client_max_body_size 10G;
        proxy_read_timeout 3600s;
        proxy_connect_timeout 75s;
        proxy_request_buffering off;

        # WebSocket support
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }

    # Main API (Express)
    location / {
        set $upstream_api http://changemaker-v2-api:4000;
        proxy_pass $upstream_api;
        # ... (proxy headers)
        proxy_read_timeout 300s;
        proxy_connect_timeout 75s;
    }
}
```

**URL Mapping**:
- `http://api.cmlite.org/media/videos` → `http://changemaker-media-api:4100/api/videos`
- `http://api.cmlite.org/auth/login` → `http://changemaker-v2-api:4000/auth/login`

**Critical**: Media API location includes `/api/` in `proxy_pass` to rewrite path.

---

### Service Subdomains

**File**: `nginx/conf.d/services.conf`

#### Gitea (git.cmlite.org)

```nginx
server {
    listen 80;
    server_name git.cmlite.org;
    add_header Content-Security-Policy "frame-ancestors 'self' app.cmlite.org" always;

    # Increase max body size for large git pushes (2GB)
    client_max_body_size 2048M;

    location / {
        set $upstream_gitea http://gitea-changemaker:3000;
        proxy_pass $upstream_gitea;
        proxy_hide_header X-Frame-Options;  # Allow iframe embedding
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

**Key Features**:
- **CSP `frame-ancestors`**: Allows embedding in `app.cmlite.org` (admin GUI)
- **proxy_hide_header X-Frame-Options**: Strips Gitea's default DENY policy
- **2GB upload limit**: For large repository pushes

---

#### n8n (n8n.cmlite.org)

```nginx
server {
    listen 80;
    server_name n8n.cmlite.org;
    add_header Content-Security-Policy "frame-ancestors 'self' app.cmlite.org" always;

    location / {
        set $upstream_n8n http://n8n-changemaker:5678;
        proxy_pass $upstream_n8n;
        proxy_hide_header X-Frame-Options;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";  # WebSocket support
    }
}
```

**WebSocket Headers**:
- `Upgrade: $http_upgrade`: Passes WebSocket upgrade header
- `Connection: "upgrade"`: Indicates protocol upgrade

**Required for**: n8n workflow editor, MailHog live updates, MkDocs live reload

---

#### NocoDB (db.cmlite.org)

```nginx
server {
    listen 80;
    server_name db.cmlite.org;
    add_header Content-Security-Policy "frame-ancestors 'self' app.cmlite.org" always;

    location / {
        set $upstream_nocodb http://changemaker-v2-nocodb:8080;
        proxy_pass $upstream_nocodb;
        proxy_hide_header X-Frame-Options;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

**Iframe Embedding**:
- `frame-ancestors 'self' app.cmlite.org`: Allows admin GUI to embed NocoDB
- `proxy_hide_header X-Frame-Options`: Removes NocoDB's default SAMEORIGIN policy

---

#### MkDocs (docs.cmlite.org)

```nginx
server {
    listen 80;
    server_name docs.cmlite.org;
    add_header Content-Security-Policy "frame-ancestors 'self' app.cmlite.org" always;

    location / {
        set $upstream_mkdocs http://mkdocs-changemaker:8000;
        proxy_pass $upstream_mkdocs;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";  # Live reload WebSocket
    }
}
```

**Live Reload**: MkDocs Material theme uses WebSocket for live reload during development.

---

#### Code Server (code.cmlite.org)

```nginx
server {
    listen 80;
    server_name code.cmlite.org;
    add_header Content-Security-Policy "frame-ancestors 'self' app.cmlite.org" always;

    location / {
        set $upstream_code http://code-server-changemaker:8080;
        proxy_pass $upstream_code;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";  # VS Code WebSocket
    }
}
```

**WebSocket Usage**: Code Server uses WebSockets for terminal, file watching, language server.

---

#### MailHog (mail.cmlite.org)

```nginx
server {
    listen 80;
    server_name mail.cmlite.org;
    add_header Content-Security-Policy "frame-ancestors 'self' app.cmlite.org" always;

    location / {
        set $upstream_mailhog http://mailhog-changemaker:8025;
        proxy_pass $upstream_mailhog;
        proxy_hide_header X-Frame-Options;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        # WebSocket support for MailHog live updates
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}
```

**Live Updates**: MailHog uses WebSocket to push new emails to browser without polling.

---

#### Listmonk (listmonk.cmlite.org)

```nginx
server {
    listen 80;
    server_name listmonk.cmlite.org;
    add_header X-Frame-Options "SAMEORIGIN" always;

    location / {
        set $upstream_listmonk http://listmonk-app:9000;
        proxy_pass $upstream_listmonk;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

**No Iframe**: Listmonk not embedded in admin (accessed directly), so `SAMEORIGIN` policy kept.

---

#### Grafana (grafana.cmlite.org)

```nginx
server {
    listen 80;
    server_name grafana.cmlite.org;
    add_header X-Frame-Options "SAMEORIGIN" always;

    location / {
        set $upstream_grafana http://grafana-changemaker:3000;
        proxy_pass $upstream_grafana;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";  # Grafana live updates
    }
}
```

**WebSocket**: Grafana uses WebSocket for live dashboard updates.

---

#### Mini QR (qr.cmlite.org)

```nginx
server {
    listen 80;
    server_name qr.cmlite.org;
    add_header Content-Security-Policy "frame-ancestors 'self' app.cmlite.org" always;

    location / {
        set $upstream_miniqr http://mini-qr:8080;
        proxy_pass $upstream_miniqr;
        proxy_hide_header X-Frame-Options;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

**Iframe Embedding**: Admin GUI embeds Mini QR for walk sheet previews.

---

#### Root Domain (cmlite.org)

```nginx
server {
    listen 80;
    server_name cmlite.org;

    location / {
        set $upstream_site http://mkdocs-site-server-changemaker:80;
        proxy_pass $upstream_site;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

**Purpose**: Serves MkDocs static site (production build) on root domain.

---

#### Homepage (home.cmlite.org)

```nginx
server {
    listen 80;
    server_name home.cmlite.org;
    add_header X-Frame-Options "SAMEORIGIN" always;

    location / {
        set $upstream_homepage http://homepage-changemaker:3000;
        proxy_pass $upstream_homepage;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

**Dashboard**: Service status dashboard with Docker integration.

---

## Embed Proxy Ports

**Purpose**: Allow admin GUI to iframe services via localhost ports (bypassing subdomain requirements).

**Ports**: 8881-8885 (NocoDB, n8n, Gitea, MailHog, Mini QR)

**Configuration** (in `services.conf`):

```nginx
# NocoDB embed proxy (port 8881)
server {
    listen 8881;
    location / {
        set $upstream_nocodb http://changemaker-v2-nocodb:8080;
        proxy_pass $upstream_nocodb;
        proxy_hide_header X-Frame-Options;
        proxy_hide_header Content-Security-Policy;  # Strip all frame restrictions
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

# n8n embed proxy (port 8882)
server {
    listen 8882;
    location / {
        set $upstream_n8n http://n8n-changemaker:5678;
        proxy_pass $upstream_n8n;
        proxy_hide_header X-Frame-Options;
        proxy_hide_header Content-Security-Policy;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

# Gitea embed proxy (port 8883)
server {
    listen 8883;
    client_max_body_size 2048M;  # Large git pushes
    location / {
        set $upstream_gitea http://gitea-changemaker:3000;
        proxy_pass $upstream_gitea;
        proxy_hide_header X-Frame-Options;
        proxy_hide_header Content-Security-Policy;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

# MailHog embed proxy (port 8884)
server {
    listen 8884;
    location / {
        set $upstream_mailhog http://mailhog-changemaker:8025;
        proxy_pass $upstream_mailhog;
        proxy_hide_header X-Frame-Options;
        proxy_hide_header Content-Security-Policy;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

# Mini QR embed proxy (port 8885)
server {
    listen 8885;
    location / {
        set $upstream_miniqr http://mini-qr:8080;
        proxy_pass $upstream_miniqr;
        proxy_hide_header X-Frame-Options;
        proxy_hide_header Content-Security-Policy;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

**Usage in Admin GUI**:
```tsx
<iframe src="http://localhost:8881" />  {/* NocoDB */}
<iframe src="http://localhost:8882" />  {/* n8n */}
<iframe src="http://localhost:8883" />  {/* Gitea */}
<iframe src="http://localhost:8884" />  {/* MailHog */}
<iframe src="http://localhost:8885" />  {/* Mini QR */}
```

**Exposed in docker-compose.yml**:
```yaml
nginx:
  ports:
    - "80:80"
    - "443:443"
    - "8881:8881"  # NocoDB
    - "8882:8882"  # n8n
    - "8883:8883"  # Gitea
    - "8884:8884"  # MailHog
    - "8885:8885"  # Mini QR
```

---

## Proxy Configuration

### Standard Proxy Headers

**All proxy locations** should include:

```nginx
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
```

**Header Explanation**:
- `Host`: Preserves original hostname (e.g., `api.cmlite.org`)
- `X-Real-IP`: Client's IP address
- `X-Forwarded-For`: Chain of proxy IPs (adds to existing list)
- `X-Forwarded-Proto`: HTTP or HTTPS (used by backend for redirect logic)

---

### WebSocket Upgrade

**Required for**: n8n, MailHog, MkDocs, Code Server, Grafana

```nginx
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
```

**Explanation**:
- `Upgrade: websocket`: Browser requests protocol upgrade
- `Connection: upgrade`: Indicates connection will persist

**Without these headers**: WebSocket connections fail with 400 Bad Request.

---

### Timeouts

**Default timeouts**:
```nginx
proxy_read_timeout 300s;     # 5 minutes
proxy_connect_timeout 75s;   # 75 seconds
```

**Media API timeouts** (video uploads):
```nginx
proxy_read_timeout 3600s;    # 1 hour
proxy_connect_timeout 75s;
```

**Why longer**: FFprobe video analysis + large file uploads take time.

---

### Upload Size Limits

**Global default** (`nginx.conf`):
```nginx
client_max_body_size 50m;
```

**Per-location overrides**:
- **Media API**: `client_max_body_size 10G;` (video uploads)
- **Gitea**: `client_max_body_size 2048M;` (large git pushes)

---

### Request Buffering

**Media API** (disable buffering for streaming uploads):
```nginx
proxy_request_buffering off;
```

**Effect**: Nginx streams request body directly to backend (no temp file).

**Benefits**:
- Lower disk I/O on Nginx server
- Faster upload start time
- Reduced memory usage

**Trade-off**: Backend must handle slow clients (Fastify multipart does this).

---

## SSL/TLS Configuration

### Certificate Paths

**Recommended structure**:
```
/etc/letsencrypt/live/cmlite.org/
├─ fullchain.pem  (certificate + intermediate)
├─ privkey.pem    (private key)
└─ chain.pem      (intermediate CA)
```

**Nginx SSL block**:
```nginx
server {
    listen 443 ssl http2;
    server_name api.cmlite.org;

    ssl_certificate /etc/letsencrypt/live/cmlite.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/cmlite.org/privkey.pem;

    # Strong TLS configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;

    # ... location blocks
}
```

---

### HTTP to HTTPS Redirect

```nginx
server {
    listen 80;
    server_name api.cmlite.org;

    # Redirect all HTTP to HTTPS
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    server_name api.cmlite.org;
    # ... SSL config + locations
}
```

---

### HSTS Header

**Already applied globally** (in `nginx.conf`):
```nginx
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
```

**Effect**: Browser caches HTTPS requirement for 1 year.

**Important**: Only enable after verifying HTTPS works (can't easily undo).

---

### Wildcard Certificates

**For `*.cmlite.org`** (Let's Encrypt DNS challenge):
```bash
certbot certonly --dns-cloudflare \
  --dns-cloudflare-credentials ~/.secrets/cloudflare.ini \
  -d cmlite.org -d "*.cmlite.org"
```

**Single cert** covers all subdomains:
- api.cmlite.org
- app.cmlite.org
- db.cmlite.org
- etc.

See [SSL/TLS](ssl-tls.md) for complete certificate management.

---

## Static File Serving

### Admin GUI Production Build

**Dockerfile multi-stage build** (admin/Dockerfile):
```dockerfile
# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Production stage
FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
```

**Nginx serves static files** (no Node.js in production):
```nginx
server {
    listen 80;
    server_name app.cmlite.org;

    root /usr/share/nginx/html;
    index index.html;

    # React Router support (all routes → index.html)
    location / {
        try_files $uri $uri/ /index.html;
    }

    # API proxy
    location /api/ {
        proxy_pass http://changemaker-v2-api:4000;
        # ... proxy headers
    }

    # Cache static assets
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
}
```

---

### MkDocs Static Site

**Build process** (via admin GUI or CLI):
```bash
docker compose exec mkdocs mkdocs build
```

**Output**: `mkdocs/site/` directory with static HTML

**Served by** `mkdocs-site-server` (Nginx Alpine container):
```yaml
mkdocs-site-server:
  image: lscr.io/linuxserver/nginx:latest
  volumes:
    - ./mkdocs/site:/config/www
  ports:
    - "4004:80"
```

**Nginx config** (in `configs/mkdocs-site/default.conf`):
```nginx
server {
    listen 80;
    root /config/www;
    index index.html;

    location / {
        try_files $uri $uri/ =404;
    }
}
```

---

## Performance Optimization

### Gzip Compression

**Already enabled globally** (see nginx.conf above).

**Compression ratio**:
- JSON responses: ~75% reduction
- HTML/CSS/JS: ~60-70% reduction
- Images/video: No compression (already compressed)

**Trade-off**: Slight CPU increase (~5-10%) for bandwidth savings.

---

### Caching Static Assets

```nginx
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
    expires 1y;
    add_header Cache-Control "public, immutable";
}
```

**Effect**: Browsers cache static assets for 1 year.

**Caveat**: Use content hashing in filenames (Vite does this automatically).

---

### Proxy Caching

**Optional** (not enabled by default):
```nginx
# In http block
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:10m max_size=1g inactive=60m;

# In location block
location /api/campaigns {
    proxy_cache api_cache;
    proxy_cache_valid 200 10m;
    proxy_cache_key "$scheme$request_method$host$request_uri";
    proxy_pass http://changemaker-v2-api:4000;
}
```

**Use cases**:
- Public campaign listing (10-minute cache)
- Public map data (5-minute cache)
- Representative lookup (1-hour cache)

**Avoid caching**:
- Authenticated endpoints
- POST/PUT/DELETE requests
- Real-time data (canvass sessions, email queue)

---

### Connection Pooling

**Keep-alive to backends**:
```nginx
upstream api {
    server changemaker-v2-api:4000;
    keepalive 32;  # Maintain 32 idle connections
}

location /api/ {
    proxy_pass http://api;
    proxy_http_version 1.1;
    proxy_set_header Connection "";  # Clear close header
}
```

**Benefits**:
- Reduced latency (no TCP handshake)
- Lower CPU (fewer connection setups)
- Better throughput under load

---

## Troubleshooting

### 502 Bad Gateway

**Symptoms**: `502 Bad Gateway` error

**Causes**:
1. Backend container not running
2. Backend healthcheck failing
3. Backend listening on wrong port
4. Network connectivity issue

**Diagnosis**:
```bash
# Check backend status
docker compose ps api

# Check backend logs
docker compose logs --tail=50 api

# Test backend directly
docker compose exec nginx curl http://changemaker-v2-api:4000/api/health

# Check Nginx error log
docker compose exec nginx cat /var/log/nginx/error.log
```

**Solution**:
```bash
# Restart backend
docker compose restart api

# Check healthcheck
docker inspect changemaker-v2-api | jq '.[0].State.Health'

# Verify port in docker-compose.yml
grep -A5 "api:" docker-compose.yml
```

---

### 504 Gateway Timeout

**Symptoms**: Request times out after 60 seconds

**Cause**: Backend processing too slow, proxy timeout too short

**Solution**:
```nginx
# Increase timeout for slow endpoints
location /api/locations/geocode {
    proxy_read_timeout 300s;  # 5 minutes
    proxy_pass http://changemaker-v2-api:4000;
}
```

---

### SSL Certificate Errors

**Symptoms**: `SSL_ERROR_RX_RECORD_TOO_LONG` or `ERR_SSL_PROTOCOL_ERROR`

**Cause**: Accessing HTTPS port via HTTP or vice versa

**Diagnosis**:
```bash
# Test HTTPS
curl -I https://api.cmlite.org

# Check certificate
openssl s_client -connect api.cmlite.org:443 -servername api.cmlite.org

# Verify Nginx config
docker compose exec nginx nginx -t
```

**Solution**:
```bash
# Reload Nginx after cert renewal
docker compose exec nginx nginx -s reload

# Check cert paths in config
grep ssl_certificate /path/to/nginx/conf.d/*.conf
```

---

### CORS Errors

**Symptoms**: Browser console shows `CORS policy: No 'Access-Control-Allow-Origin' header`

**Cause**: Backend not setting CORS headers

**Diagnosis**:
```bash
# Test from browser console
fetch('http://api.cmlite.org/api/campaigns')

# Check response headers
curl -H "Origin: http://example.com" -I http://api.cmlite.org/api/campaigns
```

**Solution**: CORS headers set by **backend** (not Nginx). Check `api/src/server.ts`:
```typescript
app.use(cors({
  origin: process.env.CORS_ORIGINS?.split(',') || ['http://localhost:3000'],
  credentials: true,
}));
```

**Nginx passthrough** (don't modify CORS headers):
```nginx
# DO NOT add these in Nginx (backend handles CORS)
# add_header Access-Control-Allow-Origin "*";  # ❌ WRONG
```

---

### WebSocket Connection Failures

**Symptoms**: WebSocket upgrade fails with `400 Bad Request`

**Cause**: Missing `Upgrade`/`Connection` headers

**Diagnosis**:
```bash
# Check Nginx config
grep -A5 "Upgrade" nginx/conf.d/services.conf

# Test WebSocket
wscat -c ws://localhost:5678
```

**Solution**:
```nginx
# Add to location block
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
```

---

### Large Upload Failures

**Symptoms**: Upload fails with `413 Request Entity Too Large`

**Cause**: `client_max_body_size` too small

**Solution**:
```nginx
# Increase limit for specific location
location /api/media/videos {
    client_max_body_size 10G;
    proxy_pass http://changemaker-media-api:4100;
}
```

---

### Iframe Not Displaying

**Symptoms**: Service loads in new tab but not in iframe

**Cause**: `X-Frame-Options: DENY` or CSP `frame-ancestors` blocking

**Diagnosis**:
```bash
# Check response headers
curl -I http://db.cmlite.org

# Look for X-Frame-Options or Content-Security-Policy
```

**Solution**:
```nginx
# Hide backend's X-Frame-Options
proxy_hide_header X-Frame-Options;

# Add CSP allowing admin
add_header Content-Security-Policy "frame-ancestors 'self' app.cmlite.org" always;
```

---

### Nginx Won't Start

**Symptoms**: `docker compose up` fails with Nginx error

**Diagnosis**:
```bash
# Test config syntax
docker compose run --rm nginx nginx -t

# Check for duplicate server_name
grep server_name nginx/conf.d/*.conf | sort

# Check for port conflicts
docker compose config | grep -A2 "ports:"
```

**Common mistakes**:
- Missing semicolon
- Duplicate `server_name` (same subdomain in multiple files)
- Invalid regex in `location`
- Unclosed `{` bracket

---

## Production Best Practices

### Rate Limiting

**Limit requests per IP** (prevents abuse):
```nginx
# In http block
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;

# In location block
location /api/ {
    limit_req zone=api_limit burst=20 nodelay;
    proxy_pass http://changemaker-v2-api:4000;
}
```

**Explanation**:
- `rate=10r/s`: 10 requests per second average
- `burst=20`: Allow bursts up to 20 requests
- `nodelay`: Process burst immediately (don't queue)

---

### Security Headers Review

**Production checklist**:
- [x] HSTS enabled (`max-age=31536000`)
- [x] `X-Content-Type-Options: nosniff`
- [x] `X-XSS-Protection: 1; mode=block`
- [x] CSP `frame-ancestors` for embeddable services
- [x] `X-Frame-Options: SAMEORIGIN` for non-embedded services
- [x] `Referrer-Policy: strict-origin-when-cross-origin`
- [x] `Permissions-Policy` restricts sensors

**Optional enhancements**:
```nginx
# Stricter CSP
add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline';" always;

# Expect-CT (certificate transparency)
add_header Expect-CT "max-age=86400, enforce" always;
```

---

### Access Logging

**Production log format** (JSON for parsing):
```nginx
log_format json_combined escape=json
  '{'
    '"time_local":"$time_local",'
    '"remote_addr":"$remote_addr",'
    '"request":"$request",'
    '"status": $status,'
    '"body_bytes_sent":$body_bytes_sent,'
    '"request_time":$request_time,'
    '"http_referrer":"$http_referer",'
    '"http_user_agent":"$http_user_agent"'
  '}';

access_log /var/log/nginx/access.log json_combined;
```

**Benefits**: Easy parsing with tools like `jq`, Logstash, Loki.

---

### Error Page Customization

**Custom error pages**:
```nginx
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;

location = /404.html {
    root /usr/share/nginx/html;
    internal;
}

location = /50x.html {
    root /usr/share/nginx/html;
    internal;
}
```

**Create files**:
```bash
cat > nginx/html/404.html <<EOF
<!DOCTYPE html>
<html>
<head><title>404 Not Found</title></head>
<body>
<h1>404 - Page Not Found</h1>
<p>Return to <a href="/">homepage</a></p>
</body>
</html>
EOF
```

---

## Related Documentation

- **[Docker Compose](docker-compose.md)** — Container orchestration
- **[Environment Variables](environment-variables.md)** — Configuration reference
- **[SSL/TLS](ssl-tls.md)** — Certificate management
- **[Tunneling](tunneling.md)** — Pangolin tunnel setup
- **[Scaling](scaling.md)** — Load balancing strategies