# Phase 16: Federation — Instance-to-Instance Campaign Network

## Context

Changemaker Lite instances are currently isolated islands. This feature introduces a **federated discovery network** where any instance can act as a **hub** (accepting registrations, serving a directory) and/or a **spoke** (registering with hubs, sharing campaigns). The goal is organic, admin-to-admin networking with public campaign discoverability as a secondary benefit.

**Design principles:**
- Any instance can be a hub, spoke, or both — no central authority
- Medium-depth campaign sharing: enough metadata for discovery, click-through to source
- Per-campaign federation toggle — admins choose what's shared
- Strict privacy boundary: **never** share emails, participant data, queue data, addresses, volunteer/canvass data, or credentials
- Hub admins curate their own directories — organic > control

---

## Prisma Schema Changes

**File:** `api/prisma/schema.prisma`

### New enums
```
FederationPeerStatus: PENDING | ACTIVE | REJECTED | SUSPENDED | OFFLINE
FederationRole: HUB | SPOKE
```

### New models

**FederationIdentity** (singleton — this instance's federation profile)
- `enabled`, `hubEnabled`, `hubAutoApprove`
- Instance profile: `instanceName`, `instanceDescription`, `instanceUrl`, `instanceRegion`, `instanceTags` (Json), `instanceLogoUrl`
- Ed25519 keypair: `publicKey`, `privateKey` (encrypted at rest)
- Hub description, sync interval, last sync timestamp/error

**FederationPeer** (one record per connection, in either direction)
- `role` (HUB or SPOKE), `remoteUrl` (unique per role+url)
- Remote instance profile fields (name, description, region, tags, logo, publicKey)
- Auth: `apiKey` (ours for them), `remoteApiKey` (theirs for us) — both encrypted
- Status tracking: `status`, `statusMessage`, `lastSeenAt`, `lastSyncAt`, `failureCount`
- Stats: `campaignsShared`, `responsesShared`
- Relation to `FederatedCampaign[]`

**FederatedCampaign** (cached campaign metadata from peers)
- `peerId` → FederationPeer
- Remote identifiers: `remoteCampaignId`, `remoteCampaignSlug`
- Safe metadata: title, description, emailSubject (NOT body), callToAction, coverPhoto, status, targetGovernmentLevels, featureFlags (Json), createdByName
- Aggregate stats: `emailCount`, `responseCount`
- Source instance info (denormalized): `sourceInstanceName`, `sourceInstanceUrl`, `sourceInstanceRegion`
- Staleness tracking: `lastSyncedAt`, `isStale`
- Future adoption: `adoptedAsCampaignId` (nullable FK to local Campaign)
- Unique constraint: `[peerId, remoteCampaignId]`

### Modifications to existing models

**Campaign** — add `federated Boolean @default(false)` field

**SiteSettings** — add `enableFederation Boolean @default(false)` feature toggle

---

## API Module Structure

**New directory:** `api/src/modules/federation/`

| File | Purpose |
|------|---------|
| `federation.schemas.ts` | Zod schemas: identity update, peer registration, campaign sync, directory query, list filters |
| `federation.service.ts` | Core business logic: identity CRUD, peer management, `buildSafeCampaignPayload()`, campaign sync, directory serving |
| `federation-admin.routes.ts` | SUPER_ADMIN routes: identity management, peer approve/reject/suspend, manual sync trigger |
| `federation-peer.routes.ts` | Inter-instance routes: inbound registration, campaign sync, directory, heartbeat (API-key auth) |
| `federation-public.routes.ts` | Public browsing: federated campaigns list, instance directory (no auth) |
| `federation-crypto.service.ts` | Ed25519 keypair generation, request signing/verification |

**New file:** `api/src/services/federation-sync-queue.service.ts` — BullMQ repeatable job for periodic sync

### Route table

**Admin routes** (`/api/federation/...`, SUPER_ADMIN + JWT auth):
| Method | Path | Description |
|--------|------|-------------|
| GET | `/identity` | Get federation config |
| PUT | `/identity` | Update config/profile |
| POST | `/identity/generate-keypair` | Generate Ed25519 keypair |
| GET | `/peers` | List all peers |
| POST | `/peers/register` | Register with a remote hub |
| POST | `/peers/:id/approve` | Approve incoming spoke |
| POST | `/peers/:id/reject` | Reject incoming spoke |
| POST | `/peers/:id/suspend` | Suspend peer |
| DELETE | `/peers/:id` | Remove peer |
| POST | `/sync` | Trigger manual sync |
| GET | `/sync/status` | Sync status + history |

**Peer routes** (`/api/federation/peer/...`, API-key auth via `X-Federation-Key` header):
| Method | Path | Description |
|--------|------|-------------|
| POST | `/register` | Inbound spoke registration |
| POST | `/sync` | Inbound campaign metadata push |
| GET | `/directory` | Serve campaign directory |
| GET | `/profile` | Return instance profile |
| POST | `/heartbeat` | Liveness check |

**Public routes** (`/api/federation/...`, no auth):
| Method | Path | Description |
|--------|------|-------------|
| GET | `/campaigns` | Browse federated campaigns (paginated, searchable) |
| GET | `/campaigns/:id` | Single federated campaign detail |
| GET | `/instances` | List known network instances |

### Mounting in server.ts
```
app.use('/api/federation', federationPublicRouter);   // No auth — first
app.use('/api/federation', federationPeerRouter);      // API-key auth
app.use('/api/federation', federationAdminRouter);     // SUPER_ADMIN JWT
```

---

## Federation Protocol

### Registration handshake
1. Spoke admin enters hub URL, clicks "Register"
2. Spoke sends `POST /api/federation/peer/register` to hub with instance profile + generated API key
3. Hub creates peer record (PENDING or ACTIVE if `hubAutoApprove`)
4. Hub responds with its own API key + peer ID
5. If approved (now or later), hub calls back to spoke's `/peer/register` to complete mutual registration
6. Both instances now have each other as peers (Spoke→HUB role, Hub→SPOKE role)

### Campaign sync
- Spokes push federated campaigns to hubs on schedule (BullMQ repeatable job)
- Payload: array of safe campaign metadata + array of un-federated campaign IDs (for removal)
- Hub stores/updates `FederatedCampaign` records
- Sync includes heartbeat (updates `lastSeenAt`)

### Privacy boundary enforcement
`buildSafeCampaignPayload()` in the service layer filters campaigns to only safe fields. **Never included:** emailBody, any email addresses, user IDs, participant data, moderation internals, custom recipients, calls data.

### Offline handling
- Increment `failureCount` on sync failure; after 5 consecutive failures → status `OFFLINE`
- Mark federated campaigns as `isStale` after 24h offline
- Keep checking with exponential backoff (max 24h)
- Auto-recover when heartbeat succeeds

---

## Security

- **API-key auth:** `crypto.randomBytes(32).toString('hex')`, encrypted at rest with existing `encrypt()`/`decrypt()` utility
- **Custom middleware:** `authenticatePeer` checks `X-Federation-Key` header, verifies peer exists + is ACTIVE
- **Request signing (optional):** Ed25519 signatures on `X-Federation-Signature` header for non-repudiation (configurable, not enforced in MVP)
- **Rate limiting:** 30 req/min for peer routes, 60 req/min for public routes (separate Redis prefixes)
- **CORS:** Peer routes need permissive CORS (cross-domain by nature)
- **Input validation:** All incoming peer data Zod-validated + HTML-escaped before storage

---

## Environment Variables

Add to `api/src/config/env.ts`:
```
ENABLE_FEDERATION: z.string().default('false')
FEDERATION_SYNC_INTERVAL_MINUTES: z.coerce.number().default(60)
FEDERATION_MAX_CAMPAIGNS_PER_SYNC: z.coerce.number().default(500)
FEDERATION_PEER_TIMEOUT_MS: z.coerce.number().default(15000)
FEDERATION_MAX_PEERS: z.coerce.number().default(50)
```

---

## Admin UI

### FederationPage (`admin/src/pages/FederationPage.tsx`)

4-tab page following PangolinPage pattern:

**Tab 1 — Identity & Settings:** Toggle federation, instance profile form, keypair management, hub/spoke settings

**Tab 2 — Connected Peers:** Table of peers (name, URL, role tag, status tag, campaigns shared, last sync, actions). "Register with Hub" button opens modal. Pending incoming registrations highlighted.

**Tab 3 — Federated Campaigns:** Card grid/table of federated campaigns with search + filter (region, tags, government level). Click-through links to source instances.

**Tab 4 — Sync Status:** Last/next sync, per-peer status, manual sync button, sync history.

### Sidebar
Add to `buildMenuItems()` in `AppLayout.tsx`, gated on `settings?.enableFederation`:
```typescript
{ key: '/app/federation', icon: <GlobalOutlined />, label: 'Federation' }
```
(Using `<GlobalOutlined />` since `<GlobalOutlined />` is already imported but used for Web submenu — may use `<ClusterOutlined />` or `<DeploymentUnitOutlined />` instead)

### Route in App.tsx
```tsx
<Route path="federation" element={<ProtectedRoute requiredRoles={['SUPER_ADMIN']}><FederationPage /></ProtectedRoute>} />
```

### Campaign form integration
Add `federated` checkbox to campaign create/edit form in CampaignsPage, visible only when `settings.enableFederation` is true.

### TypeScript types
Add `FederationIdentity`, `FederationPeer`, `FederatedCampaign`, `FederationSyncStatus` interfaces to `admin/src/types/api.ts`.

### Public network page (stretch goal in MVP)
`admin/src/pages/public/FederatedCampaignsPage.tsx` at `/network` route — card grid of federated campaigns with PublicLayout dark theme.

---

## Prometheus Metrics

Add to `api/src/utils/metrics.ts`:
- `cm_federation_peers_active` (Gauge)
- `cm_federation_campaigns_shared` (Gauge)
- `cm_federation_sync_duration_seconds` (Histogram)
- `cm_federation_sync_errors_total` (Counter with `peer_id` label)

---

## Implementation Order

| Step | Description | Files Created/Modified | Depends On |
|------|-------------|----------------------|------------|
| 1 | **Prisma schema** — Add enums, 3 new models, Campaign.federated, SiteSettings.enableFederation | `api/prisma/schema.prisma` | — |
| 2 | **Migration** — `npx prisma migrate dev --name add-federation` | `api/prisma/migrations/` | Step 1 |
| 3 | **Env vars** — Add federation config to env.ts + .env.example | `api/src/config/env.ts`, `.env.example` | — |
| 4 | **Crypto service** — Ed25519 keypair, sign/verify | `api/src/modules/federation/federation-crypto.service.ts` | — |
| 5 | **Schemas** — Zod validation for all federation endpoints | `api/src/modules/federation/federation.schemas.ts` | Step 1 |
| 6 | **Core service** — Identity CRUD, peer management, buildSafeCampaignPayload, campaign sync logic | `api/src/modules/federation/federation.service.ts` | Steps 2, 4, 5 |
| 7 | **Admin routes** — SUPER_ADMIN federation management | `api/src/modules/federation/federation-admin.routes.ts` | Step 6 |
| 8 | **Peer routes** — Inter-instance API with authenticatePeer middleware | `api/src/modules/federation/federation-peer.routes.ts` | Step 6 |
| 9 | **Public routes** — Browsing federated campaigns | `api/src/modules/federation/federation-public.routes.ts` | Step 6 |
| 10 | **Rate limiting** — Add federation rate limiters | `api/src/middleware/rate-limit.ts` | — |
| 11 | **Server mounting** — Import + mount routers, start sync queue | `api/src/server.ts` | Steps 7-10 |
| 12 | **Sync queue** — BullMQ repeatable job for periodic sync | `api/src/services/federation-sync-queue.service.ts` | Step 6 |
| 13 | **Metrics** — Prometheus counters/gauges | `api/src/utils/metrics.ts` | — |
| 14 | **Campaign form** — Add `federated` to schemas + service + CampaignsPage checkbox | `api/src/modules/influence/campaigns/campaigns.schemas.ts`, `campaigns.service.ts`, `admin/src/pages/CampaignsPage.tsx` | Step 2 |
| 15 | **Frontend types** — Federation TypeScript interfaces | `admin/src/types/api.ts` | — |
| 16 | **FederationPage** — 4-tab admin page | `admin/src/pages/FederationPage.tsx` | Steps 7, 15 |
| 17 | **Sidebar + routing** — Menu item + route in AppLayout/App.tsx | `admin/src/components/AppLayout.tsx`, `admin/src/App.tsx` | Step 16 |
| 18 | **Public network page** (stretch) — Federated campaigns browse | `admin/src/pages/public/FederatedCampaignsPage.tsx` | Steps 9, 15 |

---

## Future Extensions (not in MVP, but models accommodate)

- **Campaign adoption** — "Fork" a federated campaign locally (`FederatedCampaign.adoptedAsCampaignId`)
- **Cross-instance response sharing** — New `FederatedResponse` model synced alongside campaigns
- **Named networks/coalitions** — `FederationNetwork` + `FederationNetworkMember` models for named alliances
- **Hub-of-hubs discovery** — Hubs share known-hub lists for transitive discovery (gossip protocol)

---

## Verification

1. **Two-instance test:** Run two API instances on different ports, enable federation on both, register one with the other
2. **Campaign sync:** Create a federated campaign on spoke, verify it appears in hub's directory
3. **Privacy boundary:** Inspect sync payloads — verify no emails, user IDs, or email bodies leak
4. **Offline handling:** Stop one instance, verify the other marks it OFFLINE after 5 failed syncs, then recovers on restart
5. **Rate limiting:** Hit peer endpoints rapidly, verify 429 responses after threshold
6. **Feature gate:** Disable federation in settings, verify all routes return 403/hidden
7. **UI:** Verify sidebar item appears/hides with feature toggle, all 4 tabs functional