bunker-admin 7895ce683e Tonne of debugging - getting ready for the production builds

2026-02-16 10:44:18 -07:00

25 KiB

Raw Blame History

Representatives Module

Overview

The Representatives module integrates with the Canadian Represent API to provide elected official lookup by postal code. It features intelligent caching, rate limiting, deduplication, and both public and admin endpoints for managing representative data.

Key Features:

Canadian representative lookup via Represent API (MPs, MPPs, councillors)
Intelligent cache-first strategy with fire-and-forget cache writes
Rate limiting (55 requests/minute, under Represent API's 60/min limit)
Representative deduplication (centroid + concordance results)
Public postal code lookup (no auth required)
Admin cache management (view, clear, stats)
Integration with postal codes module for location metadata
Health check endpoint for API connectivity testing

File Paths

File	Purpose
`api/src/modules/influence/representatives/representatives.routes.ts`	Router with 8 endpoints (2 public, 6 admin)
`api/src/modules/influence/representatives/representatives.service.ts`	Representative business logic + Represent API integration
`api/src/modules/influence/representatives/representatives.schemas.ts`	Zod validation schemas
`api/src/modules/influence/representatives/represent-api.client.ts`	Represent API HTTP client with rate limiting

Database Model

model Representative {
  id                    String    @id @default(cuid())
  postalCode            String
  name                  String?
  email                 String?
  districtName          String?
  electedOffice         String?
  partyName             String?
  representativeSetName String?
  url                   String?
  photoUrl              String?
  offices               Json?     // JSON array of office contact info
  cachedAt              DateTime  @default(now())

  @@index([postalCode])
  @@map("representatives")
}

Field Descriptions:

postalCode — Canadian postal code (e.g., "M5H 2N2")
name — Representative's full name
email — Contact email address
districtName — Electoral district name (e.g., "Toronto Centre")
electedOffice — Position (e.g., "MP", "MPP", "Councillor")
partyName — Political party affiliation
representativeSetName — Data source identifier (e.g., "House of Commons")
url — Representative's official website
photoUrl — Profile photo URL
offices — JSON array of office locations with contact info
cachedAt — Timestamp when cached from Represent API

API Endpoints

Public Endpoints (No Authentication)

Method	Path	Description
GET	`/api/representatives/by-postal/:postalCode`	Lookup representatives by postal code (cache-first)
GET	`/api/representatives/test-connection`	Test Represent API connectivity

Admin Endpoints (Authentication Required)

Method	Path	Auth	Description
GET	`/api/representatives/cache-stats`	Admin roles	Get cache statistics
GET	`/api/representatives`	Admin roles	List all cached representatives (paginated)
GET	`/api/representatives/:id`	Admin roles	Get single cached representative
DELETE	`/api/representatives/by-postal/:postalCode`	Admin roles	Clear cache for postal code
DELETE	`/api/representatives/:id`	Admin roles	Delete single cached representative

Admin Roles: SUPER_ADMIN, INFLUENCE_ADMIN, MAP_ADMIN

Public Endpoint Details

GET /api/representatives/by-postal/:postalCode

Lookup representatives by Canadian postal code. Uses cache-first strategy: returns cached results if available, otherwise calls Represent API and caches results asynchronously.

Path Parameters:

postalCode (string): Canadian postal code (e.g., "M5H2N2" or "M5H 2N2")

Query Parameters:

Parameter	Type	Required	Default	Description
refresh	boolean	No	false	Force API call even if cached data exists

Example Request:

# Cache-first lookup
curl "http://api.cmlite.org/api/representatives/by-postal/M5H2N2"

# Force refresh from API
curl "http://api.cmlite.org/api/representatives/by-postal/M5H2N2?refresh=true"

Response (200 OK):

{
  "source": "cache",
  "postalCode": "M5H2N2",
  "location": {
    "city": "Toronto",
    "province": "ON"
  },
  "representatives": [
    {
      "id": "clx1234567890",
      "postalCode": "M5H2N2",
      "name": "Chrystia Freeland",
      "email": "chrystia.freeland@parl.gc.ca",
      "districtName": "University—Rosedale",
      "electedOffice": "MP",
      "partyName": "Liberal",
      "representativeSetName": "House of Commons",
      "url": "https://www.ourcommons.ca/members/en/chrystia-freeland(71619)",
      "photoUrl": "https://www.ourcommons.ca/Content/Parliamentarians/Images/OfficialMPPhotos/44/FreelendeC_Lib.jpg",
      "offices": [
        {
          "type": "constituency",
          "tel": "416-656-2424",
          "fax": "416-656-2425",
          "postal": "703-2005 Sheppard Ave E, Toronto ON M2J 5B4"
        }
      ],
      "cachedAt": "2026-02-11T12:00:00.000Z"
    },
    {
      "id": "clx0987654321",
      "postalCode": "M5H2N2",
      "name": "Suze Morrison",
      "email": "smorrisons@ola.org",
      "districtName": "Toronto Centre",
      "electedOffice": "MPP",
      "partyName": "NDP",
      "representativeSetName": "Legislative Assembly of Ontario",
      "url": "https://www.ola.org/en/members/all/suze-morrison",
      "photoUrl": null,
      "offices": [],
      "cachedAt": "2026-02-11T12:00:00.000Z"
    }
  ]
}

Response Fields:

source — Data source: "cache" (from database) or "api" (fresh from Represent API)
postalCode — Normalized postal code
location — City and province from PostalCodeCache table
representatives — Array of representative objects

Error Responses:

400 Bad Request: Invalid postal code format
404 Not Found: Postal code not found in Represent API
429 Too Many Requests: Rate limit exceeded (55/min)
500 Internal Server Error: Represent API unreachable or other error

Caching Strategy:

// 1. Check cache first (unless forceRefresh)
const cached = await prisma.representative.findMany({ where: { postalCode: code } });
if (cached.length > 0 && !forceRefresh) {
  return { source: 'cache', representatives: cached };
}

// 2. Call Represent API
const apiResponse = await representApiClient.getByPostalCode(code);

// 3. Fire-and-forget cache write (don't await)
cacheWrite(); // Deletes old cache, creates new entries

// 4. Return API results immediately (don't wait for cache)
return { source: 'api', representatives: uniqueReps };

Deduplication:

Representatives from both representatives_centroid and representatives_concordance are merged and deduplicated by name|elected_office key to avoid duplicate entries.

function deduplicateReps(reps: RepresentRepresentative[]): RepresentRepresentative[] {
  const seen = new Set<string>();
  return reps.filter((rep) => {
    const key = `${rep.name}|${rep.elected_office}`;
    if (seen.has(key)) return false;
    seen.add(key);
    return true;
  });
}

GET /api/representatives/test-connection

Test connectivity to the Represent API.

Example Request:

curl "http://api.cmlite.org/api/representatives/test-connection"

Response (200 OK):

{
  "ok": true,
  "message": "Represent API is reachable"
}

Response (200 OK, API Down):

{
  "ok": false,
  "message": "HTTP 503"
}

Use Cases:

Health checks for monitoring dashboards
Troubleshooting representative lookup issues
Verifying API configuration in admin settings

Admin Endpoint Details

GET /api/representatives/cache-stats

Get cache statistics for the representatives cache.

Authentication: Required (Admin roles)

Example Request:

curl -H "Authorization: Bearer <token>" \
  "http://api.cmlite.org/api/representatives/cache-stats"

Response (200 OK):

{
  "totalRepresentatives": 1247,
  "postalCodesWithRepresentatives": 412,
  "totalPostalCodes": 450
}

Field Descriptions:

totalRepresentatives — Total cached representative records
postalCodesWithRepresentatives — Unique postal codes with cached representatives
totalPostalCodes — Total postal codes in PostalCodeCache table (includes codes without representatives)

GET /api/representatives

List all cached representatives with pagination and search.

Authentication: Required (Admin roles)

Query Parameters:

Parameter	Type	Required	Default	Description
page	number	No	1	Page number
limit	number	No	20	Results per page (max 100)
search	string	No	-	Search name, email, district, or office
postalCode	string	No	-	Filter by postal code

Example Request:

curl -H "Authorization: Bearer <token>" \
  "http://api.cmlite.org/api/representatives?page=1&limit=10&search=Toronto&postalCode=M5H2N2"

Response (200 OK):

{
  "representatives": [
    {
      "id": "clx1234567890",
      "postalCode": "M5H2N2",
      "name": "Chrystia Freeland",
      "email": "chrystia.freeland@parl.gc.ca",
      "districtName": "University—Rosedale",
      "electedOffice": "MP",
      "partyName": "Liberal",
      "representativeSetName": "House of Commons",
      "url": "https://www.ourcommons.ca/members/en/chrystia-freeland(71619)",
      "photoUrl": "https://www.ourcommons.ca/Content/Parliamentarians/Images/OfficialMPPhotos/44/FreelendeC_Lib.jpg",
      "offices": [...],
      "cachedAt": "2026-02-11T12:00:00.000Z"
    }
  ],
  "pagination": {
    "page": 1,
    "limit": 10,
    "total": 15,
    "totalPages": 2
  }
}

Search Logic:

Search term is matched against name, email, district name, or elected office (case-insensitive):

if (search) {
  where.OR = [
    { name: { contains: search, mode: 'insensitive' } },
    { email: { contains: search, mode: 'insensitive' } },
    { districtName: { contains: search, mode: 'insensitive' } },
    { electedOffice: { contains: search, mode: 'insensitive' } },
  ];
}

GET /api/representatives/:id

Get single cached representative by ID.

Authentication: Required (Admin roles)

Path Parameters:

id (string): Representative ID (cuid)

Example Request:

curl -H "Authorization: Bearer <token>" \
  "http://api.cmlite.org/api/representatives/clx1234567890"

Response (200 OK):

Returns single representative object (same format as list).

Error Responses:

404 Not Found: Representative not found

DELETE /api/representatives/by-postal/:postalCode

Clear all cached representatives for a specific postal code.

Authentication: Required (Admin roles)

Path Parameters:

postalCode (string): Canadian postal code

Example Request:

curl -X DELETE -H "Authorization: Bearer <token>" \
  "http://api.cmlite.org/api/representatives/by-postal/M5H2N2"

Response (200 OK):

{
  "deleted": 3,
  "postalCode": "M5H2N2"
}

Use Cases:

Force cache refresh for specific postal code
Remove stale data after election
Troubleshoot incorrect representative data

DELETE /api/representatives/:id

Delete single cached representative by ID.

Authentication: Required (Admin roles)

Path Parameters:

id (string): Representative ID (cuid)

Example Request:

curl -X DELETE -H "Authorization: Bearer <token>" \
  "http://api.cmlite.org/api/representatives/clx1234567890"

Response (204 No Content):

No response body.

Error Responses:

404 Not Found: Representative not found

Represent API Integration

API Client

The represent-api.client.ts file provides a typed HTTP client for the Represent API.

Base URL:

const REPRESENT_API_URL = 'https://represent.opennorth.ca';

Configuration:

Set REPRESENT_API_URL in .env to override (default: https://represent.opennorth.ca).

Methods:

class RepresentApiClient {
  // Lookup by postal code
  async getByPostalCode(code: string): Promise<RepresentPostalCodeResponse>;

  // Health check
  async testConnection(): Promise<{ ok: boolean; message: string }>;
}

Rate Limiting

Limits:

Represent API: 60 requests/minute
Changemaker Lite: 55 requests/minute (safety margin)

Implementation:

In-memory sliding window rate limiter:

const RATE_LIMIT = 55;
const RATE_WINDOW_MS = 60_000;
const requestTimestamps: number[] = [];

function checkRateLimit(): boolean {
  const now = Date.now();
  // Remove timestamps outside the window
  while (requestTimestamps.length > 0 && requestTimestamps[0] < now - RATE_WINDOW_MS) {
    requestTimestamps.shift();
  }
  return requestTimestamps.length < RATE_LIMIT;
}

function recordRequest(): void {
  requestTimestamps.push(Date.now());
}

Behavior:

If rate limit exceeded: throws Error('Represent API rate limit reached. Please try again in a minute.')
Returns 429 status to client
Resets after 1 minute

Response Schema

interface RepresentPostalCodeResponse {
  city: string | null;
  province: string | null;
  centroid: { type: string; coordinates: [number, number] } | null;
  representatives_centroid: RepresentRepresentative[];
  representatives_concordance: RepresentRepresentative[];
}

interface RepresentRepresentative {
  name: string;
  email: string | null;
  elected_office: string;
  district_name: string;
  party_name: string | null;
  representative_set_name: string;
  url: string;
  photo_url: string | null;
  offices: RepresentOffice[];
}

interface RepresentOffice {
  type?: string;       // "constituency" or "legislature"
  tel?: string;        // Phone number
  fax?: string;        // Fax number
  postal?: string;     // Mailing address
}

Centroid vs. Concordance:

representatives_centroid — Representatives found using the postal code's geographic centroid
representatives_concordance — Representatives found using postal code concordance tables (may be more accurate for boundary-edge postal codes)
Both arrays are merged and deduplicated by Changemaker Lite

Service Functions

representativesService.lookupByPostalCode(code, forceRefresh)

Cache-first representative lookup.

Parameters:

code (string): Canadian postal code
forceRefresh (boolean, default: false): Skip cache and force API call

Returns:

{
  source: 'cache' | 'api';
  postalCode: string;
  location: { city: string | null; province: string | null };
  representatives: Representative[];
}

Logic Flow:

Check cache unless forceRefresh=true
If cached data found, return immediately with source: 'cache'
If no cache or forceRefresh, call Represent API
Merge centroid + concordance representatives and deduplicate
Fire-and-forget cache write (delete old, insert new, upsert postal code)
Return API results with source: 'api' (don't wait for cache)

Fire-and-Forget Caching:

const cacheWrite = async () => {
  try {
    // Delete old cached reps for this postal code
    await prisma.representative.deleteMany({ where: { postalCode: code } });

    // Cache new reps
    await prisma.representative.createMany({
      data: uniqueReps.map((rep) => ({
        postalCode: code,
        name: rep.name || null,
        email: rep.email || null,
        districtName: rep.district_name || null,
        electedOffice: rep.elected_office || null,
        partyName: rep.party_name || null,
        representativeSetName: rep.representative_set_name || null,
        url: rep.url || null,
        photoUrl: rep.photo_url || null,
        offices: rep.offices ? (rep.offices as unknown as Prisma.InputJsonValue) : Prisma.JsonNull,
      })),
    });

    // Upsert postal code cache (city, province, centroid)
    await postalCodesService.upsert({
      postalCode: code,
      city: apiResponse.city,
      province: apiResponse.province,
      centroidLat: coords ? coords[1] : null,
      centroidLng: coords ? coords[0] : null,
    });
  } catch (err) {
    logger.error('Failed to cache representatives', { postalCode: code, error: err });
  }
};

// Don't await — fire and forget
cacheWrite();

Why Fire-and-Forget?

Returns API results to user immediately (faster response)
Cache failures don't block user requests
Next lookup will use cached data if write succeeds
Errors logged for monitoring but don't propagate to user

representativesService.findAll(filters)

List cached representatives with pagination and search.

Parameters:

{
  page: number;      // Page number (default: 1)
  limit: number;     // Results per page (max 100, default: 20)
  search?: string;   // Search term (optional)
  postalCode?: string; // Filter by postal code (optional)
}

Returns:

{
  representatives: Representative[];
  pagination: {
    page: number;
    limit: number;
    total: number;
    totalPages: number;
  };
}

representativesService.findById(id)

Get single cached representative by ID.

Throws: AppError(404) if not found

representativesService.clearByPostalCode(code)

Delete all cached representatives for a postal code.

Returns:

{
  deleted: number;      // Count of deleted records
  postalCode: string;
}

representativesService.deleteById(id)

Delete single cached representative by ID.

Throws: AppError(404) if not found

representativesService.testApiConnection()

Test connectivity to Represent API.

Returns:

{
  ok: boolean;
  message: string;
}

Implementation:

Calls Represent API's /boundary-sets/?limit=1 endpoint (lightweight health check).

representativesService.getCacheStats()

Get cache statistics.

Returns:

{
  totalRepresentatives: number;         // Total cached representative records
  postalCodesWithRepresentatives: number; // Unique postal codes with reps
  totalPostalCodes: number;              // Total postal codes in cache
}

Implementation:

const [totalReps, postalCodesWithReps, totalPostalCodes] = await Promise.all([
  prisma.representative.count(),
  prisma.representative.groupBy({ by: ['postalCode'] }).then((g) => g.length),
  prisma.postalCodeCache.count(),
]);

return {
  totalRepresentatives: totalReps,
  postalCodesWithRepresentatives: postalCodesWithReps,
  totalPostalCodes,
};

Validation Schemas

List Representatives Schema

export const listRepresentativesSchema = z.object({
  page: z.coerce.number().int().positive().default(1),
  limit: z.coerce.number().int().positive().max(100).default(20),
  search: z.string().optional(),
  postalCode: z.string().optional(),
});

export type ListRepresentativesInput = z.infer<typeof listRepresentativesSchema>;

Coercion:

page and limit coerced from query string to number
Invalid values fallback to defaults

Integration with Postal Codes Module

The representatives module integrates with the postal codes module (api/src/modules/influence/postal-codes/) for location metadata.

PostalCodeCache Model:

model PostalCodeCache {
  id          String    @id @default(cuid())
  postalCode  String    @unique
  city        String?
  province    String?
  centroidLat Float?
  centroidLng Float?
  cachedAt    DateTime  @default(now())
}

Integration Points:

Lookup: When returning cached representatives, fetch city/province from PostalCodeCache:

const postalInfo = await postalCodesService.findByPostalCode(code);
return {
  source: 'cache',
  location: {
    city: postalInfo?.city ?? null,
    province: postalInfo?.province ?? null,
  },
  representatives: cached,
};

Cache Write: After calling Represent API, upsert postal code with location data:

await postalCodesService.upsert({
  postalCode: code,
  city: apiResponse.city,
  province: apiResponse.province,
  centroidLat: coords ? coords[1] : null,
  centroidLng: coords ? coords[0] : null,
});

Code Examples

Public: Lookup Representatives by Postal Code

import axios from 'axios';

const lookupRepresentatives = async (postalCode: string) => {
  const { data } = await axios.get(
    `/api/representatives/by-postal/${postalCode}`
  );

  console.log(`Source: ${data.source}`); // "cache" or "api"
  console.log(`Location: ${data.location.city}, ${data.location.province}`);

  data.representatives.forEach((rep) => {
    console.log(`${rep.name} (${rep.electedOffice}) - ${rep.email}`);
  });

  return data;
};

// Cache-first lookup
await lookupRepresentatives('M5H2N2');

// Force refresh from API
const { data } = await axios.get('/api/representatives/by-postal/M5H2N2?refresh=true');

Admin: Get Cache Statistics

import { api } from '@/lib/api';

const getCacheStats = async () => {
  const { data } = await api.get('/api/representatives/cache-stats');

  console.log(`Total Representatives: ${data.totalRepresentatives}`);
  console.log(`Postal Codes with Reps: ${data.postalCodesWithRepresentatives}`);
  console.log(`Total Postal Codes: ${data.totalPostalCodes}`);

  return data;
};

Admin: Clear Cache for Postal Code

import { api } from '@/lib/api';
import { message } from 'antd';

const clearPostalCodeCache = async (postalCode: string) => {
  try {
    const { data } = await api.delete(`/api/representatives/by-postal/${postalCode}`);
    message.success(`Cleared ${data.deleted} representatives for ${postalCode}`);
  } catch (error) {
    message.error('Failed to clear cache');
  }
};

Admin: Search Cached Representatives

import { api } from '@/lib/api';

const searchRepresentatives = async (search: string, page: number = 1) => {
  const { data } = await api.get('/api/representatives', {
    params: { search, page, limit: 20 },
  });

  return {
    representatives: data.representatives,
    pagination: data.pagination,
  };
};

Frontend Integration

The RepresentativesPage component (admin/src/pages/RepresentativesPage.tsx) provides:

Cache statistics dashboard (total reps, postal codes, coverage)
Representative cache table with pagination
Search by name, email, district, or office
Filter by postal code
Clear cache by postal code (bulk action)
Delete individual cached representatives
Postal code lookup tool (test Represent API)
Connection test (verify API reachability)
Refresh button (force API call for postal code)

State Management:

const [representatives, setRepresentatives] = useState<Representative[]>([]);
const [pagination, setPagination] = useState({ page: 1, limit: 20, total: 0 });
const [filters, setFilters] = useState({ search: '', postalCode: '' });
const [stats, setStats] = useState({ totalRepresentatives: 0, postalCodesWithRepresentatives: 0, totalPostalCodes: 0 });

Performance Considerations

Cache-First Strategy:

Cached lookups: <10ms (database query)
API lookups: 200-500ms (external API call)
Fire-and-forget writes don't block user response

Rate Limiting:

55 requests/minute limit prevents Represent API 429 errors
In-memory sliding window (no Redis overhead)
Returns 429 status to client when limit exceeded

Database Indexing:

@@index([postalCode]) — Fast lookup by postal code
Ordered by cachedAt DESC — Recent lookups first

Deduplication:

Prevents duplicate representatives from centroid + concordance results
Reduces database storage and frontend rendering load

Troubleshooting

Issue: "Represent API rate limit reached"

Cause: More than 55 requests in 60-second window

Solution:

Wait 1 minute and retry
Use cached data (don't force refresh)
Batch postal code lookups instead of sequential

Issue: Cached data is stale

Cause: Representative changed after election

Solution:

Force refresh: GET /api/representatives/by-postal/:postalCode?refresh=true
Admin clear cache: DELETE /api/representatives/by-postal/:postalCode
Cache will be refreshed on next lookup

Issue: Postal code returns no representatives

Cause: Invalid postal code or Represent API doesn't have data

Solution:

Verify postal code format (e.g., "M5H2N2" or "M5H 2N2")
Check Represent API directly: https://represent.opennorth.ca/postcodes/M5H2N2/
Ensure postal code is Canadian (Represent API is Canada-only)

Issue: Duplicate representatives in cache

Cause: Deduplication bug or manual database insertion

Solution:

Clear cache: DELETE /api/representatives/by-postal/:postalCode
Next lookup will re-deduplicate from API

Postal Codes Module - Postal code cache integration
Campaigns Module - Campaign email sending to representatives
Frontend: RepresentativesPage - Cache management UI
Frontend: Public Campaign Page - Public representative lookup
API Reference: Representatives - Complete endpoint reference
Feature: Influence System - Representative lookup feature guide
Represent API Documentation - Official Represent API docs

25 KiB Raw Blame History

Representatives Module

Overview

File Paths

Database Model

API Endpoints

Public Endpoints (No Authentication)

Admin Endpoints (Authentication Required)

Public Endpoint Details

GET /api/representatives/by-postal/:postalCode

GET /api/representatives/test-connection

Admin Endpoint Details

GET /api/representatives/cache-stats

GET /api/representatives

GET /api/representatives/:id

DELETE /api/representatives/by-postal/:postalCode

DELETE /api/representatives/:id

Represent API Integration

API Client

Rate Limiting

Response Schema

Service Functions

representativesService.lookupByPostalCode(code, forceRefresh)

representativesService.findAll(filters)

representativesService.findById(id)

representativesService.clearByPostalCode(code)

representativesService.deleteById(id)

representativesService.testApiConnection()

representativesService.getCacheStats()

Validation Schemas

List Representatives Schema

Integration with Postal Codes Module

Code Examples

Public: Lookup Representatives by Postal Code

Admin: Get Cache Statistics

Admin: Clear Cache for Postal Code

Admin: Search Cached Representatives

Frontend Integration

Performance Considerations

Troubleshooting

Issue: "Represent API rate limit reached"

Issue: Cached data is stale

Issue: Postal code returns no representatives

Issue: Duplicate representatives in cache

Related Documentation

25 KiB

Raw Blame History