925 lines
25 KiB
Markdown
925 lines
25 KiB
Markdown
# Representative Lookup System
|
|
|
|
## Overview
|
|
|
|
The representative lookup system integrates with the Represent API (Open North) to provide real-time postal code-based representative lookups for advocacy campaigns. It includes intelligent caching to minimize API calls, support for all Canadian government levels, and admin tools for cache management.
|
|
|
|
**Key Capabilities:**
|
|
|
|
- **Represent API integration**: Real-time lookup of elected officials by postal code
|
|
- **Multi-level support**: Federal, provincial, and municipal representatives
|
|
- **Intelligent caching**: Reduce API calls and improve performance
|
|
- **Cache invalidation**: Manual and automatic cache refresh
|
|
- **Admin tools**: Cache statistics, manual lookup, bulk operations
|
|
- **Error handling**: Graceful fallback for API failures
|
|
|
|
**Use Cases:**
|
|
|
|
- Email-your-MP campaigns
|
|
- Multi-level government outreach
|
|
- Representative contact information lookup
|
|
- Geographic representation analysis
|
|
- Campaign targeting by electoral district
|
|
|
|
## Architecture
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Public User] -->|Enter Postal Code| B[CampaignPage]
|
|
B -->|POST /api/public/representatives/lookup| C[Representative Service]
|
|
|
|
C -->|Check Cache| D{Cache Hit?}
|
|
D -->|Yes| E[Return Cached Reps]
|
|
D -->|No| F[Represent API Client]
|
|
|
|
F -->|GET /postcodes/:code| G[Represent API]
|
|
G -->|Return Reps| F
|
|
F -->|Parse & Save| H[(Representative Model)]
|
|
H -->|Return| E
|
|
|
|
I[Admin User] -->|View Cache| J[RepresentativesPage]
|
|
J -->|GET /api/representatives| C
|
|
J -->|Manual Lookup| C
|
|
J -->|Clear Cache| K[Delete Service]
|
|
K -->|Delete| H
|
|
|
|
L[Cache Invalidation Job] -->|Check lastUpdated| H
|
|
L -->|Delete Stale| H
|
|
|
|
style H fill:#e1f5ff
|
|
style G fill:#fff4e1
|
|
```
|
|
|
|
**Flow Description:**
|
|
|
|
1. **User enters postal code** → Representative service checks cache
|
|
2. **Cache miss** → Represent API client fetches representatives
|
|
3. **API response** → Parse representatives, save to cache
|
|
4. **Cache hit** → Return cached representatives (skip API call)
|
|
5. **Admin management** → View cache stats, manual lookup, clear cache
|
|
6. **Cache invalidation** → Automatic cleanup of stale entries (>30 days)
|
|
|
|
## Database Models
|
|
|
|
### Representative Model
|
|
|
|
See [Representative Model Documentation](../../database/models/representative.md) for full schema.
|
|
|
|
**Key Fields:**
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `id` | String (UUID) | Primary key |
|
|
| `representId` | String | Represent API unique identifier |
|
|
| `name` | String | Full name of representative |
|
|
| `email` | String | Email address |
|
|
| `districtName` | String | Electoral district name |
|
|
| `electedOffice` | String | Office held (MP, MPP, Mayor, etc.) |
|
|
| `partyName` | String? | Political party affiliation |
|
|
| `photoUrl` | String? | Profile photo URL |
|
|
| `postalCode` | String | Associated postal code (cache key) |
|
|
| `level` | String | Government level (federal, provincial, municipal) |
|
|
| `lastUpdated` | DateTime | Cache timestamp |
|
|
|
|
**Indexes:**
|
|
|
|
- `postalCode, level` — Composite index for fast lookups
|
|
- `representId` — Unique constraint
|
|
- `lastUpdated` — For cache invalidation queries
|
|
|
|
**Related Models:**
|
|
|
|
- [Campaign](../../database/models/campaign.md) — Campaigns target representatives
|
|
- [CampaignEmail](../../database/models/campaign-email.md) — Emails sent to representatives
|
|
|
|
## API Endpoints
|
|
|
|
### Admin Endpoints
|
|
|
|
See [Representatives Module API Reference](../../backend/modules/representatives.md#endpoints) for full details.
|
|
|
|
| Method | Endpoint | Auth | Description |
|
|
|--------|----------|------|-------------|
|
|
| GET | `/api/representatives` | SUPER_ADMIN, INFLUENCE_ADMIN | List all cached representatives |
|
|
| GET | `/api/representatives/stats` | SUPER_ADMIN, INFLUENCE_ADMIN | Get cache statistics |
|
|
| POST | `/api/representatives/lookup` | SUPER_ADMIN, INFLUENCE_ADMIN | Manual postal code lookup |
|
|
| DELETE | `/api/representatives/:id` | SUPER_ADMIN, INFLUENCE_ADMIN | Delete cached representative |
|
|
| DELETE | `/api/representatives/postal-code/:postalCode` | SUPER_ADMIN, INFLUENCE_ADMIN | Delete all reps for postal code |
|
|
|
|
### Public Endpoints
|
|
|
|
See [Representatives Module API Reference](../../backend/modules/representatives.md#public-endpoints).
|
|
|
|
| Method | Endpoint | Auth | Description |
|
|
|--------|----------|------|-------------|
|
|
| POST | `/api/public/representatives/lookup` | None | Lookup representatives by postal code |
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
| Variable | Type | Default | Description |
|
|
|----------|------|---------|-------------|
|
|
| `REPRESENT_API_URL` | string | https://represent.opennorth.ca | Represent API base URL |
|
|
| `REPRESENT_CACHE_TTL` | number | 2592000 | Cache TTL in seconds (30 days) |
|
|
| `REPRESENT_RATE_LIMIT` | number | 60 | Max requests per minute |
|
|
|
|
### Represent API
|
|
|
|
The Represent API is a public service provided by Open North. No API key required.
|
|
|
|
**API Documentation**: https://represent.opennorth.ca/api/
|
|
|
|
**Endpoints Used:**
|
|
|
|
- `GET /postcodes/:postalCode/` — Lookup representatives by postal code
|
|
- `GET /representatives/` — List representatives (unused, direct lookups only)
|
|
|
|
**Rate Limits:**
|
|
|
|
- 60 requests per minute per IP address
|
|
- Exceeding limit returns HTTP 429
|
|
|
|
**Postal Code Format:**
|
|
|
|
- Canadian postal codes only
|
|
- Format: `K1A 0A1` or `K1A0A1` (space optional)
|
|
- Normalized to uppercase without spaces for API calls
|
|
|
|
## Admin Workflow
|
|
|
|
### 1. View Cache Statistics
|
|
|
|
[Screenshot: RepresentativesPage with cache stats cards]
|
|
|
|
**Steps:**
|
|
|
|
1. Navigate to **Influence > Representatives**
|
|
2. View cache statistics:
|
|
- **Total Cached**: Total representatives in cache
|
|
- **Unique Postal Codes**: Number of postal codes cached
|
|
- **Cache Hit Rate**: Percentage of lookups served from cache
|
|
- **Stale Entries**: Entries older than 30 days
|
|
|
|
**Code Example (RepresentativesPage.tsx):**
|
|
|
|
```typescript
|
|
const [stats, setStats] = useState({
|
|
totalCached: 0,
|
|
uniquePostalCodes: 0,
|
|
cacheHitRate: 0,
|
|
staleEntries: 0
|
|
});
|
|
|
|
useEffect(() => {
|
|
const fetchStats = async () => {
|
|
const { data } = await api.get('/representatives/stats');
|
|
setStats(data);
|
|
};
|
|
|
|
fetchStats();
|
|
}, []);
|
|
|
|
return (
|
|
<Row gutter={16}>
|
|
<Col span={6}>
|
|
<Card>
|
|
<Statistic title="Total Cached" value={stats.totalCached} />
|
|
</Card>
|
|
</Col>
|
|
<Col span={6}>
|
|
<Card>
|
|
<Statistic title="Unique Postal Codes" value={stats.uniquePostalCodes} />
|
|
</Card>
|
|
</Col>
|
|
<Col span={6}>
|
|
<Card>
|
|
<Statistic
|
|
title="Cache Hit Rate"
|
|
value={stats.cacheHitRate}
|
|
suffix="%"
|
|
precision={1}
|
|
/>
|
|
</Card>
|
|
</Col>
|
|
<Col span={6}>
|
|
<Card>
|
|
<Statistic
|
|
title="Stale Entries"
|
|
value={stats.staleEntries}
|
|
valueStyle={{ color: stats.staleEntries > 0 ? '#cf1322' : undefined }}
|
|
/>
|
|
</Card>
|
|
</Col>
|
|
</Row>
|
|
);
|
|
```
|
|
|
|
### 2. Manual Postal Code Lookup
|
|
|
|
[Screenshot: RepresentativesPage with postal code search form]
|
|
|
|
**Steps:**
|
|
|
|
1. Enter postal code in search box (e.g., "K1A 0A1")
|
|
2. Click **Lookup** button
|
|
3. View results:
|
|
- Representative name, office, party
|
|
- Electoral district
|
|
- Email address (if available)
|
|
4. Results automatically cached for future lookups
|
|
|
|
**Use Cases:**
|
|
|
|
- Pre-populate cache for campaign areas
|
|
- Verify representative information
|
|
- Test postal code validation
|
|
- Troubleshoot lookup issues
|
|
|
|
**Code Example (representatives.service.ts):**
|
|
|
|
```typescript
|
|
async lookupByPostalCode(postalCode: string): Promise<Representative[]> {
|
|
// Normalize postal code
|
|
const normalized = postalCode.toUpperCase().replace(/\s/g, '');
|
|
|
|
// Check cache first (within last 30 days)
|
|
const cached = await this.prisma.representative.findMany({
|
|
where: {
|
|
postalCode: normalized,
|
|
lastUpdated: {
|
|
gte: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000) // 30 days
|
|
}
|
|
}
|
|
});
|
|
|
|
if (cached.length > 0) {
|
|
logger.info(`Cache hit for postal code ${normalized}`);
|
|
return cached;
|
|
}
|
|
|
|
// Cache miss - fetch from Represent API
|
|
logger.info(`Cache miss for postal code ${normalized}, fetching from API`);
|
|
|
|
const representatives = await this.representApiClient.getRepresentativesByPostalCode(
|
|
normalized
|
|
);
|
|
|
|
// Save to cache
|
|
const saved = await Promise.all(
|
|
representatives.map(rep =>
|
|
this.prisma.representative.upsert({
|
|
where: { representId: rep.representId },
|
|
update: {
|
|
...rep,
|
|
postalCode: normalized,
|
|
lastUpdated: new Date()
|
|
},
|
|
create: {
|
|
...rep,
|
|
postalCode: normalized,
|
|
lastUpdated: new Date()
|
|
}
|
|
})
|
|
)
|
|
);
|
|
|
|
return saved;
|
|
}
|
|
```
|
|
|
|
### 3. Clear Stale Cache Entries
|
|
|
|
[Screenshot: RepresentativesPage with "Clear Stale Cache" button]
|
|
|
|
**Steps:**
|
|
|
|
1. Click **Clear Stale Cache** button
|
|
2. Confirm deletion in modal
|
|
3. System deletes all entries older than 30 days
|
|
4. View updated cache statistics
|
|
|
|
**Automatic Cleanup:**
|
|
|
|
Cache invalidation also runs automatically via cron job (daily at 2 AM):
|
|
|
|
```typescript
|
|
// api/src/server.ts
|
|
|
|
import cron from 'node-cron';
|
|
|
|
// Clean stale representative cache daily at 2 AM
|
|
cron.schedule('0 2 * * *', async () => {
|
|
try {
|
|
const thirtyDaysAgo = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000);
|
|
|
|
const result = await prisma.representative.deleteMany({
|
|
where: {
|
|
lastUpdated: {
|
|
lt: thirtyDaysAgo
|
|
}
|
|
}
|
|
});
|
|
|
|
logger.info(`Deleted ${result.count} stale representative cache entries`);
|
|
} catch (error) {
|
|
logger.error('Failed to clean representative cache:', error);
|
|
}
|
|
});
|
|
```
|
|
|
|
### 4. Delete Specific Cache Entries
|
|
|
|
[Screenshot: RepresentativesPage table with delete buttons]
|
|
|
|
**Steps:**
|
|
|
|
1. Browse cached representatives table
|
|
2. Click **Delete** button on specific row
|
|
3. Confirm deletion
|
|
4. Representative removed from cache (will be re-fetched on next lookup)
|
|
|
|
**Bulk Delete by Postal Code:**
|
|
|
|
1. Click **Delete All** button on postal code group
|
|
2. Confirm deletion
|
|
3. All representatives for that postal code removed from cache
|
|
|
|
## Public Workflow
|
|
|
|
### 1. Enter Postal Code
|
|
|
|
[Screenshot: CampaignPage with postal code input field]
|
|
|
|
**User Journey:**
|
|
|
|
1. User visits campaign page (`/campaigns/{slug}`)
|
|
2. Enters postal code in lookup form
|
|
3. Clicks **Find My Representatives**
|
|
4. System performs lookup (cache or API)
|
|
5. Representatives displayed below form
|
|
|
|
**Code Example (CampaignPage.tsx):**
|
|
|
|
```typescript
|
|
const [representatives, setRepresentatives] = useState<Representative[]>([]);
|
|
const [loading, setLoading] = useState(false);
|
|
|
|
const handleLookup = async (values: { postalCode: string }) => {
|
|
setLoading(true);
|
|
|
|
try {
|
|
const { data } = await axios.post('/api/public/representatives/lookup', {
|
|
postalCode: values.postalCode
|
|
});
|
|
|
|
setRepresentatives(data);
|
|
|
|
if (data.length === 0) {
|
|
message.warning('No representatives found for this postal code');
|
|
}
|
|
} catch (error) {
|
|
message.error('Failed to lookup representatives');
|
|
} finally {
|
|
setLoading(false);
|
|
}
|
|
};
|
|
|
|
return (
|
|
<Form onFinish={handleLookup}>
|
|
<Form.Item
|
|
name="postalCode"
|
|
label="Postal Code"
|
|
rules={[
|
|
{ required: true, message: 'Please enter your postal code' },
|
|
{
|
|
pattern: /^[A-Za-z]\d[A-Za-z][ -]?\d[A-Za-z]\d$/,
|
|
message: 'Please enter a valid Canadian postal code'
|
|
}
|
|
]}
|
|
>
|
|
<Input placeholder="K1A 0A1" maxLength={7} />
|
|
</Form.Item>
|
|
|
|
<Form.Item>
|
|
<Button type="primary" htmlType="submit" loading={loading}>
|
|
Find My Representatives
|
|
</Button>
|
|
</Form.Item>
|
|
</Form>
|
|
);
|
|
```
|
|
|
|
### 2. View Representatives
|
|
|
|
[Screenshot: Representative cards with contact information]
|
|
|
|
**Display Fields:**
|
|
|
|
- Representative name
|
|
- Elected office (MP, MPP, Mayor, Councillor)
|
|
- Political party (if applicable)
|
|
- Electoral district name
|
|
- Photo (if available)
|
|
- Email button (if email available)
|
|
|
|
**Filtering:**
|
|
|
|
Representatives filtered by campaign's `targetGovernmentLevels`:
|
|
|
|
```typescript
|
|
// Filter representatives by campaign levels
|
|
const filteredRepresentatives = representatives.filter(rep =>
|
|
campaign.targetGovernmentLevels.includes(rep.level)
|
|
);
|
|
```
|
|
|
|
### 3. Select Representatives to Email
|
|
|
|
[Screenshot: Representative list with checkboxes]
|
|
|
|
**User Journey:**
|
|
|
|
1. User reviews list of representatives
|
|
2. Selects representatives to email (checkboxes)
|
|
3. Clicks **Continue** to email form
|
|
4. System pre-populates recipient list
|
|
|
|
**Code Example:**
|
|
|
|
```typescript
|
|
const [selectedReps, setSelectedReps] = useState<string[]>([]);
|
|
|
|
const handleSelectAll = () => {
|
|
setSelectedReps(representatives.map(r => r.id));
|
|
};
|
|
|
|
const handleSelectNone = () => {
|
|
setSelectedReps([]);
|
|
};
|
|
|
|
return (
|
|
<Space direction="vertical" style={{ width: '100%' }}>
|
|
<Space>
|
|
<Button onClick={handleSelectAll}>Select All</Button>
|
|
<Button onClick={handleSelectNone}>Select None</Button>
|
|
</Space>
|
|
|
|
<Checkbox.Group
|
|
value={selectedReps}
|
|
onChange={setSelectedReps}
|
|
style={{ width: '100%' }}
|
|
>
|
|
{representatives.map(rep => (
|
|
<Card key={rep.id} style={{ marginBottom: 16 }}>
|
|
<Checkbox value={rep.id}>
|
|
<Space>
|
|
{rep.photoUrl && (
|
|
<Avatar src={rep.photoUrl} size={64} />
|
|
)}
|
|
<Space direction="vertical" size={0}>
|
|
<Typography.Text strong>{rep.name}</Typography.Text>
|
|
<Typography.Text type="secondary">{rep.electedOffice}</Typography.Text>
|
|
<Typography.Text type="secondary">{rep.districtName}</Typography.Text>
|
|
{rep.partyName && <Tag>{rep.partyName}</Tag>}
|
|
</Space>
|
|
</Space>
|
|
</Checkbox>
|
|
</Card>
|
|
))}
|
|
</Checkbox.Group>
|
|
</Space>
|
|
);
|
|
```
|
|
|
|
## Volunteer Workflow
|
|
|
|
Not applicable — representative lookup is public-facing and admin-managed.
|
|
|
|
## Code Examples
|
|
|
|
### Backend: Represent API Client
|
|
|
|
```typescript
|
|
// api/src/modules/influence/representatives/represent-api.client.ts
|
|
|
|
import axios from 'axios';
|
|
import { logger } from '../../../utils/logger';
|
|
|
|
const REPRESENT_API_URL = process.env.REPRESENT_API_URL || 'https://represent.opennorth.ca';
|
|
|
|
interface RepresentApiResponse {
|
|
objects: Array<{
|
|
name: string;
|
|
email: string;
|
|
district_name: string;
|
|
elected_office: string;
|
|
party_name?: string;
|
|
photo_url?: string;
|
|
url: string;
|
|
representative_set_name: string;
|
|
}>;
|
|
}
|
|
|
|
export class RepresentApiClient {
|
|
async getRepresentativesByPostalCode(postalCode: string): Promise<any[]> {
|
|
try {
|
|
const { data } = await axios.get<RepresentApiResponse>(
|
|
`${REPRESENT_API_URL}/postcodes/${postalCode}/`,
|
|
{
|
|
headers: {
|
|
'Accept': 'application/json'
|
|
},
|
|
timeout: 10000
|
|
}
|
|
);
|
|
|
|
return data.objects.map(rep => ({
|
|
representId: this.extractRepresentId(rep.url),
|
|
name: rep.name,
|
|
email: rep.email || null,
|
|
districtName: rep.district_name,
|
|
electedOffice: rep.elected_office,
|
|
partyName: rep.party_name || null,
|
|
photoUrl: rep.photo_url || null,
|
|
level: this.mapGovernmentLevel(rep.representative_set_name)
|
|
}));
|
|
} catch (error) {
|
|
if (axios.isAxiosError(error)) {
|
|
if (error.response?.status === 404) {
|
|
logger.warn(`No representatives found for postal code: ${postalCode}`);
|
|
return [];
|
|
}
|
|
|
|
if (error.response?.status === 429) {
|
|
logger.error('Represent API rate limit exceeded');
|
|
throw new Error('Rate limit exceeded. Please try again later.');
|
|
}
|
|
}
|
|
|
|
logger.error('Represent API error:', error);
|
|
throw new Error('Failed to fetch representatives');
|
|
}
|
|
}
|
|
|
|
private extractRepresentId(url: string): string {
|
|
// Extract ID from URL: /representatives/house-of-commons/123/
|
|
const match = url.match(/\/representatives\/[^\/]+\/(\d+)\//);
|
|
return match ? match[1] : url;
|
|
}
|
|
|
|
private mapGovernmentLevel(setName: string): string {
|
|
// Map representative set names to standard levels
|
|
const lowerSetName = setName.toLowerCase();
|
|
|
|
if (lowerSetName.includes('house-of-commons')) return 'federal';
|
|
if (lowerSetName.includes('legislative-assembly')) return 'provincial';
|
|
if (lowerSetName.includes('council')) return 'municipal';
|
|
|
|
return 'other';
|
|
}
|
|
}
|
|
```
|
|
|
|
### Frontend: Representative Card Component
|
|
|
|
```typescript
|
|
// admin/src/components/influence/RepresentativeCard.tsx
|
|
|
|
import React from 'react';
|
|
import { Card, Avatar, Space, Typography, Tag, Button } from 'antd';
|
|
import { MailOutlined, UserOutlined } from '@ant-design/icons';
|
|
import type { Representative } from '../../types/api';
|
|
|
|
interface RepresentativeCardProps {
|
|
representative: Representative;
|
|
onSelect?: (id: string) => void;
|
|
selected?: boolean;
|
|
}
|
|
|
|
const RepresentativeCard: React.FC<RepresentativeCardProps> = ({
|
|
representative,
|
|
onSelect,
|
|
selected
|
|
}) => {
|
|
const levelColors: Record<string, string> = {
|
|
federal: 'blue',
|
|
provincial: 'green',
|
|
municipal: 'orange'
|
|
};
|
|
|
|
return (
|
|
<Card
|
|
hoverable={!!onSelect}
|
|
onClick={() => onSelect?.(representative.id)}
|
|
style={{
|
|
borderColor: selected ? '#1890ff' : undefined,
|
|
borderWidth: selected ? 2 : 1
|
|
}}
|
|
>
|
|
<Space align="start" size="large">
|
|
<Avatar
|
|
src={representative.photoUrl}
|
|
icon={<UserOutlined />}
|
|
size={80}
|
|
/>
|
|
|
|
<Space direction="vertical" size={0} style={{ flex: 1 }}>
|
|
<Typography.Title level={5} style={{ margin: 0 }}>
|
|
{representative.name}
|
|
</Typography.Title>
|
|
|
|
<Typography.Text type="secondary">
|
|
{representative.electedOffice}
|
|
</Typography.Text>
|
|
|
|
<Typography.Text type="secondary">
|
|
{representative.districtName}
|
|
</Typography.Text>
|
|
|
|
<Space size="small" style={{ marginTop: 8 }}>
|
|
<Tag color={levelColors[representative.level] || 'default'}>
|
|
{representative.level.toUpperCase()}
|
|
</Tag>
|
|
|
|
{representative.partyName && (
|
|
<Tag>{representative.partyName}</Tag>
|
|
)}
|
|
</Space>
|
|
|
|
{representative.email && (
|
|
<Button
|
|
type="link"
|
|
icon={<MailOutlined />}
|
|
href={`mailto:${representative.email}`}
|
|
style={{ padding: 0, marginTop: 8 }}
|
|
>
|
|
{representative.email}
|
|
</Button>
|
|
)}
|
|
</Space>
|
|
</Space>
|
|
</Card>
|
|
);
|
|
};
|
|
|
|
export default RepresentativeCard;
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### No Representatives Found
|
|
|
|
**Symptoms:**
|
|
- Lookup returns empty array
|
|
- Error: "No representatives found for this postal code"
|
|
|
|
**Solutions:**
|
|
|
|
1. **Verify postal code format** → Must be valid Canadian postal code
|
|
2. **Check Represent API status** → Visit https://represent.opennorth.ca/health
|
|
3. **Test postal code manually** → Try https://represent.opennorth.ca/postcodes/K1A0A1/
|
|
4. **Review API logs** → Check for rate limit errors
|
|
|
|
**Debugging:**
|
|
|
|
```bash
|
|
# Test Represent API directly
|
|
curl https://represent.opennorth.ca/postcodes/K1A0A1/ | jq
|
|
|
|
# Check representative cache
|
|
docker compose exec v2-postgres psql -U changemaker -d changemaker_lite -c \
|
|
"SELECT * FROM representatives WHERE postal_code = 'K1A0A1';"
|
|
|
|
# Check API logs
|
|
docker compose logs api | grep "Represent API"
|
|
```
|
|
|
|
### Rate Limit Exceeded
|
|
|
|
**Symptoms:**
|
|
- HTTP 429 error
|
|
- Error: "Rate limit exceeded. Please try again later."
|
|
|
|
**Solutions:**
|
|
|
|
1. **Implement exponential backoff** → Retry with increasing delays
|
|
2. **Use cache more aggressively** → Increase cache TTL to 60 days
|
|
3. **Batch lookups** → Avoid rapid repeated lookups
|
|
4. **Contact Open North** → Request rate limit increase if needed
|
|
|
|
**Code Fix (represent-api.client.ts):**
|
|
|
|
```typescript
|
|
async getRepresentativesByPostalCodeWithRetry(
|
|
postalCode: string,
|
|
maxRetries = 3
|
|
): Promise<any[]> {
|
|
for (let i = 0; i < maxRetries; i++) {
|
|
try {
|
|
return await this.getRepresentativesByPostalCode(postalCode);
|
|
} catch (error) {
|
|
if (error.message.includes('Rate limit exceeded')) {
|
|
const delay = Math.pow(2, i) * 1000; // Exponential backoff
|
|
logger.warn(`Rate limit hit, retrying in ${delay}ms...`);
|
|
await new Promise(resolve => setTimeout(resolve, delay));
|
|
continue;
|
|
}
|
|
throw error;
|
|
}
|
|
}
|
|
|
|
throw new Error('Max retries exceeded');
|
|
}
|
|
```
|
|
|
|
### Stale Representative Information
|
|
|
|
**Symptoms:**
|
|
- Representative email bounces
|
|
- Representative no longer in office
|
|
|
|
**Solutions:**
|
|
|
|
1. **Clear cache for postal code** → Delete and re-fetch
|
|
2. **Reduce cache TTL** → Set `REPRESENT_CACHE_TTL` to 7 days (604800)
|
|
3. **Manual verification** → Check official government websites
|
|
4. **Report to Represent API** → If data is incorrect, report to Open North
|
|
|
|
**Manual Cache Clear:**
|
|
|
|
```typescript
|
|
// Via admin UI
|
|
// Navigate to Influence > Representatives
|
|
// Find postal code in table
|
|
// Click "Delete All" for that postal code
|
|
|
|
// Via API
|
|
await api.delete(`/representatives/postal-code/${postalCode}`);
|
|
```
|
|
|
|
### Missing Email Addresses
|
|
|
|
**Symptoms:**
|
|
- Representative has no email address
|
|
- Cannot send campaign email
|
|
|
|
**Solutions:**
|
|
|
|
1. **Check Represent API data** → Some reps don't provide email publicly
|
|
2. **Use manual email field** → Allow admins to add email addresses
|
|
3. **Fallback to constituency office** → Use office email if available
|
|
4. **Skip representative** → Don't include in email recipients
|
|
|
|
**Code Fix (representative.service.ts):**
|
|
|
|
```typescript
|
|
async updateRepresentativeEmail(
|
|
representId: string,
|
|
email: string
|
|
): Promise<Representative> {
|
|
return this.prisma.representative.update({
|
|
where: { representId },
|
|
data: {
|
|
email,
|
|
lastUpdated: new Date() // Reset cache timestamp
|
|
}
|
|
});
|
|
}
|
|
```
|
|
|
|
## Performance Considerations
|
|
|
|
### Cache Strategy
|
|
|
|
**TTL Configuration:**
|
|
|
|
- **Default**: 30 days (2,592,000 seconds)
|
|
- **Aggressive**: 60 days for stable electoral districts
|
|
- **Conservative**: 7 days during election periods
|
|
|
|
**Cache Warming:**
|
|
|
|
Pre-populate cache for common postal codes:
|
|
|
|
```typescript
|
|
// api/src/scripts/warm-representative-cache.ts
|
|
|
|
import { RepresentativeService } from '../modules/influence/representatives/representatives.service';
|
|
import { PrismaClient } from '@prisma/client';
|
|
|
|
const prisma = new PrismaClient();
|
|
const representativeService = new RepresentativeService(prisma);
|
|
|
|
// Common postal codes from campaign participation data
|
|
const commonPostalCodes = [
|
|
'K1A0A1', 'M5H2N2', 'V6B1A1', // Federal capitals
|
|
'T2P2M5', 'H3B1A1', 'S7K0J5' // Provincial capitals
|
|
];
|
|
|
|
async function warmCache() {
|
|
for (const postalCode of commonPostalCodes) {
|
|
try {
|
|
await representativeService.lookupByPostalCode(postalCode);
|
|
console.log(`Cached representatives for ${postalCode}`);
|
|
} catch (error) {
|
|
console.error(`Failed to cache ${postalCode}:`, error);
|
|
}
|
|
|
|
// Rate limit: 1 request per second
|
|
await new Promise(resolve => setTimeout(resolve, 1000));
|
|
}
|
|
}
|
|
|
|
warmCache();
|
|
```
|
|
|
|
### Query Optimization
|
|
|
|
**Index Usage:**
|
|
|
|
```sql
|
|
-- Composite index for fast lookups
|
|
CREATE INDEX idx_representative_postal_code_level
|
|
ON representatives (postal_code, level);
|
|
|
|
-- Index for cache invalidation
|
|
CREATE INDEX idx_representative_last_updated
|
|
ON representatives (last_updated);
|
|
```
|
|
|
|
**Query Pattern:**
|
|
|
|
```typescript
|
|
// Optimized cache lookup with index
|
|
const cached = await prisma.representative.findMany({
|
|
where: {
|
|
postalCode: normalized,
|
|
level: { in: targetLevels }, // Use index
|
|
lastUpdated: {
|
|
gte: new Date(Date.now() - CACHE_TTL * 1000)
|
|
}
|
|
}
|
|
});
|
|
```
|
|
|
|
### API Rate Limiting
|
|
|
|
**Client-Side Rate Limiter:**
|
|
|
|
```typescript
|
|
import Bottleneck from 'bottleneck';
|
|
|
|
const limiter = new Bottleneck({
|
|
maxConcurrent: 1,
|
|
minTime: 1000 // 1 request per second
|
|
});
|
|
|
|
const getRepresentativesRateLimited = limiter.wrap(
|
|
representApiClient.getRepresentativesByPostalCode.bind(representApiClient)
|
|
);
|
|
```
|
|
|
|
**Redis-Based Distributed Rate Limiting:**
|
|
|
|
```typescript
|
|
import { RateLimiterRedis } from 'rate-limiter-flexible';
|
|
|
|
const rateLimiter = new RateLimiterRedis({
|
|
storeClient: redisClient,
|
|
keyPrefix: 'represent-api',
|
|
points: 60, // 60 requests
|
|
duration: 60 // per minute
|
|
});
|
|
|
|
await rateLimiter.consume('represent-api-key');
|
|
```
|
|
|
|
## Related Documentation
|
|
|
|
### Backend Modules
|
|
|
|
- [Representatives Module](../../backend/modules/representatives.md) — Full API reference
|
|
- [Campaigns Module](../../backend/modules/campaigns.md) — Campaign integration
|
|
- [Postal Codes Module](../../backend/modules/postal-codes.md) — Postal code caching
|
|
|
|
### Frontend Pages
|
|
|
|
- [RepresentativesPage](../../frontend/pages/admin/representatives-page.md) — Admin cache management
|
|
- [CampaignPage](../../frontend/pages/public/campaign-page.md) — Public representative lookup
|
|
|
|
### Database Models
|
|
|
|
- [Representative](../../database/models/representative.md) — Representative schema
|
|
- [Campaign](../../database/models/campaign.md) — Campaign schema
|
|
- [CampaignEmail](../../database/models/campaign-email.md) — Email tracking schema
|
|
|
|
### External APIs
|
|
|
|
- [Represent API Documentation](https://represent.opennorth.ca/api/) — Official API docs
|
|
- [Open North](https://www.opennorth.ca/) — Represent API provider
|
|
|
|
### Configuration
|
|
|
|
- [Environment Variables](../../getting-started/configuration.md#represent-api) — Represent API settings
|