Tonne of debugging - getting ready for the production builds
This commit is contained in:
753
mkdocs/docs/v2/features/influence/campaigns.md
Normal file
753
mkdocs/docs/v2/features/influence/campaigns.md
Normal file
@@ -0,0 +1,753 @@
|
||||
# Campaign Management System
|
||||
|
||||
## Overview
|
||||
|
||||
The campaign management system is the core of Changemaker Lite's advocacy email platform. It enables organizations to create, configure, and manage advocacy campaigns that allow supporters to contact elected representatives via email. The system supports multiple campaign types, customizable features via feature flags, and a complete lifecycle from draft to archived status.
|
||||
|
||||
**Key Capabilities:**
|
||||
|
||||
- **Multi-status lifecycle**: Draft → Active → Paused → Archived workflow
|
||||
- **12 feature flags**: Granular control over campaign behavior
|
||||
- **Government level filtering**: Target specific levels (federal, provincial, municipal)
|
||||
- **Cover photo uploads**: Visual campaign branding
|
||||
- **Slug-based routing**: SEO-friendly public URLs
|
||||
- **Response wall integration**: Public display of campaign responses
|
||||
- **Email tracking**: Monitor sent emails and campaign effectiveness
|
||||
|
||||
**Use Cases:**
|
||||
|
||||
- Advocacy campaigns targeting elected officials
|
||||
- Public awareness campaigns with response sharing
|
||||
- Email-your-MP initiatives
|
||||
- Multi-level government outreach
|
||||
- Time-limited advocacy actions
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Admin User] -->|Creates Campaign| B[CampaignsPage]
|
||||
B -->|POST /api/campaigns| C[Campaign Service]
|
||||
C -->|Save| D[(Campaign Model)]
|
||||
|
||||
E[Public User] -->|Browses| F[CampaignsListPage]
|
||||
F -->|GET /api/public/campaigns| C
|
||||
|
||||
E -->|Views Campaign| G[CampaignPage]
|
||||
G -->|GET /api/public/campaigns/:slug| C
|
||||
G -->|Lookup Reps| H[Representatives Service]
|
||||
G -->|Send Email| I[Email Queue Service]
|
||||
I -->|Add Job| J[(BullMQ Redis)]
|
||||
|
||||
K[Email Worker] -->|Process Jobs| J
|
||||
K -->|Send SMTP| L[Email Recipients]
|
||||
K -->|Track| M[(CampaignEmail Model)]
|
||||
|
||||
D -->|1:N| M
|
||||
D -->|1:N| N[(Response Model)]
|
||||
|
||||
style D fill:#e1f5ff
|
||||
style M fill:#e1f5ff
|
||||
style N fill:#e1f5ff
|
||||
style J fill:#fff4e1
|
||||
```
|
||||
|
||||
**Flow Description:**
|
||||
|
||||
1. **Admin creates campaign** → Campaign service validates and saves to database
|
||||
2. **Public user browses** → Campaign service returns active campaigns
|
||||
3. **User views campaign** → Representatives service looks up postal code
|
||||
4. **User sends email** → Email queue service adds job to BullMQ
|
||||
5. **Worker processes job** → Email sent via SMTP, tracked in CampaignEmail model
|
||||
6. **User submits response** → Response service creates response for moderation
|
||||
|
||||
## Database Models
|
||||
|
||||
### Campaign Model
|
||||
|
||||
See [Campaign Model Documentation](../../database/models/campaign.md) for full schema.
|
||||
|
||||
**Key Fields:**
|
||||
|
||||
- `status`: DRAFT | ACTIVE | PAUSED | ARCHIVED
|
||||
- `targetGovernmentLevels`: Array of government levels (federal, provincial, municipal)
|
||||
- `emailSubjectTemplate`: Subject line with {{VAR}} placeholders
|
||||
- `emailBodyTemplate`: Email body with {{VAR}} placeholders
|
||||
- `coverPhotoUrl`: Campaign hero image URL
|
||||
- `slug`: URL-friendly identifier
|
||||
|
||||
**Feature Flags (12 total):**
|
||||
|
||||
| Flag | Type | Default | Description |
|
||||
|------|------|---------|-------------|
|
||||
| `allowSmtpEmail` | boolean | true | Enable email sending |
|
||||
| `allowCallTracking` | boolean | false | Enable phone call logging |
|
||||
| `showResponseWall` | boolean | true | Display response wall |
|
||||
| `requireEmailVerification` | boolean | true | Verify response emails |
|
||||
| `allowAnonymousResponses` | boolean | false | Allow responses without login |
|
||||
| `highlightCampaign` | boolean | false | Feature on homepage |
|
||||
| `showProgressBar` | boolean | true | Display response count progress |
|
||||
| `allowSharing` | boolean | true | Enable social sharing buttons |
|
||||
| `requirePostalCode` | boolean | true | Require postal code for lookup |
|
||||
| `allowCustomMessage` | boolean | true | Users can edit email text |
|
||||
| `trackEmailOpens` | boolean | false | Track email opens (future) |
|
||||
| `notifyOnResponse` | boolean | true | Email admin on new responses |
|
||||
|
||||
**Related Models:**
|
||||
|
||||
- [CampaignEmail](../../database/models/campaign-email.md) — Tracks sent emails
|
||||
- [Response](../../database/models/response.md) — Public responses to campaign
|
||||
- [Representative](../../database/models/representative.md) — Email recipients
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Admin Endpoints
|
||||
|
||||
See [Campaigns Module API Reference](../../backend/modules/campaigns.md#endpoints) for full details.
|
||||
|
||||
| Method | Endpoint | Auth | Description |
|
||||
|--------|----------|------|-------------|
|
||||
| GET | `/api/campaigns` | SUPER_ADMIN, INFLUENCE_ADMIN | List all campaigns (paginated) |
|
||||
| GET | `/api/campaigns/:id` | SUPER_ADMIN, INFLUENCE_ADMIN | Get campaign details |
|
||||
| POST | `/api/campaigns` | SUPER_ADMIN, INFLUENCE_ADMIN | Create new campaign |
|
||||
| PUT | `/api/campaigns/:id` | SUPER_ADMIN, INFLUENCE_ADMIN | Update campaign |
|
||||
| PATCH | `/api/campaigns/:id/status` | SUPER_ADMIN, INFLUENCE_ADMIN | Update campaign status |
|
||||
| DELETE | `/api/campaigns/:id` | SUPER_ADMIN | Delete campaign |
|
||||
|
||||
### Public Endpoints
|
||||
|
||||
See [Campaigns Public API Reference](../../backend/modules/campaigns.md#public-endpoints).
|
||||
|
||||
| Method | Endpoint | Auth | Description |
|
||||
|--------|----------|------|-------------|
|
||||
| GET | `/api/public/campaigns` | None | List active campaigns |
|
||||
| GET | `/api/public/campaigns/:slug` | None | Get campaign by slug |
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `EMAIL_TEST_MODE` | boolean | false | Send emails to MailHog instead of SMTP |
|
||||
| `SMTP_HOST` | string | - | SMTP server hostname |
|
||||
| `SMTP_PORT` | number | 587 | SMTP server port |
|
||||
| `SMTP_USER` | string | - | SMTP username |
|
||||
| `SMTP_PASS` | string | - | SMTP password |
|
||||
| `SMTP_FROM_EMAIL` | string | - | Default sender email |
|
||||
| `SMTP_FROM_NAME` | string | - | Default sender name |
|
||||
|
||||
### Site Settings
|
||||
|
||||
SMTP settings can be configured via Site Settings (overrides env vars):
|
||||
|
||||
```typescript
|
||||
{
|
||||
smtpHost: string | null,
|
||||
smtpPort: number | null,
|
||||
smtpUser: string | null,
|
||||
smtpPass: string | null,
|
||||
smtpFromEmail: string | null,
|
||||
smtpFromName: string | null
|
||||
}
|
||||
```
|
||||
|
||||
### Upload Configuration
|
||||
|
||||
Cover photos uploaded to `/uploads/campaigns/{campaignId}/{filename}`.
|
||||
|
||||
**Limits:**
|
||||
- Max file size: 10MB
|
||||
- Allowed formats: jpg, jpeg, png, gif, webp
|
||||
|
||||
## Admin Workflow
|
||||
|
||||
### 1. Create Campaign
|
||||
|
||||
[Screenshot: CampaignsPage with "Create Campaign" button]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to **Influence > Campaigns**
|
||||
2. Click **Create Campaign** button
|
||||
3. Fill in campaign details:
|
||||
- Title (required)
|
||||
- Description (required)
|
||||
- Target government levels (select all that apply)
|
||||
- Email subject template (use {{VAR}} for dynamic content)
|
||||
- Email body template (HTML supported)
|
||||
4. Upload cover photo (optional)
|
||||
5. Click **Save** (saves as DRAFT)
|
||||
|
||||
**Code Example (CampaignsPage.tsx):**
|
||||
|
||||
```typescript
|
||||
const handleCreate = async (values: any) => {
|
||||
try {
|
||||
const formData = new FormData();
|
||||
formData.append('title', values.title);
|
||||
formData.append('description', values.description);
|
||||
formData.append('targetGovernmentLevels', JSON.stringify(values.targetGovernmentLevels));
|
||||
formData.append('emailSubjectTemplate', values.emailSubjectTemplate);
|
||||
formData.append('emailBodyTemplate', values.emailBodyTemplate);
|
||||
|
||||
if (values.coverPhoto?.[0]?.originFileObj) {
|
||||
formData.append('coverPhoto', values.coverPhoto[0].originFileObj);
|
||||
}
|
||||
|
||||
await api.post('/campaigns', formData, {
|
||||
headers: { 'Content-Type': 'multipart/form-data' }
|
||||
});
|
||||
|
||||
message.success('Campaign created successfully');
|
||||
fetchCampaigns();
|
||||
} catch (error) {
|
||||
message.error('Failed to create campaign');
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### 2. Configure Feature Flags
|
||||
|
||||
[Screenshot: Campaign edit modal with feature flags section]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Click **Edit** on campaign row
|
||||
2. Scroll to **Feature Flags** section
|
||||
3. Toggle flags as needed:
|
||||
- **allowSmtpEmail**: Enable email sending (required for email campaigns)
|
||||
- **showResponseWall**: Display public response wall
|
||||
- **requireEmailVerification**: Require email verification for responses
|
||||
- **highlightCampaign**: Feature on homepage
|
||||
- **allowCustomMessage**: Let users edit email text before sending
|
||||
4. Click **Save**
|
||||
|
||||
**Best Practices:**
|
||||
|
||||
- Enable `requireEmailVerification` for public response walls
|
||||
- Disable `allowCustomMessage` if you want consistent messaging
|
||||
- Use `highlightCampaign` sparingly (max 2-3 campaigns)
|
||||
- Enable `showProgressBar` to encourage participation
|
||||
|
||||
### 3. Test Campaign
|
||||
|
||||
[Screenshot: Campaign preview with test email form]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Set campaign status to **ACTIVE**
|
||||
2. Navigate to public campaign page: `/campaigns/{slug}`
|
||||
3. Enter test postal code
|
||||
4. Review representative lookup results
|
||||
5. Send test email to your own email address
|
||||
6. Verify email content and formatting
|
||||
|
||||
**Troubleshooting:**
|
||||
|
||||
- If no representatives found → Check Represent API cache
|
||||
- If email not received → Check Email Queue page for job status
|
||||
- If email formatting broken → Review HTML template syntax
|
||||
|
||||
### 4. Publish Campaign
|
||||
|
||||
[Screenshot: Campaign status dropdown]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Return to **Campaigns** page
|
||||
2. Click **Status** dropdown on campaign row
|
||||
3. Select **ACTIVE**
|
||||
4. Campaign now visible on public campaigns page
|
||||
|
||||
**Status Lifecycle:**
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> DRAFT: Create
|
||||
DRAFT --> ACTIVE: Publish
|
||||
ACTIVE --> PAUSED: Pause
|
||||
PAUSED --> ACTIVE: Resume
|
||||
ACTIVE --> ARCHIVED: Archive
|
||||
PAUSED --> ARCHIVED: Archive
|
||||
ARCHIVED --> [*]
|
||||
```
|
||||
|
||||
### 5. Monitor Campaign
|
||||
|
||||
[Screenshot: Campaign emails drawer with stats]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Click **View Emails** on campaign row
|
||||
2. Review email stats:
|
||||
- Total sent
|
||||
- Success rate
|
||||
- Failed emails
|
||||
3. View individual email details (recipient, status, sent date)
|
||||
4. Retry failed emails if needed
|
||||
|
||||
**Metrics to Track:**
|
||||
|
||||
- Emails sent per day
|
||||
- Response wall submissions
|
||||
- Verification rate (if enabled)
|
||||
- Geographic distribution (via postal codes)
|
||||
|
||||
## Public Workflow
|
||||
|
||||
### 1. Browse Campaigns
|
||||
|
||||
[Screenshot: Public campaigns list page with featured campaigns]
|
||||
|
||||
**User Journey:**
|
||||
|
||||
1. User visits `/campaigns`
|
||||
2. Sees featured campaigns (if `highlightCampaign` enabled)
|
||||
3. Browses active campaigns grid
|
||||
4. Clicks campaign card to view details
|
||||
|
||||
**Code Example (CampaignsListPage.tsx):**
|
||||
|
||||
```typescript
|
||||
const CampaignsListPage: React.FC = () => {
|
||||
const [campaigns, setCampaigns] = useState<Campaign[]>([]);
|
||||
const [featured, setFeatured] = useState<Campaign[]>([]);
|
||||
|
||||
useEffect(() => {
|
||||
const fetchCampaigns = async () => {
|
||||
const { data } = await axios.get('/api/public/campaigns');
|
||||
|
||||
const featuredCampaigns = data.filter((c: Campaign) =>
|
||||
c.highlightCampaign && c.status === 'ACTIVE'
|
||||
);
|
||||
const regularCampaigns = data.filter((c: Campaign) =>
|
||||
!c.highlightCampaign && c.status === 'ACTIVE'
|
||||
);
|
||||
|
||||
setFeatured(featuredCampaigns);
|
||||
setCampaigns(regularCampaigns);
|
||||
};
|
||||
|
||||
fetchCampaigns();
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<PublicLayout>
|
||||
{featured.length > 0 && (
|
||||
<FeaturedCampaigns campaigns={featured} />
|
||||
)}
|
||||
<CampaignGrid campaigns={campaigns} />
|
||||
</PublicLayout>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### 2. View Campaign Details
|
||||
|
||||
[Screenshot: Campaign detail page with postal code lookup form]
|
||||
|
||||
**User Journey:**
|
||||
|
||||
1. User clicks campaign card
|
||||
2. Navigated to `/campaigns/{slug}`
|
||||
3. Reads campaign description
|
||||
4. Enters postal code in lookup form
|
||||
5. System fetches representatives from Represent API
|
||||
6. User selects representatives to email
|
||||
|
||||
### 3. Send Email
|
||||
|
||||
[Screenshot: Email form with representative selection]
|
||||
|
||||
**User Journey:**
|
||||
|
||||
1. User reviews list of representatives
|
||||
2. Selects representatives to email (checkboxes)
|
||||
3. Reviews email subject and body
|
||||
4. Edits message if `allowCustomMessage` enabled
|
||||
5. Adds personal details (name, email)
|
||||
6. Clicks **Send Email**
|
||||
7. Email jobs added to BullMQ queue
|
||||
8. User sees confirmation message
|
||||
|
||||
**Code Example (CampaignPage.tsx):**
|
||||
|
||||
```typescript
|
||||
const handleSendEmails = async (values: any) => {
|
||||
try {
|
||||
const payload = {
|
||||
campaignId: campaign.id,
|
||||
senderName: values.senderName,
|
||||
senderEmail: values.senderEmail,
|
||||
postalCode: values.postalCode,
|
||||
representativeIds: values.representativeIds,
|
||||
customMessage: campaign.allowCustomMessage ? values.customMessage : null
|
||||
};
|
||||
|
||||
await axios.post('/api/public/campaigns/send-email', payload);
|
||||
|
||||
message.success('Your emails have been sent!');
|
||||
|
||||
if (campaign.showResponseWall) {
|
||||
message.info('Share your response on the Response Wall!');
|
||||
}
|
||||
} catch (error) {
|
||||
message.error('Failed to send emails');
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### 4. Submit Response (Optional)
|
||||
|
||||
[Screenshot: Response submission form]
|
||||
|
||||
**User Journey:**
|
||||
|
||||
1. After sending email, user clicks **Share Your Response**
|
||||
2. Navigated to `/responses/{campaignId}/submit`
|
||||
3. Fills in response form:
|
||||
- Type (EMAIL, LETTER, PHONE_CALL, etc.)
|
||||
- Message
|
||||
- Screenshot (optional)
|
||||
4. Submits response
|
||||
5. If `requireEmailVerification` enabled → verification email sent
|
||||
6. User clicks verification link in email
|
||||
7. Response appears on public response wall (after admin approval if moderation enabled)
|
||||
|
||||
## Volunteer Workflow
|
||||
|
||||
Not applicable — campaigns are admin-managed and public-facing.
|
||||
|
||||
## Code Examples
|
||||
|
||||
### Backend: Create Campaign
|
||||
|
||||
```typescript
|
||||
// api/src/modules/influence/campaigns/campaigns.service.ts
|
||||
|
||||
async createCampaign(
|
||||
data: Prisma.CampaignUncheckedCreateInput,
|
||||
createdByUserId: string
|
||||
): Promise<Campaign> {
|
||||
// Generate slug from title
|
||||
const baseSlug = data.title
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '-')
|
||||
.replace(/^-|-$/g, '');
|
||||
|
||||
let slug = baseSlug;
|
||||
let counter = 1;
|
||||
|
||||
// Ensure unique slug
|
||||
while (await this.prisma.campaign.findUnique({ where: { slug } })) {
|
||||
slug = `${baseSlug}-${counter}`;
|
||||
counter++;
|
||||
}
|
||||
|
||||
return this.prisma.campaign.create({
|
||||
data: {
|
||||
...data,
|
||||
slug,
|
||||
createdByUserId,
|
||||
status: 'DRAFT',
|
||||
// Default feature flags
|
||||
allowSmtpEmail: data.allowSmtpEmail ?? true,
|
||||
showResponseWall: data.showResponseWall ?? true,
|
||||
requireEmailVerification: data.requireEmailVerification ?? true,
|
||||
allowCustomMessage: data.allowCustomMessage ?? true,
|
||||
showProgressBar: data.showProgressBar ?? true,
|
||||
allowSharing: data.allowSharing ?? true,
|
||||
requirePostalCode: data.requirePostalCode ?? true,
|
||||
notifyOnResponse: data.notifyOnResponse ?? true
|
||||
}
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend: Campaign Card Component
|
||||
|
||||
```typescript
|
||||
// admin/src/pages/public/CampaignsListPage.tsx
|
||||
|
||||
const CampaignCard: React.FC<{ campaign: Campaign }> = ({ campaign }) => {
|
||||
const navigate = useNavigate();
|
||||
|
||||
return (
|
||||
<Card
|
||||
hoverable
|
||||
cover={
|
||||
campaign.coverPhotoUrl && (
|
||||
<img
|
||||
alt={campaign.title}
|
||||
src={campaign.coverPhotoUrl}
|
||||
style={{ height: 200, objectFit: 'cover' }}
|
||||
/>
|
||||
)
|
||||
}
|
||||
onClick={() => navigate(`/campaigns/${campaign.slug}`)}
|
||||
>
|
||||
<Card.Meta
|
||||
title={campaign.title}
|
||||
description={
|
||||
<Space direction="vertical" size="small">
|
||||
<Typography.Paragraph ellipsis={{ rows: 3 }}>
|
||||
{campaign.description}
|
||||
</Typography.Paragraph>
|
||||
|
||||
{campaign.showProgressBar && (
|
||||
<Progress
|
||||
percent={Math.min(
|
||||
(campaign._count?.responses || 0) / (campaign.responseGoal || 100) * 100,
|
||||
100
|
||||
)}
|
||||
status="active"
|
||||
/>
|
||||
)}
|
||||
|
||||
<Space>
|
||||
{campaign.targetGovernmentLevels.map(level => (
|
||||
<Tag key={level} color="blue">{level}</Tag>
|
||||
))}
|
||||
</Space>
|
||||
</Space>
|
||||
}
|
||||
/>
|
||||
</Card>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Campaign Not Visible on Public Page
|
||||
|
||||
**Symptoms:**
|
||||
- Campaign exists in admin but doesn't appear on `/campaigns`
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Check campaign status → must be `ACTIVE`
|
||||
2. Verify no draft campaigns leaked → filter by status in query
|
||||
3. Check Nginx caching → clear cache or disable for `/api/public/campaigns`
|
||||
|
||||
**Debugging:**
|
||||
|
||||
```bash
|
||||
# Check campaign status
|
||||
docker compose exec v2-postgres psql -U changemaker -d changemaker_lite -c \
|
||||
"SELECT id, title, status, slug FROM campaigns WHERE slug = 'your-slug';"
|
||||
|
||||
# Check public endpoint response
|
||||
curl http://localhost:4000/api/public/campaigns | jq
|
||||
```
|
||||
|
||||
### Email Template Variables Not Replaced
|
||||
|
||||
**Symptoms:**
|
||||
- Email sent with `{{senderName}}` instead of actual name
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Verify variable syntax → must use double curly braces `{{VAR}}`
|
||||
2. Check email service interpolation → ensure `processTemplate()` called
|
||||
3. Verify variable names match → `senderName`, `senderEmail`, `postalCode`, `recipientName`, `recipientEmail`
|
||||
|
||||
**Code Fix (email.service.ts):**
|
||||
|
||||
```typescript
|
||||
private processTemplate(template: string, variables: Record<string, string>): string {
|
||||
let processed = template;
|
||||
|
||||
Object.entries(variables).forEach(([key, value]) => {
|
||||
const regex = new RegExp(`{{${key}}}`, 'g');
|
||||
processed = processed.replace(regex, value || '');
|
||||
});
|
||||
|
||||
return processed;
|
||||
}
|
||||
```
|
||||
|
||||
### Cover Photo Upload Fails
|
||||
|
||||
**Symptoms:**
|
||||
- Upload spinner never completes
|
||||
- Error: "File too large"
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Check file size → max 10MB
|
||||
2. Verify file format → must be jpg/jpeg/png/gif/webp
|
||||
3. Check upload directory permissions → `/uploads/campaigns` must be writable
|
||||
4. Increase Nginx upload limit → `client_max_body_size 20M;`
|
||||
|
||||
**Docker Volume Fix:**
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
api:
|
||||
volumes:
|
||||
- ./uploads:/app/uploads:rw # Ensure :rw (read-write)
|
||||
```
|
||||
|
||||
### Representatives Not Loading
|
||||
|
||||
**Symptoms:**
|
||||
- Postal code lookup returns empty array
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Check Represent API status → visit https://represent.opennorth.ca/health
|
||||
2. Verify postal code format → must be valid Canadian postal code (K1A 0A1)
|
||||
3. Check representative cache → may need refresh
|
||||
4. Review API rate limits → Represent API has rate limits
|
||||
|
||||
**Manual Cache Refresh:**
|
||||
|
||||
```bash
|
||||
# Via admin UI
|
||||
# Navigate to Influence > Representatives
|
||||
# Enter postal code in search box
|
||||
# Click "Lookup"
|
||||
|
||||
# Via API
|
||||
curl -X POST http://localhost:4000/api/representatives/lookup \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"postalCode": "K1A0A1"}'
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Campaign Listing Optimization
|
||||
|
||||
**Query Optimization:**
|
||||
|
||||
```typescript
|
||||
// Include response count for progress bar
|
||||
const campaigns = await prisma.campaign.findMany({
|
||||
where: { status: 'ACTIVE' },
|
||||
include: {
|
||||
_count: {
|
||||
select: { responses: true }
|
||||
}
|
||||
},
|
||||
orderBy: [
|
||||
{ highlightCampaign: 'desc' }, // Featured first
|
||||
{ createdAt: 'desc' }
|
||||
]
|
||||
});
|
||||
```
|
||||
|
||||
**Caching Strategy:**
|
||||
|
||||
- Cache active campaigns list for 5 minutes (Redis)
|
||||
- Invalidate cache on campaign status change
|
||||
- Use ETags for HTTP caching
|
||||
|
||||
### Email Queue Scaling
|
||||
|
||||
**BullMQ Configuration:**
|
||||
|
||||
```typescript
|
||||
// api/src/services/email-queue.service.ts
|
||||
|
||||
const queue = new Queue('campaign-emails', {
|
||||
connection: redisConnection,
|
||||
defaultJobOptions: {
|
||||
attempts: 3,
|
||||
backoff: {
|
||||
type: 'exponential',
|
||||
delay: 5000 // 5s, 25s, 125s
|
||||
},
|
||||
removeOnComplete: {
|
||||
age: 86400, // Keep completed jobs for 24h
|
||||
count: 1000
|
||||
},
|
||||
removeOnFail: {
|
||||
age: 604800 // Keep failed jobs for 7 days
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Worker concurrency
|
||||
const worker = new Worker('campaign-emails', processCampaignEmail, {
|
||||
connection: redisConnection,
|
||||
concurrency: 5 // Process 5 emails simultaneously
|
||||
});
|
||||
```
|
||||
|
||||
**Monitoring:**
|
||||
|
||||
- Track queue size with Prometheus `cm_email_queue_size` metric
|
||||
- Alert if queue size > 1000
|
||||
- Monitor worker processing rate
|
||||
|
||||
### Cover Photo Optimization
|
||||
|
||||
**Image Processing:**
|
||||
|
||||
```typescript
|
||||
// api/src/modules/influence/campaigns/campaigns.service.ts
|
||||
|
||||
import sharp from 'sharp';
|
||||
|
||||
async uploadCoverPhoto(file: Express.Multer.File, campaignId: string): Promise<string> {
|
||||
const filename = `${Date.now()}-${file.originalname}`;
|
||||
const uploadPath = `/uploads/campaigns/${campaignId}`;
|
||||
|
||||
// Create directory
|
||||
await fs.mkdir(uploadPath, { recursive: true });
|
||||
|
||||
// Optimize image
|
||||
await sharp(file.buffer)
|
||||
.resize(1200, 630, { // Open Graph ratio
|
||||
fit: 'cover',
|
||||
position: 'center'
|
||||
})
|
||||
.jpeg({ quality: 85 })
|
||||
.toFile(`${uploadPath}/${filename}`);
|
||||
|
||||
return `${uploadPath}/${filename}`;
|
||||
}
|
||||
```
|
||||
|
||||
**CDN Integration:**
|
||||
|
||||
- Serve cover photos via CDN (Cloudflare, CloudFront)
|
||||
- Use responsive images with `srcset`
|
||||
- Lazy load images below fold
|
||||
|
||||
## Related Documentation
|
||||
|
||||
### Backend Modules
|
||||
|
||||
- [Campaigns Module](../../backend/modules/campaigns.md) — Full API reference
|
||||
- [Representatives Module](../../backend/modules/representatives.md) — Represent API integration
|
||||
- [Responses Module](../../backend/modules/responses.md) — Response wall system
|
||||
- [Email Queue Module](../../backend/modules/email-queue.md) — BullMQ email processing
|
||||
|
||||
### Frontend Pages
|
||||
|
||||
- [CampaignsPage](../../frontend/pages/admin/campaigns-page.md) — Admin campaign management
|
||||
- [CampaignPage](../../frontend/pages/public/campaign-page.md) — Public campaign view
|
||||
- [CampaignsListPage](../../frontend/pages/public/campaigns-list-page.md) — Public campaign listing
|
||||
- [ResponsesPage](../../frontend/pages/admin/responses-page.md) — Response moderation
|
||||
|
||||
### Database Models
|
||||
|
||||
- [Campaign](../../database/models/campaign.md) — Campaign schema
|
||||
- [CampaignEmail](../../database/models/campaign-email.md) — Email tracking schema
|
||||
- [Response](../../database/models/response.md) — Response schema
|
||||
- [Representative](../../database/models/representative.md) — Representative schema
|
||||
|
||||
### Configuration
|
||||
|
||||
- [Environment Variables](../../getting-started/configuration.md#email-settings) — SMTP configuration
|
||||
- [Site Settings](../../backend/modules/settings.md) — Global settings API
|
||||
|
||||
### Guides
|
||||
|
||||
- [Email Sending Guide](../influence/email-queue.md) — Email queue and BullMQ
|
||||
- [Response Wall Guide](../influence/responses.md) — Response moderation workflow
|
||||
- [Representative Lookup Guide](../influence/representatives.md) — Represent API integration
|
||||
994
mkdocs/docs/v2/features/influence/email-queue.md
Normal file
994
mkdocs/docs/v2/features/influence/email-queue.md
Normal file
@@ -0,0 +1,994 @@
|
||||
# Email Queue System
|
||||
|
||||
## Overview
|
||||
|
||||
The email queue system manages asynchronous email sending for advocacy campaigns using BullMQ and Redis. It provides reliable email delivery, retry logic, job monitoring, and comprehensive tracking of email campaign effectiveness.
|
||||
|
||||
**Key Capabilities:**
|
||||
|
||||
- **BullMQ integration**: Redis-backed job queue for email processing
|
||||
- **Automatic retry logic**: Failed emails retried with exponential backoff
|
||||
- **Job status tracking**: Monitor queued, active, completed, and failed jobs
|
||||
- **Rate limiting**: Prevent SMTP server overload
|
||||
- **Email tracking**: Track sent emails per campaign
|
||||
- **Admin monitoring**: Real-time queue statistics and job management
|
||||
- **Test mode**: Send to MailHog instead of SMTP for testing
|
||||
|
||||
**Use Cases:**
|
||||
|
||||
- Bulk email sending for advocacy campaigns
|
||||
- Reliable email delivery with retry
|
||||
- Email campaign effectiveness tracking
|
||||
- SMTP server load management
|
||||
- Development email testing
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Public User] -->|Send Email| B[CampaignPage]
|
||||
B -->|POST /api/public/campaigns/send-email| C[Campaign Service]
|
||||
C -->|Add Job| D[Email Queue Service]
|
||||
D -->|Create Job| E[(BullMQ Redis)]
|
||||
|
||||
F[Email Worker] -->|Poll Jobs| E
|
||||
F -->|Process Job| G{Send Email}
|
||||
G -->|Success| H[Email Service - SMTP]
|
||||
G -->|Failure| I[Retry Logic]
|
||||
|
||||
H -->|Track| J[(CampaignEmail Model)]
|
||||
I -->|Backoff| E
|
||||
|
||||
K[Admin User] -->|Monitor| L[EmailQueuePage]
|
||||
L -->|GET /api/email-queue/stats| D
|
||||
L -->|Pause/Resume| D
|
||||
L -->|Clean Jobs| D
|
||||
|
||||
M[Prometheus] -->|Scrape| N[Metrics Endpoint]
|
||||
N -->|cm_email_queue_size| E
|
||||
|
||||
style E fill:#fff4e1
|
||||
style J fill:#e1f5ff
|
||||
```
|
||||
|
||||
**Flow Description:**
|
||||
|
||||
1. **User sends email** → Campaign service adds job to BullMQ queue
|
||||
2. **Worker polls queue** → Picks up job for processing
|
||||
3. **Email sent via SMTP** → Nodemailer sends email
|
||||
4. **Success** → Job marked completed, email tracked in database
|
||||
5. **Failure** → Job retried with exponential backoff (3 attempts)
|
||||
6. **Admin monitors** → View queue stats, pause/resume, clean old jobs
|
||||
|
||||
## Database Models
|
||||
|
||||
### CampaignEmail Model
|
||||
|
||||
See [CampaignEmail Model Documentation](../../database/models/campaign-email.md) for full schema.
|
||||
|
||||
**Key Fields:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | String (UUID) | Primary key |
|
||||
| `campaignId` | String | Associated campaign |
|
||||
| `recipientEmail` | String | Email recipient |
|
||||
| `recipientName` | String? | Recipient name |
|
||||
| `senderEmail` | String | Sender email address |
|
||||
| `senderName` | String | Sender name |
|
||||
| `subject` | String | Email subject line |
|
||||
| `body` | String (Text) | Email body content |
|
||||
| `status` | Enum | QUEUED, SENT, FAILED |
|
||||
| `jobId` | String? | BullMQ job ID |
|
||||
| `sentAt` | DateTime? | When email was sent |
|
||||
| `failureReason` | String? | Error message if failed |
|
||||
|
||||
**Indexes:**
|
||||
|
||||
- `campaignId, status` — For campaign email stats
|
||||
- `jobId` — For job status lookups
|
||||
- `sentAt` — For time-based queries
|
||||
|
||||
**Related Models:**
|
||||
|
||||
- [Campaign](../../database/models/campaign.md) — Campaign association
|
||||
- [Representative](../../database/models/representative.md) — Email recipients
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Admin Endpoints
|
||||
|
||||
See [Email Queue Module API Reference](../../backend/modules/email-queue.md#endpoints) for full details.
|
||||
|
||||
| Method | Endpoint | Auth | Description |
|
||||
|--------|----------|------|-------------|
|
||||
| GET | `/api/email-queue/stats` | SUPER_ADMIN, INFLUENCE_ADMIN | Get queue statistics |
|
||||
| POST | `/api/email-queue/pause` | SUPER_ADMIN, INFLUENCE_ADMIN | Pause queue processing |
|
||||
| POST | `/api/email-queue/resume` | SUPER_ADMIN, INFLUENCE_ADMIN | Resume queue processing |
|
||||
| POST | `/api/email-queue/clean` | SUPER_ADMIN | Clean completed/failed jobs |
|
||||
| POST | `/api/email-queue/retry/:jobId` | SUPER_ADMIN, INFLUENCE_ADMIN | Retry failed job |
|
||||
|
||||
### Public Endpoints
|
||||
|
||||
Email queue jobs are created via campaign email endpoints (no direct public access).
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `REDIS_HOST` | string | localhost | Redis hostname |
|
||||
| `REDIS_PORT` | number | 6379 | Redis port |
|
||||
| `REDIS_PASSWORD` | string | - | Redis password (required) |
|
||||
| `SMTP_HOST` | string | - | SMTP server hostname |
|
||||
| `SMTP_PORT` | number | 587 | SMTP server port |
|
||||
| `SMTP_USER` | string | - | SMTP username |
|
||||
| `SMTP_PASS` | string | - | SMTP password |
|
||||
| `SMTP_FROM_EMAIL` | string | - | Default sender email |
|
||||
| `SMTP_FROM_NAME` | string | - | Default sender name |
|
||||
| `EMAIL_TEST_MODE` | boolean | false | Send to MailHog instead of SMTP |
|
||||
| `EMAIL_QUEUE_CONCURRENCY` | number | 5 | Max concurrent email workers |
|
||||
|
||||
### BullMQ Configuration
|
||||
|
||||
```typescript
|
||||
// api/src/services/email-queue.service.ts
|
||||
|
||||
const queueOptions = {
|
||||
connection: {
|
||||
host: process.env.REDIS_HOST,
|
||||
port: parseInt(process.env.REDIS_PORT || '6379'),
|
||||
password: process.env.REDIS_PASSWORD
|
||||
},
|
||||
defaultJobOptions: {
|
||||
attempts: 3,
|
||||
backoff: {
|
||||
type: 'exponential',
|
||||
delay: 5000 // 5s, 25s, 125s
|
||||
},
|
||||
removeOnComplete: {
|
||||
age: 86400, // Keep completed jobs for 24h
|
||||
count: 1000
|
||||
},
|
||||
removeOnFail: {
|
||||
age: 604800 // Keep failed jobs for 7 days
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### Worker Configuration
|
||||
|
||||
```typescript
|
||||
const workerOptions = {
|
||||
connection: queueOptions.connection,
|
||||
concurrency: parseInt(process.env.EMAIL_QUEUE_CONCURRENCY || '5'),
|
||||
limiter: {
|
||||
max: 60, // Max 60 emails
|
||||
duration: 60000 // per minute
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
## Admin Workflow
|
||||
|
||||
### 1. View Queue Statistics
|
||||
|
||||
[Screenshot: EmailQueuePage with queue stats cards]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to **Influence > Email Queue**
|
||||
2. View queue statistics:
|
||||
- **Waiting**: Jobs queued for processing
|
||||
- **Active**: Jobs currently being processed
|
||||
- **Completed**: Successfully sent emails
|
||||
- **Failed**: Failed emails requiring attention
|
||||
3. Monitor queue health (green if waiting < 100)
|
||||
|
||||
**Code Example (EmailQueuePage.tsx):**
|
||||
|
||||
```typescript
|
||||
const [stats, setStats] = useState({
|
||||
waiting: 0,
|
||||
active: 0,
|
||||
completed: 0,
|
||||
failed: 0,
|
||||
paused: false
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
const fetchStats = async () => {
|
||||
const { data } = await api.get('/email-queue/stats');
|
||||
setStats(data);
|
||||
};
|
||||
|
||||
fetchStats();
|
||||
|
||||
// Refresh every 5 seconds
|
||||
const interval = setInterval(fetchStats, 5000);
|
||||
return () => clearInterval(interval);
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<Row gutter={16}>
|
||||
<Col span={6}>
|
||||
<Card>
|
||||
<Statistic
|
||||
title="Waiting"
|
||||
value={stats.waiting}
|
||||
valueStyle={{ color: stats.waiting > 100 ? '#cf1322' : '#3f8600' }}
|
||||
/>
|
||||
</Card>
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Card>
|
||||
<Statistic title="Active" value={stats.active} />
|
||||
</Card>
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Card>
|
||||
<Statistic title="Completed" value={stats.completed} />
|
||||
</Card>
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Card>
|
||||
<Statistic
|
||||
title="Failed"
|
||||
value={stats.failed}
|
||||
valueStyle={{ color: stats.failed > 0 ? '#cf1322' : undefined }}
|
||||
/>
|
||||
</Card>
|
||||
</Col>
|
||||
</Row>
|
||||
);
|
||||
```
|
||||
|
||||
### 2. Pause/Resume Queue
|
||||
|
||||
[Screenshot: EmailQueuePage with pause/resume buttons]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Click **Pause Queue** button
|
||||
2. Queue stops processing new jobs
|
||||
3. Active jobs complete normally
|
||||
4. Status indicator shows "Paused"
|
||||
5. Click **Resume Queue** to restart processing
|
||||
|
||||
**Use Cases:**
|
||||
|
||||
- Temporary SMTP server maintenance
|
||||
- Stop email sending during testing
|
||||
- Prevent email sending during off-hours
|
||||
|
||||
**Code Example (email-queue.service.ts):**
|
||||
|
||||
```typescript
|
||||
async pauseQueue(): Promise<void> {
|
||||
await this.queue.pause();
|
||||
logger.info('Email queue paused');
|
||||
}
|
||||
|
||||
async resumeQueue(): Promise<void> {
|
||||
await this.queue.resume();
|
||||
logger.info('Email queue resumed');
|
||||
}
|
||||
|
||||
async isPaused(): Promise<boolean> {
|
||||
return this.queue.isPaused();
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Clean Completed Jobs
|
||||
|
||||
[Screenshot: EmailQueuePage with clean jobs button]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Click **Clean Jobs** dropdown
|
||||
2. Select cleanup type:
|
||||
- **Completed (>24h)**: Remove old successful jobs
|
||||
- **Failed (>7d)**: Remove old failed jobs
|
||||
- **All Completed**: Remove all successful jobs
|
||||
3. Confirm cleanup
|
||||
4. Jobs removed from queue, stats updated
|
||||
|
||||
**Code Example (email-queue.routes.ts):**
|
||||
|
||||
```typescript
|
||||
router.post('/clean', requireRole('SUPER_ADMIN', 'INFLUENCE_ADMIN'), async (req, res) => {
|
||||
try {
|
||||
const { type } = req.body; // 'completed', 'failed', 'all-completed'
|
||||
|
||||
let count = 0;
|
||||
|
||||
if (type === 'completed') {
|
||||
count = await queue.clean(86400000, 1000, 'completed'); // 24h
|
||||
} else if (type === 'failed') {
|
||||
count = await queue.clean(604800000, 1000, 'failed'); // 7d
|
||||
} else if (type === 'all-completed') {
|
||||
count = await queue.clean(0, 0, 'completed'); // All
|
||||
}
|
||||
|
||||
logger.info(`Cleaned ${count} ${type} jobs`);
|
||||
|
||||
res.json({ count });
|
||||
} catch (error) {
|
||||
logger.error('Failed to clean jobs:', error);
|
||||
res.status(500).json({ error: 'Failed to clean jobs' });
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### 4. Retry Failed Jobs
|
||||
|
||||
[Screenshot: Failed jobs table with retry buttons]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Scroll to **Failed Jobs** section
|
||||
2. View failed job details (error message, recipient)
|
||||
3. Click **Retry** button on specific job
|
||||
4. Job re-queued for processing
|
||||
5. Monitor in **Active** tab
|
||||
|
||||
**Bulk Retry:**
|
||||
|
||||
1. Select multiple failed jobs (checkboxes)
|
||||
2. Click **Retry Selected** button
|
||||
3. All selected jobs re-queued
|
||||
|
||||
**Code Example (email-queue.service.ts):**
|
||||
|
||||
```typescript
|
||||
async retryFailedJob(jobId: string): Promise<void> {
|
||||
const job = await this.queue.getJob(jobId);
|
||||
|
||||
if (!job) {
|
||||
throw new Error('Job not found');
|
||||
}
|
||||
|
||||
if (await job.isFailed()) {
|
||||
await job.retry();
|
||||
logger.info(`Retrying job ${jobId}`);
|
||||
} else {
|
||||
throw new Error('Job is not failed');
|
||||
}
|
||||
}
|
||||
|
||||
async retryAllFailed(): Promise<number> {
|
||||
const failed = await this.queue.getFailed();
|
||||
let count = 0;
|
||||
|
||||
for (const job of failed) {
|
||||
await job.retry();
|
||||
count++;
|
||||
}
|
||||
|
||||
logger.info(`Retried ${count} failed jobs`);
|
||||
return count;
|
||||
}
|
||||
```
|
||||
|
||||
## Public Workflow
|
||||
|
||||
### 1. Send Campaign Email
|
||||
|
||||
[Screenshot: CampaignPage with email sending form]
|
||||
|
||||
**User Journey:**
|
||||
|
||||
1. User selects representatives to email
|
||||
2. Fills in sender details (name, email)
|
||||
3. Reviews/edits email content (if allowed)
|
||||
4. Clicks **Send Email** button
|
||||
5. System creates email jobs (one per recipient)
|
||||
6. Jobs added to BullMQ queue
|
||||
7. User sees confirmation message
|
||||
|
||||
**Code Example (campaigns-public.routes.ts):**
|
||||
|
||||
```typescript
|
||||
router.post('/send-email', async (req, res) => {
|
||||
try {
|
||||
const {
|
||||
campaignId,
|
||||
senderName,
|
||||
senderEmail,
|
||||
postalCode,
|
||||
representativeIds,
|
||||
customMessage
|
||||
} = req.body;
|
||||
|
||||
const campaign = await prisma.campaign.findUnique({
|
||||
where: { id: campaignId }
|
||||
});
|
||||
|
||||
if (!campaign || campaign.status !== 'ACTIVE') {
|
||||
return res.status(400).json({ error: 'Campaign not active' });
|
||||
}
|
||||
|
||||
const representatives = await prisma.representative.findMany({
|
||||
where: { id: { in: representativeIds } }
|
||||
});
|
||||
|
||||
// Create email jobs
|
||||
const emailJobs = [];
|
||||
|
||||
for (const rep of representatives) {
|
||||
const emailData = {
|
||||
campaignId,
|
||||
recipientEmail: rep.email,
|
||||
recipientName: rep.name,
|
||||
senderEmail,
|
||||
senderName,
|
||||
subject: processTemplate(campaign.emailSubjectTemplate, {
|
||||
senderName,
|
||||
recipientName: rep.name,
|
||||
postalCode
|
||||
}),
|
||||
body: customMessage || processTemplate(campaign.emailBodyTemplate, {
|
||||
senderName,
|
||||
senderEmail,
|
||||
recipientName: rep.name,
|
||||
recipientEmail: rep.email,
|
||||
postalCode
|
||||
})
|
||||
};
|
||||
|
||||
// Add to queue
|
||||
const job = await emailQueueService.addEmail(emailData);
|
||||
|
||||
emailJobs.push(job);
|
||||
}
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
emailsQueued: emailJobs.length
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to queue campaign emails:', error);
|
||||
res.status(500).json({ error: 'Failed to send emails' });
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### 2. Job Processing
|
||||
|
||||
**Worker Processing Logic:**
|
||||
|
||||
```typescript
|
||||
// api/src/services/email-queue.service.ts
|
||||
|
||||
import { Worker } from 'bullmq';
|
||||
import { emailService } from './email.service';
|
||||
|
||||
const worker = new Worker('campaign-emails', async (job) => {
|
||||
const {
|
||||
campaignId,
|
||||
recipientEmail,
|
||||
recipientName,
|
||||
senderEmail,
|
||||
senderName,
|
||||
subject,
|
||||
body
|
||||
} = job.data;
|
||||
|
||||
try {
|
||||
// Send email via nodemailer
|
||||
await emailService.send({
|
||||
to: recipientEmail,
|
||||
from: {
|
||||
email: process.env.SMTP_FROM_EMAIL!,
|
||||
name: process.env.SMTP_FROM_NAME!
|
||||
},
|
||||
replyTo: {
|
||||
email: senderEmail,
|
||||
name: senderName
|
||||
},
|
||||
subject,
|
||||
html: body
|
||||
});
|
||||
|
||||
// Update database record
|
||||
await prisma.campaignEmail.update({
|
||||
where: { jobId: job.id },
|
||||
data: {
|
||||
status: 'SENT',
|
||||
sentAt: new Date()
|
||||
}
|
||||
});
|
||||
|
||||
logger.info(`Sent campaign email ${job.id} to ${recipientEmail}`);
|
||||
|
||||
// Update Prometheus metric
|
||||
metrics.campaignEmailsSent.inc({ campaign_id: campaignId });
|
||||
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
logger.error(`Failed to send email ${job.id}:`, error);
|
||||
|
||||
// Update database record
|
||||
await prisma.campaignEmail.update({
|
||||
where: { jobId: job.id },
|
||||
data: {
|
||||
status: 'FAILED',
|
||||
failureReason: error.message
|
||||
}
|
||||
});
|
||||
|
||||
throw error; // Let BullMQ handle retry
|
||||
}
|
||||
}, workerOptions);
|
||||
|
||||
worker.on('completed', (job) => {
|
||||
logger.info(`Job ${job.id} completed`);
|
||||
});
|
||||
|
||||
worker.on('failed', (job, err) => {
|
||||
logger.error(`Job ${job?.id} failed:`, err);
|
||||
});
|
||||
```
|
||||
|
||||
## Volunteer Workflow
|
||||
|
||||
Not applicable — email queue is system-level.
|
||||
|
||||
## Code Examples
|
||||
|
||||
### Backend: Email Queue Service
|
||||
|
||||
```typescript
|
||||
// api/src/services/email-queue.service.ts
|
||||
|
||||
import { Queue, QueueEvents } from 'bullmq';
|
||||
import { logger } from '../utils/logger';
|
||||
import { prisma } from '../config/database';
|
||||
|
||||
export class EmailQueueService {
|
||||
private queue: Queue;
|
||||
private queueEvents: QueueEvents;
|
||||
|
||||
constructor() {
|
||||
const connection = {
|
||||
host: process.env.REDIS_HOST!,
|
||||
port: parseInt(process.env.REDIS_PORT || '6379'),
|
||||
password: process.env.REDIS_PASSWORD
|
||||
};
|
||||
|
||||
this.queue = new Queue('campaign-emails', {
|
||||
connection,
|
||||
defaultJobOptions: {
|
||||
attempts: 3,
|
||||
backoff: {
|
||||
type: 'exponential',
|
||||
delay: 5000
|
||||
},
|
||||
removeOnComplete: {
|
||||
age: 86400,
|
||||
count: 1000
|
||||
},
|
||||
removeOnFail: {
|
||||
age: 604800
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
this.queueEvents = new QueueEvents('campaign-emails', { connection });
|
||||
|
||||
this.setupEventHandlers();
|
||||
}
|
||||
|
||||
private setupEventHandlers(): void {
|
||||
this.queueEvents.on('completed', ({ jobId }) => {
|
||||
logger.info(`Email job ${jobId} completed`);
|
||||
});
|
||||
|
||||
this.queueEvents.on('failed', ({ jobId, failedReason }) => {
|
||||
logger.error(`Email job ${jobId} failed: ${failedReason}`);
|
||||
});
|
||||
}
|
||||
|
||||
async addEmail(data: any): Promise<{ jobId: string }> {
|
||||
// Create database record
|
||||
const emailRecord = await prisma.campaignEmail.create({
|
||||
data: {
|
||||
...data,
|
||||
status: 'QUEUED'
|
||||
}
|
||||
});
|
||||
|
||||
// Add job to queue
|
||||
const job = await this.queue.add('send-email', data, {
|
||||
jobId: emailRecord.id
|
||||
});
|
||||
|
||||
// Update database with job ID
|
||||
await prisma.campaignEmail.update({
|
||||
where: { id: emailRecord.id },
|
||||
data: { jobId: job.id }
|
||||
});
|
||||
|
||||
logger.info(`Queued email job ${job.id}`);
|
||||
|
||||
return { jobId: job.id! };
|
||||
}
|
||||
|
||||
async getStats(): Promise<any> {
|
||||
const counts = await this.queue.getJobCounts();
|
||||
|
||||
return {
|
||||
waiting: counts.waiting || 0,
|
||||
active: counts.active || 0,
|
||||
completed: counts.completed || 0,
|
||||
failed: counts.failed || 0,
|
||||
paused: await this.queue.isPaused()
|
||||
};
|
||||
}
|
||||
|
||||
async pauseQueue(): Promise<void> {
|
||||
await this.queue.pause();
|
||||
}
|
||||
|
||||
async resumeQueue(): Promise<void> {
|
||||
await this.queue.resume();
|
||||
}
|
||||
|
||||
async clean(grace: number, limit: number, type: string): Promise<number> {
|
||||
return this.queue.clean(grace, limit, type as any);
|
||||
}
|
||||
}
|
||||
|
||||
export const emailQueueService = new EmailQueueService();
|
||||
```
|
||||
|
||||
### Frontend: Queue Stats Dashboard
|
||||
|
||||
```typescript
|
||||
// admin/src/pages/EmailQueuePage.tsx
|
||||
|
||||
import React, { useState, useEffect } from 'react';
|
||||
import { Card, Row, Col, Statistic, Button, Space, message } from 'antd';
|
||||
import { PlayCircleOutlined, PauseCircleOutlined, ClearOutlined } from '@ant-design/icons';
|
||||
import { api } from '../../lib/api';
|
||||
|
||||
const EmailQueuePage: React.FC = () => {
|
||||
const [stats, setStats] = useState<any>(null);
|
||||
const [loading, setLoading] = useState(false);
|
||||
|
||||
const fetchStats = async () => {
|
||||
const { data } = await api.get('/email-queue/stats');
|
||||
setStats(data);
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
fetchStats();
|
||||
const interval = setInterval(fetchStats, 5000);
|
||||
return () => clearInterval(interval);
|
||||
}, []);
|
||||
|
||||
const handlePause = async () => {
|
||||
setLoading(true);
|
||||
try {
|
||||
await api.post('/email-queue/pause');
|
||||
message.success('Queue paused');
|
||||
fetchStats();
|
||||
} catch (error) {
|
||||
message.error('Failed to pause queue');
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleResume = async () => {
|
||||
setLoading(true);
|
||||
try {
|
||||
await api.post('/email-queue/resume');
|
||||
message.success('Queue resumed');
|
||||
fetchStats();
|
||||
} catch (error) {
|
||||
message.error('Failed to resume queue');
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleClean = async (type: string) => {
|
||||
setLoading(true);
|
||||
try {
|
||||
const { data } = await api.post('/email-queue/clean', { type });
|
||||
message.success(`Cleaned ${data.count} jobs`);
|
||||
fetchStats();
|
||||
} catch (error) {
|
||||
message.error('Failed to clean jobs');
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
if (!stats) return <Card loading />;
|
||||
|
||||
return (
|
||||
<Space direction="vertical" size="large" style={{ width: '100%' }}>
|
||||
<Card title="Queue Statistics">
|
||||
<Row gutter={16}>
|
||||
<Col span={6}>
|
||||
<Statistic
|
||||
title="Waiting"
|
||||
value={stats.waiting}
|
||||
valueStyle={{ color: stats.waiting > 100 ? '#cf1322' : '#3f8600' }}
|
||||
/>
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Statistic title="Active" value={stats.active} />
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Statistic title="Completed" value={stats.completed} />
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Statistic
|
||||
title="Failed"
|
||||
value={stats.failed}
|
||||
valueStyle={{ color: stats.failed > 0 ? '#cf1322' : undefined }}
|
||||
/>
|
||||
</Col>
|
||||
</Row>
|
||||
</Card>
|
||||
|
||||
<Card title="Queue Controls">
|
||||
<Space>
|
||||
{stats.paused ? (
|
||||
<Button
|
||||
type="primary"
|
||||
icon={<PlayCircleOutlined />}
|
||||
onClick={handleResume}
|
||||
loading={loading}
|
||||
>
|
||||
Resume Queue
|
||||
</Button>
|
||||
) : (
|
||||
<Button
|
||||
icon={<PauseCircleOutlined />}
|
||||
onClick={handlePause}
|
||||
loading={loading}
|
||||
>
|
||||
Pause Queue
|
||||
</Button>
|
||||
)}
|
||||
|
||||
<Button
|
||||
icon={<ClearOutlined />}
|
||||
onClick={() => handleClean('completed')}
|
||||
loading={loading}
|
||||
>
|
||||
Clean Completed
|
||||
</Button>
|
||||
|
||||
<Button
|
||||
danger
|
||||
icon={<ClearOutlined />}
|
||||
onClick={() => handleClean('failed')}
|
||||
loading={loading}
|
||||
>
|
||||
Clean Failed
|
||||
</Button>
|
||||
</Space>
|
||||
</Card>
|
||||
</Space>
|
||||
);
|
||||
};
|
||||
|
||||
export default EmailQueuePage;
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Emails Stuck in Queue
|
||||
|
||||
**Symptoms:**
|
||||
- Waiting count increases but active/completed don't
|
||||
- Jobs not processing
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Check worker status → `docker compose logs api | grep "Worker"`
|
||||
2. Verify Redis connection → `docker compose exec redis redis-cli ping`
|
||||
3. Check SMTP configuration → test with `/api/auth/test-email`
|
||||
4. Restart worker → `docker compose restart api`
|
||||
|
||||
**Debugging:**
|
||||
|
||||
```bash
|
||||
# Check Redis keys
|
||||
docker compose exec redis redis-cli --pass $REDIS_PASSWORD
|
||||
> KEYS bull:campaign-emails:*
|
||||
|
||||
# Check worker logs
|
||||
docker compose logs -f api | grep "Email worker"
|
||||
|
||||
# Check queue status
|
||||
curl -H "Authorization: Bearer $TOKEN" http://localhost:4000/api/email-queue/stats
|
||||
```
|
||||
|
||||
### High Failure Rate
|
||||
|
||||
**Symptoms:**
|
||||
- Many jobs failing
|
||||
- Failed count increasing rapidly
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Check SMTP credentials → verify username/password
|
||||
2. Review failure reasons → check `failureReason` field in database
|
||||
3. Check SMTP server status → verify server is reachable
|
||||
4. Review rate limits → may be hitting SMTP server limits
|
||||
|
||||
**Common Failure Reasons:**
|
||||
|
||||
- **535 Authentication failed** → Invalid SMTP credentials
|
||||
- **550 Mailbox unavailable** → Recipient email doesn't exist
|
||||
- **421 Too many connections** → Reduce concurrency
|
||||
- **Connection timeout** → SMTP server unreachable
|
||||
|
||||
**Code Fix (email.service.ts):**
|
||||
|
||||
```typescript
|
||||
// Add better error handling
|
||||
async send(options: EmailOptions): Promise<void> {
|
||||
try {
|
||||
await this.transporter.sendMail(options);
|
||||
} catch (error) {
|
||||
if (error.responseCode === 535) {
|
||||
throw new Error('SMTP authentication failed - check credentials');
|
||||
} else if (error.responseCode === 550) {
|
||||
throw new Error('Recipient mailbox unavailable');
|
||||
} else if (error.code === 'ETIMEDOUT') {
|
||||
throw new Error('SMTP server connection timeout');
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Redis Connection Issues
|
||||
|
||||
**Symptoms:**
|
||||
- Error: "ECONNREFUSED" or "NOAUTH"
|
||||
- Queue operations fail
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Verify Redis is running → `docker compose ps redis`
|
||||
2. Check Redis password → ensure `REDIS_PASSWORD` matches docker-compose.yml
|
||||
3. Check Redis port → default 6379
|
||||
4. Verify Redis auth → `docker compose exec redis redis-cli --pass $REDIS_PASSWORD ping`
|
||||
|
||||
**Fix Redis Auth:**
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
command: redis-server --requirepass ${REDIS_PASSWORD}
|
||||
ports:
|
||||
- "6379:6379"
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Concurrency Tuning
|
||||
|
||||
**Worker Concurrency:**
|
||||
|
||||
```typescript
|
||||
// Adjust based on SMTP server limits
|
||||
const workerOptions = {
|
||||
concurrency: 5, // Process 5 emails simultaneously
|
||||
limiter: {
|
||||
max: 60, // Max 60 emails per minute
|
||||
duration: 60000
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**SMTP Server Limits:**
|
||||
|
||||
- Gmail: 100 emails/day (consumer), 2000/day (Workspace)
|
||||
- SendGrid: Varies by plan (40k/day free tier)
|
||||
- AWS SES: 14 emails/second, 200 emails/day (sandbox)
|
||||
|
||||
### Queue Monitoring
|
||||
|
||||
**Prometheus Metrics:**
|
||||
|
||||
```typescript
|
||||
import { Counter, Gauge } from 'prom-client';
|
||||
|
||||
export const campaignEmailsQueued = new Counter({
|
||||
name: 'cm_campaign_emails_queued_total',
|
||||
help: 'Total campaign emails queued',
|
||||
labelNames: ['campaign_id']
|
||||
});
|
||||
|
||||
export const campaignEmailsSent = new Counter({
|
||||
name: 'cm_campaign_emails_sent_total',
|
||||
help: 'Total campaign emails sent',
|
||||
labelNames: ['campaign_id']
|
||||
});
|
||||
|
||||
export const emailQueueSize = new Gauge({
|
||||
name: 'cm_email_queue_size',
|
||||
help: 'Current email queue size',
|
||||
labelNames: ['status']
|
||||
});
|
||||
|
||||
// Update gauge every 30 seconds
|
||||
setInterval(async () => {
|
||||
const stats = await emailQueueService.getStats();
|
||||
|
||||
emailQueueSize.set({ status: 'waiting' }, stats.waiting);
|
||||
emailQueueSize.set({ status: 'active' }, stats.active);
|
||||
emailQueueSize.set({ status: 'failed' }, stats.failed);
|
||||
}, 30000);
|
||||
```
|
||||
|
||||
### Database Optimization
|
||||
|
||||
**Index Strategy:**
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_campaign_email_status ON campaign_emails (status);
|
||||
CREATE INDEX idx_campaign_email_campaign_id ON campaign_emails (campaign_id);
|
||||
CREATE INDEX idx_campaign_email_sent_at ON campaign_emails (sent_at);
|
||||
```
|
||||
|
||||
**Query Optimization:**
|
||||
|
||||
```typescript
|
||||
// Paginated campaign email stats
|
||||
const emails = await prisma.campaignEmail.findMany({
|
||||
where: { campaignId },
|
||||
select: {
|
||||
id: true,
|
||||
recipientEmail: true,
|
||||
status: true,
|
||||
sentAt: true,
|
||||
failureReason: true
|
||||
},
|
||||
orderBy: { createdAt: 'desc' },
|
||||
take: 100,
|
||||
skip: page * 100
|
||||
});
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
### Backend Modules
|
||||
|
||||
- [Email Queue Module](../../backend/modules/email-queue.md) — Full API reference
|
||||
- [Email Service](../../backend/modules/email.md) — SMTP configuration
|
||||
- [Campaigns Module](../../backend/modules/campaigns.md) — Campaign integration
|
||||
|
||||
### Frontend Pages
|
||||
|
||||
- [EmailQueuePage](../../frontend/pages/admin/email-queue-page.md) — Admin queue monitoring
|
||||
- [CampaignsPage](../../frontend/pages/admin/campaigns-page.md) — Campaign management
|
||||
|
||||
### Database Models
|
||||
|
||||
- [CampaignEmail](../../database/models/campaign-email.md) — Email tracking schema
|
||||
- [Campaign](../../database/models/campaign.md) — Campaign schema
|
||||
|
||||
### Configuration
|
||||
|
||||
- [Environment Variables](../../getting-started/configuration.md#email-settings) — SMTP/Redis configuration
|
||||
- [BullMQ Documentation](https://docs.bullmq.io/) — Official BullMQ docs
|
||||
|
||||
### Monitoring
|
||||
|
||||
- [Prometheus Metrics](../observability/prometheus-metrics.md) — Email queue metrics
|
||||
- [Grafana Dashboards](../observability/grafana-dashboards.md) — Queue visualization
|
||||
233
mkdocs/docs/v2/features/influence/index.md
Normal file
233
mkdocs/docs/v2/features/influence/index.md
Normal file
@@ -0,0 +1,233 @@
|
||||
# Influence Module
|
||||
|
||||
The Influence module provides a complete advocacy campaign platform for email campaigns, representative lookup, response walls, and engagement tracking. It enables supporters to contact their elected officials on issues that matter.
|
||||
|
||||
## Overview
|
||||
|
||||
The Influence module consists of five integrated components:
|
||||
|
||||
1. **[Campaigns](campaigns.md)** - Create and manage advocacy email campaigns
|
||||
2. **[Representatives](representatives.md)** - Lookup representatives by postal code
|
||||
3. **[Postal Codes](postal-codes.md)** - Postal code caching service
|
||||
4. **[Email Queue](email-queue.md)** - Async email sending with BullMQ
|
||||
5. **[Responses](responses.md)** - Public response wall with moderation
|
||||
|
||||
## Features
|
||||
|
||||
### Campaign Management
|
||||
|
||||
- Create campaigns with title, description, and email template
|
||||
- Target federal, provincial, or municipal representatives
|
||||
- Track campaign statistics (emails sent, responses)
|
||||
- Public/private campaign visibility
|
||||
- Featured campaign highlighting
|
||||
|
||||
### Representative Lookup
|
||||
|
||||
- Represent API integration (federal/provincial)
|
||||
- Postal code → representative matching
|
||||
- Representative information caching
|
||||
- Multiple representative levels
|
||||
- District boundary support
|
||||
|
||||
### Email Sending
|
||||
|
||||
- Async email queue with BullMQ
|
||||
- Template processing with variable substitution
|
||||
- SMTP delivery with retry logic
|
||||
- Email tracking and statistics
|
||||
- Test mode support (MailHog)
|
||||
|
||||
### Response Wall
|
||||
|
||||
- Public response submissions
|
||||
- Email verification flow
|
||||
- Moderation dashboard
|
||||
- Upvoting system
|
||||
- Response filtering and export
|
||||
|
||||
## User Flow
|
||||
|
||||
### Public User Experience
|
||||
|
||||
1. **Browse Campaigns** (`/campaigns`)
|
||||
- View featured campaigns
|
||||
- Search and filter (future)
|
||||
- Click campaign to learn more
|
||||
|
||||
2. **Campaign Detail** (`/campaigns/:id`)
|
||||
- Read campaign description
|
||||
- Enter postal code
|
||||
- View matched representatives
|
||||
- Customize email message
|
||||
- Send email
|
||||
|
||||
3. **Response Wall** (`/responses/:campaignId`)
|
||||
- Submit public response
|
||||
- Verify email address
|
||||
- View verified responses
|
||||
- Upvote responses
|
||||
|
||||
### Admin Experience
|
||||
|
||||
1. **Campaign Management** (`/app/influence/campaigns`)
|
||||
- Create campaigns
|
||||
- Edit templates
|
||||
- Configure targeting
|
||||
- View statistics
|
||||
- Manage visibility
|
||||
|
||||
2. **Response Moderation** (`/app/influence/responses`)
|
||||
- Review submissions
|
||||
- Verify/reject responses
|
||||
- Export data
|
||||
- Monitor engagement
|
||||
|
||||
3. **Representative Cache** (`/app/influence/representatives`)
|
||||
- View cached representatives
|
||||
- Refresh cache
|
||||
- Monitor lookup statistics
|
||||
|
||||
4. **Email Queue** (`/app/influence/email-queue`)
|
||||
- Monitor queue status
|
||||
- View failed jobs
|
||||
- Retry failed emails
|
||||
- Pause/resume queue
|
||||
|
||||
## Architecture
|
||||
|
||||
### Backend Components
|
||||
|
||||
**Modules:**
|
||||
- `api/src/modules/influence/campaigns/` - Campaign CRUD + public routes
|
||||
- `api/src/modules/influence/representatives/` - Represent API integration
|
||||
- `api/src/modules/influence/postal-codes/` - Postal code cache service
|
||||
- `api/src/modules/influence/responses/` - Response CRUD + verification
|
||||
- `api/src/modules/influence/campaign-emails/` - Email tracking
|
||||
- `api/src/modules/influence/email-queue/` - Queue admin routes
|
||||
|
||||
**Services:**
|
||||
- `api/src/services/email.service.ts` - Nodemailer wrapper
|
||||
- `api/src/services/email-queue.service.ts` - BullMQ queue + worker
|
||||
|
||||
**Database Models:**
|
||||
- `Campaign` - Campaign definitions
|
||||
- `CampaignEmail` - Sent email tracking
|
||||
- `Response` - Public response submissions
|
||||
- `PostalCodeCache` - Cached representative data
|
||||
|
||||
### Frontend Components
|
||||
|
||||
**Admin Pages:**
|
||||
- `admin/src/pages/CampaignsPage.tsx` - Campaign management
|
||||
- `admin/src/pages/ResponsesPage.tsx` - Response moderation
|
||||
- `admin/src/pages/RepresentativesPage.tsx` - Cache admin
|
||||
- `admin/src/pages/EmailQueuePage.tsx` - Queue monitoring
|
||||
|
||||
**Public Pages:**
|
||||
- `admin/src/pages/public/CampaignsListPage.tsx` - Campaign listing
|
||||
- `admin/src/pages/public/CampaignPage.tsx` - Campaign detail + email form
|
||||
- `admin/src/pages/public/ResponseWallPage.tsx` - Response submissions
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# Email
|
||||
EMAIL_TEST_MODE=true # Use MailHog instead of SMTP
|
||||
SMTP_HOST=smtp.example.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=user@example.com
|
||||
SMTP_PASS=password
|
||||
|
||||
# Represent API (optional)
|
||||
REPRESENT_API_KEY=your_api_key
|
||||
|
||||
# Redis (required for BullMQ)
|
||||
REDIS_PASSWORD=your_password
|
||||
```
|
||||
|
||||
### Feature Flags
|
||||
|
||||
Email sending can be toggled via `EMAIL_TEST_MODE`:
|
||||
- `true` - Emails sent to MailHog (localhost:8025)
|
||||
- `false` - Emails sent via SMTP
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Represent API
|
||||
|
||||
Represent API (https://represent.opennorth.ca/) provides:
|
||||
- Federal MP lookup by postal code
|
||||
- Provincial MLA/MPP lookup
|
||||
- District boundaries
|
||||
- Representative contact info
|
||||
|
||||
**Rate Limits:** 60 requests/minute
|
||||
|
||||
**Caching Strategy:**
|
||||
- Cache postal code → representative mappings
|
||||
- Refresh cache on 404 (postal code not found)
|
||||
- Cache expiration: 30 days
|
||||
|
||||
### Listmonk Newsletter Sync
|
||||
|
||||
Campaign participants can be synced to Listmonk:
|
||||
- Email submissions → subscribers
|
||||
- Campaign → list assignment
|
||||
- Opt-in sync via `LISTMONK_SYNC_ENABLED`
|
||||
|
||||
### Email Queue (BullMQ)
|
||||
|
||||
BullMQ provides:
|
||||
- Async email processing
|
||||
- Job retry with exponential backoff
|
||||
- Queue monitoring and statistics
|
||||
- Job persistence in Redis
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Public Endpoints
|
||||
|
||||
```
|
||||
GET /api/campaigns/public # List public campaigns
|
||||
GET /api/campaigns/public/:id # Get campaign details
|
||||
POST /api/campaigns/:id/send-email # Send campaign email
|
||||
GET /api/representatives/:postalCode # Lookup representatives
|
||||
POST /api/responses # Submit response
|
||||
GET /api/responses/verify/:token # Verify email
|
||||
GET /api/responses/campaign/:id # Get campaign responses
|
||||
POST /api/responses/:id/upvote # Upvote response
|
||||
```
|
||||
|
||||
### Admin Endpoints
|
||||
|
||||
```
|
||||
GET /api/campaigns # List all campaigns
|
||||
POST /api/campaigns # Create campaign
|
||||
GET /api/campaigns/:id # Get campaign
|
||||
PATCH /api/campaigns/:id # Update campaign
|
||||
DELETE /api/campaigns/:id # Delete campaign
|
||||
GET /api/campaigns/:id/emails # Get campaign emails
|
||||
GET /api/responses # List responses (admin)
|
||||
PATCH /api/responses/:id # Update response
|
||||
DELETE /api/responses/:id # Delete response
|
||||
GET /api/representatives/cache # View cache
|
||||
POST /api/representatives/cache/refresh # Refresh cache
|
||||
GET /api/email-queue/stats # Queue statistics
|
||||
POST /api/email-queue/pause # Pause queue
|
||||
POST /api/email-queue/resume # Resume queue
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Campaigns](campaigns.md)
|
||||
- [Representatives](representatives.md)
|
||||
- [Email Queue](email-queue.md)
|
||||
- [Responses](responses.md)
|
||||
- [Backend Campaign Module](../../backend/modules/campaigns.md)
|
||||
- [Backend Representatives Module](../../backend/modules/representatives.md)
|
||||
- [Backend Responses Module](../../backend/modules/responses.md)
|
||||
- [Email Service](../../backend/services/index.md)
|
||||
- [Campaign Manager Guide](../../user-guides/campaign-manager-guide.md)
|
||||
151
mkdocs/docs/v2/features/influence/postal-codes.md
Normal file
151
mkdocs/docs/v2/features/influence/postal-codes.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# Postal Code Geocoding Cache
|
||||
|
||||
## Overview
|
||||
|
||||
The postal code geocoding cache system stores geographic coordinates for Canadian postal codes, enabling faster representative lookups and reducing external API calls. It integrates with the multi-provider geocoding service to provide reliable centroid calculations for postal code-based geographic queries.
|
||||
|
||||
**Key Capabilities:**
|
||||
|
||||
- **Postal code caching**: Store lat/lng centroids for postal codes
|
||||
- **Geocoding integration**: Automatic geocoding via multi-provider service
|
||||
- **Cache hit optimization**: Reduce external API calls
|
||||
- **Administrative data**: City and province extraction
|
||||
- **Representative lookup**: Fast postal code → representative mapping
|
||||
|
||||
**Use Cases:**
|
||||
|
||||
- Campaign postal code lookups
|
||||
- Geographic representative mapping
|
||||
- Postal code validation
|
||||
- Centroid-based spatial queries
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Campaign Service] -->|Lookup Postal Code| B[Postal Code Service]
|
||||
B -->|Check Cache| C{Cache Hit?}
|
||||
C -->|Yes| D[Return Cached Centroid]
|
||||
C -->|No| E[Geocoding Service]
|
||||
|
||||
E -->|Geocode| F[Multi-Provider Geocoding]
|
||||
F -->|Parse Result| G[Extract Centroid]
|
||||
G -->|Save| H[(PostalCodeCache Model)]
|
||||
H -->|Return| D
|
||||
|
||||
I[Admin] -->|View Stats| J[RepresentativesPage]
|
||||
J -->|Display| K[Cache Statistics]
|
||||
|
||||
style H fill:#e1f5ff
|
||||
style F fill:#fff4e1
|
||||
```
|
||||
|
||||
## Database Models
|
||||
|
||||
### PostalCodeCache Model
|
||||
|
||||
See [PostalCodeCache Model Documentation](../../database/models/postal-code-cache.md) for full schema.
|
||||
|
||||
**Key Fields:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `postalCode` | String | Normalized postal code (primary key) |
|
||||
| `latitude` | Float | Centroid latitude |
|
||||
| `longitude` | Float | Centroid longitude |
|
||||
| `city` | String? | City name |
|
||||
| `province` | String? | Province abbreviation |
|
||||
|
||||
**Indexes:**
|
||||
|
||||
- `postalCode` — Primary key, unique constraint
|
||||
|
||||
**Related Models:**
|
||||
|
||||
- [Representative](../../database/models/representative.md) — Uses postal codes for caching
|
||||
- [Location](../../database/models/location.md) — Uses postal codes for geocoding
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Admin Endpoints
|
||||
|
||||
| Method | Endpoint | Auth | Description |
|
||||
|--------|----------|------|-------------|
|
||||
| GET | `/api/postal-codes/stats` | SUPER_ADMIN, INFLUENCE_ADMIN | Get cache statistics |
|
||||
| POST | `/api/postal-codes/lookup` | SUPER_ADMIN, INFLUENCE_ADMIN | Manual postal code lookup |
|
||||
|
||||
### Public Endpoints
|
||||
|
||||
Postal code lookups are performed automatically via representative lookup (no direct public access).
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `GEOCODING_PROVIDER` | string | nominatim | Default geocoding provider |
|
||||
| `GEOCODING_FALLBACK_PROVIDERS` | string | - | Comma-separated fallback providers |
|
||||
|
||||
## Admin Workflow
|
||||
|
||||
### 1. View Cache Statistics
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to **Influence > Representatives**
|
||||
2. View postal code cache statistics
|
||||
3. Monitor cache hit rate
|
||||
|
||||
## Public Workflow
|
||||
|
||||
Postal code caching is automatic and transparent to public users.
|
||||
|
||||
## Code Examples
|
||||
|
||||
### Backend: Postal Code Caching
|
||||
|
||||
```typescript
|
||||
// api/src/modules/influence/postal-codes/postal-codes.service.ts
|
||||
|
||||
export class PostalCodeService {
|
||||
async getOrCreateCache(postalCode: string): Promise<PostalCodeCache> {
|
||||
const normalized = postalCode.toUpperCase().replace(/\s/g, '');
|
||||
|
||||
// Check cache
|
||||
const cached = await prisma.postalCodeCache.findUnique({
|
||||
where: { postalCode: normalized }
|
||||
});
|
||||
|
||||
if (cached) {
|
||||
return cached;
|
||||
}
|
||||
|
||||
// Geocode postal code
|
||||
const result = await geocodingService.geocode({
|
||||
query: postalCode,
|
||||
country: 'CA'
|
||||
});
|
||||
|
||||
if (!result) {
|
||||
throw new Error('Failed to geocode postal code');
|
||||
}
|
||||
|
||||
// Create cache entry
|
||||
return prisma.postalCodeCache.create({
|
||||
data: {
|
||||
postalCode: normalized,
|
||||
latitude: result.latitude,
|
||||
longitude: result.longitude,
|
||||
city: result.city,
|
||||
province: result.province
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Representatives Module](../../backend/modules/representatives.md)
|
||||
- [Geocoding Service](../map/geocoding.md)
|
||||
924
mkdocs/docs/v2/features/influence/representatives.md
Normal file
924
mkdocs/docs/v2/features/influence/representatives.md
Normal file
@@ -0,0 +1,924 @@
|
||||
# Representative Lookup System
|
||||
|
||||
## Overview
|
||||
|
||||
The representative lookup system integrates with the Represent API (Open North) to provide real-time postal code-based representative lookups for advocacy campaigns. It includes intelligent caching to minimize API calls, support for all Canadian government levels, and admin tools for cache management.
|
||||
|
||||
**Key Capabilities:**
|
||||
|
||||
- **Represent API integration**: Real-time lookup of elected officials by postal code
|
||||
- **Multi-level support**: Federal, provincial, and municipal representatives
|
||||
- **Intelligent caching**: Reduce API calls and improve performance
|
||||
- **Cache invalidation**: Manual and automatic cache refresh
|
||||
- **Admin tools**: Cache statistics, manual lookup, bulk operations
|
||||
- **Error handling**: Graceful fallback for API failures
|
||||
|
||||
**Use Cases:**
|
||||
|
||||
- Email-your-MP campaigns
|
||||
- Multi-level government outreach
|
||||
- Representative contact information lookup
|
||||
- Geographic representation analysis
|
||||
- Campaign targeting by electoral district
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Public User] -->|Enter Postal Code| B[CampaignPage]
|
||||
B -->|POST /api/public/representatives/lookup| C[Representative Service]
|
||||
|
||||
C -->|Check Cache| D{Cache Hit?}
|
||||
D -->|Yes| E[Return Cached Reps]
|
||||
D -->|No| F[Represent API Client]
|
||||
|
||||
F -->|GET /postcodes/:code| G[Represent API]
|
||||
G -->|Return Reps| F
|
||||
F -->|Parse & Save| H[(Representative Model)]
|
||||
H -->|Return| E
|
||||
|
||||
I[Admin User] -->|View Cache| J[RepresentativesPage]
|
||||
J -->|GET /api/representatives| C
|
||||
J -->|Manual Lookup| C
|
||||
J -->|Clear Cache| K[Delete Service]
|
||||
K -->|Delete| H
|
||||
|
||||
L[Cache Invalidation Job] -->|Check lastUpdated| H
|
||||
L -->|Delete Stale| H
|
||||
|
||||
style H fill:#e1f5ff
|
||||
style G fill:#fff4e1
|
||||
```
|
||||
|
||||
**Flow Description:**
|
||||
|
||||
1. **User enters postal code** → Representative service checks cache
|
||||
2. **Cache miss** → Represent API client fetches representatives
|
||||
3. **API response** → Parse representatives, save to cache
|
||||
4. **Cache hit** → Return cached representatives (skip API call)
|
||||
5. **Admin management** → View cache stats, manual lookup, clear cache
|
||||
6. **Cache invalidation** → Automatic cleanup of stale entries (>30 days)
|
||||
|
||||
## Database Models
|
||||
|
||||
### Representative Model
|
||||
|
||||
See [Representative Model Documentation](../../database/models/representative.md) for full schema.
|
||||
|
||||
**Key Fields:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | String (UUID) | Primary key |
|
||||
| `representId` | String | Represent API unique identifier |
|
||||
| `name` | String | Full name of representative |
|
||||
| `email` | String | Email address |
|
||||
| `districtName` | String | Electoral district name |
|
||||
| `electedOffice` | String | Office held (MP, MPP, Mayor, etc.) |
|
||||
| `partyName` | String? | Political party affiliation |
|
||||
| `photoUrl` | String? | Profile photo URL |
|
||||
| `postalCode` | String | Associated postal code (cache key) |
|
||||
| `level` | String | Government level (federal, provincial, municipal) |
|
||||
| `lastUpdated` | DateTime | Cache timestamp |
|
||||
|
||||
**Indexes:**
|
||||
|
||||
- `postalCode, level` — Composite index for fast lookups
|
||||
- `representId` — Unique constraint
|
||||
- `lastUpdated` — For cache invalidation queries
|
||||
|
||||
**Related Models:**
|
||||
|
||||
- [Campaign](../../database/models/campaign.md) — Campaigns target representatives
|
||||
- [CampaignEmail](../../database/models/campaign-email.md) — Emails sent to representatives
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Admin Endpoints
|
||||
|
||||
See [Representatives Module API Reference](../../backend/modules/representatives.md#endpoints) for full details.
|
||||
|
||||
| Method | Endpoint | Auth | Description |
|
||||
|--------|----------|------|-------------|
|
||||
| GET | `/api/representatives` | SUPER_ADMIN, INFLUENCE_ADMIN | List all cached representatives |
|
||||
| GET | `/api/representatives/stats` | SUPER_ADMIN, INFLUENCE_ADMIN | Get cache statistics |
|
||||
| POST | `/api/representatives/lookup` | SUPER_ADMIN, INFLUENCE_ADMIN | Manual postal code lookup |
|
||||
| DELETE | `/api/representatives/:id` | SUPER_ADMIN, INFLUENCE_ADMIN | Delete cached representative |
|
||||
| DELETE | `/api/representatives/postal-code/:postalCode` | SUPER_ADMIN, INFLUENCE_ADMIN | Delete all reps for postal code |
|
||||
|
||||
### Public Endpoints
|
||||
|
||||
See [Representatives Module API Reference](../../backend/modules/representatives.md#public-endpoints).
|
||||
|
||||
| Method | Endpoint | Auth | Description |
|
||||
|--------|----------|------|-------------|
|
||||
| POST | `/api/public/representatives/lookup` | None | Lookup representatives by postal code |
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `REPRESENT_API_URL` | string | https://represent.opennorth.ca | Represent API base URL |
|
||||
| `REPRESENT_CACHE_TTL` | number | 2592000 | Cache TTL in seconds (30 days) |
|
||||
| `REPRESENT_RATE_LIMIT` | number | 60 | Max requests per minute |
|
||||
|
||||
### Represent API
|
||||
|
||||
The Represent API is a public service provided by Open North. No API key required.
|
||||
|
||||
**API Documentation**: https://represent.opennorth.ca/api/
|
||||
|
||||
**Endpoints Used:**
|
||||
|
||||
- `GET /postcodes/:postalCode/` — Lookup representatives by postal code
|
||||
- `GET /representatives/` — List representatives (unused, direct lookups only)
|
||||
|
||||
**Rate Limits:**
|
||||
|
||||
- 60 requests per minute per IP address
|
||||
- Exceeding limit returns HTTP 429
|
||||
|
||||
**Postal Code Format:**
|
||||
|
||||
- Canadian postal codes only
|
||||
- Format: `K1A 0A1` or `K1A0A1` (space optional)
|
||||
- Normalized to uppercase without spaces for API calls
|
||||
|
||||
## Admin Workflow
|
||||
|
||||
### 1. View Cache Statistics
|
||||
|
||||
[Screenshot: RepresentativesPage with cache stats cards]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to **Influence > Representatives**
|
||||
2. View cache statistics:
|
||||
- **Total Cached**: Total representatives in cache
|
||||
- **Unique Postal Codes**: Number of postal codes cached
|
||||
- **Cache Hit Rate**: Percentage of lookups served from cache
|
||||
- **Stale Entries**: Entries older than 30 days
|
||||
|
||||
**Code Example (RepresentativesPage.tsx):**
|
||||
|
||||
```typescript
|
||||
const [stats, setStats] = useState({
|
||||
totalCached: 0,
|
||||
uniquePostalCodes: 0,
|
||||
cacheHitRate: 0,
|
||||
staleEntries: 0
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
const fetchStats = async () => {
|
||||
const { data } = await api.get('/representatives/stats');
|
||||
setStats(data);
|
||||
};
|
||||
|
||||
fetchStats();
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<Row gutter={16}>
|
||||
<Col span={6}>
|
||||
<Card>
|
||||
<Statistic title="Total Cached" value={stats.totalCached} />
|
||||
</Card>
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Card>
|
||||
<Statistic title="Unique Postal Codes" value={stats.uniquePostalCodes} />
|
||||
</Card>
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Card>
|
||||
<Statistic
|
||||
title="Cache Hit Rate"
|
||||
value={stats.cacheHitRate}
|
||||
suffix="%"
|
||||
precision={1}
|
||||
/>
|
||||
</Card>
|
||||
</Col>
|
||||
<Col span={6}>
|
||||
<Card>
|
||||
<Statistic
|
||||
title="Stale Entries"
|
||||
value={stats.staleEntries}
|
||||
valueStyle={{ color: stats.staleEntries > 0 ? '#cf1322' : undefined }}
|
||||
/>
|
||||
</Card>
|
||||
</Col>
|
||||
</Row>
|
||||
);
|
||||
```
|
||||
|
||||
### 2. Manual Postal Code Lookup
|
||||
|
||||
[Screenshot: RepresentativesPage with postal code search form]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Enter postal code in search box (e.g., "K1A 0A1")
|
||||
2. Click **Lookup** button
|
||||
3. View results:
|
||||
- Representative name, office, party
|
||||
- Electoral district
|
||||
- Email address (if available)
|
||||
4. Results automatically cached for future lookups
|
||||
|
||||
**Use Cases:**
|
||||
|
||||
- Pre-populate cache for campaign areas
|
||||
- Verify representative information
|
||||
- Test postal code validation
|
||||
- Troubleshoot lookup issues
|
||||
|
||||
**Code Example (representatives.service.ts):**
|
||||
|
||||
```typescript
|
||||
async lookupByPostalCode(postalCode: string): Promise<Representative[]> {
|
||||
// Normalize postal code
|
||||
const normalized = postalCode.toUpperCase().replace(/\s/g, '');
|
||||
|
||||
// Check cache first (within last 30 days)
|
||||
const cached = await this.prisma.representative.findMany({
|
||||
where: {
|
||||
postalCode: normalized,
|
||||
lastUpdated: {
|
||||
gte: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000) // 30 days
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if (cached.length > 0) {
|
||||
logger.info(`Cache hit for postal code ${normalized}`);
|
||||
return cached;
|
||||
}
|
||||
|
||||
// Cache miss - fetch from Represent API
|
||||
logger.info(`Cache miss for postal code ${normalized}, fetching from API`);
|
||||
|
||||
const representatives = await this.representApiClient.getRepresentativesByPostalCode(
|
||||
normalized
|
||||
);
|
||||
|
||||
// Save to cache
|
||||
const saved = await Promise.all(
|
||||
representatives.map(rep =>
|
||||
this.prisma.representative.upsert({
|
||||
where: { representId: rep.representId },
|
||||
update: {
|
||||
...rep,
|
||||
postalCode: normalized,
|
||||
lastUpdated: new Date()
|
||||
},
|
||||
create: {
|
||||
...rep,
|
||||
postalCode: normalized,
|
||||
lastUpdated: new Date()
|
||||
}
|
||||
})
|
||||
)
|
||||
);
|
||||
|
||||
return saved;
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Clear Stale Cache Entries
|
||||
|
||||
[Screenshot: RepresentativesPage with "Clear Stale Cache" button]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Click **Clear Stale Cache** button
|
||||
2. Confirm deletion in modal
|
||||
3. System deletes all entries older than 30 days
|
||||
4. View updated cache statistics
|
||||
|
||||
**Automatic Cleanup:**
|
||||
|
||||
Cache invalidation also runs automatically via cron job (daily at 2 AM):
|
||||
|
||||
```typescript
|
||||
// api/src/server.ts
|
||||
|
||||
import cron from 'node-cron';
|
||||
|
||||
// Clean stale representative cache daily at 2 AM
|
||||
cron.schedule('0 2 * * *', async () => {
|
||||
try {
|
||||
const thirtyDaysAgo = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000);
|
||||
|
||||
const result = await prisma.representative.deleteMany({
|
||||
where: {
|
||||
lastUpdated: {
|
||||
lt: thirtyDaysAgo
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
logger.info(`Deleted ${result.count} stale representative cache entries`);
|
||||
} catch (error) {
|
||||
logger.error('Failed to clean representative cache:', error);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### 4. Delete Specific Cache Entries
|
||||
|
||||
[Screenshot: RepresentativesPage table with delete buttons]
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Browse cached representatives table
|
||||
2. Click **Delete** button on specific row
|
||||
3. Confirm deletion
|
||||
4. Representative removed from cache (will be re-fetched on next lookup)
|
||||
|
||||
**Bulk Delete by Postal Code:**
|
||||
|
||||
1. Click **Delete All** button on postal code group
|
||||
2. Confirm deletion
|
||||
3. All representatives for that postal code removed from cache
|
||||
|
||||
## Public Workflow
|
||||
|
||||
### 1. Enter Postal Code
|
||||
|
||||
[Screenshot: CampaignPage with postal code input field]
|
||||
|
||||
**User Journey:**
|
||||
|
||||
1. User visits campaign page (`/campaigns/{slug}`)
|
||||
2. Enters postal code in lookup form
|
||||
3. Clicks **Find My Representatives**
|
||||
4. System performs lookup (cache or API)
|
||||
5. Representatives displayed below form
|
||||
|
||||
**Code Example (CampaignPage.tsx):**
|
||||
|
||||
```typescript
|
||||
const [representatives, setRepresentatives] = useState<Representative[]>([]);
|
||||
const [loading, setLoading] = useState(false);
|
||||
|
||||
const handleLookup = async (values: { postalCode: string }) => {
|
||||
setLoading(true);
|
||||
|
||||
try {
|
||||
const { data } = await axios.post('/api/public/representatives/lookup', {
|
||||
postalCode: values.postalCode
|
||||
});
|
||||
|
||||
setRepresentatives(data);
|
||||
|
||||
if (data.length === 0) {
|
||||
message.warning('No representatives found for this postal code');
|
||||
}
|
||||
} catch (error) {
|
||||
message.error('Failed to lookup representatives');
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<Form onFinish={handleLookup}>
|
||||
<Form.Item
|
||||
name="postalCode"
|
||||
label="Postal Code"
|
||||
rules={[
|
||||
{ required: true, message: 'Please enter your postal code' },
|
||||
{
|
||||
pattern: /^[A-Za-z]\d[A-Za-z][ -]?\d[A-Za-z]\d$/,
|
||||
message: 'Please enter a valid Canadian postal code'
|
||||
}
|
||||
]}
|
||||
>
|
||||
<Input placeholder="K1A 0A1" maxLength={7} />
|
||||
</Form.Item>
|
||||
|
||||
<Form.Item>
|
||||
<Button type="primary" htmlType="submit" loading={loading}>
|
||||
Find My Representatives
|
||||
</Button>
|
||||
</Form.Item>
|
||||
</Form>
|
||||
);
|
||||
```
|
||||
|
||||
### 2. View Representatives
|
||||
|
||||
[Screenshot: Representative cards with contact information]
|
||||
|
||||
**Display Fields:**
|
||||
|
||||
- Representative name
|
||||
- Elected office (MP, MPP, Mayor, Councillor)
|
||||
- Political party (if applicable)
|
||||
- Electoral district name
|
||||
- Photo (if available)
|
||||
- Email button (if email available)
|
||||
|
||||
**Filtering:**
|
||||
|
||||
Representatives filtered by campaign's `targetGovernmentLevels`:
|
||||
|
||||
```typescript
|
||||
// Filter representatives by campaign levels
|
||||
const filteredRepresentatives = representatives.filter(rep =>
|
||||
campaign.targetGovernmentLevels.includes(rep.level)
|
||||
);
|
||||
```
|
||||
|
||||
### 3. Select Representatives to Email
|
||||
|
||||
[Screenshot: Representative list with checkboxes]
|
||||
|
||||
**User Journey:**
|
||||
|
||||
1. User reviews list of representatives
|
||||
2. Selects representatives to email (checkboxes)
|
||||
3. Clicks **Continue** to email form
|
||||
4. System pre-populates recipient list
|
||||
|
||||
**Code Example:**
|
||||
|
||||
```typescript
|
||||
const [selectedReps, setSelectedReps] = useState<string[]>([]);
|
||||
|
||||
const handleSelectAll = () => {
|
||||
setSelectedReps(representatives.map(r => r.id));
|
||||
};
|
||||
|
||||
const handleSelectNone = () => {
|
||||
setSelectedReps([]);
|
||||
};
|
||||
|
||||
return (
|
||||
<Space direction="vertical" style={{ width: '100%' }}>
|
||||
<Space>
|
||||
<Button onClick={handleSelectAll}>Select All</Button>
|
||||
<Button onClick={handleSelectNone}>Select None</Button>
|
||||
</Space>
|
||||
|
||||
<Checkbox.Group
|
||||
value={selectedReps}
|
||||
onChange={setSelectedReps}
|
||||
style={{ width: '100%' }}
|
||||
>
|
||||
{representatives.map(rep => (
|
||||
<Card key={rep.id} style={{ marginBottom: 16 }}>
|
||||
<Checkbox value={rep.id}>
|
||||
<Space>
|
||||
{rep.photoUrl && (
|
||||
<Avatar src={rep.photoUrl} size={64} />
|
||||
)}
|
||||
<Space direction="vertical" size={0}>
|
||||
<Typography.Text strong>{rep.name}</Typography.Text>
|
||||
<Typography.Text type="secondary">{rep.electedOffice}</Typography.Text>
|
||||
<Typography.Text type="secondary">{rep.districtName}</Typography.Text>
|
||||
{rep.partyName && <Tag>{rep.partyName}</Tag>}
|
||||
</Space>
|
||||
</Space>
|
||||
</Checkbox>
|
||||
</Card>
|
||||
))}
|
||||
</Checkbox.Group>
|
||||
</Space>
|
||||
);
|
||||
```
|
||||
|
||||
## Volunteer Workflow
|
||||
|
||||
Not applicable — representative lookup is public-facing and admin-managed.
|
||||
|
||||
## Code Examples
|
||||
|
||||
### Backend: Represent API Client
|
||||
|
||||
```typescript
|
||||
// api/src/modules/influence/representatives/represent-api.client.ts
|
||||
|
||||
import axios from 'axios';
|
||||
import { logger } from '../../../utils/logger';
|
||||
|
||||
const REPRESENT_API_URL = process.env.REPRESENT_API_URL || 'https://represent.opennorth.ca';
|
||||
|
||||
interface RepresentApiResponse {
|
||||
objects: Array<{
|
||||
name: string;
|
||||
email: string;
|
||||
district_name: string;
|
||||
elected_office: string;
|
||||
party_name?: string;
|
||||
photo_url?: string;
|
||||
url: string;
|
||||
representative_set_name: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
export class RepresentApiClient {
|
||||
async getRepresentativesByPostalCode(postalCode: string): Promise<any[]> {
|
||||
try {
|
||||
const { data } = await axios.get<RepresentApiResponse>(
|
||||
`${REPRESENT_API_URL}/postcodes/${postalCode}/`,
|
||||
{
|
||||
headers: {
|
||||
'Accept': 'application/json'
|
||||
},
|
||||
timeout: 10000
|
||||
}
|
||||
);
|
||||
|
||||
return data.objects.map(rep => ({
|
||||
representId: this.extractRepresentId(rep.url),
|
||||
name: rep.name,
|
||||
email: rep.email || null,
|
||||
districtName: rep.district_name,
|
||||
electedOffice: rep.elected_office,
|
||||
partyName: rep.party_name || null,
|
||||
photoUrl: rep.photo_url || null,
|
||||
level: this.mapGovernmentLevel(rep.representative_set_name)
|
||||
}));
|
||||
} catch (error) {
|
||||
if (axios.isAxiosError(error)) {
|
||||
if (error.response?.status === 404) {
|
||||
logger.warn(`No representatives found for postal code: ${postalCode}`);
|
||||
return [];
|
||||
}
|
||||
|
||||
if (error.response?.status === 429) {
|
||||
logger.error('Represent API rate limit exceeded');
|
||||
throw new Error('Rate limit exceeded. Please try again later.');
|
||||
}
|
||||
}
|
||||
|
||||
logger.error('Represent API error:', error);
|
||||
throw new Error('Failed to fetch representatives');
|
||||
}
|
||||
}
|
||||
|
||||
private extractRepresentId(url: string): string {
|
||||
// Extract ID from URL: /representatives/house-of-commons/123/
|
||||
const match = url.match(/\/representatives\/[^\/]+\/(\d+)\//);
|
||||
return match ? match[1] : url;
|
||||
}
|
||||
|
||||
private mapGovernmentLevel(setName: string): string {
|
||||
// Map representative set names to standard levels
|
||||
const lowerSetName = setName.toLowerCase();
|
||||
|
||||
if (lowerSetName.includes('house-of-commons')) return 'federal';
|
||||
if (lowerSetName.includes('legislative-assembly')) return 'provincial';
|
||||
if (lowerSetName.includes('council')) return 'municipal';
|
||||
|
||||
return 'other';
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend: Representative Card Component
|
||||
|
||||
```typescript
|
||||
// admin/src/components/influence/RepresentativeCard.tsx
|
||||
|
||||
import React from 'react';
|
||||
import { Card, Avatar, Space, Typography, Tag, Button } from 'antd';
|
||||
import { MailOutlined, UserOutlined } from '@ant-design/icons';
|
||||
import type { Representative } from '../../types/api';
|
||||
|
||||
interface RepresentativeCardProps {
|
||||
representative: Representative;
|
||||
onSelect?: (id: string) => void;
|
||||
selected?: boolean;
|
||||
}
|
||||
|
||||
const RepresentativeCard: React.FC<RepresentativeCardProps> = ({
|
||||
representative,
|
||||
onSelect,
|
||||
selected
|
||||
}) => {
|
||||
const levelColors: Record<string, string> = {
|
||||
federal: 'blue',
|
||||
provincial: 'green',
|
||||
municipal: 'orange'
|
||||
};
|
||||
|
||||
return (
|
||||
<Card
|
||||
hoverable={!!onSelect}
|
||||
onClick={() => onSelect?.(representative.id)}
|
||||
style={{
|
||||
borderColor: selected ? '#1890ff' : undefined,
|
||||
borderWidth: selected ? 2 : 1
|
||||
}}
|
||||
>
|
||||
<Space align="start" size="large">
|
||||
<Avatar
|
||||
src={representative.photoUrl}
|
||||
icon={<UserOutlined />}
|
||||
size={80}
|
||||
/>
|
||||
|
||||
<Space direction="vertical" size={0} style={{ flex: 1 }}>
|
||||
<Typography.Title level={5} style={{ margin: 0 }}>
|
||||
{representative.name}
|
||||
</Typography.Title>
|
||||
|
||||
<Typography.Text type="secondary">
|
||||
{representative.electedOffice}
|
||||
</Typography.Text>
|
||||
|
||||
<Typography.Text type="secondary">
|
||||
{representative.districtName}
|
||||
</Typography.Text>
|
||||
|
||||
<Space size="small" style={{ marginTop: 8 }}>
|
||||
<Tag color={levelColors[representative.level] || 'default'}>
|
||||
{representative.level.toUpperCase()}
|
||||
</Tag>
|
||||
|
||||
{representative.partyName && (
|
||||
<Tag>{representative.partyName}</Tag>
|
||||
)}
|
||||
</Space>
|
||||
|
||||
{representative.email && (
|
||||
<Button
|
||||
type="link"
|
||||
icon={<MailOutlined />}
|
||||
href={`mailto:${representative.email}`}
|
||||
style={{ padding: 0, marginTop: 8 }}
|
||||
>
|
||||
{representative.email}
|
||||
</Button>
|
||||
)}
|
||||
</Space>
|
||||
</Space>
|
||||
</Card>
|
||||
);
|
||||
};
|
||||
|
||||
export default RepresentativeCard;
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### No Representatives Found
|
||||
|
||||
**Symptoms:**
|
||||
- Lookup returns empty array
|
||||
- Error: "No representatives found for this postal code"
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Verify postal code format** → Must be valid Canadian postal code
|
||||
2. **Check Represent API status** → Visit https://represent.opennorth.ca/health
|
||||
3. **Test postal code manually** → Try https://represent.opennorth.ca/postcodes/K1A0A1/
|
||||
4. **Review API logs** → Check for rate limit errors
|
||||
|
||||
**Debugging:**
|
||||
|
||||
```bash
|
||||
# Test Represent API directly
|
||||
curl https://represent.opennorth.ca/postcodes/K1A0A1/ | jq
|
||||
|
||||
# Check representative cache
|
||||
docker compose exec v2-postgres psql -U changemaker -d changemaker_lite -c \
|
||||
"SELECT * FROM representatives WHERE postal_code = 'K1A0A1';"
|
||||
|
||||
# Check API logs
|
||||
docker compose logs api | grep "Represent API"
|
||||
```
|
||||
|
||||
### Rate Limit Exceeded
|
||||
|
||||
**Symptoms:**
|
||||
- HTTP 429 error
|
||||
- Error: "Rate limit exceeded. Please try again later."
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Implement exponential backoff** → Retry with increasing delays
|
||||
2. **Use cache more aggressively** → Increase cache TTL to 60 days
|
||||
3. **Batch lookups** → Avoid rapid repeated lookups
|
||||
4. **Contact Open North** → Request rate limit increase if needed
|
||||
|
||||
**Code Fix (represent-api.client.ts):**
|
||||
|
||||
```typescript
|
||||
async getRepresentativesByPostalCodeWithRetry(
|
||||
postalCode: string,
|
||||
maxRetries = 3
|
||||
): Promise<any[]> {
|
||||
for (let i = 0; i < maxRetries; i++) {
|
||||
try {
|
||||
return await this.getRepresentativesByPostalCode(postalCode);
|
||||
} catch (error) {
|
||||
if (error.message.includes('Rate limit exceeded')) {
|
||||
const delay = Math.pow(2, i) * 1000; // Exponential backoff
|
||||
logger.warn(`Rate limit hit, retrying in ${delay}ms...`);
|
||||
await new Promise(resolve => setTimeout(resolve, delay));
|
||||
continue;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error('Max retries exceeded');
|
||||
}
|
||||
```
|
||||
|
||||
### Stale Representative Information
|
||||
|
||||
**Symptoms:**
|
||||
- Representative email bounces
|
||||
- Representative no longer in office
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Clear cache for postal code** → Delete and re-fetch
|
||||
2. **Reduce cache TTL** → Set `REPRESENT_CACHE_TTL` to 7 days (604800)
|
||||
3. **Manual verification** → Check official government websites
|
||||
4. **Report to Represent API** → If data is incorrect, report to Open North
|
||||
|
||||
**Manual Cache Clear:**
|
||||
|
||||
```typescript
|
||||
// Via admin UI
|
||||
// Navigate to Influence > Representatives
|
||||
// Find postal code in table
|
||||
// Click "Delete All" for that postal code
|
||||
|
||||
// Via API
|
||||
await api.delete(`/representatives/postal-code/${postalCode}`);
|
||||
```
|
||||
|
||||
### Missing Email Addresses
|
||||
|
||||
**Symptoms:**
|
||||
- Representative has no email address
|
||||
- Cannot send campaign email
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Check Represent API data** → Some reps don't provide email publicly
|
||||
2. **Use manual email field** → Allow admins to add email addresses
|
||||
3. **Fallback to constituency office** → Use office email if available
|
||||
4. **Skip representative** → Don't include in email recipients
|
||||
|
||||
**Code Fix (representative.service.ts):**
|
||||
|
||||
```typescript
|
||||
async updateRepresentativeEmail(
|
||||
representId: string,
|
||||
email: string
|
||||
): Promise<Representative> {
|
||||
return this.prisma.representative.update({
|
||||
where: { representId },
|
||||
data: {
|
||||
email,
|
||||
lastUpdated: new Date() // Reset cache timestamp
|
||||
}
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Cache Strategy
|
||||
|
||||
**TTL Configuration:**
|
||||
|
||||
- **Default**: 30 days (2,592,000 seconds)
|
||||
- **Aggressive**: 60 days for stable electoral districts
|
||||
- **Conservative**: 7 days during election periods
|
||||
|
||||
**Cache Warming:**
|
||||
|
||||
Pre-populate cache for common postal codes:
|
||||
|
||||
```typescript
|
||||
// api/src/scripts/warm-representative-cache.ts
|
||||
|
||||
import { RepresentativeService } from '../modules/influence/representatives/representatives.service';
|
||||
import { PrismaClient } from '@prisma/client';
|
||||
|
||||
const prisma = new PrismaClient();
|
||||
const representativeService = new RepresentativeService(prisma);
|
||||
|
||||
// Common postal codes from campaign participation data
|
||||
const commonPostalCodes = [
|
||||
'K1A0A1', 'M5H2N2', 'V6B1A1', // Federal capitals
|
||||
'T2P2M5', 'H3B1A1', 'S7K0J5' // Provincial capitals
|
||||
];
|
||||
|
||||
async function warmCache() {
|
||||
for (const postalCode of commonPostalCodes) {
|
||||
try {
|
||||
await representativeService.lookupByPostalCode(postalCode);
|
||||
console.log(`Cached representatives for ${postalCode}`);
|
||||
} catch (error) {
|
||||
console.error(`Failed to cache ${postalCode}:`, error);
|
||||
}
|
||||
|
||||
// Rate limit: 1 request per second
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
}
|
||||
}
|
||||
|
||||
warmCache();
|
||||
```
|
||||
|
||||
### Query Optimization
|
||||
|
||||
**Index Usage:**
|
||||
|
||||
```sql
|
||||
-- Composite index for fast lookups
|
||||
CREATE INDEX idx_representative_postal_code_level
|
||||
ON representatives (postal_code, level);
|
||||
|
||||
-- Index for cache invalidation
|
||||
CREATE INDEX idx_representative_last_updated
|
||||
ON representatives (last_updated);
|
||||
```
|
||||
|
||||
**Query Pattern:**
|
||||
|
||||
```typescript
|
||||
// Optimized cache lookup with index
|
||||
const cached = await prisma.representative.findMany({
|
||||
where: {
|
||||
postalCode: normalized,
|
||||
level: { in: targetLevels }, // Use index
|
||||
lastUpdated: {
|
||||
gte: new Date(Date.now() - CACHE_TTL * 1000)
|
||||
}
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### API Rate Limiting
|
||||
|
||||
**Client-Side Rate Limiter:**
|
||||
|
||||
```typescript
|
||||
import Bottleneck from 'bottleneck';
|
||||
|
||||
const limiter = new Bottleneck({
|
||||
maxConcurrent: 1,
|
||||
minTime: 1000 // 1 request per second
|
||||
});
|
||||
|
||||
const getRepresentativesRateLimited = limiter.wrap(
|
||||
representApiClient.getRepresentativesByPostalCode.bind(representApiClient)
|
||||
);
|
||||
```
|
||||
|
||||
**Redis-Based Distributed Rate Limiting:**
|
||||
|
||||
```typescript
|
||||
import { RateLimiterRedis } from 'rate-limiter-flexible';
|
||||
|
||||
const rateLimiter = new RateLimiterRedis({
|
||||
storeClient: redisClient,
|
||||
keyPrefix: 'represent-api',
|
||||
points: 60, // 60 requests
|
||||
duration: 60 // per minute
|
||||
});
|
||||
|
||||
await rateLimiter.consume('represent-api-key');
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
### Backend Modules
|
||||
|
||||
- [Representatives Module](../../backend/modules/representatives.md) — Full API reference
|
||||
- [Campaigns Module](../../backend/modules/campaigns.md) — Campaign integration
|
||||
- [Postal Codes Module](../../backend/modules/postal-codes.md) — Postal code caching
|
||||
|
||||
### Frontend Pages
|
||||
|
||||
- [RepresentativesPage](../../frontend/pages/admin/representatives-page.md) — Admin cache management
|
||||
- [CampaignPage](../../frontend/pages/public/campaign-page.md) — Public representative lookup
|
||||
|
||||
### Database Models
|
||||
|
||||
- [Representative](../../database/models/representative.md) — Representative schema
|
||||
- [Campaign](../../database/models/campaign.md) — Campaign schema
|
||||
- [CampaignEmail](../../database/models/campaign-email.md) — Email tracking schema
|
||||
|
||||
### External APIs
|
||||
|
||||
- [Represent API Documentation](https://represent.opennorth.ca/api/) — Official API docs
|
||||
- [Open North](https://www.opennorth.ca/) — Represent API provider
|
||||
|
||||
### Configuration
|
||||
|
||||
- [Environment Variables](../../getting-started/configuration.md#represent-api) — Represent API settings
|
||||
1033
mkdocs/docs/v2/features/influence/responses.md
Normal file
1033
mkdocs/docs/v2/features/influence/responses.md
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user