admin 05dba72389 Fixed a bunch of the template stuff

2025-08-25 11:46:19 -06:00

34 KiB

Raw Blame History

Development Instructions for AI Assistant

User Instructions

The user likes to develop using plain javascript, hmtl, and css for most of their applications. They are getting used to python and are comfortable with it, but prefer javascript for frontend work. They use Ubuntu as their main development environment and like to use Docker for containerization. They also have an interest in Android development, particularly using Termux for lightweight solutions.

They like to do production driven development - using docker containers to quickly push updates. They use newer docker compose commands.

No file should be more then 1000 lines and if they approach that length lets make sure to implement a refactoring and breaking up of the file.

--- Everything past this line is instructions from past LLM's. If you want to pass along updated development instructions, please only update past this line ---

SMS Campaign Manager + Android-Homelab Integration Project

📁 Project Structure: This project is now organized into logical directories. See ../PROJECT_STRUCTURE.md for the complete file layout.

🚀 Quick Start: Use ../run.sh start from project root to launch the application.

Developer Profile & Preferences

Primary Technology Stack

Frontend: JavaScript (ES6+), HTML5, CSS3
- Prefer vanilla JS or lightweight frameworks
- Responsive design with mobile-first approach
- Clean, semantic HTML structure
- Modern CSS features (Grid, Flexbox, Custom Properties)
Backend: Python (comfortable, secondary to JS)
- Flask for web services and APIs
- SQLite for development, PostgreSQL for production
- pip for package management
- Virtual environments for isolation
- Request/Response pattern with proper error handling
Development Environment: Ubuntu Linux
- bash shell scripting
- Docker for containerization and deployment
- VS Code or similar editors (with Remote SSH support)
- Git for version control
- SSH-based remote development on Android

Android/Mobile Development

Lightweight solutions preferred - Battery and performance conscious
Termux environment - Linux tools on Android with SSH access
Minimal dependencies - Essential packages only
Background service efficiency - Proper power management
Remote development - SSH-based coding from Ubuntu homelab
Dual connection patterns - Primary/fallback reliability strategies

Development Methodology

Test-Driven Development (TDD)

Write tests first - Define expected behavior before implementation
Red-Green-Refactor cycle - Fail, pass, optimize
Unit tests for individual functions and modules
Integration tests for API endpoints and workflows
Connection testing - Verify dual SMS connection reliability
End-to-end validation - Complete workflow testing including Android integration
End-to-end tests for complete user journeys

Testing Frameworks

JavaScript: Jest, Mocha, or native browser testing
Python: pytest, unittest (built-in)
API Testing: Postman/Newman, curl scripts
Docker Testing: Container health checks, multi-stage builds

Code Quality Standards

JavaScript Guidelines

// Use const/let, avoid var
const apiEndpoint = '/api/campaign/status';
let currentCampaign = null;

// Async/await over promises when possible
async function getCampaignStatus() {
    try {
        const response = await fetch(apiEndpoint);
        return await response.json();
    } catch (error) {
        console.error('Campaign status fetch failed:', error);
        throw error;
    }
}

// Modular structure with clear separation of concerns
class CampaignManager {
    constructor(apiClient) {
        this.api = apiClient;
        this.status = 'idle';
    }
    
    async startCampaign(campaignId) {
        // Implementation with error handling
    }
}

Python Guidelines

# Type hints and docstrings
from typing import Dict, List, Optional
import pytest

def send_sms(phone: str, message: str, name: Optional[str] = None) -> Dict:
    """Send SMS message with optional name substitution.
    
    Args:
        phone: Target phone number
        message: Message text with possible {name} placeholder
        name: Optional name for substitution
    
    Returns:
        Dict with status, timestamp, and any errors
    
    Raises:
        ConnectionError: If phone connection fails
    """
    # Implementation
    
# Test-driven development
def test_sms_sending():
    """Test SMS sending with mock phone connection."""
    # Arrange
    phone = "7802921731"
    message = "Hello {name}!"
    name = "Reed"
    
    # Act
    result = send_sms(phone, message, name)
    
    # Assert
    assert result['status'] == 'success'
    assert result['sent_message'] == "Hello Reed!"

Architecture Preferences

Frontend Architecture

Progressive Enhancement - Works without JavaScript, enhanced with it
Component-based thinking - Reusable, self-contained modules
State management - Clear data flow, avoid global state
API-first design - Frontend consumes backend APIs
Real-time updates - Polling for live connection status (as seen in dashboard)

Backend Architecture

RESTful APIs - Clear, predictable endpoints
Dependency injection - Testable, modular code
Configuration externalization - Environment variables, config files
Error handling - Comprehensive, user-friendly error responses
Connection management - Health monitoring and automatic failover
Dual service patterns - Primary/fallback reliability (Termux API + ADB)

Docker Development Patterns

Ubuntu Host Environment

# Multi-stage builds for efficiency
FROM python:3.11-slim as builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

FROM python:3.11-slim as runtime
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
WORKDIR /app
COPY . .
# Network host mode for ADB and SSH connectivity
CMD ["python", "app.py"]

Development vs Production

Development: Volume mounts, hot reload, debug logging, SSH tunnels
Production: Minimal images, health checks, proper logging, automated restarts
Testing: Test-specific containers, isolated databases, mock services
Remote Integration: SSH connections to Android devices, API tunneling

Android/Termux Constraints

Resource Management

Minimal memory footprint - Avoid heavy frameworks
Battery optimization - Efficient polling, proper sleep modes
Storage awareness - Clean up temporary files, log rotation
Network efficiency - Compress data, batch operations
SSH optimization - Persistent connections, multiplexing

Service Architecture

# Lightweight Flask server for Termux with SSH integration
from flask import Flask, jsonify, request
import subprocess
import logging
import os
import json
import time

# Minimal logging configuration
logging.basicConfig(level=logging.INFO, 
                   format='%(asctime)s - %(levelname)s - %(message)s')

app = Flask(__name__)
app.config['SECRET_KEY'] = os.urandom(24)  # Random key for security

@app.route('/health')
def health_check():
    """Lightweight health check endpoint."""
    return jsonify({
        'status': 'healthy', 
        'timestamp': time.time(),
        'uptime_seconds': get_uptime(),
        'ssh_active': check_ssh_connection()
    })

@app.route('/api/sms/send', methods=['POST'])
def send_sms():
    """Native SMS sending via Termux API."""
    data = request.get_json()
    phone = data.get('phone')
    message = data.get('message', '')
    name = data.get('name')
    
    # Substitute name if provided
    if name:
        message = message.replace('{name}', name)
    
    try:
        # Use termux-sms-send for native SMS
        result = subprocess.run([
            'termux-sms-send', 
            '-n', phone, 
            message
        ], capture_output=True, text=True, timeout=30)
        
        if result.returncode == 0:
            return jsonify({
                'success': True,
                'message': 'SMS sent successfully',
                'phone': phone,
                'sent_message': message,
                'timestamp': time.time()
            })
        else:
            return jsonify({
                'success': False,
                'error': result.stderr or 'SMS send failed',
                'phone': phone
            }), 500
            
    except Exception as e:
        return jsonify({
            'success': False,
            'error': str(e),
            'phone': phone
        }), 500

@app.route('/api/device/battery')
def get_battery():
    """Get device battery status."""
    try:
        result = subprocess.run(['termux-battery-status'], 
                              capture_output=True, text=True)
        battery_data = json.loads(result.stdout)
        return jsonify({'success': True, 'battery': battery_data})
    except Exception as e:
        return jsonify({'success': False, 'error': str(e)}), 500

if __name__ == '__main__':
    # Bind to all interfaces to allow SSH tunneled connections
    app.run(host='0.0.0.0', port=5001, debug=False)

Specific Project Guidelines

SMS Campaign Manager Integration

Dual connection support - Termux API primary, ADB fallback
Graceful failover - Automatic switching between connection types
Real-time status - WebSocket or polling for live updates
User experience - Clear feedback, error recovery
Remote development - SSH-based coding from Ubuntu homelab
Connection monitoring - Health checks and performance metrics

Testing Strategy

Mock phone connections - Unit tests without hardware
Docker test environments - Isolated, reproducible tests
API contract testing - Ensure frontend/backend compatibility
End-to-end automation - Full workflow validation including Android device
SSH connection testing - Verify remote development environment
Dual connection validation - Test failover scenarios

Security Considerations

Input validation - Sanitize all user inputs
Authentication - HMAC signatures for API requests
Command whitelisting - Prevent dangerous operations
Error information - Avoid exposing system details
SSH security - Key-based authentication, proper permissions
Network isolation - Secure tunneling for API communications

Communication Style

Code Documentation

Clear variable names - Self-documenting code
Function docstrings - Purpose, parameters, return values
Inline comments - For complex logic only
README files - Setup, usage, troubleshooting
Integration guides - Detailed setup for Android/SSH components

Problem-Solving Approach

Break down complexity - Divide large tasks into smaller steps
Iterative development - Working increments, continuous testing
Error-first thinking - What can go wrong? How to handle it?
Performance awareness - Resource usage, optimization opportunities
Connection reliability - Always plan for network failures and recoveries

Development Workflow

Git Practices

Feature branches - One feature per branch
Descriptive commits - Clear, actionable commit messages
Pull requests - Code review and discussion
Tag releases - Version management
Integration testing - Verify Android components before merging

Docker Development

# Development workflow with Android integration
docker-compose -f docker-compose.dev.yml up --build
docker-compose exec app pytest tests/
docker-compose logs -f app

# Test Android connectivity
ssh android-dev "curl http://localhost:5001/health"

# Production deployment
docker-compose up -d
docker-compose exec app python -m pytest

Remote Android Development

# SSH into Android device for development
ssh android-dev

# Start development server on Android
cd ~/sms-campaign-manager
python termux-sms-api-server.py

# Test from Ubuntu homelab
curl -X GET http://10.0.0.193:5001/health
curl -X GET http://10.0.0.193:5001/api/device/battery

Expected Deliverables

Code Quality

Test coverage >80% - Comprehensive test suites including Android integration
Working Docker setup - Easy deployment and development
Clear documentation - Setup, API reference, troubleshooting, Android guides
Error handling - Graceful failure recovery and connection failover

User Experience

Responsive interface - Works on desktop and mobile
Real-time feedback - Status updates and progress indicators including connection health
Intuitive workflows - Minimal learning curve
Accessibility - WCAG guidelines compliance
Connection transparency - Clear indication of which SMS method is active

Performance

Fast page loads - Optimized assets, efficient queries
Minimal resource usage - Especially on Android side
Scalable architecture - Handle increasing message volumes
Monitoring integration - Health checks and metrics for both Ubuntu and Android components
Connection reliability - Sub-second failover between Termux API and ADB

Anti-Patterns to Avoid

Technical

Heavy frameworks on Android - Keep it lightweight for battery life
Synchronous operations - Use async/await patterns
Hardcoded values - Use configuration and environment variables
Ignore error handling - Always plan for failure scenarios
Single connection dependency - Always have fallback methods
Blocking SSH operations - Use timeouts and proper connection management

Development Process

Big-bang releases - Prefer incremental updates
Testing as afterthought - Test-driven development
Monolithic architecture - Modular, testable components
Poor documentation - Code should be self-explanatory with good docs
Ignore Android constraints - Battery, memory, and storage limitations
Network assumptions - Always plan for connectivity issues

Current Project Status Integration Notes

Completed Components

✅ SSH Development Environment: Full remote coding setup from Ubuntu to Android
✅ Dual SMS Connections: Termux API primary with ADB fallback
✅ Flask Enhancement: Connection manager with health monitoring
✅ Docker Integration: Network host mode for ADB and SSH connectivity
✅ Documentation: Comprehensive guides covering 472+ pages of setup procedures

Development Environment Ready

SSH Access: ssh android-dev (10.0.0.193:8022)
API Server: Running on Android at http://10.0.0.193:5001
Web Interface: Ubuntu homelab at http://localhost:5000
Connection Monitoring: Real-time status via /api/connections/status

This instruction set ensures consistent, high-quality development aligned with your preferences for JavaScript, HTML, CSS frontend work, comfortable Python backend development, Ubuntu environments, Docker usage, and lightweight Android solutions with reliable remote development capabilities.

Database Schema & API Endpoints Reference

Database Architecture

The SMS Campaign Manager uses SQLite with TRUNCATE journal mode to avoid WAL file locking issues in Docker environments. The database is located at ./data/campaign.db with proper permissions and backup strategies.

Core Tables Schema

-- Campaigns table - Main campaign tracking
CREATE TABLE campaigns (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,                    -- Campaign display name
    template TEXT NOT NULL,                -- Message template with {name} placeholders
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    started_at TIMESTAMP,                  -- When campaign execution began
    completed_at TIMESTAMP,               -- When campaign finished
    status TEXT DEFAULT 'draft',          -- 'draft', 'pending', 'running', 'completed', 'paused'
    total_recipients INTEGER DEFAULT 0,    -- Expected recipient count
    total_sent INTEGER DEFAULT 0           -- Actually sent message count
);

-- Recipients table - Individual campaign targets
CREATE TABLE recipients (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    campaign_id INTEGER,                   -- Foreign key to campaigns
    phone TEXT NOT NULL,                   -- Target phone number (e.g., '7801234567')
    name TEXT,                            -- Contact name for {name} substitution
    status TEXT DEFAULT 'pending',        -- 'pending', 'sent', 'failed'
    sent_at TIMESTAMP,                    -- When message was delivered
    error_message TEXT,                   -- Error details if failed
    FOREIGN KEY (campaign_id) REFERENCES campaigns (id)
);

-- Messages table - SMS tracking and conversation history
CREATE TABLE messages (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    phone TEXT NOT NULL,                   -- Phone number
    message TEXT NOT NULL,                 -- Actual message content sent
    direction TEXT DEFAULT 'outbound',     -- 'outbound' (sent) or 'inbound' (received)
    status TEXT DEFAULT 'pending',         -- 'pending', 'sent', 'delivered', 'failed'
    campaign_id INTEGER,                   -- Optional: link to campaign
    name TEXT,                            -- Contact name
    timestamp REAL,                       -- Unix timestamp from phone
    sent_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    is_read INTEGER DEFAULT 0,            -- Read status for conversations
    conversation_id TEXT,                 -- Conversation grouping identifier
    FOREIGN KEY (campaign_id) REFERENCES campaigns (id)
);

-- Contact lists table - Reusable contact groups
CREATE TABLE contact_lists (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,                    -- List display name
    filename TEXT,                        -- Original CSV filename
    contacts TEXT,                        -- JSON array of contact objects
    contact_count INTEGER DEFAULT 0,      -- Cached count for performance
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

Database Connection Best Practices

# Avoid WAL mode issues in Docker - use TRUNCATE mode
def get_db_connection(db_path):
    """Get database connection with proper settings to avoid WAL issues"""
    conn = sqlite3.connect(db_path, timeout=30.0, isolation_level='DEFERRED')
    
    # Use TRUNCATE journal mode instead of WAL to avoid file locking issues
    conn.execute("PRAGMA journal_mode=TRUNCATE")
    conn.execute("PRAGMA synchronous=NORMAL")
    conn.execute("PRAGMA temp_store=MEMORY")
    conn.execute("PRAGMA busy_timeout=30000")
    
    return conn

# Retry logic for database operations
def execute_with_retry(operation, max_retries=3):
    """Execute database operation with retry logic"""
    for attempt in range(max_retries):
        try:
            return operation()
        except sqlite3.OperationalError as e:
            if "disk I/O error" in str(e) and attempt < max_retries - 1:
                logger.warning(f"Database I/O error, retrying... (attempt {attempt + 1}/{max_retries})")
                time.sleep(1)
                continue
            else:
                raise

API Endpoints Documentation

Campaign Management

POST /api/campaign/upload

// Upload CSV file with contact preview
const formData = new FormData();
formData.append('file', csvFile);

const response = await fetch('/api/campaign/upload', {
    method: 'POST',
    body: formData
});

// Response format:
{
    "success": true,
    "total_contacts": 4,
    "contacts": [
        {"name": "John Doe", "phone": "7801234567"},
        {"name": "Jane Smith", "phone": "7801234568"}
    ],
    "preview": [/* First 10 contacts for UI display */],
    "message": "Successfully loaded 4 contacts"
}

POST /api/campaign/create

// Create new campaign with recipients
const campaignData = {
    name: "Weekend Volunteer Outreach",
    message: "Hi {name}! Hope all is well. Are you available this weekend?",
    csv_data: [
        {"name": "John Doe", "phone": "7801234567"},
        {"name": "Jane Smith", "phone": "7801234568"}
    ]
};

const response = await fetch('/api/campaign/create', {
    method: 'POST',
    headers: {'Content-Type': 'application/json'},
    body: JSON.stringify(campaignData)
});

// Response with contact preview:
{
    "success": true,
    "campaign_id": 24,
    "campaign_name": "Weekend Volunteer Outreach",
    "total_recipients": 2,
    "contacts_preview": [
        {
            "name": "John Doe",
            "phone": "7801234567", 
            "preview_message": "Hi John Doe! Hope all is well. Are you available this weekend?"
        }
    ],
    "message": "Campaign created with 2 recipients"
}

POST /api/campaign/start

// Start campaign execution (fetches recipients from database)
const startData = {
    campaign_id: 24
};

const response = await fetch('/api/campaign/start', {
    method: 'POST',
    headers: {'Content-Type': 'application/json'},
    body: JSON.stringify(startData)
});

// Response:
{
    "success": true,
    "status": "started", 
    "total": 2
}

GET /api/campaigns/recent

// Get recent campaign list with status
const response = await fetch('/api/campaigns/recent');
const campaigns = await response.json();

// Response format:
[
    {
        "id": 24,
        "name": "Weekend Volunteer Outreach",
        "total_recipients": 2,
        "sent_count": 2,               // Actual messages sent
        "status": "completed",
        "created_at": "2025-08-25 16:17:00.364348"
    }
]

Connection & Status Monitoring

GET /api/phone/status

// Check dual SMS connection status
const response = await fetch('/api/phone/status');
const status = await response.json();

// Response format:
{
    "termux_connected": true,          // Termux API health check
    "adb_connected": true,            // ADB connection status  
    "connected": true,                // Either connection available
    "ip": "10.0.0.193",
    "port": "5555",
    "prefer_termux": true             // Primary connection preference
}

GET /api/connections/status

// Detailed connection diagnostics
const response = await fetch('/api/connections/status');
const connections = await response.json();

// Response format:
{
    "connections": {
        "termux_api": {
            "available": true,
            "url": "http://10.0.0.193:5001",
            "type": "primary",
            "last_check": "2025-08-25T16:20:00Z"
        },
        "adb": {
            "available": true,
            "target": "10.0.0.193:5555", 
            "type": "fallback"
        }
    },
    "optimal_connection": "termux_api"
}

Contact List Management

GET /api/lists

// Get saved contact lists
const response = await fetch('/api/lists');
const lists = await response.json();

// Response format:
[
    {
        "id": 1,
        "name": "volunteers_2025.csv_20250825_120000",
        "contact_count": 15,
        "created_at": "2025-08-25T12:00:00Z"
    }
]

GET /api/lists/:id

// Load specific contact list with full contact data
const response = await fetch(`/api/lists/${listId}`);
const list = await response.json();

// Response format:
{
    "id": 1,
    "name": "volunteers_2025.csv",
    "contacts": [
        {"name": "John Doe", "phone": "7801234567"},
        {"name": "Jane Smith", "phone": "7801234568"}
    ],
    "contact_count": 2
}

Templates Management

GET /api/templates

// Get all message templates
const response = await fetch('/api/templates');
const templates = await response.json();

// Response format:
[
    {
        "id": 1,
        "name": "Volunteer Check-In",
        "content": "Hi {name}! Hope all is well. Are you available this weekend?",
        "description": "Check availability for volunteer events",
        "category": "volunteer",
        "is_favorite": 0,
        "times_used": 0,
        "created_at": "2025-08-25 16:39:21",
        "updated_at": "2025-08-25 16:39:21"
    }
]

POST /api/templates

// Create new template
const templateData = {
    name: "Follow-up Message",
    content: "Hi {name}! Following up on our conversation...",
    description: "Follow up template",
    category: "followup",
    is_favorite: 0
};

const response = await fetch('/api/templates', {
    method: 'POST',
    headers: {'Content-Type': 'application/json'},
    body: JSON.stringify(templateData)
});

// Response:
{
    "success": true,
    "template_id": 6,
    "message": "Template created successfully"
}

GET /api/templates/:id

// Get specific template
const response = await fetch('/api/templates/1');
const data = await response.json();

// Response:
{
    "success": true,
    "template": {
        "id": 1,
        "name": "Volunteer Check-In",
        "template": "Hi {name}! Hope all is well. Are you available this weekend?",
        "description": "Check availability for volunteer events",
        "category": "volunteer",
        "usage_count": 3
    }
}

PUT /api/templates/:id

// Update template
const updateData = {
    name: "Updated Template Name",
    content: "Updated message content",
    is_favorite: 1
};

const response = await fetch('/api/templates/1', {
    method: 'PUT',
    headers: {'Content-Type': 'application/json'},
    body: JSON.stringify(updateData)
});

DELETE /api/templates/:id

// Delete template
const response = await fetch('/api/templates/1', { method: 'DELETE' });
const result = await response.json();

// Response:
{
    "success": true,
    "message": "Template deleted successfully"
}

POST /api/templates/:id/use

// Mark template as used (increments usage_count)
await fetch('/api/templates/1/use', { method: 'POST' });

GET /api/analytics

// Campaign performance metrics
const response = await fetch('/api/analytics');
const analytics = await response.json();

// Response format:
{
    "total_sent": 14,
    "total_responded": 3,
    "total_opted_out": 0,
    "follow_ups": 2,
    "response_types": [
        {"type": "positive", "count": 2},
        {"type": "neutral", "count": 1}
    ],
    "recent_campaigns": [/* Latest 5 campaigns */]
}

Frontend Integration Patterns

Alpine.js Data Structure

function campaignApp() {
    return {
        // Connection monitoring
        phoneStatus: {
            termux_connected: false,
            adb_connected: false,
            connected: false,
            last_check: null
        },
        
        // Campaign creation
        campaignName: '',
        messageTemplate: '',
        contactsPreview: [],          // First 10 contacts for preview
        totalContacts: 0,            // Full contact count
        uploadedContacts: [],        // All uploaded contacts
        campaignReady: false,        // Upload validation flag
        
        // Recent campaigns (auto-refreshed)
        recentCampaigns: [],
        
        // Auto-refresh intervals
        async init() {
            await this.checkConnectionStatus();
            await this.loadRecentCampaigns();
            
            // Periodic updates
            setInterval(() => this.checkConnectionStatus(), 10000);
            setInterval(() => this.loadRecentCampaigns(), 15000);
        },
        
        // File upload with preview
        async handleFileUpload(event) {
            const file = event.target.files[0];
            const formData = new FormData();
            formData.append('file', file);
            
            const response = await fetch('/api/campaign/upload', {
                method: 'POST',
                body: formData
            });
            
            const data = await response.json();
            if (data.success) {
                this.contactsPreview = data.preview;
                this.totalContacts = data.total_contacts;
                this.uploadedContacts = data.contacts;
                this.campaignReady = true;
                
                // Store for campaign creation
                window.campaignContacts = data.contacts;
            }
        }
    }
}

Error Handling Patterns

// Consistent error handling across API calls
async function apiCall(endpoint, options = {}) {
    try {
        const response = await fetch(endpoint, {
            headers: {
                'Content-Type': 'application/json',
                ...options.headers
            },
            ...options
        });
        
        const data = await response.json();
        
        if (!response.ok) {
            throw new Error(data.error || `HTTP ${response.status}`);
        }
        
        return data;
    } catch (error) {
        console.error(`API call failed: ${endpoint}`, error);
        
        // User-friendly error display
        if (error.message.includes('Failed to fetch')) {
            throw new Error('Connection lost. Please check your internet connection.');
        }
        
        throw error;
    }
}

// Usage example
async function startCampaign() {
    try {
        const result = await apiCall('/api/campaign/create', {
            method: 'POST',
            body: JSON.stringify({
                name: this.campaignName,
                message: this.messageTemplate,
                csv_data: this.uploadedContacts
            })
        });
        
        if (result.success) {
            alert(`Campaign "${result.campaign_name}" created with ${result.total_recipients} recipients!`);
            
            // Start the campaign
            const startResult = await apiCall('/api/campaign/start', {
                method: 'POST',
                body: JSON.stringify({campaign_id: result.campaign_id})
            });
            
            if (startResult.success) {
                this.campaignState.status = 'running';
                this.campaignState.total = startResult.total;
            }
        }
    } catch (error) {
        alert(`Failed to start campaign: ${error.message}`);
    }
}

Database Maintenance & Troubleshooting

Common Issues & Solutions

WAL File Locking (Fixed)

# Database cleanup script (./fix-database.sh)
#!/bin/bash
echo "🔧 Fixing database and permissions..."

# Stop container
docker compose down

# Remove WAL and SHM files
rm -f data/campaign.db-wal data/campaign.db-shm

# Convert to TRUNCATE mode
sqlite3 data/campaign.db << EOF
PRAGMA journal_mode=TRUNCATE;
VACUUM;
.exit
EOF

# Fix permissions
chmod 777 data/
chmod 666 data/campaign.db

# Restart
docker compose up -d

Campaign Data Integrity

-- Verify campaign consistency
SELECT 
    c.id,
    c.name,
    c.total_recipients as expected,
    COUNT(r.id) as actual_recipients,
    c.total_sent,
    COUNT(CASE WHEN r.status = 'sent' THEN 1 END) as actual_sent
FROM campaigns c 
LEFT JOIN recipients r ON c.id = r.campaign_id 
GROUP BY c.id 
HAVING expected != actual_recipients;

-- Fix recipient counts
UPDATE campaigns 
SET total_recipients = (
    SELECT COUNT(*) FROM recipients WHERE campaign_id = campaigns.id
);

Performance Optimization

Database Indexing

-- Add indexes for common queries
CREATE INDEX IF NOT EXISTS idx_recipients_campaign_status 
ON recipients(campaign_id, status);

CREATE INDEX IF NOT EXISTS idx_messages_phone_timestamp 
ON messages(phone, timestamp DESC);

CREATE INDEX IF NOT EXISTS idx_campaigns_status_created 
ON campaigns(status, created_at DESC);

Connection Pooling

# SQLite connection management
import sqlite3
from contextlib import contextmanager

@contextmanager
def get_db_transaction():
    """Context manager for database transactions with proper cleanup"""
    conn = None
    try:
        conn = get_db_connection(app.config['DATABASE'])
        conn.execute("BEGIN")
        yield conn
        conn.commit()
    except Exception as e:
        if conn:
            conn.rollback()
        raise
    finally:
        if conn:
            conn.close()

# Usage
async def create_campaign_with_recipients(campaign_data, recipients):
    with get_db_transaction() as conn:
        cursor = conn.cursor()
        
        # Create campaign
        cursor.execute(
            "INSERT INTO campaigns (name, template, total_recipients, status) VALUES (?, ?, ?, ?)",
            (campaign_data['name'], campaign_data['message'], len(recipients), 'pending')
        )
        campaign_id = cursor.lastrowid
        
        # Batch insert recipients
        cursor.executemany(
            "INSERT INTO recipients (campaign_id, phone, name, status) VALUES (?, ?, ?, ?)",
            [(campaign_id, r.get('phone'), r.get('name'), 'pending') for r in recipients]
        )
        
        return campaign_id

Testing Database Operations

# Test campaign creation end-to-end
def test_campaign_workflow():
    """Test complete campaign creation and execution workflow"""
    # Setup
    test_contacts = [
        {"name": "Test User 1", "phone": "7801234567"},
        {"name": "Test User 2", "phone": "7801234568"}
    ]
    
    # Test campaign creation
    response = client.post('/api/campaign/create', json={
        'name': 'Test Campaign',
        'message': 'Hello {name}!',
        'csv_data': test_contacts
    })
    
    assert response.json['success'] == True
    campaign_id = response.json['campaign_id']
    
    # Verify database state
    with get_db_connection() as conn:
        cursor = conn.cursor()
        
        # Check campaign record
        cursor.execute("SELECT * FROM campaigns WHERE id = ?", (campaign_id,))
        campaign = cursor.fetchone()
        assert campaign['total_recipients'] == 2
        
        # Check recipient records
        cursor.execute("SELECT * FROM recipients WHERE campaign_id = ?", (campaign_id,))
        recipients = cursor.fetchall()
        assert len(recipients) == 2
        assert all(r['status'] == 'pending' for r in recipients)
    
    # Test campaign start
    response = client.post('/api/campaign/start', json={
        'campaign_id': campaign_id
    })
    
    assert response.json['success'] == True
    assert response.json['total'] == 2

This comprehensive database and API reference ensures consistent implementation patterns, proper error handling, and maintainable code architecture aligned with the project's lightweight, Docker-based approach.

34 KiB Raw Blame History