campaign_connector/docs/enhanced-conversations-readme.md

Enhanced Conversations - WhatsApp-Style Messaging Interface

🎯 Overview

The Enhanced Conversations feature transforms the SMS Campaign Manager into a WhatsApp-style messaging interface with real-time bidirectional sync, message status tracking, and advanced conversation management.

✨ Key Features

🔄 Bidirectional SMS Sync

• Automatically syncs SMS messages from your Android device
• Real-time message updates via WebSocket connection
• Support for both campaign and manual messages
• Smart conversation threading by phone number

📱 WhatsApp-Style Interface

• Clean, modern messaging UI with message bubbles
• Contact avatars with initials
• Message timestamps and status indicators
• Scrollable message history with pagination
• Real-time typing and connection status

📊 Message Status Tracking

• Pending ⏳ - Message queued for sending
• Sent ✓ - Successfully sent via Termux API
• Delivered ✓✓ - Confirmed delivery (when available)
• Failed ❌ - Send attempt failed

🌟 Advanced Conversation Management

• Star/unstar important conversations
• Mark conversations as read/unread
• Search conversations by name or phone number
• Filter by status (All, Unread, Starred)
• Contact name resolution from phone contacts

⚡ Real-Time Updates

• WebSocket-powered live messaging
• Instant message delivery notifications
• Automatic conversation sync
• Connection status indicators

🏗️ Architecture

Backend Components

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Main App      │    │  Sync Service    │    │ Android Device  │
│   (Flask)       │◄──►│  (Background)    │◄──►│ (Termux API)    │
│                 │    │                  │    │                 │
│ • API Routes    │    │ • SMS Pulling    │    │ • SMS History   │
│ • WebSocket     │    │ • Message Queue  │    │ • Contact Names │
│ • Database      │    │ • Status Updates │    │ • Send SMS      │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Database Schema

Enhanced Conversations Table:

• is_starred - Boolean flag for important conversations
• contact_name - Resolved contact name from phone
• last_sync_timestamp - Track sync progress
• total_message_count - Message count optimization

Enhanced Messages Table:

• status - Message delivery status
• direction - Inbound/outbound classification
• timestamp - Unix timestamp for sorting
• external_message_id - Phone's SMS ID for deduplication
• sync_status - Sync state tracking

🚀 Installation & Setup

1. Quick Setup

# Run the automated deployment
./scripts/deploy_enhanced_conversations.sh

2. Manual Setup

# 1. Install dependencies
cd src && pip install -r requirements.txt

# 2. Run database migration
python scripts/migrate_conversations_db.py

# 3. Integrate with main app
python scripts/integrate_enhanced_conversations.py

# 4. Update Android Termux API server
scp android/termux-sms-api-server.py android-dev@your-phone-ip:~/projects/sms-campaign-manager/

# 5. Restart services
docker-compose restart  # OR python src/app.py

3. Android Device Setup

The enhanced system requires updated Termux API endpoints:

# On your Android device (via SSH)
cd ~/projects/sms-campaign-manager
pkill -f termux-sms-api-server.py
python termux-sms-api-server.py &

🎮 Usage Guide

Starting a Conversation

1. From Campaign: Conversations automatically created when sending campaign messages
2. Manual Sync: Click the sync button to pull message history from phone
3. Direct Access: Navigate to Conversations tab in dashboard

Messaging Interface

┌─────────────────────────────────────────────────────────────────┐
│ [★] Contact Name              🔄 Sync    [Last seen: 2m ago]     │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Hello! How can I help?                               [9:15 AM] │
│      [You]: Thanks for reaching out ✓✓               [9:16 AM]  │
│                                                                 │
├─────────────────────────────────────────────────────────────────┤
│ Type a message...                                     [Send] 📤  │
└─────────────────────────────────────────────────────────────────┘

Message Status Indicators

• ⏳ Pending - Message is being processed
• ✓ Sent - Successfully sent to phone
• ✓✓ Delivered - Confirmed delivery (when available)
• ❌ Failed - Send attempt failed

Conversation Management

Filtering:

• All - Show all conversations
• Unread - Only conversations with unread messages
• Starred - Important/flagged conversations

Actions:

• Star/Unstar - Mark conversations as important
• Mark as Read - Clear unread indicators
• Sync History - Pull latest messages from phone
• Search - Find conversations by name or content

🔧 API Reference

Enhanced Endpoints

# Get conversations with enhanced data
GET /api/conversations/enhanced?search=john&starred=true

# Get paginated messages
GET /api/conversations/{id}/messages?page=1&per_page=50

# Send message
POST /api/conversations/{id}/send
Content-Type: application/json
{
  "message": "Hello there!"
}

# Toggle star status
PUT /api/conversations/{id}/star

# Mark as read
PUT /api/conversations/{id}/mark-read

# Manual sync
POST /api/conversations/{id}/sync
POST /api/conversations/sync-all

# Get statistics
GET /api/conversations/stats

WebSocket Events

Client → Server:

socket.emit('join_conversation', { conversation_id: 'conv_123' });
socket.emit('send_message', { 
  conversation_id: 'conv_123', 
  phone: '5551234567',
  message: 'Hello!' 
});
socket.emit('sync_conversation', { conversation_id: 'conv_123' });

Server → Client:

socket.on('new_message', (data) => { /* Handle new message */ });
socket.on('message_status_update', (data) => { /* Update status */ });
socket.on('conversation_update', (data) => { /* Update conversation */ });
socket.on('sync_status', (data) => { /* Sync progress */ });

🧪 Testing

Run Test Suite

# Full test suite
python scripts/test_enhanced_conversations.py

# Verbose output
python scripts/test_enhanced_conversations.py --verbose

# Custom endpoints
python scripts/test_enhanced_conversations.py --base-url http://localhost:5000 --phone-ip 192.168.1.100

Manual Testing Checklist

• 🔌 WebSocket connection establishes
• 📱 Android device API responds
• 📤 Messages send successfully
• 📥 Incoming messages appear in real-time
• ⭐ Star/unstar functionality works
• 🔍 Search and filters work
• 📊 Status indicators update correctly
• 🔄 Manual sync pulls message history
• 📱 Contact names resolve from phone

🐛 Troubleshooting

Common Issues

WebSocket Connection Fails:

# Check if Socket.IO is installed
pip show flask-socketio

# Test WebSocket endpoint
curl http://localhost:5000/socket.io/?EIO=4&transport=polling

Android API Not Responding:

# Test connectivity
ping your-phone-ip

# Check Termux API server
curl http://your-phone-ip:5001/health

# Restart server on Android
ssh android-dev "pkill -f termux-sms-api && python ~/projects/sms-campaign-manager/termux-sms-api-server.py &"

Messages Not Syncing:

# Check sync service logs
docker-compose logs -f sms-campaign

# Manual database inspection
sqlite3 data/campaign.db "SELECT * FROM messages ORDER BY timestamp DESC LIMIT 10;"

# Test SMS history endpoint
curl "http://your-phone-ip:5001/api/sms/history?limit=5"

Database Errors:

# Re-run migration
python scripts/migrate_conversations_db.py

# Check schema
sqlite3 data/campaign.db ".schema conversations"

# Verify permissions
ls -la data/campaign.db

Debug Mode

Enable debug logging:

# In app.py
logging.basicConfig(level=logging.DEBUG)

# In JavaScript console
localStorage.debug = 'socket.io-client:*';

📈 Performance Optimization

Database Optimization

• Indexes: Automatic indexes on conversation_id, timestamp, status
• Pagination: Messages loaded in chunks of 50
• Caching: Conversation list cached in memory

Network Optimization

• WebSocket: Persistent connection reduces HTTP overhead
• Compression: Message payloads automatically compressed
• Rate Limiting: 2-second delay between SMS sends

Memory Management

• Connection Pooling: SQLite WAL mode for concurrent access
• Message Cleanup: Old messages archived after 30 days
• Client-side: Virtual scrolling for large message lists

🔒 Security Considerations

Authentication

• HMAC Signatures: All Android API calls signed
• Whitelisted Commands: Only approved Termux commands allowed
• Rate Limiting: Prevents SMS spam and abuse

Data Protection

• Local Database: Messages stored locally, not in cloud
• Encrypted Transport: HTTPS/WSS for all communications
• Permission Model: Respects Android SMS permissions

🤝 Contributing

Development Setup

# Clone and setup
git clone <repository>
cd sms-campaign-manager

# Install development dependencies
pip install -r src/requirements.txt
pip install -r requirements-dev.txt

# Run in development mode
python src/app.py

# Run tests
python scripts/test_enhanced_conversations.py --verbose

Architecture Guidelines

• Backend: Python with Flask, SQLite, asyncio
• Frontend: Vanilla JavaScript with Alpine.js
• Real-time: WebSocket via Socket.IO
• Styling: Tailwind CSS utility classes
• Testing: Python unittest + manual browser testing

📚 Additional Resources

• Termux API Documentation
• Flask-SocketIO Guide
• Alpine.js Documentation
• SMS Campaign Manager Wiki

---

Built with ❤️ for efficient SMS campaign management and real-time messaging.