admin/changemaker.lite
1
0
Fork 0
Code Issues Packages Projects Releases 35 Wiki Activity

listmonk sync

Browse Source
This commit is contained in:
admin
2025-08-15 11:14:38 -06:00
parent e36610ceed
commit 6ac45f7adf
20 changed files with 3416 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

318
map/instruct/LISTMONK_INTEGRATION_GUIDE.md Normal file
View File

@@ -0,0 +1,318 @@
# Listmonk Integration Guide
This guide covers the complete setup and usage of the Listmonk integration for the Map application.
## Overview
The Map application integrates seamlessly with [Listmonk](https://listmonk.app/), a self-hosted newsletter and mailing list manager, to provide advanced email marketing capabilities. This integration enables:
- **Real-time subscriber synchronization**
- **Automated list segmentation**
- **Professional email campaigns**
- **Advanced analytics and reporting**
- **Bi-directional data sync**
## Prerequisites
1. **Listmonk Instance**: You need a running Listmonk instance (typically deployed via Docker Compose)
2. **Admin Access**: Listmonk admin credentials for API access
3. **Network Connectivity**: Map application must be able to reach Listmonk instance
4. **Environment Configuration**: Proper `.env` setup
## Quick Setup
### 1. Configure Environment Variables
Add these variables to your `.env` file:
```env
# Listmonk Integration
LISTMONK_URL=http://listmonk:9000
LISTMONK_USERNAME=admin
LISTMONK_PASSWORD=your-secure-password
LISTMONK_ENABLED=true
LISTMONK_SYNC_ON_STARTUP=true
LISTMONK_AUTO_CREATE_LISTS=true
```
**Important Notes:**
- Use `http://listmonk:9000` for Docker Compose deployments (service name)
- Use `http://localhost:9000` for local development
- Ensure the password matches your Listmonk admin password
### 2. Restart the Application
```bash
docker-compose restart map-viewer
```
### 3. Verify Integration
1. **Check Admin Panel**: Go to `/admin.html` and look for the "Listmonk Integration" section
2. **Status Indicator**: Green status means successful connection
3. **View Logs**: Check terminal for sync messages
4. **Listmonk Dashboard**: Verify lists were created in Listmonk
## Automated List Management
The integration automatically creates and maintains these subscriber lists:
### Core Lists
- **All Locations** - Every location with contact information
- **All Users** - Every registered Map application user
### Support Level Lists
- **Support Level 1** - Strongest supporters (Green markers)
- **Support Level 2** - Moderate supporters (Yellow markers)
- **Support Level 3** - Weak supporters (Orange markers)
- **Support Level 4** - Opponents (Red markers)
### Sign Status Lists
- **Has Sign** - Locations with campaign signs
- **No Sign** - Locations without signs
### Combined Segmentation Lists
- **Support 1 with Signs** - Strong supporters who have signs
- **Support 2 with Signs** - Moderate supporters who have signs
- **Support 3 with Signs** - Weak supporters who have signs
- **Support 4 with Signs** - Opponents who have signs (for tracking)
- **Support 1-3** - All positive supporters (Levels 1, 2, and 3)
- **Sign Requests** - Supporters without signs (potential sign placements)
## Real-Time Sync Features
### Automatic Synchronization
- **New Location Added**: Instantly added to appropriate lists
- **Location Updated**: Support level or sign status changes trigger list updates
- **Location Deleted**: Removed from all lists
- **User Created**: Added to "All Users" list
- **User Deleted**: Removed from all lists
### Bi-Directional Sync
- **Unsubscribes**: When users unsubscribe in Listmonk, their records are updated in the Map database
- **Email Updates**: Email address changes sync both ways
- **List Management**: Changes in Listmonk are reflected in Map data
## Admin Management Features
### Status Dashboard
- **Connection Status**: Live indicator (🟢 Connected, 🔴 Error, ⚪ Disabled)
- **Sync Statistics**: Total subscribers, lists, and sync operations
- **Last Sync Times**: When each data type was last synchronized
- **Error Notifications**: Pop-up alerts for sync failures
### Manual Operations
- **Full Resync**: Completely rebuild all lists from current data
- **Selective Sync**: Sync only locations or users
- **Test Connection**: Verify Listmonk connectivity
- **Reinitialize**: Delete and recreate all lists
- **Progress Tracking**: Visual progress bars for bulk operations
### Bulk Sync Controls
```javascript
// Available admin actions
POST /api/listmonk/sync/full // Complete resync
POST /api/listmonk/sync/locations // Sync locations only
POST /api/listmonk/sync/users // Sync users only
POST /api/listmonk/test-connection // Test connectivity
POST /api/listmonk/reinitialize // Recreate lists
```
## Error Handling & Troubleshooting
### Common Issues
#### Connection Failed
**Symptoms**: Red status indicator, "Connection failed" in logs
**Solutions**:
1. Verify `LISTMONK_URL` is correct
2. Check Listmonk is running: `docker-compose ps`
3. Test network connectivity: `curl http://listmonk:9000`
4. Verify credentials in Listmonk admin panel
#### Sync Errors
**Symptoms**: Yellow status indicator, specific error messages in logs
**Solutions**:
1. Check Listmonk API limits and performance
2. Verify subscriber data format (valid emails, etc.)
3. Review Listmonk logs: `docker-compose logs listmonk`
4. Try manual resync from admin panel
#### Missing Lists
**Symptoms**: Lists not appearing in Listmonk
**Solutions**:
1. Ensure `LISTMONK_AUTO_CREATE_LISTS=true`
2. Restart application to trigger initialization
3. Use "Reinitialize" button in admin panel
4. Check Listmonk permissions and API access
#### Slow Sync Performance
**Symptoms**: Long delays during sync operations
**Solutions**:
1. Monitor Listmonk server resources
2. Check network latency between services
3. Consider reducing sync frequency for large datasets
4. Review database performance (both NocoDB and Listmonk)
### Debug Logging
Enable detailed logging by setting:
```env
NODE_ENV=development
```
This provides verbose sync operation details in the terminal.
## Email Campaign Workflow
### 1. Segment Selection
Use the automatically created lists in Listmonk to target specific audience segments:
- **Fundraising**: Target "Support Level 1" for donation requests
- **Volunteer Recruitment**: Target "Support 1-3" for volunteer opportunities
- **Sign Placement**: Target "Sign Requests" for new sign installations
- **Opposition Research**: Monitor "Support Level 4" engagement
### 2. Campaign Creation
Create campaigns in Listmonk using the segmented lists:
1. Go to Listmonk admin panel
2. Create new campaign
3. Select appropriate subscriber lists
4. Design email content
5. Schedule or send immediately
### 3. Analytics & Follow-up
- **Track Opens**: Monitor engagement by support level
- **Click Tracking**: Measure action rates across segments
- **Unsubscribe Monitoring**: Identify potential support level changes
- **A/B Testing**: Test messaging effectiveness across segments
## Advanced Configuration
### Custom List Creation
To create additional custom lists, modify the Listmonk service configuration:
```javascript
// app/services/listmonk.js
const customLists = [
{
name: 'High Value Voters',
description: 'Support Level 1 & 2 with voting history',
// Add custom segmentation logic
}
];
```
### Sync Frequency Tuning
Adjust sync timing based on your needs:
```env
# Sync immediately on data changes (default: true)
LISTMONK_REALTIME_SYNC=true
# Startup sync (default: true)
LISTMONK_SYNC_ON_STARTUP=true
# Batch size for large syncs (default: 100)
LISTMONK_BATCH_SIZE=50
```
### Webhook Integration
For advanced use cases, consider setting up Listmonk webhooks to trigger Map application updates:
```env
# Enable webhook endpoint
LISTMONK_WEBHOOK_ENABLED=true
LISTMONK_WEBHOOK_SECRET=your-webhook-secret
```
## Security Considerations
### Credentials Management
- **Store securely**: Use environment variables, never commit passwords
- **Rotate regularly**: Change Listmonk passwords periodically
- **Limit access**: Create dedicated API user with minimal required permissions
- **Network security**: Use HTTPS in production, restrict network access
### Data Privacy
- **GDPR Compliance**: Ensure proper consent and unsubscribe handling
- **Data Minimization**: Only sync necessary contact information
- **Retention Policies**: Configure appropriate data retention in Listmonk
- **Audit Logging**: Monitor access and changes to subscriber data
## Integration Monitoring
### Health Checks
The system provides several monitoring endpoints:
```javascript
GET /api/listmonk/status // Connection and sync status
GET /api/listmonk/lists // List information and counts
GET /health // Overall application health
```
### Performance Metrics
Monitor these key metrics:
- **Sync Duration**: Time to complete full synchronization
- **Error Rate**: Percentage of failed sync operations
- **List Growth**: Subscriber count changes over time
- **API Response Time**: Listmonk API performance
### Alerting
Set up monitoring for:
- **Connection failures** (implement external monitoring)
- **Sync errors** (check application logs)
- **List count discrepancies** (compare Map vs Listmonk counts)
- **Performance degradation** (sync time increases)
## Backup & Recovery
### Data Backup
- **Listmonk Data**: Regular database backups of Listmonk
- **Map Data**: NocoDB backup includes all source data
- **Configuration**: Version control `.env` files (without secrets)
### Disaster Recovery
In case of data loss:
1. **Restore Listmonk**: From database backup
2. **Reinitialize**: Use admin panel "Reinitialize" function
3. **Full Resync**: Rebuild all lists from Map data
4. **Verify**: Check list counts and subscriber data
## Best Practices
### Campaign Management
- **Consistent Messaging**: Maintain brand voice across all campaigns
- **Segmentation Strategy**: Use meaningful, actionable segments
- **Testing**: A/B test subject lines and content
- **Compliance**: Follow CAN-SPAM and GDPR requirements
- **Analytics**: Track and analyze campaign performance
### Data Quality
- **Email Validation**: Verify email addresses before adding to lists
- **Duplicate Prevention**: System handles duplicates automatically
- **Regular Cleanup**: Remove bounced and inactive subscribers
- **Contact Updates**: Keep contact information current
### Performance Optimization
- **Batch Operations**: Use bulk sync for large data changes
- **Off-Peak Sync**: Schedule heavy operations during low-usage periods
- **Resource Monitoring**: Watch server resources during sync operations
- **Network Optimization**: Ensure reliable connectivity between services
## Support & Documentation
### Additional Resources
- **Listmonk Documentation**: https://listmonk.app/docs/
- **Map Application Docs**: See main README.md
- **API Reference**: Available at `/api` endpoints
- **Community Support**: GitHub issues and discussions
### Getting Help
If you encounter issues:
1. **Check Status**: Admin panel shows current integration status
2. **Review Logs**: Terminal output provides detailed error information
3. **Test Connection**: Use admin panel connection test
4. **Documentation**: Refer to this guide and Listmonk docs
5. **Community**: Ask questions in project GitHub issues
This integration significantly enhances the Map application's email marketing capabilities while maintaining data consistency and providing powerful segmentation options for targeted campaigns.