changemaker.lite/mkdocs/docs/v2/development/docker-workflow.md

# Docker Development Workflow

Guide to developing Changemaker Lite V2 using Docker containers for consistent, reproducible development environments.

## Overview

Docker-based development provides:

- **Consistency:** Same environment across all developer machines
- **Isolation:** Services don't interfere with host system
- **Production Parity:** Dev environment matches production
- **Easy Reset:** Rebuild containers for clean state

This guide covers Docker development workflows, from basic container operations to advanced debugging techniques.

## Docker vs Local Development

### When to Use Docker

**Advantages:**
- Consistent Node.js/PostgreSQL/Redis versions
- No need to install services on host machine
- Easy onboarding for new developers
- Production-like environment
- Volume mounts still support hot reload

**Disadvantages:**
- Slightly slower hot reload (especially macOS/Windows)
- More complex debugging setup
- Volume mount performance overhead
- Larger disk space usage

### When to Use Local npm

**Advantages:**
- Faster hot reload (native file system)
- Direct access to Node.js processes
- Simpler debugging (VSCode attach)
- Better performance on macOS/Windows

**Disadvantages:**
- Must install Node.js, PostgreSQL, Redis locally
- Version inconsistencies between developers
- Host system configuration required

### Hybrid Approach (Recommended)

Run databases in Docker, API/Admin locally:

```bash
# Docker: Databases only
docker compose up -d v2-postgres redis mailhog

# Local: Development servers
cd api && npm run dev
cd admin && npm run dev
```

This combines benefits of both approaches.

## Starting Development Services

### Full Docker Development

Start all development services:

```bash
# Core services (API, Admin, Databases)
docker compose up -d api admin v2-postgres redis

# Optional: MailHog for email testing
docker compose up -d mailhog

# Optional: Media API
docker compose up -d media-api
```

**Verify services started:**

```bash
docker compose ps
```

**Expected output:**
```
NAME                  STATUS    PORTS
api                   running   0.0.0.0:4000->4000/tcp
admin                 running   0.0.0.0:3000->3000/tcp
v2-postgres           running   0.0.0.0:5433->5432/tcp
redis                 running   0.0.0.0:6379->6379/tcp
mailhog               running   0.0.0.0:1025->1025/tcp, 0.0.0.0:8025->8025/tcp
```

### Selective Service Start

Start only what you need:

```bash
# Just databases (for local npm development)
docker compose up -d v2-postgres redis

# Just API (admin running locally)
docker compose up -d api v2-postgres redis

# Just Admin (API running locally)
docker compose up -d admin
```

### Start with Monitoring Stack

Enable monitoring services:

```bash
# Start with monitoring profile
docker compose --profile monitoring up -d

# Or specific monitoring services
docker compose up -d prometheus grafana
```

## Watching Logs

### View Service Logs

Real-time log streaming:

```bash
# All services
docker compose logs -f

# Specific service
docker compose logs -f api

# Multiple services
docker compose logs -f api admin

# Last 100 lines, then follow
docker compose logs -f --tail=100 api
```

**Log output example (API):**
```
api  | Server running on port 4000
api  | Database connected
api  | Redis connected
api  | BullMQ worker started
api  | GET /api/users 200 45ms
```

**Log output example (Admin):**
```
admin  | VITE v5.x.x ready in 500 ms
admin  | ➜  Local:   http://localhost:3000/
admin  | ➜  Network: http://172.18.0.5:3000/
```

### Filter Logs

Use grep to filter log output:

```bash
# Show only errors
docker compose logs -f api | grep ERROR

# Show only database queries
docker compose logs -f api | grep "SELECT\|INSERT\|UPDATE\|DELETE"

# Show only HTTP requests
docker compose logs -f api | grep "GET\|POST\|PUT\|DELETE"
```

### Export Logs

Save logs to file:

```bash
# All services
docker compose logs > logs.txt

# Specific service with timestamps
docker compose logs -t api > api-logs.txt

# Last 24 hours
docker compose logs --since 24h > recent-logs.txt
```

## Executing Commands in Containers

### Using docker compose exec

Run commands inside running containers:

```bash
# General syntax
docker compose exec <service> <command>

# Examples:
docker compose exec api npm run type-check
docker compose exec admin npm run lint
docker compose exec v2-postgres psql -U changemaker_v2 -d changemaker_v2_db
```

### Common API Commands

```bash
# Type-check
docker compose exec api npm run type-check

# Prisma migrate
docker compose exec api npx prisma migrate dev --name add_field

# Prisma Studio
docker compose exec api npx prisma studio

# Seed database
docker compose exec api npx prisma db seed

# Drizzle push (Media API)
docker compose exec api npx drizzle-kit push

# Node REPL
docker compose exec api node

# Shell access
docker compose exec api sh
```

### Common Admin Commands

```bash
# Type-check
docker compose exec admin npm run type-check

# Build
docker compose exec admin npm run build

# Lint
docker compose exec admin npm run lint

# Shell access
docker compose exec admin sh
```

### Database Commands

```bash
# PostgreSQL shell
docker compose exec v2-postgres psql -U changemaker_v2 -d changemaker_v2_db

# Run SQL query
docker compose exec v2-postgres psql -U changemaker_v2 -d changemaker_v2_db -c "SELECT COUNT(*) FROM users;"

# Dump database
docker compose exec v2-postgres pg_dump -U changemaker_v2 changemaker_v2_db > backup.sql

# Restore database
cat backup.sql | docker compose exec -T v2-postgres psql -U changemaker_v2 changemaker_v2_db
```

### Redis Commands

```bash
# Redis CLI
docker compose exec redis redis-cli -a your_redis_password

# Ping
docker compose exec redis redis-cli -a your_redis_password ping

# Get all keys
docker compose exec redis redis-cli -a your_redis_password KEYS '*'

# Monitor commands
docker compose exec redis redis-cli -a your_redis_password MONITOR
```

## Hot Reload in Containers

### How Volume Mounts Enable Hot Reload

Docker Compose volume mounts sync code between host and container:

```yaml
# docker-compose.yml
api:
  volumes:
    - ./api:/app                # Syncs code changes
    - /app/node_modules         # Preserves container's node_modules
    - /app/dist                 # Preserves build output
```

**When you edit a file on host:**
1. File change detected by host file system
2. Change synced to container via volume mount
3. `tsx watch` (API) or Vite (Admin) detects change
4. Service restarts (API) or HMR updates (Admin)

### API Hot Reload

API uses `tsx watch` for auto-restart:

```bash
# Start API in Docker
docker compose up -d api

# Watch logs
docker compose logs -f api

# Edit file: api/src/modules/auth/auth.service.ts
# Logs show:
# api  | File changed: src/modules/auth/auth.service.ts
# api  | Restarting server...
# api  | Server running on port 4000
```

**What triggers reload:**
- `.ts` file changes in `src/`
- Schema changes (after Prisma migrate)

**What does NOT trigger reload:**
- `.env` changes (restart container manually)
- `package.json` changes (rebuild container)

### Admin Hot Reload (Vite HMR)

Admin uses Vite Hot Module Replacement:

```bash
# Start Admin in Docker
docker compose up -d admin

# Watch logs
docker compose logs -f admin

# Edit file: admin/src/pages/UsersPage.tsx
# Logs show:
# admin  | 10:30:45 AM [vite] hmr update /src/pages/UsersPage.tsx
# Browser updates WITHOUT full reload
```

**HMR behavior:**
- Component changes: Updates component only
- CSS changes: Updates styles instantly
- Store changes: May require full reload

### Performance Considerations

**Linux:** Volume mounts are native, excellent performance.

**macOS/Windows:** Volume mounts use virtualization layer, slower performance.

**Optimization for macOS/Windows:**

1. **Use delegated volume mounts** (docker-compose.yml):

```yaml
api:
  volumes:
    - ./api:/app:delegated  # Slightly better performance
```

2. **Reduce watched files** (.dockerignore):

```
node_modules
dist
coverage
.git
*.log
```

3. **Use local development** for intensive work:

```bash
# Stop Docker services
docker compose stop api admin

# Run locally
cd api && npm run dev
cd admin && npm run dev
```

## Database Operations

### Running Migrations in Docker

```bash
# Create migration
docker compose exec api npx prisma migrate dev --name add_user_field

# Apply migrations (production)
docker compose exec api npx prisma migrate deploy

# Check migration status
docker compose exec api npx prisma migrate status
```

### Seeding Database

```bash
# Run seed script
docker compose exec api npx prisma db seed

# Or run custom script
docker compose exec api npx tsx prisma/custom-seed.ts
```

### Resetting Database

**WARNING: Deletes all data!**

```bash
# Reset and re-seed
docker compose exec api npx prisma migrate reset

# Confirm when prompted:
# ⚠️  All data will be lost. Continue? [y/N]: y
```

### Prisma Studio in Docker

```bash
# Start Prisma Studio
docker compose exec api npx prisma studio

# Access at http://localhost:5555
```

**Note:** Port forwarding must be configured (already set in docker-compose.yml).

### Manual Database Access

```bash
# Open PostgreSQL shell
docker compose exec v2-postgres psql -U changemaker_v2 -d changemaker_v2_db

# Run queries
changemaker_v2_db=# SELECT * FROM users;
changemaker_v2_db=# \dt  -- List tables
changemaker_v2_db=# \q   -- Exit
```

## Rebuilding Containers

### When to Rebuild

Rebuild containers when:
- `package.json` dependencies change
- `Dockerfile` changes
- Base image needs update
- Container is in corrupted state

### Rebuild Commands

```bash
# Rebuild all services
docker compose build

# Rebuild specific service
docker compose build api

# Rebuild without cache (clean build)
docker compose build --no-cache api

# Rebuild and restart
docker compose up -d --build api
```

### Full Rebuild Workflow

```bash
# 1. Stop services
docker compose down

# 2. Rebuild (no cache)
docker compose build --no-cache

# 3. Start services
docker compose up -d

# 4. Verify
docker compose ps
docker compose logs -f api admin
```

### After Package Changes

When `package.json` changes (new dependencies):

```bash
# Option 1: Rebuild container
docker compose build --no-cache api
docker compose restart api

# Option 2: Install in running container
docker compose exec api npm install
docker compose restart api

# Option 3: Remove and recreate
docker compose rm -sf api
docker compose up -d api
```

## Cleaning Up

### Stop Services

```bash
# Stop all services
docker compose stop

# Stop specific service
docker compose stop api

# Stop and remove containers
docker compose down
```

### Remove Containers

```bash
# Remove containers (keeps volumes)
docker compose down

# Remove containers and volumes (DELETES DATA)
docker compose down -v

# Remove containers, volumes, and images
docker compose down -v --rmi all
```

### Clean Docker System

```bash
# Remove stopped containers
docker container prune

# Remove unused images
docker image prune

# Remove unused volumes
docker volume prune

# Remove everything (DANGEROUS)
docker system prune -a --volumes
```

### Clean Project Volumes

```bash
# List project volumes
docker volume ls | grep changemaker

# Remove specific volume
docker volume rm changemaker-lite_v2-postgres-data

# Remove all project volumes (DELETES DATABASE)
docker compose down -v
```

### Reset Development Environment

Complete reset (deletes all data):

```bash
# 1. Stop and remove everything
docker compose down -v --rmi all

# 2. Clean Docker system
docker system prune -a --volumes -f

# 3. Rebuild from scratch
docker compose build --no-cache

# 4. Start services
docker compose up -d

# 5. Run migrations
docker compose exec api npx prisma migrate deploy
docker compose exec api npx prisma db seed
```

## Debugging in Docker

### Attach to Running Container

```bash
# Get shell in running container
docker compose exec api sh

# Or bash (if available)
docker compose exec api bash

# Inside container:
# - Explore file system
# - Run commands
# - Check environment variables
```

### Inspect Container

```bash
# View container details
docker inspect api

# View container environment variables
docker inspect api | grep -A 20 "Env"

# View container mounts
docker inspect api | grep -A 50 "Mounts"
```

### VSCode Remote Containers

Install "Remote - Containers" extension, then:

1. Open Command Palette (Cmd+Shift+P / Ctrl+Shift+P)
2. Select "Remote-Containers: Attach to Running Container"
3. Choose `api` or `admin` container
4. VSCode opens new window attached to container
5. Open `/app` folder in container
6. Set breakpoints and debug normally

### Debug Logs

Enable verbose logging:

```bash
# API with debug logs
docker compose exec api npm run dev -- --inspect

# Watch logs with timestamp
docker compose logs -f -t api

# Filter errors only
docker compose logs -f api 2>&1 | grep -i error
```

### Network Debugging

```bash
# Test container connectivity
docker compose exec api ping v2-postgres
docker compose exec api ping redis

# Check listening ports
docker compose exec api netstat -tuln

# Test API from inside container
docker compose exec api wget -O- http://localhost:4000/health
```

### Performance Debugging

```bash
# Container stats
docker stats

# Specific service stats
docker stats api admin

# Container resource limits
docker inspect api | grep -A 10 "Memory\|Cpu"
```

## Advanced Workflows

### Multi-Stage Development

Run different service combinations:

```bash
# Frontend development (local Admin, Docker API)
docker compose up -d api v2-postgres redis
cd admin && npm run dev

# Backend development (local API, Docker Admin)
docker compose up -d admin v2-postgres redis
cd api && npm run dev

# Full-stack (everything in Docker)
docker compose up -d api admin v2-postgres redis
```

### Custom Docker Compose Files

Create `docker-compose.dev.yml` for dev overrides:

```yaml
# docker-compose.dev.yml
services:
  api:
    command: npm run dev -- --inspect=0.0.0.0:9229
    ports:
      - "9229:9229"  # Debug port
    environment:
      - LOG_LEVEL=debug
```

**Usage:**
```bash
# Use both files
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d

# Or set COMPOSE_FILE env var
export COMPOSE_FILE=docker-compose.yml:docker-compose.dev.yml
docker compose up -d
```

### Docker Profiles for Optional Services

Start monitoring stack:

```bash
# With monitoring services
docker compose --profile monitoring up -d

# Without monitoring (default)
docker compose up -d
```

**Monitoring services:**
- Prometheus (port 9090)
- Grafana (port 3001)
- Alertmanager (port 9093)
- cAdvisor (port 8080)

### Build Arguments

Pass build-time arguments:

```bash
# Build with Node.js version argument
docker compose build --build-arg NODE_VERSION=20.11.0 api

# Set in docker-compose.yml
services:
  api:
    build:
      context: ./api
      args:
        - NODE_VERSION=${NODE_VERSION:-20}
```

### Health Checks

Check service health:

```bash
# View health status
docker compose ps

# Inspect health check
docker inspect --format='{{json .State.Health}}' api | jq

# Wait for healthy
docker compose up -d api
docker compose exec api sh -c 'while ! wget -q -O- http://localhost:4000/health; do sleep 1; done'
```

## Troubleshooting

### Container Exits Immediately

**Problem:** Container starts then stops.

**Solution:**

```bash
# Check logs for errors
docker compose logs api

# Common causes:
# 1. Missing .env file
# 2. Database connection failed
# 3. Syntax error in code
# 4. Port already in use

# Start with interactive mode to see error
docker compose run --rm api npm run dev
```

### Volume Mount Not Working

**Problem:** Code changes don't appear in container.

**Solution:**

```bash
# Check volume mounts
docker inspect api | grep -A 20 "Mounts"

# Verify volume path
docker compose exec api ls -la /app

# Recreate container
docker compose rm -sf api
docker compose up -d api
```

### Permission Errors

**Problem:** Permission denied errors in container.

**Solution:**

```bash
# Check file ownership
docker compose exec api ls -la /app

# Fix permissions on host
sudo chown -R $(whoami):$(whoami) ./api

# Or run container as current user (docker-compose.yml)
services:
  api:
    user: "${UID}:${GID}"
```

### Port Conflicts

**Problem:** Port already in use.

**Solution:**

```bash
# Find process using port
lsof -ti:4000 | xargs kill -9

# Or change port in docker-compose.yml
services:
  api:
    ports:
      - "4002:4000"  # Host:Container

# Or use .env
API_PORT=4002
```

### Database Connection Failed

**Problem:** API cannot connect to PostgreSQL.

**Solution:**

```bash
# Check database is running
docker compose ps v2-postgres

# Check database logs
docker compose logs v2-postgres

# Test connection
docker compose exec api sh -c 'wget -qO- http://v2-postgres:5432 || echo "Not reachable"'

# Verify DATABASE_URL
docker compose exec api sh -c 'echo $DATABASE_URL'

# Restart database
docker compose restart v2-postgres
```

### Out of Disk Space

**Problem:** No space left on device.

**Solution:**

```bash
# Check Docker disk usage
docker system df

# Remove unused images
docker image prune -a

# Remove unused volumes
docker volume prune

# Remove build cache
docker builder prune

# Full cleanup
docker system prune -a --volumes
```

### Container Running Out of Memory

**Problem:** Container crashes with OOM.

**Solution:**

```bash
# Check container stats
docker stats api

# Increase Docker memory limit (Docker Desktop → Preferences → Resources)

# Or set memory limit in docker-compose.yml
services:
  api:
    mem_limit: 2g
    memswap_limit: 2g
```

### Slow Performance on macOS/Windows

**Problem:** Slow hot reload, high CPU usage.

**Solution:**

1. **Use delegated volume mounts:**

```yaml
services:
  api:
    volumes:
      - ./api:/app:delegated
```

2. **Reduce file watching:**

```javascript
// vite.config.ts
export default {
  server: {
    watch: {
      ignored: ['**/node_modules/**', '**/dist/**']
    }
  }
}
```

3. **Switch to local development:**

```bash
docker compose up -d v2-postgres redis
cd api && npm run dev
cd admin && npm run dev
```

## Best Practices

### Development Workflow

1. **Start services in background:**
   ```bash
   docker compose up -d api admin
   ```

2. **Watch logs in separate terminal:**
   ```bash
   docker compose logs -f api admin
   ```

3. **Make code changes:**
   - Hot reload picks up changes automatically

4. **Type-check before commit:**
   ```bash
   docker compose exec api npm run type-check
   docker compose exec admin npm run type-check
   ```

5. **Stop services when done:**
   ```bash
   docker compose stop
   ```

### Container Naming

Use meaningful service names in docker-compose.yml:

```yaml
services:
  api:              # Not "backend" or "server"
  admin:            # Not "frontend" or "ui"
  v2-postgres:      # Not "db" (version-specific)
  redis:            # Standard name
```

### Environment Variables

1. **Use .env file** (not docker-compose.yml):
   ```bash
   # .env
   API_PORT=4000
   ADMIN_PORT=3000
   ```

2. **Reference in docker-compose.yml:**
   ```yaml
   services:
     api:
       environment:
         - API_PORT=${API_PORT}
   ```

3. **Don't commit .env** (use .env.example).

### Volume Management

1. **Named volumes for data:**
   ```yaml
   volumes:
     v2-postgres-data:  # Persistent database
   ```

2. **Bind mounts for code:**
   ```yaml
   volumes:
     - ./api:/app  # Live code sync
   ```

3. **Anonymous volumes for dependencies:**
   ```yaml
   volumes:
     - /app/node_modules  # Isolate from host
   ```

### Log Management

1. **Use log rotation:**
   ```yaml
   services:
     api:
       logging:
         driver: "json-file"
         options:
           max-size: "10m"
           max-file: "3"
   ```

2. **Filter logs with grep:**
   ```bash
   docker compose logs -f api | grep ERROR
   ```

3. **Export logs for analysis:**
   ```bash
   docker compose logs > debug-logs.txt
   ```

## Quick Reference

### Essential Commands

```bash
# Start
docker compose up -d api admin

# Stop
docker compose stop

# Restart
docker compose restart api

# Logs
docker compose logs -f api

# Execute command
docker compose exec api npm run type-check

# Shell access
docker compose exec api sh

# Rebuild
docker compose build --no-cache api

# Clean up
docker compose down -v
```

### Service Health Checks

```bash
# Check status
docker compose ps

# Test API
curl http://localhost:4000/health

# Test Admin
curl http://localhost:3000

# Test database
docker compose exec v2-postgres psql -U changemaker_v2 -d changemaker_v2_db -c "SELECT 1"

# Test Redis
docker compose exec redis redis-cli -a password ping
```

### Quick Reset

```bash
# Full reset (DELETES DATA)
docker compose down -v
docker compose build --no-cache
docker compose up -d
docker compose exec api npx prisma migrate deploy
docker compose exec api npx prisma db seed
```

## Related Documentation

- **Setup:** [Local Development Setup](local-setup.md)
- **Commands:** [NPM Commands Reference](npm-commands.md)
- **Database:** [Migrations Guide](migrations.md)
- **Debugging:** [Debugging Guide](debugging.md)
- **Deployment:** [Docker Compose Deployment](../deployment/docker-compose.md)

## Summary

You now know:
- ✅ When to use Docker vs local development
- ✅ How to start and stop services
- ✅ How to watch and filter logs
- ✅ How to execute commands in containers
- ✅ How hot reload works with volume mounts
- ✅ How to perform database operations in Docker
- ✅ How to rebuild and clean up containers
- ✅ How to debug containerized services
- ✅ Advanced workflows and best practices

**Quick Start:**
```bash
docker compose up -d api admin v2-postgres redis
docker compose logs -f api admin
# Make changes → Hot reload!
docker compose exec api npm run type-check
docker compose stop
```