changemaker.lite/mkdocs/docs/v2/database/migrations.md

# Migration Workflow

## Overview

Changemaker Lite V2 uses a **dual ORM architecture** with separate migration workflows:

- **Prisma Migrate** — Main API (Express, 30 models)
- **Drizzle Kit** — Media API (Fastify, 3 models)

Both ORMs share the same PostgreSQL database but maintain independent migration histories.

---

## Prisma Migration Workflow

### Development Workflow

#### 1. Modify Schema
Edit `api/prisma/schema.prisma`:
```prisma
model Location {
  id       String  @id @default(cuid())
  address  String
  // Add new field:
  province String?
  // ...
}
```

#### 2. Create Migration
```bash
cd api
npx prisma migrate dev --name add_province_to_location
```

**This command:**
- Generates SQL migration file in `prisma/migrations/`
- Applies migration to database
- Regenerates Prisma Client
- Updates `_prisma_migrations` table

**Output:**
```
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "changemaker_v2", schema "public"

Applying migration `20260213120000_add_province_to_location`

The following migration(s) have been created and applied from new schema changes:

migrations/
  └─ 20260213120000_add_province_to_location/
      └─ migration.sql

Your database is now in sync with your schema.
```

#### 3. Review Migration SQL
```sql
-- migrations/20260213120000_add_province_to_location/migration.sql
-- AlterTable
ALTER TABLE "locations" ADD COLUMN "province" TEXT;
```

#### 4. Commit Migration
```bash
git add prisma/migrations/
git commit -m "Add province field to Location model"
```

### Production Workflow

#### 1. Deploy Migration
```bash
docker compose exec api npx prisma migrate deploy
```

**This command:**
- Applies pending migrations from `prisma/migrations/`
- Does NOT create new migrations
- Does NOT prompt for confirmations
- Safe for production/CI pipelines

#### 2. Verify Migration Status
```bash
docker compose exec api npx prisma migrate status
```

**Output:**
```
1 migration found in prisma/migrations

Following migration have been applied:

20260213120000_add_province_to_location

Database schema is up to date!
```

### Common Migration Scenarios

#### Add Field (Nullable)
```prisma
model Location {
  federalDistrict String?  // Add nullable field
}
```
**Migration:**
```sql
ALTER TABLE "locations" ADD COLUMN "federal_district" TEXT;
```

#### Add Field (Required with Default)
```prisma
model Location {
  buildingType BuildingType @default(SINGLE_FAMILY)
}
```
**Migration:**
```sql
ALTER TABLE "locations" ADD COLUMN "building_type" TEXT NOT NULL DEFAULT 'SINGLE_FAMILY';
```

#### Add Relation
```prisma
model Shift {
  cutId String?
  cut   Cut?    @relation(fields: [cutId], references: [id], onDelete: SetNull)
}
```
**Migration:**
```sql
ALTER TABLE "shifts" ADD COLUMN "cut_id" TEXT;
CREATE INDEX "shifts_cut_id_idx" ON "shifts"("cut_id");
ALTER TABLE "shifts" ADD CONSTRAINT "shifts_cut_id_fkey" FOREIGN KEY ("cut_id") REFERENCES "cuts"("id") ON DELETE SET NULL ON UPDATE CASCADE;
```

#### Change Field Type
```prisma
model Location {
  geocodeConfidence Int?  // Changed from String? to Int?
}
```
**Migration (requires data migration):**
```sql
-- Step 1: Add new column
ALTER TABLE "locations" ADD COLUMN "geocode_confidence_new" INTEGER;

-- Step 2: Migrate data (custom logic)
UPDATE "locations" SET "geocode_confidence_new" = CAST("geocode_confidence" AS INTEGER)
WHERE "geocode_confidence" ~ '^[0-9]+$';

-- Step 3: Drop old column
ALTER TABLE "locations" DROP COLUMN "geocode_confidence";

-- Step 4: Rename new column
ALTER TABLE "locations" RENAME COLUMN "geocode_confidence_new" TO "geocode_confidence";
```

#### Add Enum
```prisma
enum BuildingType {
  SINGLE_FAMILY
  MULTI_UNIT
  MIXED_USE
  COMMERCIAL
}
```
**Migration:**
```sql
CREATE TYPE "BuildingType" AS ENUM ('SINGLE_FAMILY', 'MULTI_UNIT', 'MIXED_USE', 'COMMERCIAL');
```

#### Add Index
```prisma
model Location {
  latitude  Decimal
  longitude Decimal

  @@index([latitude, longitude])
}
```
**Migration:**
```sql
CREATE INDEX "locations_latitude_longitude_idx" ON "locations"("latitude", "longitude");
```

### Migration Commands Reference

| Command | Description | Environment |
|---------|-------------|-------------|
| `npx prisma migrate dev` | Create + apply migration | Development |
| `npx prisma migrate deploy` | Apply pending migrations | Production/CI |
| `npx prisma migrate status` | Check migration status | All |
| `npx prisma migrate reset` | Reset DB + apply all migrations | Development only |
| `npx prisma db push` | Push schema without migrations | Prototyping only |
| `npx prisma studio` | Open Prisma Studio (DB GUI) | Development |

### Safe Migration Practices

#### ✅ DO
- Always review generated SQL before committing
- Test migrations on dev database first
- Back up production database before deploying migrations
- Use nullable fields for new columns on existing tables
- Use `@default()` for new required fields
- Commit migration files to version control

#### ❌ DON'T
- Use `prisma db push` in production (skips migrations)
- Use `prisma migrate reset` in production (deletes data)
- Manually edit migration files after applying
- Delete old migration files (breaks history)
- Change field names without data migration plan

---

## Drizzle Migration Workflow

### Development Workflow

#### 1. Modify Schema
Edit `api/src/modules/media/db/schema.ts`:
```typescript
export const videos = pgTable('videos', {
  id: serial('id').primaryKey(),
  path: text('path').notNull().unique(),
  // Add new field:
  description: text('description'),
  // ...
});
```

#### 2. Push Schema Changes
```bash
cd api
npx drizzle-kit push
```

**This command:**
- Generates SQL diff from schema
- Applies changes directly to database
- **Does NOT create migration files** (Drizzle push mode)
- Updates database schema immediately

**Output:**
```
Reading config file '/home/bunker-admin/changemaker.lite/api/drizzle.config.ts'
Pulling schema from database...✓
Applying changes...

[✓] Applying: ALTER TABLE "videos" ADD COLUMN "description" text;

Schema applied successfully!
```

#### 3. Verify Schema
```bash
npx drizzle-kit studio
```
Opens Drizzle Studio at `https://local.drizzle.studio/` for database inspection.

### Production Workflow

**Same as development:**
```bash
docker compose exec media-api npx drizzle-kit push
```

### Drizzle vs Prisma Migrate

| Feature | Prisma Migrate | Drizzle Kit Push |
|---------|----------------|------------------|
| Migration files | ✓ Generated | ✗ Not generated |
| Migration history | ✓ Tracked in `_prisma_migrations` | ✗ No history table |
| Rollback support | ✓ Via migration files | ✗ Manual only |
| Production safety | ✓ Explicit deploy step | ⚠️ Direct push |
| Best for | Main API (schema stability) | Media API (rapid iteration) |

**Why Drizzle for Media API?**
- Smaller schema (3 tables vs 30)
- Faster iteration during development
- Simpler deployment (no migration history to manage)
- Media API is newer (less risk of breaking changes)

### Drizzle Commands Reference

| Command | Description |
|---------|-------------|
| `npx drizzle-kit push` | Push schema changes to DB |
| `npx drizzle-kit studio` | Open Drizzle Studio |
| `npx drizzle-kit generate` | Generate migrations (not used) |

---

## Migration File Structure

### Prisma Migrations
```
api/prisma/migrations/
├── 20260211120000_initial/
│   └── migration.sql
├── 20260211125000_add_refresh_tokens/
│   └── migration.sql
├── 20260212100000_add_canvass_system/
│   └── migration.sql
└── migration_lock.toml
```

**File naming:** `YYYYMMDDHHMMSS_description/migration.sql`

**migration_lock.toml:**
```toml
# Please do not edit this file manually
provider = "postgresql"
```

### Drizzle Schema (No Migrations)
```
api/src/modules/media/db/
├── schema.ts          # Source of truth
└── drizzle.config.ts  # Drizzle config
```

---

## Rollback Strategies

### Prisma Rollback (Manual)

**Scenario:** Migration `20260213120000_add_province` caused issues.

**Step 1:** Identify last good migration
```bash
npx prisma migrate status
```

**Step 2:** Manually revert migration SQL
```sql
-- Reverse of migration.sql
ALTER TABLE "locations" DROP COLUMN "province";
```

**Step 3:** Mark migration as rolled back
```sql
DELETE FROM "_prisma_migrations" WHERE migration_name = '20260213120000_add_province';
```

**Step 4:** Remove migration file
```bash
rm -rf prisma/migrations/20260213120000_add_province/
```

**Step 5:** Fix schema
Edit `prisma/schema.prisma` to remove `province` field.

**Step 6:** Create new migration
```bash
npx prisma migrate dev --name remove_province_from_location
```

### Drizzle Rollback (Manual)

**Step 1:** Revert schema changes in `schema.ts`

**Step 2:** Push reverted schema
```bash
npx drizzle-kit push
```

**Step 3:** If data loss occurred, restore from backup

---

## Common Migration Errors

### Error: "Migration failed to apply cleanly"
**Cause:** Database state doesn't match expected state
**Solution:**
```bash
npx prisma migrate resolve --applied <migration-name>  # Mark as applied
# OR
npx prisma migrate resolve --rolled-back <migration-name>  # Mark as rolled back
```

### Error: "Unique constraint violation"
**Cause:** Trying to add unique constraint on column with duplicate values
**Solution:**
1. Clean up duplicate data first
2. Run migration

### Error: "Column cannot be NOT NULL"
**Cause:** Trying to add required field to table with existing rows
**Solution:** Use `@default()` or make field nullable

### Error: "Foreign key constraint failed"
**Cause:** Referencing non-existent records
**Solution:** Ensure related records exist before adding FK

---

## Database Backup Before Migration

### Development
```bash
docker compose exec v2-postgres pg_dump -U changemaker_v2 changemaker_v2 > backup.sql
```

### Production
```bash
# Via docker-compose
docker compose exec v2-postgres pg_dump -U changemaker_v2 changemaker_v2 | gzip > backup_$(date +%Y%m%d_%H%M%S).sql.gz

# Via backup script
./scripts/backup.sh
```

### Restore from Backup
```bash
# Stop API services
docker compose stop api media-api

# Restore database
docker compose exec -T v2-postgres psql -U changemaker_v2 changemaker_v2 < backup.sql

# Restart services
docker compose up -d api media-api
```

---

## CI/CD Integration

### GitHub Actions Example
```yaml
name: Deploy V2

on:
  push:
    branches: [main]

jobs:
  migrate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 20

      - name: Install dependencies
        run: cd api && npm ci

      - name: Run Prisma migrations
        run: cd api && npx prisma migrate deploy
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}

      - name: Run Drizzle push
        run: cd api && npx drizzle-kit push
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
```

---

## Related Documentation

- [Database Overview](./index.md) — Architecture and models
- [Schema Reference](./schema.md) — All model fields
- [Seeding](./seeding.md) — Default data
- [Prisma Documentation](https://www.prisma.io/docs/orm/prisma-migrate)
- [Drizzle Documentation](https://orm.drizzle.team/kit-docs/overview)