admin/changemaker.lite
1
0
Fork 0
Code Issues Packages Projects Releases 31 Wiki Activity
changemaker.lite/mkdocs/docs/v2/features/media/video-library.md

43 KiB
Raw Blame History

Video Library Management

Overview

The Video Library system provides comprehensive video asset management through a dedicated Fastify microservice running on port 4100, separate from the main Express API. This dual API architecture allows the media system to operate independently while sharing the same PostgreSQL database.

Key Features:

  • Dual API Architecture — Fastify media API (port 4100) separate from Express API (port 4000)
  • Drizzle ORM — Media tables use Drizzle ORM instead of Prisma for schema flexibility
  • 9 Directory Types — Organized library structure (studios, gifs, private, inbox, curated, playback, compilations, videos, highlights)
  • FFprobe Integration — Automatic metadata extraction (duration, dimensions, orientation, quality, audio detection)
  • Video CRUD — Full create, read, update, delete operations (admin-only)
  • Directory Scanning — Bulk import videos from filesystem with automatic record creation
  • Validation System — Re-validate videos to refresh metadata and check file integrity
  • File Hashing — Duplicate detection via SHA-256 file hashing
  • Soft Delete — Videos marked invalid instead of hard deletion (preserves history)
  • Thumbnail Support — Custom thumbnail paths for video previews

Access Control:

  • All video library operations require SUPER_ADMIN role
  • Public video viewing handled separately via Shared Media system (see public-gallery.md)

Technology Stack:

  • Fastify 4.x — High-performance Node.js web framework
  • Drizzle ORM — TypeScript-first ORM with zero-runtime overhead
  • FFprobe — FFmpeg's media file analyzer for metadata extraction
  • PostgreSQL 16 — Shared database with main API

Architecture

The Media API operates as an independent microservice while maintaining data consistency through shared database access:

flowchart TB
    subgraph "Client Layer"
        Admin[Admin GUI :3000]
        Public[Public Users]
    end

    subgraph "API Layer"
        Express[Express API :4000<br/>Prisma ORM]
        Fastify[Fastify Media API :4100<br/>Drizzle ORM]
    end

    subgraph "Data Layer"
        DB[(PostgreSQL 16<br/>v2_changemaker)]
        FS[/media/local/library/<br/>Video Files]
    end

    subgraph "Processing"
        FFprobe[FFprobe Service<br/>Metadata Extraction]
    end

    Admin -->|Media Requests| Fastify
    Admin -->|Other Requests| Express
    Public -->|View Videos| Fastify

    Fastify -->|Drizzle Queries| DB
    Express -->|Prisma Queries| DB

    Fastify -->|Read/Write| FS
    Fastify -->|Extract Metadata| FFprobe
    FFprobe -->|Analyze| FS

    style Fastify fill:#e74c3c
    style Express fill:#3498db
    style DB fill:#2ecc71
    style FS fill:#f39c12

Architecture Highlights:

  1. Port Separation — Media API on 4100, Main API on 4000
  2. ORM Independence — Drizzle for media, Prisma for everything else
  3. Shared Database — Both APIs access same PostgreSQL instance
  4. File System Access — Media API has direct volume mount to /media/local/library
  5. Nginx Routing — media.cmlite.org routes to port 4100

Why Dual API?

The media system was added after V2 launch as a self-contained enhancement. Keeping it as a separate Fastify microservice:

  • Avoids disrupting the stable Express API
  • Allows independent scaling and deployment
  • Provides testing ground for Drizzle ORM migration
  • Isolates video processing workloads from core application logic

Database Model (Drizzle)

Videos Table Schema

// api/src/modules/media/db/schema.ts
import { pgTable, uuid, text, integer, timestamp, boolean, jsonb } from 'drizzle-orm/pg-core';

export const videos = pgTable('videos', {
  id: uuid('id').primaryKey().defaultRandom(),

  // File Information
  path: text('path').notNull().unique(), // Relative path from library root
  filename: text('filename').notNull(),
  originalFilename: text('original_filename'), // User-uploaded filename
  directoryType: text('directory_type').notNull(), // studios|gifs|private|inbox|curated|playback|compilations|videos|highlights

  // Metadata
  producer: text('producer'),
  creator: text('creator'),
  title: text('title'),
  tags: jsonb('tags').$type<string[]>().default([]),

  // Video Properties
  durationSeconds: integer('duration_seconds'),
  quality: text('quality'), // SD|HD|FHD|UHD
  orientation: text('orientation'), // portrait|landscape|square
  hasAudio: boolean('has_audio').default(false),
  width: integer('width'),
  height: integer('height'),

  // File Details
  fileSize: integer('file_size'), // Bytes
  fileHash: text('file_hash'), // SHA-256 for duplicate detection

  // Validation
  isValid: boolean('is_valid').default(true),
  lastValidated: timestamp('last_validated'),
  standardizedAt: timestamp('standardized_at'), // When file was moved to standard location

  // Thumbnail
  thumbnailPath: text('thumbnail_path'),

  // Public Sharing
  publicViewCount: integer('public_view_count').default(0),
  publicUpvoteCount: integer('public_upvote_count').default(0),
  movedFromPublicAt: timestamp('moved_from_public_at'), // When video was unlocked from public

  // Timestamps
  createdAt: timestamp('created_at').defaultNow(),
  updatedAt: timestamp('updated_at').defaultNow(),
});

Directory Types Enum

Directory Type Purpose Public Eligible
studios Studio-organized content ✅
gifs Short looping videos ✅
private Private/unreleased content ❌
inbox Upload staging area ❌
curated Hand-picked highlights ✅
playback Playback-optimized encodes ✅
compilations Multi-video compilations ✅
videos General video library ✅
highlights Auto-generated highlights ✅

Quality Classifications

Quality Height Range Typical Resolution
SD < 720px 480p, 576p
HD 720px - 1079px 720p
FHD 1080px - 2159px 1080p
UHD ≥ 2160px 4K, 8K

Orientation Detection

const detectOrientation = (width: number, height: number): string => {
  const ratio = width / height;
  if (ratio > 1.1) return 'landscape';
  if (ratio < 0.9) return 'portrait';
  return 'square';
};

API Endpoints

All endpoints require authentication with SUPER_ADMIN role unless marked as public.

List Videos

GET /api/media/videos

Query Parameters:

Parameter Type Default Description
page number 1 Page number for pagination
limit number 20 Results per page (max 100)
directoryType string - Filter by directory (studios, gifs, etc.)
orientation string - Filter by orientation (portrait, landscape, square)
producer string - Filter by producer (partial match)
creator string - Filter by creator (partial match)
quality string - Filter by quality (SD, HD, FHD, UHD)
hasAudio boolean - Filter by audio presence
isValid boolean true Filter by validation status
search string - Search in title, producer, creator

Response:

{
  "data": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "path": "videos/sample.mp4",
      "filename": "sample.mp4",
      "directoryType": "videos",
      "producer": "Studio A",
      "creator": "Director B",
      "title": "Sample Video",
      "durationSeconds": 180,
      "quality": "FHD",
      "orientation": "landscape",
      "hasAudio": true,
      "width": 1920,
      "height": 1080,
      "fileSize": 52428800,
      "isValid": true,
      "createdAt": "2026-02-10T12:00:00Z"
    }
  ],
  "pagination": {
    "page": 1,
    "limit": 20,
    "total": 156,
    "totalPages": 8
  }
}

Get Video Details

GET /api/media/videos/:id

Response:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "path": "videos/sample.mp4",
  "filename": "sample.mp4",
  "originalFilename": "my-video.mp4",
  "directoryType": "videos",
  "producer": "Studio A",
  "creator": "Director B",
  "title": "Sample Video",
  "tags": ["action", "sports", "highlight"],
  "durationSeconds": 180,
  "quality": "FHD",
  "orientation": "landscape",
  "hasAudio": true,
  "width": 1920,
  "height": 1080,
  "fileSize": 52428800,
  "fileHash": "a3d2f1e8b9c7...",
  "isValid": true,
  "lastValidated": "2026-02-10T12:00:00Z",
  "thumbnailPath": "thumbnails/550e8400.jpg",
  "publicViewCount": 1250,
  "publicUpvoteCount": 85,
  "createdAt": "2026-02-10T12:00:00Z",
  "updatedAt": "2026-02-10T12:00:00Z"
}

Create Video Record

POST /api/media/videos

Request Body:

{
  "path": "videos/new-video.mp4",
  "filename": "new-video.mp4",
  "directoryType": "videos",
  "producer": "Studio A",
  "creator": "Director B",
  "title": "New Video",
  "tags": ["action", "sports"]
}

Notes:

  • File must already exist at specified path on filesystem
  • FFprobe metadata extraction runs automatically after creation
  • Use /api/media/upload/single for file upload + record creation

Response:

{
  "id": "660e8400-e29b-41d4-a716-446655440000",
  "path": "videos/new-video.mp4",
  "filename": "new-video.mp4",
  "directoryType": "videos",
  "isValid": true,
  "createdAt": "2026-02-13T10:30:00Z"
}

Update Video Metadata

PUT /api/media/videos/:id

Request Body:

{
  "producer": "Updated Studio",
  "creator": "New Director",
  "title": "Updated Title",
  "tags": ["updated", "tags"]
}

Updatable Fields:

  • producer — Video producer/studio
  • creator — Director/creator name
  • title — Display title
  • tags — Array of tag strings
  • thumbnailPath — Custom thumbnail path

Response:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "producer": "Updated Studio",
  "creator": "New Director",
  "title": "Updated Title",
  "tags": ["updated", "tags"],
  "updatedAt": "2026-02-13T10:35:00Z"
}

Delete Video

DELETE /api/media/videos/:id

Behavior:

  • Soft Delete — Sets isValid = false instead of removing record
  • File remains on filesystem (manual cleanup required)
  • Video no longer appears in default listings
  • Can be restored by setting isValid = true via database

Response:

{
  "success": true,
  "message": "Video marked as invalid"
}

Scan Directory

POST /api/media/videos/scan

Request Body:

{
  "directoryType": "videos",
  "skipExisting": true
}

Parameters:

Field Type Required Description
directoryType string ✅ Directory to scan (videos, studios, etc.)
skipExisting boolean - Skip files already in database (default: true)

Process:

  1. Reads filesystem directory /media/local/library/{directoryType}/
  2. Filters for video extensions (.mp4, .mov, .avi, .mkv, .webm, .m4v, .flv)
  3. Checks each file against database (by path)
  4. Creates records for new files
  5. Runs FFprobe metadata extraction on new records

Response:

{
  "scanned": 45,
  "created": 12,
  "skipped": 33,
  "failed": 0,
  "errors": []
}

Validate Video

POST /api/media/videos/:id/validate

Purpose:

  • Re-run FFprobe metadata extraction
  • Update video properties (duration, dimensions, etc.)
  • Verify file still exists and is readable
  • Refresh file size and hash

Response:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "isValid": true,
  "lastValidated": "2026-02-13T10:40:00Z",
  "metadata": {
    "durationSeconds": 180,
    "width": 1920,
    "height": 1080,
    "quality": "FHD",
    "orientation": "landscape",
    "hasAudio": true
  }
}

Configuration

Environment Variables

# Media API Server
MEDIA_API_PORT=4100
MEDIA_API_HOST=0.0.0.0

# File Paths
MEDIA_LIBRARY_PATH=/media/local/library
MEDIA_INBOX_PATH=/media/local/inbox

# Feature Flags
ENABLE_MEDIA_FEATURES=true

# Database (shared with main API)
DATABASE_URL=postgresql://user:pass@v2-postgres:5432/v2_changemaker

# FFprobe
FFPROBE_TIMEOUT=30000  # milliseconds
FFPROBE_PATH=/usr/bin/ffprobe  # Auto-detected if not set

Docker Volume Mounts

# docker-compose.yml
services:
  media-api:
    volumes:
      - /media/local/library:/media/local/library:ro  # Read-only library
      - /media/local/inbox:/media/local/inbox:rw      # Read-write inbox

Important: Inbox requires :rw (read-write) for uploads. Library can be :ro (read-only) for security.

Site Settings

The media system respects the global ENABLE_MEDIA_FEATURES flag in Site Settings:

SELECT * FROM settings WHERE key = 'ENABLE_MEDIA_FEATURES';

When disabled:

  • Media API still runs but returns 503 Service Unavailable
  • Admin GUI hides Media menu items
  • Public gallery shows maintenance message

Admin Workflow

Viewing the Video Library

  1. Navigate to Media → Library in admin sidebar
  2. Table displays all videos with:
    • Thumbnail preview
    • Title, producer, creator
    • Duration, quality, orientation
    • Directory type
    • File size
    • Created date
  3. Use filters at top:
    • Directory Type dropdown
    • Orientation radio buttons (All / Portrait / Landscape / Square)
    • Quality checkboxes (SD, HD, FHD, UHD)
    • Search input (searches title, producer, creator)

Scanning a Directory

When to Use:

  • After manually copying videos to library directory
  • After video processing jobs complete
  • When videos exist on filesystem but not in database

Steps:

  1. Click "Scan Directory" button in Library page toolbar
  2. Select directory type from dropdown
  3. Toggle "Skip Existing" (recommended for large libraries)
  4. Click "Start Scan"
  5. Progress modal shows:
    • Files scanned
    • New records created
    • Skipped (already in DB)
    • Failed (with error messages)
  6. Click "Close" when complete
  7. Table refreshes with new videos

Example Output:

Scanning /media/local/library/videos...
Found 45 video files
- Created 12 new records
- Skipped 33 existing records
- Failed 0 files
Scan complete in 8.3 seconds

Editing Video Metadata

  1. Click pencil icon in video row
  2. Edit modal opens with fields:
    • Producer — Studio or production company
    • Creator — Director or primary creator
    • Title — Display title
    • Tags — Comma-separated tags (auto-suggests existing tags)
  3. Click "Save" to update
  4. Metadata changes immediately visible in table

Bulk Editing:

  1. Select multiple videos using checkboxes
  2. Click "Bulk Edit" button
  3. Set common fields (producer, tags, etc.)
  4. Click "Apply to Selected"

Validating Videos

Purpose: Refresh metadata and verify file integrity

Steps:

  1. Click "Validate" button in video row (or Actions dropdown)
  2. FFprobe re-analyzes video file
  3. Database updates with fresh metadata:
    • Duration (may have changed if file was re-encoded)
    • Dimensions
    • Audio detection
    • File size and hash
  4. lastValidated timestamp updates
  5. If file missing or corrupt, isValid set to false

Bulk Validation:

  1. Select multiple videos
  2. Click "Validate Selected"
  3. Progress modal shows validation results
  4. Failed validations highlighted in red

Deleting Videos

Soft Delete (Default):

  1. Click trash icon in video row
  2. Confirm deletion dialog
  3. Video marked isValid = false
  4. Video disappears from default view
  5. File remains on filesystem
  6. Record preserved in database

Viewing Deleted Videos:

  1. Toggle "Show Invalid" filter
  2. Deleted videos appear with strikethrough
  3. Can restore by clicking "Restore" button

Hard Delete (Database Only):

  1. Filter for invalid videos
  2. Select video(s)
  3. Click "Permanently Delete"
  4. Removes database record
  5. File still on filesystem (manual cleanup required)

File System Cleanup:

Deleted video files must be manually removed from filesystem:

# SSH into media-api container
docker compose exec media-api sh

# Navigate to library
cd /media/local/library/videos

# Remove specific file
rm deleted-video.mp4

# Or find and remove all invalid videos (BE CAREFUL)
# (requires database query to get invalid file paths)

Directory Structure

/media/local/library/
├── studios/              # Studio-organized content
│   ├── studio-a/
│   │   ├── video-001.mp4
│   │   └── video-002.mp4
│   └── studio-b/
│       └── video-003.mp4
│
├── gifs/                 # Short looping videos
│   ├── loop-001.mp4
│   └── loop-002.webm
│
├── private/              # Private/unreleased content
│   └── unreleased.mp4
│
├── inbox/                # Upload staging area (READ-WRITE)
│   ├── uuid-123.mp4      # Temp uploads
│   └── uuid-456.mov
│
├── curated/              # Hand-picked highlights
│   ├── best-of-2025.mp4
│   └── top-plays.mp4
│
├── playback/             # Playback-optimized encodes
│   ├── streaming-001.mp4
│   └── streaming-002.mp4
│
├── compilations/         # Multi-video compilations
│   ├── compilation-001.mp4
│   └── mega-compilation.mp4
│
├── videos/               # General video library
│   ├── video-001.mp4
│   ├── video-002.mp4
│   └── ... (thousands of videos)
│
└── highlights/           # Auto-generated highlights
    ├── highlight-001.mp4
    └── highlight-002.mp4

Directory Guidelines:

  • studios/ — Organize by producer/studio name (subfolder structure allowed)
  • gifs/ — Short videos under 15 seconds, suitable for looping
  • private/ — Never shared publicly, admin-only access
  • inbox/ — Temporary upload location, files moved after processing
  • curated/ — High-quality selections for public gallery homepage
  • playback/ — Web-optimized encodes (H.264, web-friendly profiles)
  • compilations/ — Merged videos created by compilation jobs
  • videos/ — Main library, all-purpose storage
  • highlights/ — AI-generated or manually created highlight reels

Code Examples

List Videos with Filters (Fastify Route)

// api/src/modules/media/routes/videos.routes.ts
import { FastifyInstance } from 'fastify';
import { eq, and, like, desc, sql } from 'drizzle-orm';
import { videos } from '@/modules/media/db/schema';
import { db } from '@/modules/media/db';

export default async function (app: FastifyInstance) {
  app.get('/api/media/videos', async (req, reply) => {
    const {
      page = 1,
      limit = 20,
      directoryType,
      orientation,
      producer,
      creator,
      quality,
      hasAudio,
      isValid = true,
      search,
    } = req.query as any;

    // Build filters
    const filters = [];

    if (directoryType) {
      filters.push(eq(videos.directoryType, directoryType));
    }

    if (orientation) {
      filters.push(eq(videos.orientation, orientation));
    }

    if (producer) {
      filters.push(like(videos.producer, `%${producer}%`));
    }

    if (creator) {
      filters.push(like(videos.creator, `%${creator}%`));
    }

    if (quality) {
      filters.push(eq(videos.quality, quality));
    }

    if (typeof hasAudio === 'boolean') {
      filters.push(eq(videos.hasAudio, hasAudio));
    }

    if (typeof isValid === 'boolean') {
      filters.push(eq(videos.isValid, isValid));
    }

    if (search) {
      filters.push(
        sql`(
          ${videos.title} ILIKE ${'%' + search + '%'} OR
          ${videos.producer} ILIKE ${'%' + search + '%'} OR
          ${videos.creator} ILIKE ${'%' + search + '%'}
        )`
      );
    }

    // Count total
    const [{ count }] = await db
      .select({ count: sql<number>`count(*)` })
      .from(videos)
      .where(and(...filters));

    // Fetch paginated results
    const results = await db
      .select()
      .from(videos)
      .where(and(...filters))
      .limit(Number(limit))
      .offset((Number(page) - 1) * Number(limit))
      .orderBy(desc(videos.createdAt));

    reply.send({
      data: results,
      pagination: {
        page: Number(page),
        limit: Number(limit),
        total: Number(count),
        totalPages: Math.ceil(Number(count) / Number(limit)),
      },
    });
  });
}

Scan Directory for Videos

// api/src/modules/media/routes/videos.routes.ts
import fs from 'fs/promises';
import path from 'path';
import { eq } from 'drizzle-orm';
import { videos } from '@/modules/media/db/schema';
import { ffprobeService } from '@/modules/media/services/ffprobe.service';

app.post('/api/media/videos/scan', async (req, reply) => {
  const { directoryType, skipExisting = true } = req.body as any;

  if (!directoryType) {
    return reply.code(400).send({ error: 'directoryType required' });
  }

  const dirPath = path.join(process.env.MEDIA_LIBRARY_PATH!, directoryType);

  try {
    // Check directory exists
    await fs.access(dirPath);
  } catch {
    return reply.code(400).send({ error: `Directory not found: ${directoryType}` });
  }

  // Read directory
  const files = await fs.readdir(dirPath, { recursive: true });

  // Filter for video files
  const videoExtensions = ['.mp4', '.mov', '.avi', '.mkv', '.webm', '.m4v', '.flv'];
  const videoFiles = files.filter((f) =>
    videoExtensions.some((ext) => f.toLowerCase().endsWith(ext))
  );

  const results = {
    scanned: videoFiles.length,
    created: 0,
    skipped: 0,
    failed: 0,
    errors: [] as string[],
  };

  for (const filename of videoFiles) {
    try {
      const relativePath = path.join(directoryType, filename);

      // Check if already exists
      if (skipExisting) {
        const existing = await db
          .select()
          .from(videos)
          .where(eq(videos.path, relativePath))
          .limit(1);

        if (existing.length > 0) {
          results.skipped++;
          continue;
        }
      }

      // Extract metadata
      const fullPath = path.join(dirPath, filename);
      const metadata = await ffprobeService.extract(fullPath);

      // Create record
      await db.insert(videos).values({
        path: relativePath,
        filename: path.basename(filename),
        directoryType,
        durationSeconds: metadata.duration,
        width: metadata.width,
        height: metadata.height,
        orientation: metadata.orientation,
        quality: metadata.quality,
        hasAudio: metadata.hasAudio,
        fileSize: metadata.fileSize,
        isValid: true,
      });

      results.created++;
    } catch (error: any) {
      results.failed++;
      results.errors.push(`${filename}: ${error.message}`);
    }
  }

  reply.send(results);
});

Validate Video Metadata

// api/src/modules/media/routes/videos.routes.ts
import { eq } from 'drizzle-orm';
import { videos } from '@/modules/media/db/schema';
import { ffprobeService } from '@/modules/media/services/ffprobe.service';

app.post('/api/media/videos/:id/validate', async (req, reply) => {
  const { id } = req.params as { id: string };

  // Fetch video record
  const [video] = await db
    .select()
    .from(videos)
    .where(eq(videos.id, id))
    .limit(1);

  if (!video) {
    return reply.code(404).send({ error: 'Video not found' });
  }

  try {
    // Build full file path
    const fullPath = path.join(process.env.MEDIA_LIBRARY_PATH!, video.path);

    // Extract fresh metadata
    const metadata = await ffprobeService.extract(fullPath);

    // Update database
    const [updated] = await db
      .update(videos)
      .set({
        durationSeconds: metadata.duration,
        width: metadata.width,
        height: metadata.height,
        orientation: metadata.orientation,
        quality: metadata.quality,
        hasAudio: metadata.hasAudio,
        fileSize: metadata.fileSize,
        fileHash: metadata.fileHash,
        isValid: true,
        lastValidated: new Date(),
        updatedAt: new Date(),
      })
      .where(eq(videos.id, id))
      .returning();

    reply.send({
      id: updated.id,
      isValid: updated.isValid,
      lastValidated: updated.lastValidated,
      metadata: {
        durationSeconds: updated.durationSeconds,
        width: updated.width,
        height: updated.height,
        quality: updated.quality,
        orientation: updated.orientation,
        hasAudio: updated.hasAudio,
      },
    });
  } catch (error: any) {
    // Mark as invalid if validation fails
    await db
      .update(videos)
      .set({
        isValid: false,
        lastValidated: new Date(),
        updatedAt: new Date(),
      })
      .where(eq(videos.id, id));

    reply.code(500).send({
      error: 'Validation failed',
      message: error.message,
      isValid: false,
    });
  }
});

Frontend: Library Page Table

// admin/src/pages/media/LibraryPage.tsx
import { Table, Button, Select, Input, Tag, Space } from 'antd';
import { useEffect, useState } from 'react';
import { mediaApi } from '@/lib/media-api';

export default function LibraryPage() {
  const [videos, setVideos] = useState([]);
  const [loading, setLoading] = useState(false);
  const [pagination, setPagination] = useState({ page: 1, limit: 20, total: 0 });
  const [filters, setFilters] = useState({
    directoryType: undefined,
    orientation: undefined,
    search: '',
  });

  const fetchVideos = async () => {
    setLoading(true);
    try {
      const { data } = await mediaApi.get('/api/media/videos', {
        params: {
          page: pagination.page,
          limit: pagination.limit,
          ...filters,
        },
      });
      setVideos(data.data);
      setPagination((prev) => ({ ...prev, total: data.pagination.total }));
    } catch (error) {
      console.error('Failed to fetch videos:', error);
    } finally {
      setLoading(false);
    }
  };

  useEffect(() => {
    fetchVideos();
  }, [pagination.page, filters]);

  const columns = [
    {
      title: 'Preview',
      dataIndex: 'thumbnailPath',
      width: 100,
      render: (path: string) => (
        <img
          src={path || '/placeholder.jpg'}
          alt="Thumbnail"
          style={{ width: 80, height: 60, objectFit: 'cover' }}
        />
      ),
    },
    {
      title: 'Title',
      dataIndex: 'title',
      render: (text: string, record: any) => (
        <div>
          <div style={{ fontWeight: 600 }}>{text || record.filename}</div>
          <div style={{ fontSize: 12, color: '#888' }}>
            {record.producer} • {record.creator}
          </div>
        </div>
      ),
    },
    {
      title: 'Duration',
      dataIndex: 'durationSeconds',
      width: 100,
      render: (seconds: number) => {
        const mins = Math.floor(seconds / 60);
        const secs = seconds % 60;
        return `${mins}:${secs.toString().padStart(2, '0')}`;
      },
    },
    {
      title: 'Quality',
      dataIndex: 'quality',
      width: 80,
      render: (quality: string) => {
        const colors: Record<string, string> = {
          SD: 'default',
          HD: 'blue',
          FHD: 'green',
          UHD: 'purple',
        };
        return <Tag color={colors[quality]}>{quality}</Tag>;
      },
    },
    {
      title: 'Orientation',
      dataIndex: 'orientation',
      width: 100,
    },
    {
      title: 'Directory',
      dataIndex: 'directoryType',
      width: 120,
    },
    {
      title: 'Actions',
      width: 150,
      render: (_: any, record: any) => (
        <Space>
          <Button size="small" onClick={() => handleEdit(record.id)}>
            Edit
          </Button>
          <Button size="small" onClick={() => handleValidate(record.id)}>
            Validate
          </Button>
          <Button size="small" danger onClick={() => handleDelete(record.id)}>
            Delete
          </Button>
        </Space>
      ),
    },
  ];

  return (
    <div>
      <Space style={{ marginBottom: 16 }}>
        <Select
          placeholder="Directory Type"
          style={{ width: 200 }}
          onChange={(value) => setFilters({ ...filters, directoryType: value })}
          allowClear
        >
          <Select.Option value="videos">Videos</Select.Option>
          <Select.Option value="studios">Studios</Select.Option>
          <Select.Option value="gifs">GIFs</Select.Option>
          <Select.Option value="curated">Curated</Select.Option>
        </Select>

        <Input.Search
          placeholder="Search title, producer, creator"
          style={{ width: 300 }}
          onSearch={(value) => setFilters({ ...filters, search: value })}
          allowClear
        />

        <Button type="primary" onClick={handleScanDirectory}>
          Scan Directory
        </Button>
      </Space>

      <Table
        columns={columns}
        dataSource={videos}
        loading={loading}
        rowKey="id"
        pagination={{
          current: pagination.page,
          pageSize: pagination.limit,
          total: pagination.total,
          onChange: (page) => setPagination({ ...pagination, page }),
        }}
      />
    </div>
  );
}

Troubleshooting

Problem: Media API Not Accessible

Symptoms:

  • Admin GUI shows "Cannot connect to media API"
  • Browser console shows CORS errors or network failures
  • Public gallery doesn't load

Solutions:

  1. Check Fastify server running:
docker compose ps media-api
# Should show "Up" status

docker compose logs media-api
# Look for "Fastify server listening on port 4100"
  1. Verify port 4100 not in use:
lsof -i :4100
# Should show only media-api container

# If another process using port, stop it or change MEDIA_API_PORT in .env
  1. Check nginx proxy configuration:
# nginx/conf.d/api.conf
# Media API block must come BEFORE general API block

server {
    listen 80;
    server_name media.cmlite.org;

    location / {
        proxy_pass http://localhost:4100;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}
  1. Test direct API access:
# From host machine
curl http://localhost:4100/api/media/videos

# From inside container
docker compose exec media-api curl http://localhost:4100/api/media/videos
  1. Check Docker networking:
docker network inspect changemaker-lite
# Verify media-api container connected

Problem: Scan Finds No Videos

Symptoms:

  • Scan completes with "Created 0 new records"
  • Directory known to contain video files
  • Scan reports 0 files scanned

Solutions:

  1. Verify MEDIA_LIBRARY_PATH correct:
# Check environment variable
docker compose exec media-api printenv MEDIA_LIBRARY_PATH
# Should output: /media/local/library

# List directory contents
docker compose exec media-api ls -la /media/local/library/videos
# Should show video files
  1. Check directory exists:
# Create missing directory
docker compose exec media-api mkdir -p /media/local/library/videos

# Copy test videos
docker cp test.mp4 $(docker compose ps -q media-api):/media/local/library/videos/
  1. Verify Docker volume mounted:
# docker-compose.yml
services:
  media-api:
    volumes:
      - /media/local/library:/media/local/library:ro  # Check path correct

# Inspect volume mounts
docker compose config | grep -A 5 media-api
  1. Check file extensions supported:

Only these extensions scanned:

  • .mp4
  • .mov
  • .avi
  • .mkv
  • .webm
  • .m4v
  • .flv

Rename files if using other extensions:

# Rename .MP4 to .mp4 (case-sensitive)
docker compose exec media-api sh -c 'cd /media/local/library/videos && rename "s/.MP4$/.mp4/" *.MP4'
  1. Check file permissions:
# Verify readable by container user
docker compose exec media-api ls -la /media/local/library/videos

# Fix permissions if needed (on host)
sudo chmod -R 755 /media/local/library

Problem: FFprobe Validation Fails

Symptoms:

  • Validation returns error "FFprobe command failed"
  • Videos marked isValid = false
  • Timeout errors after 30 seconds

Solutions:

  1. Check FFmpeg installed in container:
# Verify FFprobe available
docker compose exec media-api which ffprobe
# Should output: /usr/bin/ffprobe

docker compose exec media-api ffprobe -version
# Should show FFmpeg version info
  1. Install FFmpeg if missing:
# api/Dockerfile.media
FROM node:20-alpine

# Install FFmpeg (both dev and production stages)
RUN apk add --no-cache ffmpeg

# ... rest of Dockerfile

# Rebuild container
docker compose build media-api
docker compose up -d media-api
  1. Test FFprobe directly on video:
# Run FFprobe manually
docker compose exec media-api ffprobe -v quiet -print_format json -show_streams -show_format /media/local/library/videos/test.mp4

# If this fails, video file corrupt or unsupported
  1. Check timeout not exceeded:

Default timeout: 30 seconds

# For very large files (>5GB), increase timeout
# api/src/modules/media/services/ffprobe.service.ts
const FFPROBE_TIMEOUT = 60000; // 60 seconds
  1. Verify video file not corrupt:
# Test playback
docker compose exec media-api ffplay /media/local/library/videos/test.mp4

# Or copy to host and test in VLC
docker cp $(docker compose ps -q media-api):/media/local/library/videos/test.mp4 ./test.mp4
vlc test.mp4
  1. Check for special characters in filename:
# Rename files with spaces or special chars
docker compose exec media-api sh -c 'cd /media/local/library/videos && rename "s/ /_/g" *.mp4'

Problem: Drizzle Schema Changes Not Applied

Symptoms:

  • Code references new column but database doesn't have it
  • Error: "column does not exist"
  • Schema changes made but not reflected

Solutions:

  1. Push schema changes:
# Drizzle uses push (not migrations)
cd api
npx drizzle-kit push

# Confirm changes
  1. Verify connection:
# Check DATABASE_URL correct
docker compose exec media-api printenv DATABASE_URL

# Test connection
docker compose exec media-api npx drizzle-kit studio
# Opens DB browser on http://localhost:4983
  1. Compare with Prisma migrations:

Media tables exist in same database as Prisma tables. If conflict:

# Check both schemas
npx prisma db pull  # Prisma introspection
npx drizzle-kit introspect  # Drizzle introspection

# Resolve conflicts manually

Problem: Large Library Performance

Symptoms:

  • Library page loads slowly (5+ seconds)
  • Pagination sluggish
  • Scan operations timeout

Solutions:

  1. Add database indexes:
-- Index for common filters
CREATE INDEX idx_videos_directory_type ON videos(directory_type);
CREATE INDEX idx_videos_orientation ON videos(orientation);
CREATE INDEX idx_videos_quality ON videos(quality);
CREATE INDEX idx_videos_is_valid ON videos(is_valid);
CREATE INDEX idx_videos_created_at ON videos(created_at DESC);

-- Composite index for filtered queries
CREATE INDEX idx_videos_filters ON videos(directory_type, is_valid, created_at DESC);

-- Full-text search index
CREATE INDEX idx_videos_search ON videos USING gin(to_tsvector('english', coalesce(title, '') || ' ' || coalesce(producer, '') || ' ' || coalesce(creator, '')));
  1. Reduce page size:
// admin/src/pages/media/LibraryPage.tsx
const [pagination, setPagination] = useState({ page: 1, limit: 10, total: 0 });
// Reduced from 20 to 10
  1. Enable query caching:
// api/src/modules/media/routes/videos.routes.ts
import { redisClient } from '@/config/redis';

app.get('/api/media/videos', async (req, reply) => {
  const cacheKey = `videos:list:${JSON.stringify(req.query)}`;

  // Check cache
  const cached = await redisClient.get(cacheKey);
  if (cached) {
    return reply.send(JSON.parse(cached));
  }

  // Fetch from database
  const results = await db.select()...;

  // Cache for 5 minutes
  await redisClient.setex(cacheKey, 300, JSON.stringify(results));

  reply.send(results);
});
  1. Use virtual scrolling:
// Replace Ant Design Table with react-window for large datasets
import { FixedSizeList } from 'react-window';

Performance Considerations

Directory Scans

Scaling Factors:

  • 100 files: ~2 seconds
  • 1,000 files: ~15 seconds
  • 10,000 files: ~2.5 minutes

Optimization Strategies:

  1. Incremental Scans — Use skipExisting: true to only process new files
  2. Parallel Processing — Scan multiple directories simultaneously
  3. Background Jobs — Queue scans as async jobs instead of synchronous requests
  4. Caching — Cache directory listings in Redis

FFprobe Extraction

Timing:

  • Small video (<100MB): ~50-100ms
  • Medium video (500MB): ~150-250ms
  • Large video (2GB+): ~500ms-1s

Batch Processing:

For 100 videos: ~10-20 seconds total

Optimization:

// Parallel extraction (limit concurrency)
import pLimit from 'p-limit';

const limit = pLimit(5); // Max 5 concurrent FFprobe calls

const results = await Promise.all(
  videoFiles.map((file) =>
    limit(() => ffprobeService.extract(file))
  )
);

Database Queries

Query Performance:

  • List 20 videos (no filters): ~5-10ms
  • List 20 videos (with filters): ~10-20ms
  • Full-text search: ~20-50ms
  • Count total videos: ~5ms (with index)

Optimization:

  1. Always use pagination — Never fetch all records
  2. Index heavily filtered columns — directoryType, orientation, quality, isValid
  3. Use SELECT only needed columns — Avoid SELECT * for large tables
  4. Cache counts — Total video count changes infrequently, cache in Redis

Thumbnail Generation

Deferred Loading:

Don't generate thumbnails during scan. Instead:

  1. Create video record without thumbnail
  2. Queue thumbnail generation job
  3. Worker processes job asynchronously
  4. Update record with thumbnailPath

Lazy Loading:

Frontend requests thumbnails only when visible (IntersectionObserver).

Dual API Architecture

Why Separate Fastify API?

The media system was introduced as a Phase 14 enhancement after V2 core functionality stabilized. A separate Fastify microservice was chosen to:

  1. Avoid Disrupting Stable Express API — V2 Express API battle-tested with 30+ models, introducing media directly risked regressions
  2. Test Drizzle ORM Migration — Fastify+Drizzle serves as proof-of-concept for potential future Prisma→Drizzle migration
  3. Isolate Video Processing — CPU/GPU-intensive FFprobe, encoding jobs isolated from main API request handling
  4. Independent Scaling — Media API can be horizontally scaled separately based on video processing load
  5. Technology Experimentation — Fastify's performance benefits evaluated for potential broader adoption

Database Sharing Strategy

Same PostgreSQL, Different ORMs:

┌─────────────────┐
│  PostgreSQL 16  │
│ v2_changemaker  │
└─────────────────┘
         ↑
    ┌────┴────┐
    │         │
┌───┴───┐ ┌──┴────┐
│Prisma │ │Drizzle│
│ ORM   │ │  ORM  │
└───┬───┘ └──┬────┘
    │        │
┌───┴────┐ ┌─┴─────┐
│Express │ │Fastify│
│  API   │ │ Media │
│ :4000  │ │  API  │
│        │ │ :4100 │
└────────┘ └───────┘

Benefits:

  • Single Source of Truth — All data in one database
  • Cross-API Queries — Main API can query media tables via Prisma raw queries
  • Unified Backups — One PostgreSQL dump includes both APIs
  • Shared Connections — Connection pooling optimizations benefit both

Challenges:

  • Schema Coordination — Must manually sync schema changes between Prisma migrations and Drizzle pushes
  • Type Conflicts — Same table, different type definitions (Prisma vs Drizzle types)
  • Migration Complexity — Prisma generates migrations, Drizzle uses push (no migration files)

Migration Strategy Roadmap

Short Term (Current):

  • Keep dual API architecture
  • Synchronize schemas manually
  • Document shared tables in both ORMs

Medium Term (6-12 months):

  • Evaluate Fastify+Drizzle performance vs Express+Prisma
  • If Fastify superior, migrate select Express routes to Fastify
  • If no significant benefit, consolidate media into Express+Prisma

Long Term (12+ months):

  • Unified API (either all Express or all Fastify)
  • Single ORM (either all Prisma or all Drizzle)
  • Deprecate less performant stack

Migration Effort Estimate:

  • Media to Express+Prisma: 3-5 days (convert Drizzle queries to Prisma, merge Fastify routes into Express)
  • All to Fastify+Drizzle: 2-3 weeks (convert 30+ Prisma models to Drizzle, rewrite Express routes for Fastify)

Related Documentation

Backend Documentation

  • API Server: backend/api/media-server.md — Fastify server setup, middleware, error handling
  • Videos Module: backend/modules/media/videos.md — Video routes, service layer, business logic
  • FFprobe Service: backend/modules/media/ffprobe.md — Metadata extraction implementation
  • Jobs System: backend/modules/media/jobs.md — Job queue architecture, worker processes

Frontend Documentation

  • Library Page: frontend/pages/media/library.md — Video library management UI
  • Shared Media Page: frontend/pages/media/shared.md — Public gallery admin UI
  • Media Components: frontend/components/media.md — Reusable video components

Database Documentation

  • Media Models: database/models/media.md — Drizzle schema definitions for videos, compilations, jobs
  • Drizzle Setup: database/drizzle.md — Drizzle ORM configuration, connection management

Feature Documentation

  • Video Upload: features/media/upload.md — Upload system workflow, FFprobe integration
  • Media Jobs: features/media/jobs.md — Job queue system, processing pipeline
  • Public Gallery: features/media/public-gallery.md — Public video sharing system

Integration Documentation

  • Dual API Architecture: architecture/dual-api.md — Express+Prisma vs Fastify+Drizzle comparison
  • Nginx Routing: deployment/nginx.md — Reverse proxy configuration for media.cmlite.org
  • Docker Setup: deployment/docker.md — Media API container, volume mounts, healthchecks

Next Steps

After mastering video library management:

  1. Upload System — Read features/media/upload.md to understand video upload workflow
  2. Jobs Queue — Review features/media/jobs.md for video processing automation
  3. Public Gallery — Explore features/media/public-gallery.md for sharing videos publicly
  4. Custom Integrations — Use Media API endpoints to build custom video features

For hands-on practice, try:

# 1. Upload test videos
curl -X POST http://localhost:4100/api/media/upload/single \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -F "video=@test.mp4" \
  -F "producer=Test Studio" \
  -F "title=Test Video"

# 2. Scan directory
curl -X POST http://localhost:4100/api/media/videos/scan \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"directoryType": "videos"}'

# 3. List videos
curl http://localhost:4100/api/media/videos?page=1&limit=10

# 4. Validate video
curl -X POST http://localhost:4100/api/media/videos/VIDEO_ID/validate \
  -H "Authorization: Bearer YOUR_TOKEN"

Last Updated: 2026-02-13 Version: V2.0 Maintainer: Changemaker Lite Team