feat(upgrade): Approach C - CCP-driven release upgrade (template re-render)

Adds the third upgrade path alongside Approach A (full upgrade.sh) and B
(image-only). For releases that change orchestration (new services, new
nginx routes, new compose env vars) in addition to image versions, CCP
re-renders templates server-side, sends the rendered files to the tenant
via the existing mTLS agent, then composePull + composeUp. Tenant content
(mkdocs/, custom configs/) is never touched.

Pieces:

PHASE 1 — Schema + per-instance imageTag

- prisma/schema.prisma: new Instance.imageTag column (NULL = fall back
  to env.IMAGE_TAG default).
- prisma/migrations/20260522093400_add_instance_image_tag/: SQL.
- services/template-engine.ts:
  - buildTemplateContext now uses instance.imageTag || env.IMAGE_TAG.
  - InstanceForTemplate interface gains imageTag: string | null.

PHASE 2 — Pre-flight diff (read-only "what would change?")

- agent/services/file.service.ts: new diffFiles() helper with a small
  inline LCS-based unified-diff (no new deps). Returns per-file status
  ('unchanged' | 'modified' | 'created') + truncated unified diff.
- agent/routes/files.routes.ts: POST /instance/:slug/files/diff.
- api/services/execution-driver.ts: diffFiles added to interface.
- api/services/local-driver.ts + remote-driver.ts: diffFiles methods
  (local mirrors agent helper inline; remote POSTs to the agent endpoint).
- api/services/upgrade.service.ts: previewReleaseUpgrade() — renders
  templates in-memory with the proposed imageTag, filters out .env for
  isRegistered=true tenants, calls driver.diffFiles, computes envCoverage
  (which env vars the new compose needs vs which the tenant's .env has).

PHASE 3 — Apply path (the actual upgrade)

- api/services/upgrade.service.ts: startReleaseUpgrade() and the inner
  runReleaseUpgrade() runner. Distinct from runRemoteUpgrade because CCP
  does the work directly via the mTLS driver (no agent-side script).
  Flow: persist imageTag in DB → render → writeFiles → composePull →
  composeUp → composePs verify. Status reported via InstanceUpgrade
  rows (same shape the existing CCP polling UI already uses).
- Failure handling: instance.imageTag stays at the new value on failure
  so operator can retry. Manual rollback only.

PHASE 4 — Routes + schemas

- instances.schemas.ts: startReleaseUpgradeSchema (imageTag regex).
- instances.routes.ts:
  - POST /:id/upgrade-release       (apply)
  - POST /:id/upgrade-release/preview (read-only diff)

PHASE 5 — CCP admin UI

- admin/pages/InstanceDetailPage.tsx: third "Upgrade to Release" button
  next to Quick Upgrade + Upgrade Now. Opens a modal with imageTag input,
  Preview button (calls /preview), and Apply button. Preview modal shows:
  - Red alert if envCoverage.missingInTenantEnv is non-empty (compose
    needs vars the tenant's .env doesn't define).
  - Per-file status tags (unchanged / modified / created) + truncated
    unified diff for modified files.
- admin/types/api.ts: Instance.imageTag added.

Constraints applied:
- Remote-only initial scope: throws "currently supported only for remote
  instances" if instance.isRemote === false.
- isRegistered=true tenants (install.sh fleet): .env is filtered out
  of the render set (CCP can't render env without secrets in DB), the
  tenant's existing .env stays as-is. envCoverage warns the operator
  if the new compose references env vars their .env doesn't define.
- Shared in-progress guard with Approach A/B (one upgrade at a time).

Per the plan: see ~/.claude/plans/insight-temporal-bachman.md.

All three projects type-check cleanly (api, agent, admin).

Bunker Admin

changemaker-control-panel/admin/src/pages/InstanceDetailPage.tsx

@@ -40,6 +40,7 @@ import {
   DisconnectOutlined,
   UploadOutlined,
   ThunderboltOutlined,
+  CloudUploadOutlined,
   BellOutlined,
   CheckCircleOutlined,
   WarningOutlined,
@@ -582,6 +583,53 @@ export default function InstanceDetailPage() {
     }
   };
 
+  // Release upgrade (Approach C): CCP re-renders templates with new image tag,
+  // writes them to the tenant, then composePull + composeUp. For releases
+  // that change orchestration (new services, compose config) in addition
+  // to image versions. Tenant content (mkdocs/, customized configs/) is
+  // never touched.
+  const [releaseUpgradeModalOpen, setReleaseUpgradeModalOpen] = useState(false);
+  const [releaseImageTag, setReleaseImageTag] = useState<string>('');
+  const [releasePreview, setReleasePreview] = useState<{
+    files: Array<{ path: string; status: string; diff: string | null; sizeBefore: number; sizeAfter: number }>;
+    envCoverage?: { requiredVars: string[]; presentInTenantEnv: string[]; missingInTenantEnv: string[] };
+  } | null>(null);
+  const [releasePreviewLoading, setReleasePreviewLoading] = useState(false);
+
+  const handlePreviewReleaseUpgrade = async () => {
+    setReleasePreviewLoading(true);
+    setReleasePreview(null);
+    try {
+      const body = releaseImageTag.trim() ? { imageTag: releaseImageTag.trim() } : {};
+      const { data } = await api.post(`/instances/${id}/upgrade-release/preview`, body);
+      setReleasePreview(data.data);
+    } catch (err: unknown) {
+      const resp = (err as { response?: { data?: { error?: { message?: string } } } })?.response
+        ?.data?.error;
+      message.error(resp?.message || 'Preview failed');
+    } finally {
+      setReleasePreviewLoading(false);
+    }
+  };
+
+  const handleStartReleaseUpgrade = async () => {
+    setUpgradingInstance(true);
+    try {
+      const body = releaseImageTag.trim() ? { imageTag: releaseImageTag.trim() } : {};
+      const { data } = await api.post(`/instances/${id}/upgrade-release`, body);
+      setCurrentUpgrade(data.data);
+      setReleaseUpgradeModalOpen(false);
+      setReleasePreview(null);
+      message.success('Release upgrade started');
+    } catch (err: unknown) {
+      const resp = (err as { response?: { data?: { error?: { message?: string } } } })?.response
+        ?.data?.error;
+      message.error(resp?.message || 'Failed to start release upgrade');
+    } finally {
+      setUpgradingInstance(false);
+    }
+  };
+
   // Event handlers
   const handleAcknowledgeEvent = async (eventId: string) => {
     try {
@@ -1670,6 +1718,17 @@ export default function InstanceDetailPage() {
                       Quick Upgrade
                     </Button>
                   </Popconfirm>
+                  <Button
+                    icon={<CloudUploadOutlined />}
+                    onClick={() => {
+                      setReleaseImageTag(instance.imageTag || '');
+                      setReleasePreview(null);
+                      setReleaseUpgradeModalOpen(true);
+                    }}
+                    disabled={instance.status !== 'RUNNING' && instance.status !== 'STOPPED'}
+                  >
+                    Upgrade to Release
+                  </Button>
                   <Popconfirm
                     title="Start full upgrade?"
                     description="This will pull the latest code, run database migrations, and restart all services. Brief downtime is expected."
@@ -2256,6 +2315,117 @@ export default function InstanceDetailPage() {
           </Space>
         </Modal>
       )}
+
+      {/* Approach C: Release upgrade modal (CCP template re-render) */}
+      <Modal
+        title="Upgrade to Release"
+        open={releaseUpgradeModalOpen}
+        onCancel={() => setReleaseUpgradeModalOpen(false)}
+        footer={[
+          <Button key="cancel" onClick={() => setReleaseUpgradeModalOpen(false)}>
+            Cancel
+          </Button>,
+          <Button
+            key="preview"
+            onClick={handlePreviewReleaseUpgrade}
+            loading={releasePreviewLoading}
+          >
+            Preview Changes
+          </Button>,
+          <Button
+            key="apply"
+            type="primary"
+            danger={
+              !!releasePreview?.envCoverage?.missingInTenantEnv?.length
+            }
+            loading={upgradingInstance}
+            onClick={handleStartReleaseUpgrade}
+          >
+            Apply Upgrade
+          </Button>,
+        ]}
+        width={900}
+      >
+        <Space direction="vertical" style={{ width: '100%' }} size="middle">
+          <div>
+            <Typography.Text strong>Image Tag:</Typography.Text>
+            <Input
+              value={releaseImageTag}
+              onChange={(e) => setReleaseImageTag(e.target.value)}
+              placeholder="e.g. v2.10.3 (blank = use current env.IMAGE_TAG default)"
+              style={{ marginTop: 4 }}
+            />
+            <Typography.Text type="secondary" style={{ fontSize: 12 }}>
+              Re-renders docker-compose.yml + env + nginx configs with this tag. Tenant content
+              (mkdocs/, custom configs/) is never touched. Click <em>Preview Changes</em> to see the
+              per-file diff before applying.
+            </Typography.Text>
+          </div>
+
+          {releasePreview && (
+            <>
+              {releasePreview.envCoverage?.missingInTenantEnv && releasePreview.envCoverage.missingInTenantEnv.length > 0 && (
+                <Alert
+                  type="error"
+                  showIcon
+                  message="Missing env vars in tenant .env"
+                  description={
+                    <div>
+                      <div>The new docker-compose.yml references vars the tenant&apos;s .env does NOT define:</div>
+                      <code style={{ display: 'block', marginTop: 8, fontSize: 11 }}>
+                        {releasePreview.envCoverage.missingInTenantEnv.join(', ')}
+                      </code>
+                      <div style={{ marginTop: 8 }}>
+                        Applying without these vars may break services. Add them to the tenant&apos;s .env
+                        first, or reconcile the template.
+                      </div>
+                    </div>
+                  }
+                />
+              )}
+
+              <Typography.Text strong>
+                Files: {releasePreview.files.length} total, {releasePreview.files.filter(f => f.status === 'modified').length} modified, {releasePreview.files.filter(f => f.status === 'created').length} created, {releasePreview.files.filter(f => f.status === 'unchanged').length} unchanged
+              </Typography.Text>
+
+              <div style={{ maxHeight: 500, overflow: 'auto', border: '1px solid #303030', borderRadius: 4 }}>
+                {releasePreview.files.map((f) => (
+                  <div key={f.path} style={{ padding: 8, borderBottom: '1px solid #303030' }}>
+                    <div style={{ display: 'flex', justifyContent: 'space-between', alignItems: 'center' }}>
+                      <code style={{ fontSize: 12 }}>{f.path}</code>
+                      <Tag
+                        color={
+                          f.status === 'unchanged' ? 'green' :
+                          f.status === 'modified' ? 'gold' :
+                          'blue'
+                        }
+                      >
+                        {f.status} {f.sizeBefore !== undefined && `(${f.sizeBefore} → ${f.sizeAfter} B)`}
+                      </Tag>
+                    </div>
+                    {f.diff && (
+                      <pre
+                        style={{
+                          background: '#1e1e1e',
+                          color: '#d4d4d4',
+                          padding: 8,
+                          marginTop: 8,
+                          maxHeight: 200,
+                          overflow: 'auto',
+                          fontSize: 11,
+                          borderRadius: 4,
+                        }}
+                      >
+                        {f.diff}
+                      </pre>
+                    )}
+                  </div>
+                ))}
+              </div>
+            </>
+          )}
+        </Space>
+      </Modal>
     </div>
   );
 }

changemaker-control-panel/admin/src/types/api.ts

@@ -9,6 +9,7 @@ export interface Instance {
   composeProject: string;
   gitBranch: string;
   gitCommit?: string;
+  imageTag?: string;
   portConfig: Record<string, number>;
   enableMedia: boolean;
   enableChat: boolean;

changemaker-control-panel/agent/src/routes/files.routes.ts

@@ -24,6 +24,19 @@ router.post('/instance/:slug/files', async (req: Request, res: Response) => {
   res.json({ written: files.length });
 });
 
+// POST /instance/:slug/files/diff — Approach C pre-flight: diff proposed
+// rendered files against on-disk current content. Read-only.
+router.post('/instance/:slug/files/diff', async (req: Request, res: Response) => {
+  const entry = await getSlugEntry(param(req, 'slug'));
+  const { files } = req.body;
+  if (!Array.isArray(files)) {
+    res.status(400).json({ error: 'VALIDATION', message: 'files array required' });
+    return;
+  }
+  const results = await fileService.diffFiles(entry.basePath, files);
+  res.json({ files: results });
+});
+
 // POST /instance/:slug/mkdir — Create directory
 router.post('/instance/:slug/mkdir', async (req: Request, res: Response) => {
   const entry = await getSlugEntry(param(req, 'slug'));

changemaker-control-panel/agent/src/services/file.service.ts

@@ -35,6 +35,113 @@ export async function writeFiles(
   }
 }
 
+/**
+ * Diff proposed files against current on-disk contents at basePath.
+ * For Approach C pre-flight preview: operator sees per-file change summary
+ * before applying re-rendered templates. Returns one DiffResult per proposed
+ * file. Uses a small inline LCS-based unified diff to avoid new deps.
+ */
+export interface DiffResult {
+  path: string;
+  status: 'unchanged' | 'modified' | 'created';
+  diff: string | null;
+  sizeBefore: number;
+  sizeAfter: number;
+}
+
+const DIFF_MAX_LINES = 500;
+
+function unifiedDiff(oldText: string, newText: string, relativePath: string): string {
+  // Compact unified-diff: line-level LCS, emit context + changed lines.
+  // Not a full GNU diff — adequate for compose/env/conf inspection in the UI.
+  const oldLines = oldText.split('\n');
+  const newLines = newText.split('\n');
+
+  // Build LCS table (line-level). For files up to ~1500 lines this is O(N*M)
+  // which is fine; we truncate output length not algorithm runtime.
+  const m = oldLines.length, n = newLines.length;
+  const dp: number[][] = Array.from({ length: m + 1 }, () => new Array<number>(n + 1).fill(0));
+  for (let i = m - 1; i >= 0; i--) {
+    for (let j = n - 1; j >= 0; j--) {
+      dp[i][j] = oldLines[i] === newLines[j]
+        ? dp[i + 1][j + 1] + 1
+        : Math.max(dp[i + 1][j], dp[i][j + 1]);
+    }
+  }
+  // Backtrack to emit unified-style hunks
+  const out: string[] = [`--- a/${relativePath}`, `+++ b/${relativePath}`];
+  let i = 0, j = 0, oldStart = 0, newStart = 0;
+  const hunk: string[] = [];
+  let emittedLines = 0;
+  while ((i < m || j < n) && emittedLines < DIFF_MAX_LINES) {
+    if (i < m && j < n && oldLines[i] === newLines[j]) {
+      hunk.push(` ${oldLines[i]}`);
+      i++; j++;
+    } else if (j < n && (i === m || dp[i][j + 1] >= dp[i + 1][j])) {
+      hunk.push(`+${newLines[j]}`);
+      j++; newStart++;
+    } else {
+      hunk.push(`-${oldLines[i]}`);
+      i++; oldStart++;
+    }
+    emittedLines++;
+  }
+  if (emittedLines >= DIFF_MAX_LINES) hunk.push(`... (diff truncated at ${DIFF_MAX_LINES} lines)`);
+  out.push(...hunk);
+  return out.join('\n');
+}
+
+export async function diffFiles(
+  basePath: string,
+  files: Array<{ relativePath: string; content: string }>
+): Promise<DiffResult[]> {
+  const results: DiffResult[] = [];
+  for (const file of files) {
+    const filePath = path.join(basePath, file.relativePath);
+    assertWithin(filePath, basePath);
+    const sizeAfter = Buffer.byteLength(file.content, 'utf-8');
+
+    let current: string | null = null;
+    try {
+      current = await fs.readFile(filePath, 'utf-8');
+    } catch {
+      current = null;
+    }
+
+    if (current === null) {
+      results.push({
+        path: file.relativePath,
+        status: 'created',
+        diff: null,
+        sizeBefore: 0,
+        sizeAfter,
+      });
+      continue;
+    }
+
+    const sizeBefore = Buffer.byteLength(current, 'utf-8');
+    if (current === file.content) {
+      results.push({
+        path: file.relativePath,
+        status: 'unchanged',
+        diff: null,
+        sizeBefore,
+        sizeAfter,
+      });
+      continue;
+    }
+
+    results.push({
+      path: file.relativePath,
+      status: 'modified',
+      diff: unifiedDiff(current, file.content, file.relativePath),
+      sizeBefore,
+      sizeAfter,
+    });
+  }
+  return results;
+}
+
 export async function mkdirp(basePath: string, relativePath: string): Promise<void> {
   const dirPath = path.join(basePath, relativePath);
   assertWithin(dirPath, basePath);

changemaker-control-panel/api/prisma/migrations/20260522093400_add_instance_image_tag/migration.sql

@@ -0,0 +1,4 @@
+-- Approach C: per-instance image tag override.
+-- NULL means "use env.IMAGE_TAG default". Set via CCP "Upgrade to Release"
+-- flow when operator chooses a tag for a specific tenant.
+ALTER TABLE "instances" ADD COLUMN "image_tag" TEXT;

changemaker-control-panel/api/prisma/schema.prisma

@@ -70,6 +70,13 @@ model Instance {
   gitBranch       String         @default("v2") @map("git_branch")
   gitCommit       String?        @map("git_commit")
 
+  // Per-instance image tag override (Approach C release upgrades).
+  // NULL = fall back to env.IMAGE_TAG (the CCP-wide default). When set,
+  // CCP renders this value into the tenant's .env IMAGE_TAG, and the
+  // compose template's ${IMAGE_TAG:-latest} substitution picks it up at
+  // compose-up time. Each tenant rolls forward on its own cadence.
+  imageTag        String?        @map("image_tag")
+
   // Allocated host ports (JSON: { api: 14001, admin: 13001, postgres: 15401, nginx: 10001 })
   portConfig      Json           @map("port_config")
 

changemaker-control-panel/api/src/modules/instances/instances.routes.ts

@@ -4,7 +4,7 @@ import rateLimit from 'express-rate-limit';
 import { prisma } from '../../lib/prisma';
 import { authenticate, requireRole } from '../../middleware/auth';
 import { validate } from '../../middleware/validate';
-import { createInstanceSchema, updateInstanceSchema, registerInstanceSchema, reconfigureInstanceSchema, configureTunnelSchema, importInstancesSchema, startUpgradeSchema, startImageUpgradeSchema, setupRemoteTunnelSchema } from './instances.schemas';
+import { createInstanceSchema, updateInstanceSchema, registerInstanceSchema, reconfigureInstanceSchema, configureTunnelSchema, importInstancesSchema, startUpgradeSchema, startImageUpgradeSchema, startReleaseUpgradeSchema, setupRemoteTunnelSchema } from './instances.schemas';
 import * as instancesService from './instances.service';
 import * as healthService from '../../services/health.service';
 import * as backupService from '../../services/backup.service';
@@ -381,6 +381,41 @@ router.post(
   }
 );
 
+// Release upgrade (Approach C). Re-renders templates via CCP and applies
+// them to the tenant, then composePull + composeUp. Used when a release
+// changes orchestration in addition to image versions.
+router.post(
+  '/:id/upgrade-release',
+  requireRole('SUPER_ADMIN', 'OPERATOR'),
+  validate(startReleaseUpgradeSchema),
+  async (req: Request, res: Response) => {
+    const { imageTag } = req.body || {};
+    const upgrade = await upgradeService.startReleaseUpgrade(
+      req.params.id as string,
+      req.user!.id,
+      req.ip,
+      { imageTag }
+    );
+    res.status(201).json({ data: upgrade });
+  }
+);
+
+// Approach C pre-flight: preview what re-render would change before applying.
+// READ-ONLY — tenant disk is not touched.
+router.post(
+  '/:id/upgrade-release/preview',
+  requireRole('SUPER_ADMIN', 'OPERATOR'),
+  validate(startReleaseUpgradeSchema),
+  async (req: Request, res: Response) => {
+    const { imageTag } = req.body || {};
+    const preview = await upgradeService.previewReleaseUpgrade(
+      req.params.id as string,
+      { imageTag }
+    );
+    res.json({ data: preview });
+  }
+);
+
 router.get(
   '/:id/upgrade-status',
   requireRole('SUPER_ADMIN', 'OPERATOR'),

changemaker-control-panel/api/src/modules/instances/instances.schemas.ts

@@ -132,6 +132,19 @@ export const startImageUpgradeSchema = z.object({
     .optional(),
 });
 
+// Approach C: release upgrade via CCP template re-render. CCP renders the
+// docker-compose.yml + nginx confs + pangolin resources etc. against the
+// tenant's context (with the proposed imageTag), writes them to the tenant,
+// then composePull + composeUp. Used when a release changes orchestration
+// in addition to image versions. imageTag is the new value for the
+// per-instance Instance.imageTag column (NULL falls back to env default).
+export const startReleaseUpgradeSchema = z.object({
+  imageTag: z
+    .string()
+    .regex(/^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}$/, 'Invalid imageTag')
+    .optional(),
+});
+
 export const setupRemoteTunnelSchema = z.object({
   // Empty string or omitted → resources use standard subdomains (app., api., etc.)
   // A value like "ck" → creates ck-app., ck-api., etc. for multi-tenant domains

changemaker-control-panel/api/src/services/execution-driver.ts

@@ -24,6 +24,13 @@ export interface ExecutionDriver {
   // ─── Filesystem Operations ──────────────────────────────────
   readEnvFile(basePath: string): Promise<Record<string, string> | null>;
   writeFiles(basePath: string, files: Array<{ relativePath: string; content: string }>): Promise<void>;
+  // Approach C pre-flight: diff proposed file contents against on-disk current.
+  // Returns per-file status (unchanged | modified | created) + unified diff for modified.
+  // Read-only.
+  diffFiles(
+    basePath: string,
+    files: Array<{ relativePath: string; content: string }>
+  ): Promise<Array<{ path: string; status: 'unchanged' | 'modified' | 'created'; diff: string | null; sizeBefore: number; sizeAfter: number }>>;
   mkdir(basePath: string, relativePath: string): Promise<void>;
   fileExists(basePath: string, relativePath: string): Promise<boolean>;
   deleteDirectory(dirPath: string): Promise<void>;

changemaker-control-panel/api/src/services/local-driver.ts

@@ -80,6 +80,35 @@ export class LocalDriver implements ExecutionDriver {
     }
   }
 
+  // Approach C pre-flight diff. Reads current file contents at basePath +
+  // relativePath, returns per-file status + diff. Local implementation
+  // mirrors the agent-side diffFiles helper.
+  async diffFiles(basePath: string, files: Array<{ relativePath: string; content: string }>) {
+    const results: Array<{ path: string; status: 'unchanged' | 'modified' | 'created'; diff: string | null; sizeBefore: number; sizeAfter: number }> = [];
+    for (const file of files) {
+      const filePath = path.join(basePath, file.relativePath);
+      const sizeAfter = Buffer.byteLength(file.content, 'utf-8');
+      let current: string | null = null;
+      try { current = await fs.readFile(filePath, 'utf-8'); } catch { current = null; }
+      if (current === null) {
+        results.push({ path: file.relativePath, status: 'created', diff: null, sizeBefore: 0, sizeAfter });
+      } else if (current === file.content) {
+        results.push({ path: file.relativePath, status: 'unchanged', diff: null, sizeBefore: Buffer.byteLength(current), sizeAfter });
+      } else {
+        // Minimal diff for local: full new content. Local mode is dev-only;
+        // detailed diffs come from the agent-side implementation.
+        results.push({
+          path: file.relativePath,
+          status: 'modified',
+          diff: `--- a/${file.relativePath}\n+++ b/${file.relativePath}\n(local-driver: showing new content only)\n${file.content}`,
+          sizeBefore: Buffer.byteLength(current),
+          sizeAfter,
+        });
+      }
+    }
+    return results;
+  }
+
   async mkdir(basePath: string, relativePath: string) {
     await fs.mkdir(path.join(basePath, relativePath), { recursive: true });
   }

changemaker-control-panel/api/src/services/remote-driver.ts

@@ -309,6 +309,17 @@ export class RemoteDriver implements ExecutionDriver {
     });
   }
 
+  // Approach C pre-flight diff via agent.
+  async diffFiles(_basePath: string, files: Array<{ relativePath: string; content: string }>) {
+    const resp = await this.request<{ files: Array<{ path: string; status: 'unchanged' | 'modified' | 'created'; diff: string | null; sizeBefore: number; sizeAfter: number }> }>({
+      method: 'POST',
+      path: `/instance/${this.slug}/files/diff`,
+      body: { files },
+      timeoutMs: env.AGENT_LONG_OP_TIMEOUT_MS,
+    });
+    return resp.files;
+  }
+
   async mkdir(_basePath: string, relativePath: string): Promise<void> {
     await this.request({
       method: 'POST',

changemaker-control-panel/api/src/services/template-engine.ts

@@ -135,6 +135,8 @@ export interface InstanceForTemplate {
   smtpFrom: string | null;
   emailTestMode: boolean;
   gitBranch: string;
+  // Per-instance image tag override (Approach C). NULL falls back to env.IMAGE_TAG.
+  imageTag: string | null;
 }
 
 /**
@@ -208,7 +210,9 @@ export function buildTemplateContext(
     gitBranch: instance.gitBranch,
     registryUrl: env.GITEA_REGISTRY,
     useRegistry: env.USE_REGISTRY_IMAGES,
-    imageTag: env.IMAGE_TAG,
+    // Approach C: per-instance imageTag overrides the CCP-wide env default.
+    // NULL on the Instance row falls back to env.IMAGE_TAG (typically 'latest').
+    imageTag: instance.imageTag || env.IMAGE_TAG,
   };
 }
 

changemaker-control-panel/api/src/services/upgrade.service.ts

@@ -8,6 +8,8 @@ import { logger } from '../utils/logger';
 import { createEvent } from './event.service';
 import { getRemoteDriverForInstance } from './execution-driver';
 import type { AgentUpdateStatus } from './remote-driver';
+import { buildTemplateContext, clearTemplateCache, renderAllTemplatesInMemory } from './template-engine';
+import { decryptJson } from '../utils/encryption';
 
 /**
  * Shell-injection guards. Any user- or DB-controlled value that flows into
@@ -382,6 +384,313 @@ export async function startImageUpgrade(
   return upgrade;
 }
 
+// ─── Approach C: Release upgrade (template re-render) ────────────────────────
+
+export interface StartReleaseUpgradeOptions {
+  imageTag?: string;
+}
+
+const SAFE_IMAGE_TAG = /^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}$/;
+
+/**
+ * Files that should NOT be re-rendered for tenants without encryptedSecrets
+ * (install.sh-registered tenants). Their .env was provisioned at install
+ * time and contains real secrets we can't reproduce.
+ */
+const REGISTERED_TENANT_SKIP_FILES = new Set(['.env']);
+
+/**
+ * Filter rendered file list for tenants without secrets. For install.sh
+ * tenants we keep the existing .env on disk (CCP can't render env without
+ * secrets in DB). Compose, nginx, pangolin etc. still render correctly
+ * because they only reference instance fields, not secrets directly.
+ */
+function filterRenderedFilesForRegisteredTenant(
+  files: Array<{ relativePath: string; content: string }>
+): Array<{ relativePath: string; content: string }> {
+  return files.filter(f => !REGISTERED_TENANT_SKIP_FILES.has(f.relativePath));
+}
+
+/**
+ * Extract env var names referenced by the rendered docker-compose.yml.
+ * Used to compute envCoverage for install.sh tenants — operator needs to
+ * know if any ${VAR} references won't have a value in the tenant's .env.
+ */
+function extractComposeEnvVars(composeYaml: string): string[] {
+  const vars = new Set<string>();
+  // Match ${VAR} or ${VAR:-default} or ${VAR:?required}
+  const re = /\$\{([A-Z_][A-Z0-9_]*)(?:[:-?][^}]*)?\}/g;
+  let m: RegExpExecArray | null;
+  while ((m = re.exec(composeYaml)) !== null) {
+    vars.add(m[1]);
+  }
+  return Array.from(vars).sort();
+}
+
+/**
+ * Approach C pre-flight preview. Renders templates with the proposed
+ * imageTag override and diffs against the tenant's current files. Also
+ * computes envCoverage for install.sh tenants so the operator can see
+ * if the new compose needs any env vars their .env doesn't have.
+ * READ-ONLY — touches nothing on the tenant.
+ */
+export async function previewReleaseUpgrade(
+  instanceId: string,
+  options?: StartReleaseUpgradeOptions
+): Promise<{
+  files: Array<{ path: string; status: 'unchanged' | 'modified' | 'created'; diff: string | null; sizeBefore: number; sizeAfter: number }>;
+  envCoverage?: {
+    requiredVars: string[];
+    presentInTenantEnv: string[];
+    missingInTenantEnv: string[];
+  };
+}> {
+  const instance = await prisma.instance.findUnique({ where: { id: instanceId } });
+  if (!instance) throw new Error('Instance not found');
+  if (!instance.isRemote) {
+    throw new Error('Release upgrade preview is currently supported only for remote instances');
+  }
+  if (options?.imageTag && !SAFE_IMAGE_TAG.test(options.imageTag)) {
+    throw new Error('Invalid imageTag');
+  }
+
+  // Build context with proposed imageTag override (not persisted)
+  const previewInstance = { ...instance, imageTag: options?.imageTag ?? instance.imageTag };
+  const secrets = instance.encryptedSecrets
+    ? decryptJson<Record<string, string>>(instance.encryptedSecrets)
+    : {};
+
+  clearTemplateCache();
+  const context = buildTemplateContext(previewInstance, secrets);
+  let files = await renderAllTemplatesInMemory(context);
+
+  // Skip .env for registered tenants (no secrets to render against)
+  if (!instance.encryptedSecrets) {
+    files = filterRenderedFilesForRegisteredTenant(files);
+  }
+
+  const driver = await getRemoteDriverForInstance({
+    id: instance.id,
+    slug: instance.slug,
+    isRemote: instance.isRemote,
+    agentUrl: instance.agentUrl,
+  });
+  const diffResults = await driver.diffFiles(instance.basePath, files);
+
+  // For registered tenants: report envCoverage so operator knows if any
+  // ${VAR} from the new compose isn't in their tenant .env. Required check
+  // because CCP isn't rendering their env file.
+  let envCoverage: { requiredVars: string[]; presentInTenantEnv: string[]; missingInTenantEnv: string[] } | undefined;
+  if (!instance.encryptedSecrets) {
+    const composeFile = files.find(f => f.relativePath === 'docker-compose.yml');
+    if (composeFile) {
+      const requiredVars = extractComposeEnvVars(composeFile.content);
+      // Read tenant's current .env via the agent's readEnvFile
+      const tenantEnv = await driver.readEnvFile(instance.basePath);
+      const presentKeys = new Set(Object.keys(tenantEnv || {}));
+      const presentInTenantEnv = requiredVars.filter(v => presentKeys.has(v));
+      const missingInTenantEnv = requiredVars.filter(v => !presentKeys.has(v));
+      envCoverage = { requiredVars, presentInTenantEnv, missingInTenantEnv };
+    }
+  }
+
+  return { files: diffResults, envCoverage };
+}
+
+/**
+ * Approach C apply path. Persists imageTag, re-renders templates, writes
+ * them to the tenant, then composePull + composeUp --remove-orphans.
+ * Fire-and-forget; status visible via the existing getUpgradeStatus() poll.
+ */
+export async function startReleaseUpgrade(
+  instanceId: string,
+  userId: string,
+  ipAddress?: string,
+  options?: StartReleaseUpgradeOptions
+) {
+  const instance = await prisma.instance.findUnique({ where: { id: instanceId } });
+  if (!instance) throw new Error('Instance not found');
+
+  if (!instance.isRemote) {
+    throw new Error('Release upgrade is currently supported only for remote instances');
+  }
+
+  if (instance.status !== InstanceStatus.RUNNING && instance.status !== InstanceStatus.STOPPED) {
+    throw new Error(`Cannot upgrade instance in ${instance.status} state`);
+  }
+
+  if (options?.imageTag && !SAFE_IMAGE_TAG.test(options.imageTag)) {
+    throw new Error('Invalid imageTag');
+  }
+
+  // Shared in-progress guard across all upgrade types.
+  const active = await prisma.instanceUpgrade.findFirst({
+    where: { instanceId, status: { in: [UpgradeStatus.PENDING, UpgradeStatus.IN_PROGRESS] } },
+  });
+  if (active) throw new Error('An upgrade is already in progress for this instance');
+
+  const upgrade = await prisma.instanceUpgrade.create({
+    data: {
+      instanceId,
+      status: UpgradeStatus.PENDING,
+      previousCommit: instance.imageTag ?? instance.gitCommit,
+      branch: instance.gitBranch,
+      triggeredById: userId,
+    },
+  });
+
+  await prisma.auditLog.create({
+    data: {
+      userId,
+      instanceId,
+      action: AuditAction.INSTANCE_UPGRADE,
+      details: {
+        upgradeId: upgrade.id,
+        previousImageTag: instance.imageTag,
+        newImageTag: options?.imageTag,
+        source: 'remote',
+        mode: 'release-template',
+      } as unknown as Prisma.InputJsonValue,
+      ipAddress,
+    },
+  });
+
+  // Fire-and-forget runner. Distinct from runRemoteUpgrade because we don't
+  // shell out to upgrade.sh — CCP does the render + compose orchestration
+  // directly through the mTLS driver. No agent-side script involved.
+  runReleaseUpgrade(upgrade.id, instance, options).catch((err) => {
+    logger.error(`[release-upgrade] Orchestration failed for ${instance.slug}: ${err}`);
+  });
+
+  return upgrade;
+}
+
+/**
+ * Internal: do the actual Approach C work. Updates DB, renders, writes,
+ * pulls, recreates, verifies. All non-progress reporting comes via DB
+ * status updates on the InstanceUpgrade row.
+ */
+async function runReleaseUpgrade(
+  upgradeId: string,
+  instance: Instance,
+  options?: StartReleaseUpgradeOptions
+) {
+  const slug = instance.slug;
+  const newImageTag = options?.imageTag;
+
+  const updateStatus = async (data: Prisma.InstanceUpgradeUpdateInput) => {
+    await prisma.instanceUpgrade.update({ where: { id: upgradeId }, data });
+  };
+
+  try {
+    await updateStatus({
+      status: UpgradeStatus.IN_PROGRESS,
+      currentPhase: 1,
+      phaseName: 'Render',
+      percentage: 10,
+      progressMessage: 'Rendering templates with new image tag...',
+    });
+
+    // Persist new imageTag before render so buildTemplateContext picks it up.
+    if (newImageTag) {
+      await prisma.instance.update({ where: { id: instance.id }, data: { imageTag: newImageTag } });
+    }
+    const refreshed = await prisma.instance.findUniqueOrThrow({ where: { id: instance.id } });
+
+    const secrets = refreshed.encryptedSecrets
+      ? decryptJson<Record<string, string>>(refreshed.encryptedSecrets)
+      : {};
+    clearTemplateCache();
+    const context = buildTemplateContext(refreshed, secrets);
+    let files = await renderAllTemplatesInMemory(context);
+    if (!refreshed.encryptedSecrets) {
+      files = filterRenderedFilesForRegisteredTenant(files);
+    }
+
+    const driver = await getRemoteDriverForInstance({
+      id: instance.id,
+      slug: instance.slug,
+      isRemote: instance.isRemote,
+      agentUrl: instance.agentUrl,
+    });
+
+    // Phase 2: write rendered files
+    await updateStatus({
+      currentPhase: 2,
+      phaseName: 'Write Files',
+      percentage: 30,
+      progressMessage: `Writing ${files.length} rendered file(s)...`,
+    });
+    await driver.writeFiles(instance.basePath, files);
+
+    // Phase 3: pull images per new compose
+    await updateStatus({
+      currentPhase: 3,
+      phaseName: 'Pull Images',
+      percentage: 55,
+      progressMessage: 'Pulling images from registry...',
+    });
+    await driver.composePull(instance.basePath, instance.composeProject);
+
+    // Phase 4: recreate services
+    await updateStatus({
+      currentPhase: 4,
+      phaseName: 'Recreate Services',
+      percentage: 80,
+      progressMessage: 'Recreating services with new orchestration...',
+    });
+    await driver.composeUp(instance.basePath, instance.composeProject);
+
+    // Phase 5: verify (best-effort; soft warnings only)
+    await updateStatus({
+      currentPhase: 5,
+      phaseName: 'Verify',
+      percentage: 95,
+      progressMessage: 'Verifying container health...',
+    });
+    const warnings: string[] = [];
+    try {
+      const containers = await driver.composePs(instance.basePath, instance.composeProject);
+      const unhealthy = containers.filter(c => c.status && /restarting|exited/i.test(c.status));
+      if (unhealthy.length > 0) {
+        warnings.push(`${unhealthy.length} container(s) not healthy after upgrade: ${unhealthy.map(c => c.name).join(', ')}`);
+      }
+    } catch {
+      warnings.push('composePs verification failed (services may still be starting)');
+    }
+
+    await updateStatus({
+      status: UpgradeStatus.COMPLETED,
+      currentPhase: 5,
+      phaseName: 'Complete',
+      percentage: 100,
+      progressMessage: `Release upgrade complete${newImageTag ? ` (imageTag: ${newImageTag})` : ''}`,
+      newCommit: newImageTag ?? refreshed.imageTag,
+      commitCount: 0,
+      warnings: warnings.length ? (warnings as unknown as Prisma.InputJsonValue) : undefined,
+      completedAt: new Date(),
+    });
+    logger.info(`[release-upgrade] ${slug}: completed${newImageTag ? ` → ${newImageTag}` : ''}`);
+  } catch (err) {
+    const message = (err as Error).message || 'Release upgrade failed';
+    await updateStatus({
+      status: UpgradeStatus.FAILED,
+      errorMessage: message,
+      progressMessage: `Failed: ${message}`,
+      completedAt: new Date(),
+    });
+    await createEvent(
+      instance.id,
+      'ERROR',
+      'upgrade',
+      'Release upgrade failed',
+      message,
+      { upgradeId, source: 'remote', mode: 'release-template' }
+    );
+    logger.error(`[release-upgrade] ${slug}: failed: ${message}`);
+  }
+}
+
 /**
  * Async REMOTE upgrade runner.
  *