Three fixes to harden the admin-UI upgrade path, all in scripts/upgrade.sh.
Root-caused by yesterday's v2.9.2 → v2.9.3 on marcelle which was killed by
systemd mid-Phase-4 and left the system in a misleading half-upgraded state
(VERSION bumped, container pre-upgrade, result.json stale from 24h prior).
- Fix A (failure visibility): stop silencing stderr on the five docker
compose pull sites so timeouts / auth failures / network errors flow
into upgrade-watcher.log. Add explicit SIGTERM/SIGINT traps alongside
the existing EXIT trap. Track CURRENT_PHASE_NAME globally so the
failure message reports "during Phase 4: Container Rebuild" rather
than just an exit code. Introduce write_result_force (bypasses
API_MODE guard) + archive_failure_to_history so a killed upgrade
always leaves a truthful result.json + history.json entry, and the
progress.json is cleared so the admin UI stops showing a phantom
in-progress phase.
- Fix B (atomic VERSION): Phase 3 rsync now --excludes VERSION and
stashes the new one at data/upgrade/VERSION.pending. Phase 7 promotes
it to VERSION only after VERIFY_FAILED stays false. on_failure deletes
the pending file. upgrade-check.sh needs no changes — its head -1
VERSION read sees actual state instead of a mid-upgrade promise.
- Fix C (external smoke): after Phase 7 localhost checks, curl
https://api.${DOMAIN}/api/health with --max-time 10 and warn (not
fail) on non-200. Catches Pangolin resource misassignments that the
localhost-only checks miss. Appends to UPGRADE_WARNINGS so the admin
UI surfaces it in result.json.
Bunker Admin
Six independent fixes surfaced during the v2.9.1 → v2.9.2 admin-UI
upgrade validation today. Together they make a clean install on a new
box work end-to-end without in-session patching.
- Fix 1: scripts/validate-compose-parity.sh + build-release.sh hook —
fail release builds when api/admin/media-api/nginx healthcheck
blocks drift between docker-compose.yml and docker-compose.prod.yml.
Previous boot-race fix had to be applied to both files manually.
- Fix 2: scripts/systemd/install.sh chowns logs/ to the install user
(the API container creates subdirs there as root, locking the
host-side watcher out), pre-creates logs/upgrade-watcher.log, and
changemaker-upgrade.service adds StartLimitIntervalSec=0 so a
single transient failure can't wedge the .path unit permanently.
- Fix 3: /api/upgrade/status now returns a `watcher` sub-object that
flags the host systemd watcher as stalled when trigger.json has
been pending >30s. Admin SettingsPage SystemUpgradeTab renders a
warning Alert with the systemctl recovery command when unhealthy.
- Fix 4: scripts/upgrade.sh write_result() — prefer head -1 VERSION
over `git rev-parse HEAD` so release-mode upgrades report the new
tag in result.json instead of "unknown".
- Fix 5: admin container healthcheck start_period 20s → 60s in both
compose files, same class as the earlier api fix. Matches Gancio
convention.
- Fix 7: /api/pangolin/sync now detects resources bound to a stale
siteId (common after --pangolin-site new rotations), deletes and
recreates them against the current site, and reports them under
a new `reassigned` response field.
Bunker Admin
Major additions: onboarding tour system, correlation-id middleware, media
error handler, restore script, env validation script, Dockerignore files.
Updates across 70+ admin components for improved UX and error handling.
Bunker Admin
Drop the custom Dockerfile.code-server that bundled Claude Code CLI,
Python/MkDocs tooling, and build-essential on top of codercom base.
Switch to the already-mirrored linuxserver/code-server image instead.
- Both compose files: use code-server:latest, LinuxServer env vars
(PUID/PGID/DEFAULT_WORKSPACE), port 8443, /config mount layout
- Nginx configs + templates: proxy to :8443 instead of :8080
- API env default: CODE_SERVER_URL updated to :8443
- build-and-push.sh: remove --include-code-server flag
- upgrade.sh: remove code-server conditional rebuild + registry fallback
- install.sh: add --ignore-pull-failures for optional missing images
- .env.example, CCP templates, bunker-ops template: updated
Bunker Admin
When --use-registry is set, the upgrade script tries to pull images
tagged with the current HEAD SHA. If images were built at an earlier
commit, that SHA tag won't exist. Now tries :latest before falling
back to a full source build. Also applies to nginx and code-server.
Bunker Admin
New install method: curl one-liner downloads a lightweight release
tarball (~9 MB) and runs the config wizard. No git clone needed,
no TypeScript compilation — pulls pre-built images from Gitea registry.
- docker-compose.prod.yml: production compose without build blocks or
source code volume mounts; IMAGE_TAG defaults to latest
- scripts/install.sh: curl-friendly installer (downloads tarball,
extracts, runs config.sh)
- scripts/build-release.sh: creates release tarball from dev repo
with only runtime files (configs, scripts, docs, empty data dirs)
- config.sh: release-mode detection (VERSION file + no .git dir),
auto-sets IMAGE_TAG=latest and NODE_ENV=production
- upgrade.sh: release-mode upgrade path (downloads new tarball from
Gitea Releases API instead of git pull, always uses registry mode)
- upgrade-check.sh: release-mode version check via Gitea API
- .gitignore: exclude releases/ and api/dist/
- Docs: updated getting-started with pre-built install instructions
Bunker Admin
- Fix @/utils/logger path alias (tsc doesn't transform @/ in output)
- Add JWT_INVITE_SECRET to media-api compose environment block
- Fix redis-exporter depends_on to use service name not container name
- Fix upgrade.sh: restore tracked files deleted by restore_user_paths
- Add scripts/build-and-push.sh for building + pushing production images
- Add scripts/mirror-images.sh for mirroring third-party images
Bunker Admin
Inserts Phase 5 (Database Migration) between container rebuild and service
restart. Detects failed/incomplete Prisma migrations via _prisma_migrations
query and auto-resolves them before running migrate deploy in a one-off
container — catching errors in the script rather than letting the API enter
a restart loop. Also detects when package.json/package-lock.json changed
and removes old API/admin containers to prevent stale anonymous volumes
from shadowing updated node_modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Resolve Pangolin site slug to numeric ID in sync route (fixes target creation 400 errors)
- Disable SSO on newly created Pangolin resources for public access
- Fix nginx media API proxy: use rewrite + set ordering for proper URI rewriting
- Upgrade script: clear skip-worktree flags, fix Docker-owned dir permissions, stash untracked files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
API container writes trigger files to a shared volume (data/upgrade/),
and a systemd path watcher on the host detects them and runs the
upgrade scripts. This avoids giving the container Docker socket access.
- Add upgrade-check.sh (git fetch + compare + write status.json)
- Add upgrade-watcher.sh (systemd bridge, dispatches check/upgrade)
- Add systemd path/service units with placeholder substitution
- Modify upgrade.sh with --api-mode flag (progress.json + result.json)
- Add API upgrade module (service + routes, SUPER_ADMIN only)
- Add System tab to Settings page with version info, changelog,
progress steps, and upgrade confirmation modal
- Add upgrade watcher installation to config.sh wizard
- Add data/upgrade/ shared volume to api service in docker-compose
Bunker Admin
Two issues occurred during upgrades:
1. Gancio config.json lost when Docker volume name prefix changes
(e.g., changemakerlite_ vs changemaker-lite_). Gancio finds existing
DB but no config and enters restart loop. Fix: verify_gancio_config()
checks the volume and regenerates config.json from .env if missing.
2. mkdocs-site-server (LSIO nginx) returns 403 after upgrade because
the anonymous /config volume shadows the ./mkdocs/site bind mount.
Fix: docker compose rm -sf the LSIO container before up -d so the
anonymous volume is recreated fresh.
Also adds Gancio and MkDocs site health checks to Phase 6 verification.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
