25 Commits

Author SHA1 Message Date
21208b58c7 feat(media): HLS adaptive bitrate streaming with MP4 fallback
Replaces single-MP4 + range-request streaming with HLS multi-bitrate
segments to fix video stutter through the Newt tunnel. Range-request
bursts were the root cause; HLS chunks are small and tunnel-friendly,
plus the player adapts bitrate to bandwidth.

Backend
- New BullMQ `hls-transcode` queue (in-process worker, concurrency 1)
- FFmpeg single-pass transcode → 360p/720p/1080p variants with aligned
  keyframes; output at /media/local/hls/{id}/master.m3u8
- New /api/{videos|public}/{id}/hls/* routes serving signed manifests
  and segments (URLs emitted as /media/* so nginx rewrites to media-api)
- Prisma: HlsStatus enum + 6 fields on Video + index, migration
- Upload + yt-dlp fetch paths enqueue transcode jobs
- ENABLE_HLS_TRANSCODE flag (default off; gates enqueue only)
- Backfill script: `npm run backfill:hls`
- media-api bumped to 4 CPU / 2G for FFmpeg headroom

Frontend
- New useHls hook: lazy-imports hls.js (kept out of main bundle),
  native HLS on Safari/iOS, gives up after 2 NETWORK_ERRORs so MP4
  fallback engages cleanly
- VideoPlayer, VideoViewerModal, ShortsPage, ProductDetailPage now
  prefer HLS when ready; MP4 fallback is automatic
- ShortsPage prefetches next-3 master manifests via <link rel="prefetch">
- PublicVideoCard hover preview stays MP4 (avoids hls.js init latency)

Bunker Admin
2026-04-30 19:03:29 -06:00
4ccc433eb9 fix(gitea): fall back to INITIAL_ADMIN_PASSWORD for gitea-init
Previously both compose files defaulted GITEA_ADMIN_PASSWORD to empty,
and scripts/gitea-init.sh silently skipped admin creation on blank input.
If config.sh failed to propagate the password (e.g. Docs Comments not
enabled, or --admin-password omitted), fresh installs ended up with a
Gitea container running but zero users — and the admin GUI's Gitea setup
wizard had no token to progress.

Changes:
- docker-compose.yml + docker-compose.prod.yml: GITEA_ADMIN_PASSWORD now
  falls back to INITIAL_ADMIN_PASSWORD when unset
- .env.example: declare GITEA_ADMIN_PASSWORD= with explanatory comment so
  users discover the override
- scripts/gitea-init.sh: silent skip becomes a loud WARN so a broken
  config is visible in compose logs

Bunker Admin
2026-04-23 10:54:24 -06:00
e55bc07eb6 Security hardening: red-team remediation + CCP/WIP updates
## Security (red-team audit 2026-04-12)

Public data exposure (P0):
- Public map converted to server-side heatmap, 2-decimal (~1.1km) bucketing,
  no addresses/support-levels/sign-info returned
- Petition signers endpoint strips displayName/signerComment/geoCity/geoCountry
- Petition public-stats drops recentSigners entirely
- Response wall strips userComment + submittedByName
- Campaign createdByUserEmail + moderation fields gated to SUPER_ADMIN

Access control (P1):
- Campaign findById/update/delete/email-stats enforce owner === req.user.id
  (SUPER_ADMIN bypasses), return 404 to avoid enumeration
- GPS tracking session route restricted to session owner or SUPER_ADMIN
- Canvass volunteer stats restricted to self or SUPER_ADMIN
- People household endpoints restricted to INFLUENCE + MAP roles (was ADMIN*)
- CCP upgrade.service.ts + certificate.service.ts gate user-controlled
  shell inputs (branch, path, slug, SAN hostname) behind regex validators

Token security (P2):
- Query-param JWT auth replaced with HMAC-signed short-lived URLs
  (utils/signed-url.ts + /api/media/sign endpoint); legacy ?token= removed
  from media streaming, photos, chat-notifications, and social SSE
- GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT now REQUIRED (min 32 chars);
  JWT_ACCESS_SECRET fallback removed — BREAKING for existing deployments
- Refresh tokens bound to device fingerprint (UA + /24 IP) via `df` JWT
  claim; mismatch revokes all user sessions
- Refresh expiry reduced 7d → 24h
- Refresh/logout via request body removed — httpOnly cookie only
- Password-reset + verification-resend rate limits now keyed on (IP, email)
  composite to prevent both IP rotation and email enumeration

Defense-in-depth (P3):
- DOMPurify sanitization applied to GrapesJS landing page HTML/CSS
- /api/health?detailed=true disk-space leak removed
- Password-reset/verification token log lines no longer include userId

## Deployment

- docker-compose.yml + docker-compose.prod.yml: media-api now receives
  GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT; empty fallbacks removed
- CCP templates/env.hbs adds both new secrets; refresh expiry → 24h
- CCP secret-generator.ts generates giteaSsoSecret + servicePasswordSalt
- leaflet.heat added to admin/package.json for heatmap rendering

## Operator action required on existing installs

Run `./config.sh` once (idempotent — only fills empty values) or manually
add GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT to .env via
`openssl rand -hex 32`. Startup fails with a clear Zod error otherwise.

See SECURITY_REDTEAM_2026-04-12.md for full audit and verification matrix.

## Other

Includes in-flight CCP work: instance schema tweaks, agent server updates,
health service, tunnel service, DEV_WORKFLOW doc updates, and new migration
dropping composeProject uniqueness.

Bunker Admin
2026-04-12 15:17:00 -06:00
36b709b911 Automate Gitea init, NocoDB auto sign-in, and fix prod compose
- Add scripts/gitea-init.sh: runs migrations + creates admin user on
  first boot, replacing the manual installation wizard
- Set GITEA__security__INSTALL_LOCK=true in both compose files
- Add NocoDB auth bridge (nginx) + /api/services/nocodb-auth proxy
  endpoint so the admin iframe auto-authenticates
- Update NocoDBPage.tsx to fetch token and use auth bridge flow
- Fix docker-compose.prod.yml missing Gitea env vars for API container
  (GITEA_URL, GITEA_API_TOKEN, GITEA_ADMIN_PASSWORD, etc.)
- Pass NC_ADMIN_EMAIL/PASSWORD to API for NocoDB auth proxy
- Increase Gitea auto-setup retries from 3 to 6 with admin auth check
- Update config.sh non-interactive mode to set GITEA_ADMIN_USER
- Include gitea-init.sh in release tarball (build-release.sh)

Bunker Admin
2026-04-09 12:49:33 -06:00
f0d994074d Update admin modals and page components for mobile responsiveness
Bunker Admin
2026-04-09 11:43:23 -06:00
38ccaa8a5b Add remote instance management with mTLS agent and phone-home registration
Enables the CCP to manage CML instances on remote servers via a lightweight
HTTP agent. Key components:

- ExecutionDriver abstraction (local-driver.ts / remote-driver.ts) routes
  operations to local Docker or remote agent transparently
- Remote agent package (agent/) with mTLS authentication, Docker Compose
  operations, file management, backup/upgrade delegation
- Certificate service using openssl CLI for CA management and cert issuance
- Phone-home registration: remote agents register via invite code, CCP admin
  approves, agent receives mTLS cert bundle automatically
- config.sh integration with configure_control_panel() section
- ccp-agent Docker Compose service (profile-gated)
- Frontend: AgentRegistrationsPage, InviteCodesPage, Remote Agents sidebar menu
- Security hardened: cert bundle wiped after delivery, shell injection prevention
  via execFile, command allowlist with metachar rejection, rate-limited public
  endpoints, auto-populated fingerprint pinning

Also wires ENABLE_SOCIAL/PEOPLE/ANALYTICS through env.ts, seed.ts, and
docker-compose env passthrough (from previous session).

Bunker Admin
2026-04-07 15:24:33 -06:00
08bd1f92b0 Add unified analytics system with GeoIP geo-tracking
Full analytics platform with MaxMind GeoLite2 IP-to-location resolution,
cross-module dashboard (docs, video, photo), user drill-down, volunteer
self-service stats, and ANALYTICS_ADMIN role with feature flag controls.

- ANALYTICS_ADMIN role + ANALYTICS_ROLES group across backend and frontend
- GeoIP service (MaxMind GeoLite2, lazy-loaded, graceful degradation)
- Geo fields (country, region, city, lat/lng) on DocsPageView, VideoView, PhotoView
- IP resolved to geo before SHA-256 hashing (privacy-preserving)
- Unified analytics module: overview, geo, content, user engagement endpoints
- 4 admin dashboard pages: Overview, Geography (Leaflet map), Content, Users
- Volunteer MyAnalyticsPage for self-service activity stats
- Settings UI: enableAnalytics, analyticsGeoEnabled, trackAuthenticatedUsers, retentionDays
- Scheduled cleanup job respecting configurable retention period
- config.sh: Analytics + MaxMind prompt in configure_features()
- Control panel: enableAnalytics flag, template, discovery, wizard, detail page
- Docker: geoip volume mount, MaxMind env vars, entrypoint auto-download
- Nginx: X-Forwarded-For fix ($proxy_add_x_forwarded_for) for real client IP
- Express trust proxy set to 2 for Pangolin/Newt tunnel chain
- CORS updated for docs origin (cmlite.org + docs.cmlite.org)
- Lander page: added docs-analytics tracking snippet
- Prisma migration: 20260402100000_add_analytics_system

Bunker Admin
2026-04-03 08:47:44 -06:00
f378db89b5 Separate local vs remote Gitea API tokens to prevent credential collision
GITEA_API_TOKEN is for the local platform Gitea (docs comments, user
provisioning, SSO). New GITEA_REGISTRY_API_TOKEN is for the remote
registry at gitea.bnkops.com (release uploads via build-release.sh).

Previously both contexts shared one variable, causing auth failures
when the token for one instance was used against the other.

Bunker Admin
2026-03-31 11:53:20 -06:00
91db29402c Add Gitea SSO, fix security audit findings, harden production defaults
Gitea SSO: cookie-based single sign-on via nginx auth_request — sets
cml_session cookie on login/refresh, validates via /api/auth/gitea-sso-validate,
injects X-WEBAUTH-USER header for reverse proxy auth. Dedicated GITEA_SSO_SECRET
and SERVICE_PASSWORD_SALT env vars isolate secret rotation.

Security fixes from March 30 audit: IDOR on ticketed events (requireEventOwnership
middleware), IDOR on action items (admin/assignee/creator check), path traversal
on photos (resolve-based validation), CSV upload size limit (5MB), shared calendar
email exposure removed.

Gitea provisioner: auto-sync docs repo collaborator access based on role
(CONTENT_ROLES get write, SUPER_ADMIN gets admin). Gitea client extended
with collaborator management API methods.

Production hardening: NODE_ENV defaults to production in docker-compose.prod.yml,
Grafana anonymous auth disabled, install.sh branch ref updated to main.

Admin UI: moved docs reset from toolbar to MkDocs Settings danger zone,
improved collab Ctrl+S to explicitly save + cache-bust preview.

MkDocs site rebuild with updated repo data, upgrade screenshots, and content.

Bunker Admin
2026-03-31 11:20:01 -06:00
1bf19fff0e Security audit: fix 30 findings across auth, IDOR, XSS, path traversal, infrastructure
Comprehensive 6-domain security audit addressing 8 Critical, 17 Important,
and 5 Low findings. Key fixes:

Critical:
- Strip PII from unauthenticated ticket lookup (IDOR)
- Add role+permission checks to event check-in routes
- Validate tier-to-event ownership on update/delete (IDOR)
- Fix path traversal in video replace (resolve + prefix check)
- Enable MongoDB authentication for Rocket.Chat
- Disable Grafana anonymous access
- Sanitize CSV exports against formula injection (payments)
- Apply DOMPurify to richDescription on public event page (XSS)

Important:
- Require current password for self-service password changes
- Atomic password reset token consumption (race condition fix)
- Scope postMessage to specific origin (not wildcard)
- Validate redirect parameter against open redirect
- Replace weak temp passwords (5760 values → crypto.randomBytes)
- Move shift capacity check inside transaction (TOCTOU fix)
- Fix EVENTS_ADMIN privilege inversion in ticketed events
- Make ENCRYPTION_KEY required (remove optional fallback)
- Add internal Prometheus metrics endpoint for Docker scraping
- Add nginx-level rate limiting (limit_req_zone)
- Fix X-Forwarded-For to use $remote_addr (prevents spoofing)
- Replace CSP stripping with frame-ancestors in embed proxies
- Remove error.message from Fastify 500 responses
- Strip PII from volunteer canvass address data
- Wrap GrapesJS output in {% raw %} to prevent Jinja2 SSTI
- Scope SSE token query param to /sse path only
- Sanitize Listmonk email query against injection

Bunker Admin
2026-03-27 08:47:24 -06:00
39d74e7b85 Add guided tour, media enhancements, error handling, and DevOps improvements
Major additions: onboarding tour system, correlation-id middleware, media
error handler, restore script, env validation script, Dockerignore files.
Updates across 70+ admin components for improved UX and error handling.

Bunker Admin
2026-03-26 10:31:51 -06:00
0c634e100f Replace custom code-server (9GB) with upstream LinuxServer image (~1GB)
Drop the custom Dockerfile.code-server that bundled Claude Code CLI,
Python/MkDocs tooling, and build-essential on top of codercom base.
Switch to the already-mirrored linuxserver/code-server image instead.

- Both compose files: use code-server:latest, LinuxServer env vars
  (PUID/PGID/DEFAULT_WORKSPACE), port 8443, /config mount layout
- Nginx configs + templates: proxy to :8443 instead of :8080
- API env default: CODE_SERVER_URL updated to :8443
- build-and-push.sh: remove --include-code-server flag
- upgrade.sh: remove code-server conditional rebuild + registry fallback
- install.sh: add --ignore-pull-failures for optional missing images
- .env.example, CCP templates, bunker-ops template: updated

Bunker Admin
2026-03-25 20:10:36 -06:00
7287328148 Harden install pipeline: health checks, log rotation, backup timer
- install.sh: Add Docker daemon check, 10GB disk space pre-flight,
  error handling on pull/up, post-startup health polling with crash
  detection, cleanup trap on failure
- docker-compose: Fix nginx/listmonk depends_on to service_healthy,
  add x-logging anchor (10m/3 files) to all ~39 services
- config.sh: Preserve existing secrets on re-run (reconfigure mode),
  add automated daily backup timer (systemd, 02:00, 30-day retention)
- mirror-images.sh: Fix gotify source tag (2.9.0 not v2.9.0)
- build-release.sh: Ensure mkdocs/docs and mkdocs/overrides dirs exist
- .env.example: Add COMPOSE_PROFILES variable

Bunker Admin
2026-03-25 19:33:11 -06:00
3262d92065 Remove hardcoded container names for multi-instance deployment support
- Dashboard: auto-discovers containers from Docker network via socket
  proxy API instead of hardcoded 30-name list. Labels derived from
  docker compose service metadata.
- Email/Settings: mailhog host read from env.SMTP_HOST instead of
  hardcoded 'mailhog-changemaker' string
- Pangolin: grafana container derived from env.GRAFANA_URL hostname;
  newt container/service names from NEWT_CONTAINER_NAME/NEWT_COMPOSE_SERVICE
- SSRF blocklist: built dynamically from all service URL env vars
  instead of hardcoded hostname list
- New env vars: DOCKER_NETWORK_NAME, DOCKER_PROXY_URL,
  NEWT_CONTAINER_NAME, NEWT_COMPOSE_SERVICE

Bunker Admin
2026-03-25 17:35:05 -06:00
abdfd50cb8 Make embed proxy ports configurable via env vars for multi-instance deployments
All 13 nginx embed proxy ports (8881-8895) are now driven by environment
variables instead of being hardcoded. This prevents port conflicts when
running multiple Changemaker instances on the same host.

Chain: .env → docker-compose port mappings → nginx container env →
entrypoint.sh envsubst → services.conf.template listen directives →
API /services/config endpoint → frontend buildServiceUrl().

Existing deployments are unaffected (all vars default to current values).

Bunker Admin
2026-03-25 15:25:00 -06:00
bb1935027d Upgrade system finished 2026-03-22 21:47:09 -06:00
647efffdc4 Security hardening: JWT algorithm pinning, key separation, injection fixes
- Pin HS256 algorithm on all jwt.verify() calls (9 sites) and jwt.sign()
  calls (3 sites) — prevents algorithm confusion attacks
- Add JWT_INVITE_SECRET env var; volunteer invite tokens now use a
  dedicated key separate from access/refresh secrets
- Remove req.query.secret fallback from Listmonk webhook route — secrets
  must not appear in nginx access logs
- Replace child_process.spawn in email template seed endpoint with direct
  function import; add require.main guard to seed script
- Add sanitizeCsvField() to location CSV export to prevent formula
  injection in Excel/Sheets (=, +, -, @ prefix → apostrophe prefix)
- Cap QR endpoint text input at 2000 chars to prevent DoS via large payloads
- Fix pre-existing TS errors: type participantNeeds as UpsertNeedsInput
  in meeting-planner service; add sso field to UpdateResourcePayload

Bunker Admin
2026-03-22 12:35:04 -06:00
76b87d9f3d Tonne of things 2026-03-08 18:11:26 -06:00
a37d9910af Add nocodb-init container for automatic database registration
Follows the listmonk-init pattern: an alpine:3 container that runs once
after NocoDB is healthy, calls the REST API to register changemaker_v2
as a browsable data source, and exits. Idempotent — exits immediately
if the base already has tables, and guards against duplicate sources
during async table discovery.

Bunker Admin
2026-03-05 10:59:39 -07:00
e95bc8883e scheduling features 2026-03-01 15:22:27 -07:00
9e51aac570 Okay Wish I could say I know exactly. Will do better next time promise lol 2026-02-26 17:47:04 -07:00
1a1f12c45b Tonne of updates 2026-02-18 17:15:31 -07:00
56e262ad8b Tonne of udpatess 2026-02-18 10:01:54 -07:00
99a6abab06 Add video card insert feature + MkDocs video hydration + fixes
- New video card block for GrapesJS landing pages, email templates,
  MkDocs export, and documentation editor Insert dropdown
- Shared HTML generators in admin/src/utils/videoCardHtml.ts
- MkDocs video-player.js hydrates .video-card-block elements:
  thumbnail fix via MEDIA_API_URL, click-to-play inline, Gallery link
- Media API CORS: auto-add MkDocs + docs subdomain origins
- env_config_hook.py: smart Docker hostname detection, ADMIN_PORT
  resolution, pass env vars to MkDocs container
- Gallery URL uses /gallery?expanded=ID format
- VideoPickerModal: fix double /api prefix and Docker hostname thumbs
- Seed: default-video-card PageBlock
- Remove V1 legacy code (influence/, map/)

Bunker Admin
2026-02-17 15:42:32 -07:00
cd19f8c0b9 .env example 2026-02-16 19:27:45 -07:00