26 Commits

Author SHA1 Message Date
21208b58c7 feat(media): HLS adaptive bitrate streaming with MP4 fallback
Replaces single-MP4 + range-request streaming with HLS multi-bitrate
segments to fix video stutter through the Newt tunnel. Range-request
bursts were the root cause; HLS chunks are small and tunnel-friendly,
plus the player adapts bitrate to bandwidth.

Backend
- New BullMQ `hls-transcode` queue (in-process worker, concurrency 1)
- FFmpeg single-pass transcode → 360p/720p/1080p variants with aligned
  keyframes; output at /media/local/hls/{id}/master.m3u8
- New /api/{videos|public}/{id}/hls/* routes serving signed manifests
  and segments (URLs emitted as /media/* so nginx rewrites to media-api)
- Prisma: HlsStatus enum + 6 fields on Video + index, migration
- Upload + yt-dlp fetch paths enqueue transcode jobs
- ENABLE_HLS_TRANSCODE flag (default off; gates enqueue only)
- Backfill script: `npm run backfill:hls`
- media-api bumped to 4 CPU / 2G for FFmpeg headroom

Frontend
- New useHls hook: lazy-imports hls.js (kept out of main bundle),
  native HLS on Safari/iOS, gives up after 2 NETWORK_ERRORs so MP4
  fallback engages cleanly
- VideoPlayer, VideoViewerModal, ShortsPage, ProductDetailPage now
  prefer HLS when ready; MP4 fallback is automatic
- ShortsPage prefetches next-3 master manifests via <link rel="prefetch">
- PublicVideoCard hover preview stays MP4 (avoids hls.js init latency)

Bunker Admin
2026-04-30 19:03:29 -06:00
e55bc07eb6 Security hardening: red-team remediation + CCP/WIP updates
## Security (red-team audit 2026-04-12)

Public data exposure (P0):
- Public map converted to server-side heatmap, 2-decimal (~1.1km) bucketing,
  no addresses/support-levels/sign-info returned
- Petition signers endpoint strips displayName/signerComment/geoCity/geoCountry
- Petition public-stats drops recentSigners entirely
- Response wall strips userComment + submittedByName
- Campaign createdByUserEmail + moderation fields gated to SUPER_ADMIN

Access control (P1):
- Campaign findById/update/delete/email-stats enforce owner === req.user.id
  (SUPER_ADMIN bypasses), return 404 to avoid enumeration
- GPS tracking session route restricted to session owner or SUPER_ADMIN
- Canvass volunteer stats restricted to self or SUPER_ADMIN
- People household endpoints restricted to INFLUENCE + MAP roles (was ADMIN*)
- CCP upgrade.service.ts + certificate.service.ts gate user-controlled
  shell inputs (branch, path, slug, SAN hostname) behind regex validators

Token security (P2):
- Query-param JWT auth replaced with HMAC-signed short-lived URLs
  (utils/signed-url.ts + /api/media/sign endpoint); legacy ?token= removed
  from media streaming, photos, chat-notifications, and social SSE
- GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT now REQUIRED (min 32 chars);
  JWT_ACCESS_SECRET fallback removed — BREAKING for existing deployments
- Refresh tokens bound to device fingerprint (UA + /24 IP) via `df` JWT
  claim; mismatch revokes all user sessions
- Refresh expiry reduced 7d → 24h
- Refresh/logout via request body removed — httpOnly cookie only
- Password-reset + verification-resend rate limits now keyed on (IP, email)
  composite to prevent both IP rotation and email enumeration

Defense-in-depth (P3):
- DOMPurify sanitization applied to GrapesJS landing page HTML/CSS
- /api/health?detailed=true disk-space leak removed
- Password-reset/verification token log lines no longer include userId

## Deployment

- docker-compose.yml + docker-compose.prod.yml: media-api now receives
  GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT; empty fallbacks removed
- CCP templates/env.hbs adds both new secrets; refresh expiry → 24h
- CCP secret-generator.ts generates giteaSsoSecret + servicePasswordSalt
- leaflet.heat added to admin/package.json for heatmap rendering

## Operator action required on existing installs

Run `./config.sh` once (idempotent — only fills empty values) or manually
add GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT to .env via
`openssl rand -hex 32`. Startup fails with a clear Zod error otherwise.

See SECURITY_REDTEAM_2026-04-12.md for full audit and verification matrix.

## Other

Includes in-flight CCP work: instance schema tweaks, agent server updates,
health service, tunnel service, DEV_WORKFLOW doc updates, and new migration
dropping composeProject uniqueness.

Bunker Admin
2026-04-12 15:17:00 -06:00
5f0ae6bc5a Revert NocoDB auto sign-in, keep CSP fix for embed proxy
NocoDB v2 stores auth tokens in-memory (Pinia store), not in cookies
accessible to external pages. The auth bridge approach can't inject
tokens into NocoDB's SPA state. Reverted to the original banner
approach ("sign in to NocoDB in a new tab").

Kept: CSP fix (frame-ancestors http://localhost:* instead of just
localhost, which only matched port 80).

Bunker Admin
2026-04-09 14:01:02 -06:00
36b709b911 Automate Gitea init, NocoDB auto sign-in, and fix prod compose
- Add scripts/gitea-init.sh: runs migrations + creates admin user on
  first boot, replacing the manual installation wizard
- Set GITEA__security__INSTALL_LOCK=true in both compose files
- Add NocoDB auth bridge (nginx) + /api/services/nocodb-auth proxy
  endpoint so the admin iframe auto-authenticates
- Update NocoDBPage.tsx to fetch token and use auth bridge flow
- Fix docker-compose.prod.yml missing Gitea env vars for API container
  (GITEA_URL, GITEA_API_TOKEN, GITEA_ADMIN_PASSWORD, etc.)
- Pass NC_ADMIN_EMAIL/PASSWORD to API for NocoDB auth proxy
- Increase Gitea auto-setup retries from 3 to 6 with admin auth check
- Update config.sh non-interactive mode to set GITEA_ADMIN_USER
- Include gitea-init.sh in release tarball (build-release.sh)

Bunker Admin
2026-04-09 12:49:33 -06:00
c6f8a49925 Add CCP registration page to CML admin panel
Operators can now register with a Control Panel directly from the admin
GUI (Services → Control Panel) without SSH access. Uses the existing
updateEnvFile + dockerService pattern from the Pangolin setup.

New endpoint: /api/ccp-registration (status, register, unregister)
New page: ControlPanelPage with form for CCP URL, invite code, agent URL
Also passes CCP env vars through docker-compose to the API container.

Bunker Admin
2026-04-08 15:13:28 -06:00
38ccaa8a5b Add remote instance management with mTLS agent and phone-home registration
Enables the CCP to manage CML instances on remote servers via a lightweight
HTTP agent. Key components:

- ExecutionDriver abstraction (local-driver.ts / remote-driver.ts) routes
  operations to local Docker or remote agent transparently
- Remote agent package (agent/) with mTLS authentication, Docker Compose
  operations, file management, backup/upgrade delegation
- Certificate service using openssl CLI for CA management and cert issuance
- Phone-home registration: remote agents register via invite code, CCP admin
  approves, agent receives mTLS cert bundle automatically
- config.sh integration with configure_control_panel() section
- ccp-agent Docker Compose service (profile-gated)
- Frontend: AgentRegistrationsPage, InviteCodesPage, Remote Agents sidebar menu
- Security hardened: cert bundle wiped after delivery, shell injection prevention
  via execFile, command allowlist with metachar rejection, rate-limited public
  endpoints, auto-populated fingerprint pinning

Also wires ENABLE_SOCIAL/PEOPLE/ANALYTICS through env.ts, seed.ts, and
docker-compose env passthrough (from previous session).

Bunker Admin
2026-04-07 15:24:33 -06:00
08bd1f92b0 Add unified analytics system with GeoIP geo-tracking
Full analytics platform with MaxMind GeoLite2 IP-to-location resolution,
cross-module dashboard (docs, video, photo), user drill-down, volunteer
self-service stats, and ANALYTICS_ADMIN role with feature flag controls.

- ANALYTICS_ADMIN role + ANALYTICS_ROLES group across backend and frontend
- GeoIP service (MaxMind GeoLite2, lazy-loaded, graceful degradation)
- Geo fields (country, region, city, lat/lng) on DocsPageView, VideoView, PhotoView
- IP resolved to geo before SHA-256 hashing (privacy-preserving)
- Unified analytics module: overview, geo, content, user engagement endpoints
- 4 admin dashboard pages: Overview, Geography (Leaflet map), Content, Users
- Volunteer MyAnalyticsPage for self-service activity stats
- Settings UI: enableAnalytics, analyticsGeoEnabled, trackAuthenticatedUsers, retentionDays
- Scheduled cleanup job respecting configurable retention period
- config.sh: Analytics + MaxMind prompt in configure_features()
- Control panel: enableAnalytics flag, template, discovery, wizard, detail page
- Docker: geoip volume mount, MaxMind env vars, entrypoint auto-download
- Nginx: X-Forwarded-For fix ($proxy_add_x_forwarded_for) for real client IP
- Express trust proxy set to 2 for Pangolin/Newt tunnel chain
- CORS updated for docs origin (cmlite.org + docs.cmlite.org)
- Lander page: added docs-analytics tracking snippet
- Prisma migration: 20260402100000_add_analytics_system

Bunker Admin
2026-04-03 08:47:44 -06:00
5a0c4641a1 Security audit fixes, mobile responsiveness across 40+ admin pages
Security hardening from Mar 31 audit:
- Separate login rate limit (10/15min) from general auth budget (15/15min)
- Timing-safe webhook secret comparison (Listmonk)
- Docs file creation ACL check (matches PUT/DELETE guards)
- Key separation warnings for GITEA_SSO_SECRET and SERVICE_PASSWORD_SALT
- Clear GITEA_ADMIN_PASSWORD from .env after auto-setup
- SQL injection prevention in effectiveness groupBy (pre-validated map)
- Token hashing for password reset and verification tokens

Mobile responsiveness (Phase 2C):
- Add MobilePageHeader component and useMobile hook
- Responsive table columns (hide secondary cols on mobile)
- scroll={{ x: 'max-content' }} across all data tables
- Mobile-adapted layouts for Dashboard, Settings, Calendar, SMS, Social pages
- Conditional toolbar buttons on mobile viewports

Infrastructure:
- Updated docker-compose and nginx templates
- Build script and mirror script updates

Bunker Admin
2026-03-31 18:30:17 -06:00
f378db89b5 Separate local vs remote Gitea API tokens to prevent credential collision
GITEA_API_TOKEN is for the local platform Gitea (docs comments, user
provisioning, SSO). New GITEA_REGISTRY_API_TOKEN is for the remote
registry at gitea.bnkops.com (release uploads via build-release.sh).

Previously both contexts shared one variable, causing auth failures
when the token for one instance was used against the other.

Bunker Admin
2026-03-31 11:53:20 -06:00
91db29402c Add Gitea SSO, fix security audit findings, harden production defaults
Gitea SSO: cookie-based single sign-on via nginx auth_request — sets
cml_session cookie on login/refresh, validates via /api/auth/gitea-sso-validate,
injects X-WEBAUTH-USER header for reverse proxy auth. Dedicated GITEA_SSO_SECRET
and SERVICE_PASSWORD_SALT env vars isolate secret rotation.

Security fixes from March 30 audit: IDOR on ticketed events (requireEventOwnership
middleware), IDOR on action items (admin/assignee/creator check), path traversal
on photos (resolve-based validation), CSV upload size limit (5MB), shared calendar
email exposure removed.

Gitea provisioner: auto-sync docs repo collaborator access based on role
(CONTENT_ROLES get write, SUPER_ADMIN gets admin). Gitea client extended
with collaborator management API methods.

Production hardening: NODE_ENV defaults to production in docker-compose.prod.yml,
Grafana anonymous auth disabled, install.sh branch ref updated to main.

Admin UI: moved docs reset from toolbar to MkDocs Settings danger zone,
improved collab Ctrl+S to explicitly save + cache-bust preview.

MkDocs site rebuild with updated repo data, upgrade screenshots, and content.

Bunker Admin
2026-03-31 11:20:01 -06:00
8b9ab93856 Add docs CMS: blog authoring, access policies, sharing, version history, templates, metadata, search, Gitea auto-setup
7 documentation system features:
- Blog authoring: frontmatter panel, new post wizard, authors management
- Access policies: per-file/directory edit restrictions with role/user granularity
- Public sharing: share links with collaborative editing via dual JWT auth
- Version history: Gitea auto-commit on save, diff viewer, restore
- Document templates: 8 built-in templates (blog, guide, API ref, ADR, FAQ, etc.)
- Metadata dashboard: overview of all docs with warnings (no-tags, stale, etc.)
- Content search: in-file text search with line-level matches

Gitea auto-setup: one-click configuration of API token, repos, labels, OAuth app
- Backend service + startup hook (auto-configures if GITEA_ADMIN_PASSWORD set)
- Admin GUI wizard at /app/services/gitea/setup
- config.sh now prompts for Gitea admin password

Backend: 10 new files, 5 modified (3 models, 1 enum, 2 migrations, 30+ API endpoints)
Frontend: 13 new files, 3 modified (hooks, components, pages)

Bunker Admin
2026-03-27 13:28:52 -06:00
1bf19fff0e Security audit: fix 30 findings across auth, IDOR, XSS, path traversal, infrastructure
Comprehensive 6-domain security audit addressing 8 Critical, 17 Important,
and 5 Low findings. Key fixes:

Critical:
- Strip PII from unauthenticated ticket lookup (IDOR)
- Add role+permission checks to event check-in routes
- Validate tier-to-event ownership on update/delete (IDOR)
- Fix path traversal in video replace (resolve + prefix check)
- Enable MongoDB authentication for Rocket.Chat
- Disable Grafana anonymous access
- Sanitize CSV exports against formula injection (payments)
- Apply DOMPurify to richDescription on public event page (XSS)

Important:
- Require current password for self-service password changes
- Atomic password reset token consumption (race condition fix)
- Scope postMessage to specific origin (not wildcard)
- Validate redirect parameter against open redirect
- Replace weak temp passwords (5760 values → crypto.randomBytes)
- Move shift capacity check inside transaction (TOCTOU fix)
- Fix EVENTS_ADMIN privilege inversion in ticketed events
- Make ENCRYPTION_KEY required (remove optional fallback)
- Add internal Prometheus metrics endpoint for Docker scraping
- Add nginx-level rate limiting (limit_req_zone)
- Fix X-Forwarded-For to use $remote_addr (prevents spoofing)
- Replace CSP stripping with frame-ancestors in embed proxies
- Remove error.message from Fastify 500 responses
- Strip PII from volunteer canvass address data
- Wrap GrapesJS output in {% raw %} to prevent Jinja2 SSTI
- Scope SSE token query param to /sse path only
- Sanitize Listmonk email query against injection

Bunker Admin
2026-03-27 08:47:24 -06:00
39d74e7b85 Add guided tour, media enhancements, error handling, and DevOps improvements
Major additions: onboarding tour system, correlation-id middleware, media
error handler, restore script, env validation script, Dockerignore files.
Updates across 70+ admin components for improved UX and error handling.

Bunker Admin
2026-03-26 10:31:51 -06:00
0c634e100f Replace custom code-server (9GB) with upstream LinuxServer image (~1GB)
Drop the custom Dockerfile.code-server that bundled Claude Code CLI,
Python/MkDocs tooling, and build-essential on top of codercom base.
Switch to the already-mirrored linuxserver/code-server image instead.

- Both compose files: use code-server:latest, LinuxServer env vars
  (PUID/PGID/DEFAULT_WORKSPACE), port 8443, /config mount layout
- Nginx configs + templates: proxy to :8443 instead of :8080
- API env default: CODE_SERVER_URL updated to :8443
- build-and-push.sh: remove --include-code-server flag
- upgrade.sh: remove code-server conditional rebuild + registry fallback
- install.sh: add --ignore-pull-failures for optional missing images
- .env.example, CCP templates, bunker-ops template: updated

Bunker Admin
2026-03-25 20:10:36 -06:00
3262d92065 Remove hardcoded container names for multi-instance deployment support
- Dashboard: auto-discovers containers from Docker network via socket
  proxy API instead of hardcoded 30-name list. Labels derived from
  docker compose service metadata.
- Email/Settings: mailhog host read from env.SMTP_HOST instead of
  hardcoded 'mailhog-changemaker' string
- Pangolin: grafana container derived from env.GRAFANA_URL hostname;
  newt container/service names from NEWT_CONTAINER_NAME/NEWT_COMPOSE_SERVICE
- SSRF blocklist: built dynamically from all service URL env vars
  instead of hardcoded hostname list
- New env vars: DOCKER_NETWORK_NAME, DOCKER_PROXY_URL,
  NEWT_CONTAINER_NAME, NEWT_COMPOSE_SERVICE

Bunker Admin
2026-03-25 17:35:05 -06:00
bb1935027d Upgrade system finished 2026-03-22 21:47:09 -06:00
647efffdc4 Security hardening: JWT algorithm pinning, key separation, injection fixes
- Pin HS256 algorithm on all jwt.verify() calls (9 sites) and jwt.sign()
  calls (3 sites) — prevents algorithm confusion attacks
- Add JWT_INVITE_SECRET env var; volunteer invite tokens now use a
  dedicated key separate from access/refresh secrets
- Remove req.query.secret fallback from Listmonk webhook route — secrets
  must not appear in nginx access logs
- Replace child_process.spawn in email template seed endpoint with direct
  function import; add require.main guard to seed script
- Add sanitizeCsvField() to location CSV export to prevent formula
  injection in Excel/Sheets (=, +, -, @ prefix → apostrophe prefix)
- Cap QR endpoint text input at 2000 chars to prevent DoS via large payloads
- Fix pre-existing TS errors: type participantNeeds as UpsertNeedsInput
  in meeting-planner service; add sso field to UpdateResourcePayload

Bunker Admin
2026-03-22 12:35:04 -06:00
c192c04c79 Security audit: fix 25 findings across API, nginx, and Docker
Addresses data exposure, access control, input validation, infrastructure
hardening, and supply chain security issues identified during audit.

Key changes:
- Strip internal fields from public campaign/profile/comment endpoints
- Restrict docs routes to CONTENT_ROLES, provisioning to SUPER_ADMIN
- Add SSE connection limits, social middleware fail-closed behavior
- Bind all non-nginx ports to 127.0.0.1, pin container image versions
- Add CSP header, conditional HSTS, token redaction in nginx logs
- Validate nav URLs, calendar schemas, video tracking batch events
- Reject default admin password placeholder, add SSRF protocol checks
- Exclude .env from Code Server, enforce RC admin password in compose
- Add Zod validation for achievement grant/revoke, webhook secret header
- Fix path traversal prefix attack, add calendar token expiry

Bunker Admin
2026-03-09 14:13:37 -06:00
9e51aac570 Okay Wish I could say I know exactly. Will do better next time promise lol 2026-02-26 17:47:04 -07:00
7352815e57 More control panel updates 2026-02-21 11:46:55 -07:00
1a1f12c45b Tonne of updates 2026-02-18 17:15:31 -07:00
56e262ad8b Tonne of udpatess 2026-02-18 10:01:54 -07:00
99a6abab06 Add video card insert feature + MkDocs video hydration + fixes
- New video card block for GrapesJS landing pages, email templates,
  MkDocs export, and documentation editor Insert dropdown
- Shared HTML generators in admin/src/utils/videoCardHtml.ts
- MkDocs video-player.js hydrates .video-card-block elements:
  thumbnail fix via MEDIA_API_URL, click-to-play inline, Gallery link
- Media API CORS: auto-add MkDocs + docs subdomain origins
- env_config_hook.py: smart Docker hostname detection, ADMIN_PORT
  resolution, pass env vars to MkDocs container
- Gallery URL uses /gallery?expanded=ID format
- VideoPickerModal: fix double /api prefix and Docker hostname thumbs
- Seed: default-video-card PageBlock
- Remove V1 legacy code (influence/, map/)

Bunker Admin
2026-02-17 15:42:32 -07:00
a7978de5a0 Bunch of stuff again 2026-02-16 18:48:54 -07:00
7895ce683e Tonne of debugging - getting ready for the production builds 2026-02-16 10:44:18 -07:00
a77306fac2 Initial v2 commit: complete rebuild with unified API + React admin
Phase 1-14 complete:
- Unified Express.js API (TypeScript, Prisma ORM, PostgreSQL 16)
- React Admin GUI (Vite + Ant Design + Zustand)
- JWT auth with refresh tokens
- Influence: Campaigns, Representatives, Responses, Email Queue
- Map: Locations, Cuts, Shifts, Canvassing System
- NAR data import infrastructure (2025 format)
- Listmonk newsletter integration
- Landing page builder (GrapesJS)
- MkDocs + Code Server integration
- Volunteer portal with GPS tracking
- Monitoring stack (Prometheus, Grafana, Alertmanager)
- Pangolin tunnel integration

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-11 10:05:04 -07:00