30 Commits

Author SHA1 Message Date
21208b58c7 feat(media): HLS adaptive bitrate streaming with MP4 fallback
Replaces single-MP4 + range-request streaming with HLS multi-bitrate
segments to fix video stutter through the Newt tunnel. Range-request
bursts were the root cause; HLS chunks are small and tunnel-friendly,
plus the player adapts bitrate to bandwidth.

Backend
- New BullMQ `hls-transcode` queue (in-process worker, concurrency 1)
- FFmpeg single-pass transcode → 360p/720p/1080p variants with aligned
  keyframes; output at /media/local/hls/{id}/master.m3u8
- New /api/{videos|public}/{id}/hls/* routes serving signed manifests
  and segments (URLs emitted as /media/* so nginx rewrites to media-api)
- Prisma: HlsStatus enum + 6 fields on Video + index, migration
- Upload + yt-dlp fetch paths enqueue transcode jobs
- ENABLE_HLS_TRANSCODE flag (default off; gates enqueue only)
- Backfill script: `npm run backfill:hls`
- media-api bumped to 4 CPU / 2G for FFmpeg headroom

Frontend
- New useHls hook: lazy-imports hls.js (kept out of main bundle),
  native HLS on Safari/iOS, gives up after 2 NETWORK_ERRORs so MP4
  fallback engages cleanly
- VideoPlayer, VideoViewerModal, ShortsPage, ProductDetailPage now
  prefer HLS when ready; MP4 fallback is automatic
- ShortsPage prefetches next-3 master manifests via <link rel="prefetch">
- PublicVideoCard hover preview stays MP4 (avoids hls.js init latency)

Bunker Admin
2026-04-30 19:03:29 -06:00
ac901c9e53 Update system hardening: breaking-release gate + release-mode rollback + health budgets + success archival
Four fixes building on the prior upgrade-path work. All observed on
marcelle across today's v2.9.2 → v2.9.5 cycles and addressed here.

- Fix 1 (breaking-release gate). upgrade-check.sh now parses the first
  line of each Gitea release body for `BREAKING: <reason>` and threads
  `breaking`/`breakingReason` through status.json into the API status
  response. Admin UI renders a red Alert with a typed-tag confirmation
  input and gates the Start Upgrade button. auto-upgrade.service.ts
  refuses to apply breaking releases, logging a skip and holding off
  until the operator confirms manually.

- Fix 2 (release-mode rollback). print_rollback_help and the --rollback
  flow both used `git checkout`, which silently fails in release
  installs (no .git). Added INSTALL_MODE branches: release mode
  downloads the prior tarball from Gitea using a new VERSION.rollback
  marker seeded at Phase 3 start. Source mode retains the existing
  git-based flow.

- Fix 3 (Phase 7 health budgets). admin verify_service_health budget
  30s → 90s (matches the admin container's start_period from commit
  47704667). Gancio + MkDocs switched from one-shot to the existing
  verify_service_health retry wrapper. Cuts the cry-wolf
  "services may still be starting" warning from every upgrade result.

- Fix 4 (symmetric success archival). Bash archive_failure_to_history
  already logs failures on exit; added a matching archive_success_to_
  history called after write_result on the success path. API-side
  archiveResult now dedupes on completedAt so double-recording (bash
  + post-restart handler) can't land twice in history.json.

Release the bundle as v2.9.6.

Bunker Admin
2026-04-15 16:57:13 -06:00
e55bc07eb6 Security hardening: red-team remediation + CCP/WIP updates
## Security (red-team audit 2026-04-12)

Public data exposure (P0):
- Public map converted to server-side heatmap, 2-decimal (~1.1km) bucketing,
  no addresses/support-levels/sign-info returned
- Petition signers endpoint strips displayName/signerComment/geoCity/geoCountry
- Petition public-stats drops recentSigners entirely
- Response wall strips userComment + submittedByName
- Campaign createdByUserEmail + moderation fields gated to SUPER_ADMIN

Access control (P1):
- Campaign findById/update/delete/email-stats enforce owner === req.user.id
  (SUPER_ADMIN bypasses), return 404 to avoid enumeration
- GPS tracking session route restricted to session owner or SUPER_ADMIN
- Canvass volunteer stats restricted to self or SUPER_ADMIN
- People household endpoints restricted to INFLUENCE + MAP roles (was ADMIN*)
- CCP upgrade.service.ts + certificate.service.ts gate user-controlled
  shell inputs (branch, path, slug, SAN hostname) behind regex validators

Token security (P2):
- Query-param JWT auth replaced with HMAC-signed short-lived URLs
  (utils/signed-url.ts + /api/media/sign endpoint); legacy ?token= removed
  from media streaming, photos, chat-notifications, and social SSE
- GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT now REQUIRED (min 32 chars);
  JWT_ACCESS_SECRET fallback removed — BREAKING for existing deployments
- Refresh tokens bound to device fingerprint (UA + /24 IP) via `df` JWT
  claim; mismatch revokes all user sessions
- Refresh expiry reduced 7d → 24h
- Refresh/logout via request body removed — httpOnly cookie only
- Password-reset + verification-resend rate limits now keyed on (IP, email)
  composite to prevent both IP rotation and email enumeration

Defense-in-depth (P3):
- DOMPurify sanitization applied to GrapesJS landing page HTML/CSS
- /api/health?detailed=true disk-space leak removed
- Password-reset/verification token log lines no longer include userId

## Deployment

- docker-compose.yml + docker-compose.prod.yml: media-api now receives
  GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT; empty fallbacks removed
- CCP templates/env.hbs adds both new secrets; refresh expiry → 24h
- CCP secret-generator.ts generates giteaSsoSecret + servicePasswordSalt
- leaflet.heat added to admin/package.json for heatmap rendering

## Operator action required on existing installs

Run `./config.sh` once (idempotent — only fills empty values) or manually
add GITEA_SSO_SECRET + SERVICE_PASSWORD_SALT to .env via
`openssl rand -hex 32`. Startup fails with a clear Zod error otherwise.

See SECURITY_REDTEAM_2026-04-12.md for full audit and verification matrix.

## Other

Includes in-flight CCP work: instance schema tweaks, agent server updates,
health service, tunnel service, DEV_WORKFLOW doc updates, and new migration
dropping composeProject uniqueness.

Bunker Admin
2026-04-12 15:17:00 -06:00
08bd1f92b0 Add unified analytics system with GeoIP geo-tracking
Full analytics platform with MaxMind GeoLite2 IP-to-location resolution,
cross-module dashboard (docs, video, photo), user drill-down, volunteer
self-service stats, and ANALYTICS_ADMIN role with feature flag controls.

- ANALYTICS_ADMIN role + ANALYTICS_ROLES group across backend and frontend
- GeoIP service (MaxMind GeoLite2, lazy-loaded, graceful degradation)
- Geo fields (country, region, city, lat/lng) on DocsPageView, VideoView, PhotoView
- IP resolved to geo before SHA-256 hashing (privacy-preserving)
- Unified analytics module: overview, geo, content, user engagement endpoints
- 4 admin dashboard pages: Overview, Geography (Leaflet map), Content, Users
- Volunteer MyAnalyticsPage for self-service activity stats
- Settings UI: enableAnalytics, analyticsGeoEnabled, trackAuthenticatedUsers, retentionDays
- Scheduled cleanup job respecting configurable retention period
- config.sh: Analytics + MaxMind prompt in configure_features()
- Control panel: enableAnalytics flag, template, discovery, wizard, detail page
- Docker: geoip volume mount, MaxMind env vars, entrypoint auto-download
- Nginx: X-Forwarded-For fix ($proxy_add_x_forwarded_for) for real client IP
- Express trust proxy set to 2 for Pangolin/Newt tunnel chain
- CORS updated for docs origin (cmlite.org + docs.cmlite.org)
- Lander page: added docs-analytics tracking snippet
- Prisma migration: 20260402100000_add_analytics_system

Bunker Admin
2026-04-03 08:47:44 -06:00
5a0c4641a1 Security audit fixes, mobile responsiveness across 40+ admin pages
Security hardening from Mar 31 audit:
- Separate login rate limit (10/15min) from general auth budget (15/15min)
- Timing-safe webhook secret comparison (Listmonk)
- Docs file creation ACL check (matches PUT/DELETE guards)
- Key separation warnings for GITEA_SSO_SECRET and SERVICE_PASSWORD_SALT
- Clear GITEA_ADMIN_PASSWORD from .env after auto-setup
- SQL injection prevention in effectiveness groupBy (pre-validated map)
- Token hashing for password reset and verification tokens

Mobile responsiveness (Phase 2C):
- Add MobilePageHeader component and useMobile hook
- Responsive table columns (hide secondary cols on mobile)
- scroll={{ x: 'max-content' }} across all data tables
- Mobile-adapted layouts for Dashboard, Settings, Calendar, SMS, Social pages
- Conditional toolbar buttons on mobile viewports

Infrastructure:
- Updated docker-compose and nginx templates
- Build script and mirror script updates

Bunker Admin
2026-03-31 18:30:17 -06:00
91db29402c Add Gitea SSO, fix security audit findings, harden production defaults
Gitea SSO: cookie-based single sign-on via nginx auth_request — sets
cml_session cookie on login/refresh, validates via /api/auth/gitea-sso-validate,
injects X-WEBAUTH-USER header for reverse proxy auth. Dedicated GITEA_SSO_SECRET
and SERVICE_PASSWORD_SALT env vars isolate secret rotation.

Security fixes from March 30 audit: IDOR on ticketed events (requireEventOwnership
middleware), IDOR on action items (admin/assignee/creator check), path traversal
on photos (resolve-based validation), CSV upload size limit (5MB), shared calendar
email exposure removed.

Gitea provisioner: auto-sync docs repo collaborator access based on role
(CONTENT_ROLES get write, SUPER_ADMIN gets admin). Gitea client extended
with collaborator management API methods.

Production hardening: NODE_ENV defaults to production in docker-compose.prod.yml,
Grafana anonymous auth disabled, install.sh branch ref updated to main.

Admin UI: moved docs reset from toolbar to MkDocs Settings danger zone,
improved collab Ctrl+S to explicitly save + cache-bust preview.

MkDocs site rebuild with updated repo data, upgrade screenshots, and content.

Bunker Admin
2026-03-31 11:20:01 -06:00
5d15b4cffa Add engagement scoring and homepage stats EventBus listeners
- Engagement scoring listener: 11 event subscriptions, weighted scoring
  (donation=50, subscription=40, shift=20, canvass=15, email=10, video=3),
  Redis sorted set leaderboard, per-contact score + last-activity tracking
- Homepage stats listener: 12 subscriptions, incremental Redis counters
  (emails, signups, donations, responses, canvass, videos), capped recent
  activity lists (last 20 per type), cache invalidation on data changes
- GET /api/homepage/live-stats — public real-time counters + recent activity
- GET /api/observability/engagement-leaderboard — admin top-N contacts
- Total: 8 listeners, 70 subscriptions across all modules

Bunker Admin
2026-03-31 10:21:05 -06:00
902adce646 Add Straw Polls feature: quick opinion polling with public landers, MkDocs widgets, and social integration
Full-stack implementation across 7 sprints:
- Backend: 5 Prisma models (StrawPoll, Option, Vote, Comment, Challenge), 4 enums, POLLS_ADMIN role,
  admin CRUD routes, public voting/SSE/widget endpoints, BullMQ auto-close queue, rate limiting
- Admin: StrawPollsPage with inline drawers (campaigns pattern), PollResults bar chart, sidebar under Advocacy
- Public: dedicated poll lander with real-time SSE updates, browse page, anonymous voting with token dedup
- MkDocs: straw-poll-widget.js hydration (inline vote + card link modes), GrapesJS block types
- Social: feed activity (poll_voted), friend badge integration, challenge notifications, notification preferences
- Feature flag: enablePolls toggle in Settings, FeatureGate, Zod schema

Bunker Admin
2026-03-31 10:16:56 -06:00
68434c51a6 Extend EventBus: RC notifications, CRM activity, Gancio migration, calendar source types
- Add 7 new RC notification types: campaign published, donations, subscriptions,
  SMS escalations, user approved, video published, ticketed events
- Add CRM activity entries for subscription activated and email bounced
- Migrate ticketed-events Gancio sync from inline calls to EventBus listener
- Add meeting.created/deleted events from jitsi.routes.ts
- Add SHIFT, MEETING, TICKETED_EVENT to CalendarItemSource enum (Prisma migration)
- Update calendar-sync listener to use proper source types instead of MANUAL
- Total: 45 listener subscriptions across 6 modules, zero inline sync calls remaining

Bunker Admin
2026-03-31 10:04:44 -06:00
0c2ffe754e Harden Stripe payment integration: 15 security fixes from audit
Addresses 11 original findings (1 critical, 3 high, 4 medium, 3 low)
plus 4 additional findings from security review:

- Mask secrets in PUT /settings response (was leaking decrypted keys)
- Add paymentCheckoutRateLimit (10/hr/IP) to all 5 checkout endpoints
- Implement durable audit logging to payment_audit_log table
- Pin Stripe API version to 2026-01-28.clover (SDK v20.3.1)
- Add charge.dispute.created/closed webhook handlers with DISPUTED status
- Restore tickets on dispute won, handle charge_refunded closure
- Guard against sentinel passthrough corrupting stored Stripe keys
- Wrap refund DB updates in try/catch with webhook reconciliation fallback
- Add $transaction for product maxPurchases race condition
- Remove dead Payment model lookup from handleChargeRefunded
- Cap donation amount at $100k in both schemas
- Add requirePaymentsEnabled middleware on all checkout routes
- Remove Stripe internal IDs from CSV exports
- Add Cache-Control: no-store on admin settings responses

Bunker Admin
2026-03-31 08:34:23 -06:00
1bf19fff0e Security audit: fix 30 findings across auth, IDOR, XSS, path traversal, infrastructure
Comprehensive 6-domain security audit addressing 8 Critical, 17 Important,
and 5 Low findings. Key fixes:

Critical:
- Strip PII from unauthenticated ticket lookup (IDOR)
- Add role+permission checks to event check-in routes
- Validate tier-to-event ownership on update/delete (IDOR)
- Fix path traversal in video replace (resolve + prefix check)
- Enable MongoDB authentication for Rocket.Chat
- Disable Grafana anonymous access
- Sanitize CSV exports against formula injection (payments)
- Apply DOMPurify to richDescription on public event page (XSS)

Important:
- Require current password for self-service password changes
- Atomic password reset token consumption (race condition fix)
- Scope postMessage to specific origin (not wildcard)
- Validate redirect parameter against open redirect
- Replace weak temp passwords (5760 values → crypto.randomBytes)
- Move shift capacity check inside transaction (TOCTOU fix)
- Fix EVENTS_ADMIN privilege inversion in ticketed events
- Make ENCRYPTION_KEY required (remove optional fallback)
- Add internal Prometheus metrics endpoint for Docker scraping
- Add nginx-level rate limiting (limit_req_zone)
- Fix X-Forwarded-For to use $remote_addr (prevents spoofing)
- Replace CSP stripping with frame-ancestors in embed proxies
- Remove error.message from Fastify 500 responses
- Strip PII from volunteer canvass address data
- Wrap GrapesJS output in {% raw %} to prevent Jinja2 SSTI
- Scope SSE token query param to /sse path only
- Sanitize Listmonk email query against injection

Bunker Admin
2026-03-27 08:47:24 -06:00
3262d92065 Remove hardcoded container names for multi-instance deployment support
- Dashboard: auto-discovers containers from Docker network via socket
  proxy API instead of hardcoded 30-name list. Labels derived from
  docker compose service metadata.
- Email/Settings: mailhog host read from env.SMTP_HOST instead of
  hardcoded 'mailhog-changemaker' string
- Pangolin: grafana container derived from env.GRAFANA_URL hostname;
  newt container/service names from NEWT_CONTAINER_NAME/NEWT_COMPOSE_SERVICE
- SSRF blocklist: built dynamically from all service URL env vars
  instead of hardcoded hostname list
- New env vars: DOCKER_NETWORK_NAME, DOCKER_PROXY_URL,
  NEWT_CONTAINER_NAME, NEWT_COMPOSE_SERVICE

Bunker Admin
2026-03-25 17:35:05 -06:00
a56f8446f7 Fix Pangolin setup: root domain support + disable SSO auth on resources
- Omit subdomain field for root domain resources (Pangolin rejects empty
  string but accepts absent field)
- Set sso:false + blockAccess:false after resource creation so resources
  are publicly accessible without Pangolin auth redirects
- Make subdomain optional in CreateHttpResourcePayload type
- Applied to both /setup and /sync endpoints

Bunker Admin
2026-03-23 15:47:57 -06:00
bb1935027d Upgrade system finished 2026-03-22 21:47:09 -06:00
647efffdc4 Security hardening: JWT algorithm pinning, key separation, injection fixes
- Pin HS256 algorithm on all jwt.verify() calls (9 sites) and jwt.sign()
  calls (3 sites) — prevents algorithm confusion attacks
- Add JWT_INVITE_SECRET env var; volunteer invite tokens now use a
  dedicated key separate from access/refresh secrets
- Remove req.query.secret fallback from Listmonk webhook route — secrets
  must not appear in nginx access logs
- Replace child_process.spawn in email template seed endpoint with direct
  function import; add require.main guard to seed script
- Add sanitizeCsvField() to location CSV export to prevent formula
  injection in Excel/Sheets (=, +, -, @ prefix → apostrophe prefix)
- Cap QR endpoint text input at 2000 chars to prevent DoS via large payloads
- Fix pre-existing TS errors: type participantNeeds as UpsertNeedsInput
  in meeting-planner service; add sso field to UpdateResourcePayload

Bunker Admin
2026-03-22 12:35:04 -06:00
28e4bc9475 Bunch of updates to scheduling 2026-03-15 13:50:09 -06:00
900a0affe5 Add CRM activity enrichment, notification bridging, crash-safe scheduled jobs, and quick wins
Workstream A — CRM & Notifications:
- Add fire-and-forget CRM activity helper (api/src/utils/crm-activity.ts) hooked into
  campaign email, canvass visit, donation, and purchase write sites
- Add 5 operational NotificationType enum values (shift_signup_confirmed, shift_reminder,
  shift_cancelled, canvass_session_summary, reengagement) via Prisma migration
- Bridge notification email queue to in-app notifications for volunteer-facing events
- Extend TYPE_TO_PREF map and NotificationsPage labels for new types

Workstream B — Quick Wins:
- Extract shared role constants (11 roles) to admin/src/utils/role-constants.ts,
  update 4 consuming pages
- Add Ad Analytics sidebar entry in payments submenu
- Gate 6 calendar routes with enableSocialCalendar feature flag
- Add GET /series/:id/count endpoint and fix hardcoded shiftsCount={0} in ShiftsPage
- Add influenceCampaignId to Order model for donation-campaign attribution,
  wire through Stripe checkout metadata

Workstream C — Crash-Safe Scheduled Jobs:
- Create BullMQ scheduled-jobs queue with 10 repeatable job types replacing
  setInterval blocks in server.ts (dynamic imports, concurrency: 2)
- Keep presenceService (1min) and challengeScoringService (5min) as setInterval

Bunker Admin
2026-03-09 14:15:30 -06:00
76b87d9f3d Tonne of things 2026-03-08 18:11:26 -06:00
1cca51e518 Tonne of updates to things like social systems, calendars, and the documentation system (making it mobile friendly and fixing up navigation) 2026-03-07 13:10:08 -07:00
08d8066157 Add ticketed events, Jitsi meeting integration, social features, and calendar system
- Ticketed events: full CRUD, ticket tiers (free/paid/donation), Stripe checkout,
  QR-based check-in scanner, public event pages, ticket confirmation emails
- Event formats: IN_PERSON/ONLINE/HYBRID with auto Jitsi meeting room lifecycle,
  ticket-gated meeting access, moderator JWT tokens, feature-flag guarded
- Social engagement: challenges with scoring/leaderboards, referral tracking,
  volunteer spotlight, impact stories, campaign celebrations, wall of fame
- Social calendar: personal calendar layers, shared calendar items with
  recurrence, scheduling polls, mobile day view
- MCP server: events tool pack with full admin CRUD + meeting token generation
- Unified calendar: eventFormat-aware tags, online event indicators
- Updated docs site, pangolin configs, and various admin UI improvements

Bunker Admin
2026-03-06 14:33:33 -07:00
06ce9dac1b sms updates 2026-02-27 15:02:28 -07:00
621042806a Updates to sms 2026-02-27 08:36:21 -07:00
9e51aac570 Okay Wish I could say I know exactly. Will do better next time promise lol 2026-02-26 17:47:04 -07:00
7352815e57 More control panel updates 2026-02-21 11:46:55 -07:00
1a1f12c45b Tonne of updates 2026-02-18 17:15:31 -07:00
56e262ad8b Tonne of udpatess 2026-02-18 10:01:54 -07:00
99a6abab06 Add video card insert feature + MkDocs video hydration + fixes
- New video card block for GrapesJS landing pages, email templates,
  MkDocs export, and documentation editor Insert dropdown
- Shared HTML generators in admin/src/utils/videoCardHtml.ts
- MkDocs video-player.js hydrates .video-card-block elements:
  thumbnail fix via MEDIA_API_URL, click-to-play inline, Gallery link
- Media API CORS: auto-add MkDocs + docs subdomain origins
- env_config_hook.py: smart Docker hostname detection, ADMIN_PORT
  resolution, pass env vars to MkDocs container
- Gallery URL uses /gallery?expanded=ID format
- VideoPickerModal: fix double /api prefix and Docker hostname thumbs
- Seed: default-video-card PageBlock
- Remove V1 legacy code (influence/, map/)

Bunker Admin
2026-02-17 15:42:32 -07:00
a7978de5a0 Bunch of stuff again 2026-02-16 18:48:54 -07:00
7895ce683e Tonne of debugging - getting ready for the production builds 2026-02-16 10:44:18 -07:00
a77306fac2 Initial v2 commit: complete rebuild with unified API + React admin
Phase 1-14 complete:
- Unified Express.js API (TypeScript, Prisma ORM, PostgreSQL 16)
- React Admin GUI (Vite + Ant Design + Zustand)
- JWT auth with refresh tokens
- Influence: Campaigns, Representatives, Responses, Email Queue
- Map: Locations, Cuts, Shifts, Canvassing System
- NAR data import infrastructure (2025 format)
- Listmonk newsletter integration
- Landing page builder (GrapesJS)
- MkDocs + Code Server integration
- Volunteer portal with GPS tracking
- Monitoring stack (Prometheus, Grafana, Alertmanager)
- Pangolin tunnel integration

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-11 10:05:04 -07:00