# Production 403 Errors - Root Cause & Fix

## Diagnosis Summary

**Issue:** All API endpoints returning 302 redirects to Pangolin authentication page
**Root Cause:** Pangolin tunnel resources configured with authentication enabled (should be "Not Protected")
**Status:** CORS configuration ✅ FIXED | Pangolin resources ❌ NEEDS MANUAL FIX

---

## What Was Fixed

### ✅ CORS Configuration (COMPLETED)

**File:** `/home/bunker-admin/changemaker.lite/.env`

**Changes applied:**
```bash
# Changed from development to production
NODE_ENV=production

# Added production domain to CORS whitelist
CORS_ORIGINS=http://app.betteredmonton.org,https://app.betteredmonton.org,http://localhost:3000,http://localhost
```

**API container restarted:** ✅ Done

---

## What Still Needs Manual Fix

### ❌ Pangolin Resource Authentication (REQUIRES MANUAL ACTION)

**Problem:** Resources are configured with authentication, causing 302 redirects to auth page.

**Evidence:**
```bash
$ curl -I https://api.betteredmonton.org/api/health
HTTP/2 302
location: https://pangolin.bnkserve.org/auth/resource/68488f80-b055-41ea-bc1b-0ab905fb8a53?redirect=...
```

**Fix Required:** Change authentication setting for ALL Pangolin resources to "Not Protected"

---

## Step-by-Step Fix Instructions

### 1. Log in to Pangolin Dashboard

URL: https://api.bnkserve.org (remove `/v1` from API URL)

### 2. Navigate to Resources

Dashboard → **Resources** → **Public**

### 3. Edit Each Resource

For EACH of these critical resources:
- ✅ **app.betteredmonton.org** (Admin GUI + Public Pages)
- ✅ **api.betteredmonton.org** (Main API)
- ✅ **media.betteredmonton.org** (Media API)
- db.betteredmonton.org (NocoDB)
- docs.betteredmonton.org (MkDocs)
- code.betteredmonton.org (Code Server)
- git.betteredmonton.org (Gitea)
- n8n.betteredmonton.org (n8n)
- grafana.betteredmonton.org (Grafana)
- listmonk.betteredmonton.org (Listmonk)
- qr.betteredmonton.org (Mini QR)
- home.betteredmonton.org (Homepage)

**Most critical (fix these first):**
1. **api.betteredmonton.org** - Main API (all endpoints fail without this)
2. **app.betteredmonton.org** - Admin GUI (login page won't work)
3. **media.betteredmonton.org** - Media API (video library features)

### 4. Change Authentication Setting

For each resource:
1. Click **Edit** (pencil icon)
2. Find **Authentication** or **Access Policy** section
3. Change from **"Protected"** or **"Authenticated"** to:
   - **"Not Protected"** OR
   - **"Public Access"** OR
   - **"No Authentication"**
   (exact wording depends on Pangolin UI version)
4. Click **Save**

### 5. Verify Fix

After changing authentication settings, test each endpoint:

**Test API:**
```bash
curl https://api.betteredmonton.org/api/health
# Expected: {"status":"healthy","checks":{"database":"ok","redis":"ok"}}
# NOT: 302 redirect
```

**Test Public Campaigns:**
```bash
curl https://api.betteredmonton.org/api/campaigns/public
# Expected: JSON array of campaigns
# NOT: 302 redirect
```

**Test Admin GUI:**
Visit https://app.betteredmonton.org in browser
- Should see login page
- NO redirect to Pangolin auth page

---

## Why This Happened

1. **Pangolin resources default to "Protected"** - requires manual change to "Not Protected"
2. **Manual setup process** - automated setup was removed, so resources must be configured manually
3. **No API enforcement** - Pangolin API doesn't enforce "Not Protected" when creating resources programmatically

---

## Resource Configuration Reference

**Correct settings for ALL resources:**
- **Protocol:** HTTPS (SSL enabled)
- **Target:** nginx:80 (all services route through nginx)
- **Authentication:** **Not Protected** ← THIS IS CRITICAL
- **SSL/TLS:** Enabled

---

## Troubleshooting

### Still seeing 302 redirects after changing settings?

1. **Clear browser cache** - old redirects may be cached
2. **Try incognito/private window**
3. **Wait 30-60 seconds** - Pangolin may need time to update routing
4. **Check resource status** - ensure resource shows as "Active" in Pangolin dashboard
5. **Verify target** - should point to `nginx:80` (not individual service ports)

### API works locally but not via tunnel?

Confirm:
- [ ] Newt container is running: `docker compose ps newt`
- [ ] Newt logs show connection: `docker compose logs newt --tail 50`
- [ ] PANGOLIN_SITE_ID, PANGOLIN_NEWT_ID, PANGOLIN_NEWT_SECRET are set in .env
- [ ] Nginx is running: `docker compose ps nginx`

### Health endpoint works but other endpoints fail?

Check in this order:
1. Test public endpoints (no auth): `/api/campaigns/public`, `/api/shifts/public`
2. Test protected endpoints with valid JWT: `/api/campaigns`, `/api/users`
3. Check auth store in browser DevTools: localStorage should have `auth-storage` with tokens
4. Verify JWT secrets haven't changed (would invalidate existing tokens)

---

## Post-Fix Verification Checklist

After changing Pangolin resource authentication to "Not Protected":

- [ ] Health endpoint returns JSON (not 302): `curl https://api.betteredmonton.org/api/health`
- [ ] Public campaigns endpoint works: `curl https://api.betteredmonton.org/api/campaigns/public`
- [ ] Admin GUI loads: visit https://app.betteredmonton.org
- [ ] Login works: can authenticate with admin credentials
- [ ] Campaign management page loads data (no console errors)
- [ ] Representative lookup functions
- [ ] Public campaign page accessible: https://app.betteredmonton.org/campaigns
- [ ] Map page loads: https://app.betteredmonton.org/map
- [ ] Shifts page works: https://app.betteredmonton.org/shifts

---

## Summary

**What was done:**
1. ✅ Updated `.env` with production CORS origins
2. ✅ Set NODE_ENV to production
3. ✅ Restarted API container
4. ✅ Verified API works locally

**What you need to do:**
1. ❌ Log in to Pangolin dashboard at https://api.bnkserve.org
2. ❌ Edit each resource and set Authentication to "Not Protected"
3. ❌ Verify endpoints no longer return 302 redirects
4. ❌ Test application is fully functional

**Time estimate:** 5-10 minutes to update all 12 resources

---

## Contact

If you encounter issues after following these steps:
- Check Pangolin documentation: https://pangolin.bnkserve.org/docs (if available)
- Review Newt container logs: `docker compose logs newt`
- Verify nginx routing: `docker compose logs nginx | grep betteredmonton.org`