Situation

After I got Keycloak wired up, logging into the dashboard started getting stuck. No error, no crash log. Just one of those situations where you feel like you’re missing something obvious and can’t see it. Today I tried again, and this time there was a clear error in the log: Invalid parameter: redirect_uri

Result First:

Both symptoms trace back to the same root cause.

Trap One: A Chart’s OIDC Options Aren’t All Wired The Same Way

The chart supports pointing OIDC config at an existing Secret instead of writing credentials into values.yaml directly. Sane for anything that shouldn’t sit in git. Point it at a Secret with OIDC_CLIENT_ID, OIDC_CLIENT_SECRET, OIDC_ISSUER_URL, and OIDC_CALLBACK_URL, and the assumption is the app reads all four the same way.

It doesn’t. Reading the chart’s actual deployment template (not just its values.yaml comments) turned up the real logic: in external-secret mode, client-id, client-secret, and issuer-url get wired into CLI flags unconditionally. The callback URL only gets wired if a separate values field (not anything in the Secret) is non-empty:

{{- if or (ne $oidc.callbackURL "") (ne $callbackURL "") }}
- "-oidc-callback-url=$(OIDC_CALLBACK_URL)"
{{- end }}

$oidc.callbackURL here is config.oidc.callbackURL in values, a plain string I’d never set. Because the Secret already had the value and that seemed like the obvious place for it. The Secret key existed. The chart just never looked at it.

The fix that should be step one, not step three: for any chart that supports “point me at your own Secret” mode, read the actual template logic for which fields are wired unconditionally and which need an extra trigger. Don’t assume parallel-looking config fields all follow the same code path just because they’re documented next to each other.

Trap Two: A Silent Fallback Is Worse Than A Hard Failure

With that flag missing, the backend doesn’t error out. It falls back to constructing the callback URL from the request that’s currently hitting it:

func getOidcCallbackURL(r *http.Request, config *HeadlampConfig) string {
    if config.OidcCallbackURL != "" {
        return config.OidcCallbackURL
    }
    // ...falls back to r.Host + a scheme guessed from
    // X-Forwarded-Proto, or TLS state, or a default
}

This “usually correct” fallback is exactly why the first failure, four days earlier, produced no error: most requests arrived over HTTPS, the guessed scheme happened to match what the identity provider expected, and login silently worked by accident almost every time. The one thing that actually broke it was the app being reachable over plain HTTP with no forced redirect to HTTPS, and that wasn’t something I’d have thought to check. Nothing about “OIDC login” pointed at “check whether this app forces HTTPS.”

Reproducing it made the mechanism obvious once I tried both entry points on purpose:

# over HTTPS: the callback URL happens to match what the IdP expects
curl -sk "https://headlamp.example.internal/oidc?cluster=main" \
  | grep -oE 'redirect_uri=[^&"]+'
# redirect_uri=https%3A%2F%2Fheadlamp.example.internal%2Foidc-callback

# over plain HTTP: no forced redirect, so this request actually goes through as-is
curl -s "http://headlamp.example.internal/oidc?cluster=main" \
  | grep -oE 'redirect_uri=[^&"]+'
# redirect_uri=http%3A%2F%2Fheadlamp.example.internal%2Foidc-callback  ← doesn't match what's registered

Same code path, same app. The result depends on which door you walked through: it passes testing whenever the tester happens to use the “normal” path, and only breaks for whoever doesn’t.

Notes

• A fallback that’s “usually right” is a worse bug than one that’s always wrong. Always-wrong gets caught on the first test. Usually-right survives testing and shows up for someone else, later, as something that looks unrelated.
• Config fields that live in the same block of a values file don’t necessarily follow the same code path. If a chart supports pulling secrets from an external Secret, check the template for which keys are actually read unconditionally versus which need an extra explicit trigger.
• When a bug is intermittent, ask what’s different about the environment between “works” and “doesn’t,” not just what’s different in the app’s own logs. In this case the answer wasn’t in Headlamp or Keycloak at all. It was in whether the entry point enforced HTTPS.
• A vague stuck-screen bug and a specific rejected-parameter error can be the same root cause, just surfaced through two different paths. Don’t assume a clearer error message later means a new bug. Check whether it’s the old one finally showing its face.

How To Prevent It

Reference:

• Kubernetes — Authenticating with OpenID Connect Tokens
• Keycloak Server Administration Guide
• Real Incident: Home RKE2 cluster, Headlamp OIDC login via Keycloak