feat: local Keycloak mock + backend auth env wiring

docs/architecture/keycloak-spa-authentication.md

@@ -54,10 +54,16 @@ What the backend needs to add is **token validation** — a FastAPI dependency t
 
 ## Prerequisites
 
-A **Keycloak client registration** is required — someone needs to create a `smartem-frontend` client in the DLS Keycloak realm (`identity.diamond.ac.uk`) with:
+A **Keycloak client registration** is required — someone needs to create a `SmartEM` client in the DLS Keycloak realm (`identity.diamond.ac.uk`) with:
 
 - Client type: public (SPAs can't keep secrets)
 - Valid redirect URIs (the SPA's URL)
 - Web origins (for CORS)
 
 Without this, `keycloak-js` has nowhere to redirect to.
+
+## Local development
+
+For frontend development you do not need access to the DLS identity server. A self-contained Keycloak mock lives under `keycloak-mock/` in this repository and provides a `dls` realm with a pre-configured `SmartEM` client and seeded users.
+
+See [Local Keycloak for SmartEM frontend dev](../development/local-keycloak.md) for setup and integration with the frontend.

docs/development/index.md

@@ -9,6 +9,7 @@ contributing
 tools
 generate-docs
 e2e-simulation
+local-keycloak
 github-labels
 ```
 
@@ -22,6 +23,7 @@ github-labels
 
 - [Development Tools](tools.md) - Utility tools for development, testing, and maintenance
 - [E2E Simulation](e2e-simulation.md) - End-to-end development simulation setup
+- [Local Keycloak](local-keycloak.md) - Self-contained Keycloak mock for frontend auth development
 
 ### Documentation and CI/CD
 

docs/development/local-keycloak.md

@@ -0,0 +1,122 @@
+# Local Keycloak for SmartEM frontend dev
+
+The SmartEM frontend authenticates against Keycloak. For local development there is a self-contained Keycloak mock under `keycloak-mock/` in this repository. It is offered in two equivalent forms — Docker Compose for quick standalone use, and Kubernetes manifests integrated with the rest of the dev stack — both reading from the same realm export.
+
+The architecture of the auth flow itself is documented in [Keycloak SPA authentication](../architecture/keycloak-spa-authentication.md). This page is the operational counterpart: what to run, when, and how to point the frontend at it.
+
+## Why a mock
+
+The DLS identity servers (`identity.diamond.ac.uk`, `identity-test.diamond.ac.uk`) require explicit registration of every client and origin allowed to authenticate. Local development on `http://localhost:5173` is not registered by default, and getting it added is an admin task that has to be repeated whenever the dev port changes or a new developer joins.
+
+Running a local Keycloak instead:
+
+- removes the round-trip with whoever administers the DLS realm;
+- gives every developer the same realm config (checked into version control);
+- works offline;
+- is deterministic — no stale tokens from yesterday's session, no realm config drift.
+
+The mock realm is **not** intended to be representative of the production DLS realm beyond what the frontend needs (client, scopes, a couple of users, a `fedId` claim mapper). It is a development convenience, not a staging environment.
+
+## Which form to use
+
+| Use Compose when… | Use Kubernetes when… |
+|---|---|
+| You only need the frontend running, not the backend stack. | You're already running the dev k3s stack (`./scripts/k8s/dev-k8s.sh up`) and want auth alongside it. |
+| You want the fastest possible cycle (`docker compose up -d` ≈ 20 s). | You want to match how the rest of the dev stack is deployed. |
+| You're debugging Keycloak itself and want minimal moving parts. | You want to validate behaviour close to production deployment shape. |
+
+Both forms read from the same `keycloak-mock/dls-realm.json`, so any change you make to the realm export takes effect for either on next apply.
+
+## Compose form
+
+From `smartem-devtools/`:
+
+```bash
+cd keycloak-mock
+docker compose up -d
+```
+
+Keycloak starts on `http://localhost:8080` in dev mode, imports the `dls` realm from `dls-realm.json`, and is ready in ~20 s. Bootstrap admin credentials are `admin` / `admin` (admin console at `http://localhost:8080`).
+
+State is **not** persisted between container lifecycles — each `up` reimports the realm. That keeps the mock deterministic; if you need to capture interactive admin changes, export the realm via `kc.sh export` from inside the running container and replace `dls-realm.json`.
+
+Tear down with `docker compose down`.
+
+## Kubernetes form
+
+The `keycloak-mock/kustomization.yaml` is referenced as a base by `k8s/environments/development/kustomization.yaml`. Bringing up the dev stack brings up Keycloak alongside it:
+
+```bash
+./scripts/k8s/dev-k8s.sh up
+```
+
+The Keycloak service is reachable as:
+
+- `http://keycloak-service:8080` inside the cluster (ClusterIP);
+- `http://<node-ip>:30090` from outside (NodePort).
+
+To deploy Keycloak alone — for example, on an existing cluster not managed by `dev-k8s.sh`:
+
+```bash
+kubectl apply -k keycloak-mock
+```
+
+The kustomization generates a ConfigMap from `dls-realm.json` and mounts it at `/opt/keycloak/data/import`. Updating the realm requires re-applying the kustomization so the ConfigMap is regenerated, and either deleting the pod or doing a `kubectl rollout restart deployment/keycloak`.
+
+## Realm contents
+
+The realm export at `keycloak-mock/dls-realm.json` defines:
+
+- **Realm**: `dls` — matches `VITE_KEYCLOAK_REALM` defaults so no env change needed.
+- **Client**: `SmartEM` — public client, standard flow, PKCE S256.
+- **Valid redirect URIs**: `http://localhost:5173/*` and `http://localhost:5174/*` (the smartem app and legacy app dev ports).
+- **Web origins**: same hosts (required for silent SSO iframe checks once that's enabled in the frontend).
+- **Custom claim mapper**: `fedId` — the AuthProvider in the SmartEM frontend reads `idTokenParsed.fedId`. The mapper picks up the user attribute and emits it as an ID-token claim.
+- **Seeded users**:
+  - `devuser` / `devpass` — generic, fedId `dev12345`.
+  - `valuser` / `valpass` — for Val Redchenko, fedId `val99999`.
+
+The mock does not implement the full DLS user model (groups, roles, federated identity). Add more users or roles by editing `dls-realm.json` directly.
+
+## Pointing the frontend at it
+
+In `smartem-frontend/apps/smartem/.env.local`:
+
+```env
+VITE_KEYCLOAK_URL=http://localhost:8080      # compose, or k8s with port-forward
+VITE_KEYCLOAK_REALM=dls
+VITE_KEYCLOAK_CLIENT_ID=SmartEM
+VITE_AUTH_ENABLED=true
+```
+
+For the k8s form without port-forward, substitute `http://<node-ip>:30090` for the URL. With a single-node k3s cluster, `node-ip` is the host's IP.
+
+After changing `.env.local`, restart the Vite dev server (env values are read at startup).
+
+## Disabling auth entirely
+
+For UI work that doesn't need to exercise auth, set `VITE_AUTH_ENABLED=false`. The `AuthGate` short-circuits and the SPA renders without contacting Keycloak at all. This is a deliberately separate path from "Keycloak is unavailable" — the latter is an error state to recover from, the former is a development convenience.
+
+## Editing the realm
+
+Modify `dls-realm.json` directly. Both forms read from the same file. After editing:
+
+- **Compose**: `docker compose down && docker compose up -d` — Keycloak reimports on startup.
+- **Kubernetes**: `kubectl apply -k keycloak-mock` (regenerates the ConfigMap with a new hash if `disableNameSuffixHash` is removed) followed by `kubectl rollout restart deployment/keycloak -n smartem-decisions`.
+
+For interactive admin changes you want to keep, use the admin console, then export from inside the container:
+
+```bash
+docker exec smartem-keycloak \
+  /opt/keycloak/bin/kc.sh export \
+  --realm dls \
+  --dir /tmp/export \
+  --users realm_file
+docker cp smartem-keycloak:/tmp/export/dls-realm.json ./dls-realm.json
+```
+
+## Limits and non-goals
+
+- **Not for staging or production.** Dev mode, HTTP only, bootstrap admin credentials, no TLS, no persistent storage.
+- **Not a faithful DLS realm replica.** Groups, federated identity, fine-grained roles, custom themes — all absent. The mock has just enough surface for the frontend's `AuthProvider` to function end-to-end.
+- **Realm export drift.** If the production DLS realm changes its claim shape (new mappers, scope changes), the mock won't track that automatically. Treat it as a snapshot, not a mirror.

env-examples/.env.example.k8s.development

@@ -27,3 +27,13 @@ ADMINER_PORT=8080
 # CORS configuration (comma-separated list of allowed origins)
 # Use "*" for development to allow all origins
 CORS_ALLOWED_ORIGINS=*
+
+# Keycloak OIDC integration (in-cluster mock).
+# KEYCLOAK_VERIFY_ISS=false because tokens are minted with the browser-facing
+# URL (localhost:30090) while the pod fetches JWKS via in-cluster DNS
+# (keycloak-service:8080).
+KEYCLOAK_AUTH_REQUIRED=false
+KEYCLOAK_URL=http://keycloak-service:8080
+KEYCLOAK_REALM=dls
+KEYCLOAK_CLIENT_ID=SmartEM
+KEYCLOAK_VERIFY_ISS=false

env-examples/.env.example.k8s.staging

@@ -28,3 +28,12 @@ ADMINER_PORT=8080
 # For staging, specify exact origins for security
 # Example: "https://staging.example.com,https://staging-app.example.com"
 CORS_ALLOWED_ORIGINS=https://staging.example.com
+
+# Keycloak OIDC integration
+# KEYCLOAK_AUTH_REQUIRED=true makes the backend reject non-exempt requests
+# without a valid Bearer token. KEYCLOAK_URL must reach the realm host.
+KEYCLOAK_AUTH_REQUIRED=true
+KEYCLOAK_URL=https://identity-test.diamond.ac.uk
+KEYCLOAK_REALM=dls
+KEYCLOAK_CLIENT_ID=SmartEM
+KEYCLOAK_VERIFY_ISS=true

k8s/environments/development/configmap.yaml

@@ -14,3 +14,13 @@ data:
   HTTP_API_PORT: "8000"
   ADMINER_PORT: "8080"
   CORS_ALLOWED_ORIGINS: "*"
+  # Keycloak OIDC integration. Points at the in-cluster keycloak-mock
+  # service. KEYCLOAK_VERIFY_ISS=false because tokens are issued with the
+  # browser-facing URL (localhost:30090) while the pod fetches JWKS via
+  # in-cluster DNS (keycloak-service:8080) - signing key matches but the
+  # iss claim does not.
+  KEYCLOAK_AUTH_REQUIRED: "false"
+  KEYCLOAK_URL: "http://keycloak-service:8080"
+  KEYCLOAK_REALM: "dls"
+  KEYCLOAK_CLIENT_ID: "SmartEM"
+  KEYCLOAK_VERIFY_ISS: "false"

k8s/environments/development/kustomization.yaml

@@ -11,6 +11,7 @@ resources:
 - adminer.yaml
 - smartem-http-api.yaml
 - smartem-worker.yaml
+- ../../../keycloak-mock
 
 namePrefix: ""
 namespace: smartem-decisions

k8s/environments/staging/configmap.yaml

@@ -14,3 +14,11 @@ data:
   HTTP_API_PORT: "8000"
   ADMINER_PORT: "8080"
   CORS_ALLOWED_ORIGINS: "https://staging.example.com"
+  # Keycloak OIDC integration. KEYCLOAK_AUTH_REQUIRED=true rejects all
+  # non-exempt requests that don't carry a valid Bearer token. Defaults
+  # below target the DLS test realm; override per environment via .env.
+  KEYCLOAK_AUTH_REQUIRED: "true"
+  KEYCLOAK_URL: "https://identity-test.diamond.ac.uk"
+  KEYCLOAK_REALM: "dls"
+  KEYCLOAK_CLIENT_ID: "SmartEM"
+  KEYCLOAK_VERIFY_ISS: "true"

keycloak-mock/README.md

@@ -0,0 +1,70 @@
+# Keycloak mock
+
+Local Keycloak instance for developing and testing SmartEM frontend authentication without depending on the DLS identity server.
+
+Two deployment forms; pick whichever fits your workflow.
+
+## Contents
+
+| File | Purpose |
+|------|---------|
+| `dls-realm.json` | Realm export. Single source of truth, consumed by both forms. Realm `dls`, client `SmartEM`, two seeded users. |
+| `docker-compose.yml` | Standalone Compose deployment. Fastest path to a running Keycloak. |
+| `keycloak.yaml` | Kubernetes Deployment + Services. |
+| `kustomization.yaml` | Kustomize config; loads the realm JSON as a ConfigMap. Referenced as a base by `k8s/environments/development/kustomization.yaml`, also works standalone. |
+
+## Realm contents
+
+- **Realm**: `dls`
+- **Client**: `SmartEM` (public, PKCE S256, standard flow)
+- **Redirect URIs**: `http://localhost:5173/*`, `http://localhost:5174/*`
+- **Web Origins**: same hosts
+- **Custom claim**: `fedId` (protocol mapper from user attribute), to mirror DLS realm claims
+- **Users**:
+  - `devuser` / `devpass` (Dev User, fedId `dev12345`)
+  - `valuser` / `valpass` (Val Redchenko, fedId `val99999`)
+
+## Compose
+
+```bash
+docker compose up -d
+# admin console: http://localhost:8080  (admin / admin)
+# realm endpoint: http://localhost:8080/realms/dls
+```
+
+State is **not persisted** between container restarts — each `up` reimports the realm. That's deliberate for a mock: deterministic startup, no stale state.
+
+To tear down: `docker compose down`.
+
+## Kubernetes
+
+Comes up automatically with the rest of the dev stack via `./scripts/k8s/dev-k8s.sh`. The keycloak base is wired into `k8s/environments/development/kustomization.yaml`.
+
+To deploy keycloak alone (e.g. on an existing cluster):
+
+```bash
+kubectl apply -k keycloak-mock
+```
+
+Once running, the Keycloak service is reachable inside the cluster at `http://keycloak-service:8080` (ClusterIP) and from outside at `http://<node-ip>:30090` (NodePort). (30090 not 30080 because the SmartEM HTTP API service already owns 30080 in the dev environment.)
+
+## Pointing the frontend at it
+
+In `smartem-frontend/apps/smartem/.env.local`:
+
+```env
+VITE_KEYCLOAK_URL=http://localhost:8080      # compose / port-forward
+# or
+VITE_KEYCLOAK_URL=http://<node-ip>:30090     # k8s NodePort
+VITE_KEYCLOAK_REALM=dls
+VITE_KEYCLOAK_CLIENT_ID=SmartEM
+VITE_AUTH_ENABLED=true
+```
+
+Then `npm run dev:smartem` from the smartem-frontend repo root.
+
+## Editing the realm
+
+Modify `dls-realm.json` directly. Both Compose and Kustomize pick it up on next apply. For interactive admin (UI changes you want to capture), use the admin console at `:8080`, export the realm via `kc.sh export`, and replace `dls-realm.json`.
+
+See [Local Keycloak for SmartEM frontend dev](../docs/development/local-keycloak.md) for the wider context, including which workflow to choose and the auth flow integration.

keycloak-mock/dls-realm.json

@@ -0,0 +1,105 @@
+{
+  "realm": "dls",
+  "enabled": true,
+  "displayName": "Diamond Light Source (local dev)",
+  "registrationAllowed": false,
+  "loginWithEmailAllowed": true,
+  "duplicateEmailsAllowed": false,
+  "resetPasswordAllowed": false,
+  "editUsernameAllowed": false,
+  "bruteForceProtected": false,
+  "sslRequired": "none",
+  "accessTokenLifespan": 300,
+  "ssoSessionIdleTimeout": 1800,
+  "ssoSessionMaxLifespan": 36000,
+  "clients": [
+    {
+      "clientId": "SmartEM",
+      "name": "SmartEM Frontend",
+      "enabled": true,
+      "publicClient": true,
+      "standardFlowEnabled": true,
+      "directAccessGrantsEnabled": false,
+      "implicitFlowEnabled": false,
+      "serviceAccountsEnabled": false,
+      "redirectUris": [
+        "http://localhost:5173/*",
+        "http://localhost:5174/*"
+      ],
+      "webOrigins": [
+        "http://localhost:5173",
+        "http://localhost:5174"
+      ],
+      "attributes": {
+        "pkce.code.challenge.method": "S256",
+        "post.logout.redirect.uris": "http://localhost:5173/*##http://localhost:5174/*"
+      },
+      "defaultClientScopes": [
+        "openid",
+        "profile",
+        "email",
+        "roles",
+        "web-origins"
+      ],
+      "optionalClientScopes": [
+        "offline_access"
+      ],
+      "protocolMappers": [
+        {
+          "name": "fedId",
+          "protocol": "openid-connect",
+          "protocolMapper": "oidc-usermodel-attribute-mapper",
+          "consentRequired": false,
+          "config": {
+            "user.attribute": "fedId",
+            "id.token.claim": "true",
+            "access.token.claim": "true",
+            "userinfo.token.claim": "true",
+            "claim.name": "fedId",
+            "jsonType.label": "String"
+          }
+        }
+      ]
+    }
+  ],
+  "users": [
+    {
+      "username": "devuser",
+      "enabled": true,
+      "emailVerified": true,
+      "firstName": "Dev",
+      "lastName": "User",
+      "email": "devuser@diamond.ac.uk",
+      "attributes": {
+        "fedId": ["dev12345"]
+      },
+      "credentials": [
+        {
+          "type": "password",
+          "value": "devpass",
+          "temporary": false
+        }
+      ],
+      "realmRoles": ["default-roles-dls"]
+    },
+    {
+      "username": "valuser",
+      "enabled": true,
+      "emailVerified": true,
+      "firstName": "Val",
+      "lastName": "Redchenko",
+      "email": "lazyval@gmail.com",
+      "attributes": {
+        "fedId": ["val99999"]
+      },
+      "credentials": [
+        {
+          "type": "password",
+          "value": "valpass",
+          "temporary": false
+        }
+      ],
+      "realmRoles": ["default-roles-dls"]
+    }
+  ]
+}