# Auth & TLS

## `I` Overview

myModel protects two separate surfaces with two different mechanisms:

- **The management API** (Quart, default port 5000) verifies a Keycloak-issued JWT bearer token.
  This layer is active in production mode only.
- **The vLLM inference servers** (ports 5001 and up) use a static API key passed to `vllm serve`.
  This layer is active whenever `$LOCAL_VLLM_API_KEY` is set.

TLS is a third, independent layer.
The management API serves HTTPS when a certificate pair exists under `<directory>/ssl`, and plain HTTP otherwise.

Development mode is the default.
It disables the JWT layer entirely, so the management API accepts unauthenticated requests.
Production mode wraps the whole ASGI app in `AuthMiddleware`, so every management route requires a valid token, including `/health`.

## `II` Running in production mode

The `OpsStance` enum in `mymodel/serve.py` controls the deployment posture.
The launcher selects it from two inputs:

- The `NUCLEUS_MODE` environment variable (default `dev`, from `mymodel/common.py`).
- The `--dev_mode` CLI flag, which forces development mode regardless of `NUCLEUS_MODE`.

Production mode is active when `NUCLEUS_MODE` does not start with `dev` and `--dev_mode` is not passed:

```sh
NUCLEUS_MODE=prd uv run python -m mymodel gemma_md
```

The JWT layer reads its configuration from the process environment:

| Variable          | Purpose                                             | Read by             |
| ----------------- | --------------------------------------------------- | ------------------- |
| `KEYCLOAK_SECRET` | HMAC key used to verify token signatures            | `AuthMiddleware`    |
| `KEYCLOAK_ISSUER` | Expected value of the token's `iss` claim           | `AuthMiddleware`    |
| `NUCLEUS_MODE`    | Deployment posture (`dev` prefix means development) | `mymodel/common.py` |

The `my` package's `env` interface reads these variables.
At import time, the interface also loads the nearest `.env` file via python-dotenv.
An unset variable reads as the empty string.

```{warning}
Never run production mode with `KEYCLOAK_SECRET` unset.
The verifier would then use the empty string as the HMAC key, and a token signed with an empty key would pass verification.
```

## `III` The JWT middleware

`AuthMiddleware` (`mymodel/web.py`) is an ASGI middleware that verifies a Keycloak-issued JWT on every connection.
It is vendored from the fleet's `ai` monorepo, with the `logfire` dependency replaced by stdlib `logging`.
`Semioserve.build_app()` attaches it only in production mode.

### Token extraction

The middleware extracts the token differently per connection type:

- **HTTP**: from the `Authorization` header.
  The header must match `Bearer <token>` exactly (case-sensitive, a single space), because the match is a fullmatch against the pattern `Bearer (?P<token>\S+)`.
- **Websocket**: from the `auth` query parameter, as in `ws://host:port/path?auth=<token>`.

The management API currently defines only HTTP routes (`/models` and `/health`), so the websocket path is dormant.

### Verification steps

`verify_token()` applies these checks in order:

1. A token is present at all.
2. `jwt.decode()` verifies the signature with `KEYCLOAK_SECRET` and the HS512 algorithm (HMAC-SHA-512).
   The algorithm is a constructor parameter, but `Semioserve` always uses the default.
3. The `iss` claim equals `$KEYCLOAK_ISSUER`.
4. The `exp` claim is not in the past.
   PyJWT already rejects expired tokens during decode, so this explicit check is a redundant second pass.
5. The `aud` claim equals the hardcoded string `nexus`.

A token that passes every check lets the request through to the Quart app.
The middleware logs the authenticated client address at INFO level.

## `IV` Failure modes

Rejection looks the same to the client for every failure cause.
The specific reason appears only in the server logs, at ERROR level.

| Cause                                       | HTTP response                           | Websocket response                                |
| ------------------------------------------- | --------------------------------------- | ------------------------------------------------- |
| Missing or malformed `Authorization` header | `401`, `text/plain` body `Unauthorized` | Close code `4000`, reason `Authentication failed` |
| Missing `auth` query parameter              | (n/a)                                   | Close code `4000`, reason `Authentication failed` |
| Bad signature or malformed JWT              | `401`, body `Unauthorized`              | Close code `4000`                                 |
| Expired token                               | `401`, body `Unauthorized`              | Close code `4000`                                 |
| Wrong issuer (`iss`) or audience (`aud`)    | `401`, body `Unauthorized`              | Close code `4000`                                 |

Two edge cases deserve honest treatment:

- **Missing claims crash instead of rejecting cleanly.**
  The verifier indexes `payload['iss']`, `payload['exp']`, and `payload['aud']` directly.
  A well-formed token that lacks one of these claims raises an uncaught `KeyError`.
  This surfaces as a server error rather than a clean `401`.
- **The manual expiry check is unreachable for standard tokens.**
  PyJWT validates `exp` during `jwt.decode()` and raises first.
  The explicit check in `verify_token()` only matters for tokens where PyJWT skipped expiry, which the current call never produces.

## `V` TLS for the management API

The launcher looks for a certificate pair under `<directory>/ssl`, where `<directory>` is the `--directory` CLI argument (default: the `$MY_LOCAL` path):

- `server.crt`: the server certificate.
- `server.key`: the server private key.
- `ca.crt`: an optional CA certificate, loaded only in development mode.

When both `server.crt` and `server.key` exist, `SSLConfig` (`mymodel/web.py`) builds a Hypercorn config from them.
The SSL context enforces TLS 1.2 as the minimum version.
In production mode the CA certificate is not applied, since the JWT middleware handles authentication.

When the pair is absent, the management API falls back to plain HTTP.
This is the typical setup for local development behind loopback.

## `VI` The inference-port API key

The JWT layer protects only the management API.
The vLLM servers that run the actual inference have their own, separate mechanism.
When `$LOCAL_VLLM_API_KEY` is set, `Semioserve` passes it to each `vllm serve` subprocess as `--api-key`.
vLLM then requires that key as a bearer token on its OpenAI-compatible endpoints.
When the variable is unset, the flag is omitted and the inference ports accept unauthenticated requests.

This key is also how the fleet's LiteLLM gateway reaches a local vLLM server.
The gateway lives in its own repository (`~/my/apps/gate/gateway`, `localhost:4000`) and sends `$LOCAL_VLLM_API_KEY` as the upstream credential for its local routes.
Three distinct credentials therefore appear in a full deployment:

- `LITELLM_TOKEN`: client to gateway (managed by the gateway repo, not this one).
- `LOCAL_VLLM_API_KEY`: gateway to vLLM inference servers (this repo, optional).
- Keycloak JWTs: clients to the management API (this repo, production mode only).

## `VII` Current status and limitations

- `AuthMiddleware` and `SSLConfig` have no test coverage.
  No file under `tests/` exercises the middleware, token verification, or the TLS configuration.
- The websocket branch of the middleware is dormant, because the management API has no websocket routes.
- The audience value `nexus` is hardcoded.
  Operators must configure their Keycloak client to issue tokens with that audience.

For the management API endpoints themselves, see [Python API](api).
For the launcher that wires all of this together, see [Serving with vLLM](serving).