(gateway)=
# The LiteLLM Gateway

```{important}
The LiteLLM gateway no longer lives in this repository.
It moved to its own repository, [`~/my/apps/gate`](https://gitlab.com/doering-ai/apps/gate) (`gate/gateway/`), on 2026-08-08.
This repo's old `gateway/` directory was removed.
```

The gateway is operational infrastructure shared by every harness in the fleet.
It handles routing, metering, and fleet-mode policy.
It listens on `localhost:4000` and reaches this repo's local servers over loopback (`:8000`, `:8001`).
It never runs inference itself.

The full reference lives in `~/my/apps/gate/gateway/README.md` and `~/my/apps/gate/gateway/docs/`.
It covers config, secrets, harness wiring, telemetry, and troubleshooting.

## What myModel still owns

- **`vllm/`** provides local vLLM on the RTX 4090. The gateway exposes it as the `local-coder` model alias. See [Serving](serving) and `vllm/README.md`. Start it with `task vllm:up` from this repo's root (no gateway secret needed).
- **`ternary/`** provides the session-bounded Ternary Bonsai 27B profile. The gateway exposes it as `local-bonsai-27b`. See `ternary/README.md`. Start it with `task ternary:up`.

## Connecting a harness

Point any tool at `http://localhost:4000/v1` with `LITELLM_TOKEN` as the bearer key.
Claude Code uses `http://localhost:4000` for its Anthropic surface.
Select `local-coder` or `local-bonsai-27b` by alias, the same as any other gateway route.
Harness wiring for each tool lives in `~/my/apps/gate/gateway/README.md`.

______________________________________________________________________

See also: [Quickstart](quickstart) for the myModel walkthrough, and [myGate](https://gitlab.com/doering-ai/apps/gate) for the gateway itself.