# Ternary Bonsai

## `I` Overview

The Ternary Bonsai service is the local quality-floor runtime for active advisor and critic review sessions.
It serves one pinned artifact, the Ternary Bonsai 27B model in Q2_0 GGUF form, through a pinned llama.cpp build.
The raw endpoint binds only to `127.0.0.1:8001`, and the LiteLLM gateway exposes it as `local-bonsai-27b`.

The service is session-bounded by design.
You start it for a review session and stop it afterward.
It has no `[Install]` target, so systemd never starts it at boot, and it forcibly ends any forgotten session after six hours.

Three layers cooperate:

| Layer                                | Role                                                                                |
| ------------------------------------ | ----------------------------------------------------------------------------------- |
| `mymodel/ternary.py`                 | The launcher. Validates every input, then execs the exact verified binary and model |
| `ternary/config.env`                 | The operational session bounds (ports, budgets, deadlines)                          |
| `ternary/ternary-bonsai-27b.service` | The systemd user unit that runs the launcher under cgroup and sandbox limits        |

The launcher holds all immutable identity: artifact paths, the pinned PrismML runtime revision, and the pinned SHA-256 digests.
The **PrismML runtime** is the llama.cpp source tree under `~/.local/share/prism-bonsai/llama.cpp`, pinned to one commit.
`config.env` cannot override any of these values, and `tests/test_ternary.py` rejects a profile that tries.

```{important}
The trust boundary is a single-principal workstation.
The committed loopback token prevents accidental cross-wiring between local services.
It is not isolation from another process running as you.
The project venv and its CUDA/NCCL packages count as trusted, package-managed inputs.
```

## `II` The launcher

`mymodel.ternary` is a single-file worker built on pydantic and the standard library alone.
Its `Worker` model validates one session-bounded server, and its CLI exposes four commands:

| Command     | Effect                                                               |
| ----------- | -------------------------------------------------------------------- |
| `preflight` | Validate immutable inputs and GPU admission, then exit               |
| `serve`     | Run preflight, re-verify the opened inodes, then exec the server     |
| `status`    | Report health from the loopback endpoint, exit 1 when unhealthy      |
| `wait`      | Block until the endpoint answers, with a 120-second default deadline |

### The preflight gate

Preflight **fails closed**: any failed check raises `RuntimeError` before the GPU is committed.
The checks run in this order:

1. The host stays `127.0.0.1` and parallelism stays `1`.
2. The `llama-server` binary exists and is executable, and the model file exists.
3. Each runtime-library directory exists, belongs to root or to you, and is not group- or world-writable.
4. The binary SHA-256 matches the pinned digest.
5. The model SHA-256 matches the pinned digest.
6. `git rev-parse HEAD` in the runtime tree matches the pinned PrismML commit.
7. `nvidia-smi` reports at least 16,000 MiB of free VRAM (the **admission floor**).
8. No competing process holds GPU compute.
   The display processes `kwin_wayland` and `Xorg` may hold a small CUDA context and do not block admission.
9. The loopback port binds successfully.

Every `nvidia-smi` probe also fails closed on empty or malformed output, not only on bad values.

### Verified exec

`serve` does not trust the paths preflight checked.
It opens the binary and model with `O_NOFOLLOW`, hashes the open file descriptors, and compares each digest against the pinned value again.
Hashing the open inode defeats a path swap between check and exec.
It then calls `os.execve` on the verified binary descriptor, passing the model as `/proc/self/fd/<fd>`.

The child environment is minimal.
The launcher forwards an explicit allowlist of non-secret variables (`HOME`, `PATH`, `USER`, cache paths, locale) and adds `LD_LIBRARY_PATH` plus `LLAMA_API_KEY`.
Gateway credentials such as `LITELLM_TOKEN` never cross into the runtime, and the API key never appears on the command line.

### The server contract

The launcher builds a one-slot, bounded-reasoning command:

| Setting                 | Value                | Purpose                            |
| ----------------------- | -------------------- | ---------------------------------- |
| `--host` / `--port`     | `127.0.0.1` / `8001` | Loopback ingress only              |
| `--alias`               | `ternary-bonsai-27b` | The model name clients send        |
| `--ctx-size`            | `262144`             | Full model context                 |
| `--parallel`            | `1`                  | One request at a time              |
| `--reasoning-budget`    | `65536`              | Bounded reasoning effort           |
| `--timeout`             | `1800`               | Thirty-minute per-request deadline |
| `--cache-type-k` / `-v` | `q8_0`               | Quantized KV cache                 |
| `--cache-ram`           | `0`                  | Prompt cache disabled              |
| `--n-gpu-layers all`    | fixed                | Full GPU offload                   |
| `--flash-attn on`       | fixed                | Flash attention                    |
| `--reasoning-format`    | `deepseek`           | Reasoning content format           |

## `III` The systemd unit

`ternary/ternary-bonsai-27b.service` is a user unit of `Type=exec` that runs the launcher three times:

1. `ExecStartPre` runs `preflight`, so a failed gate stops the unit before exec.
2. `ExecStart` runs `serve`, which replaces the process with the verified `llama-server`.
3. `ExecStartPost` runs `wait --timeout 120`, so the unit reports active only when the endpoint answers.

### Session bounds

| Directive                        | Value                | Effect                                               |
| -------------------------------- | -------------------- | ---------------------------------------------------- |
| `RuntimeMaxSec`                  | `6h`                 | Forgotten sessions end after six hours               |
| `StartLimitIntervalSec`/`Burst`  | `6h`/`3`             | Crash loops stop after three starts                  |
| `Restart`                        | `no`                 | No automatic restart                                 |
| `MemoryHigh` / `MemoryMax`       | `20G` / `28G`        | cgroup memory throttle and hard cap                  |
| `TasksMax`                       | `512`                | Thread and process cap                               |
| `OOMPolicy`                      | `stop`               | OOM kills the unit, not the desktop                  |
| `KillMode`                       | `control-group`      | Stop kills the whole process group                   |
| `KillSignal` / `FinalKillSignal` | `SIGINT` / `SIGKILL` | Graceful stop, then force after `TimeoutStopSec=45s` |

### Sandboxing

The unit hides the host from the runtime.
`ProtectHome=tmpfs` mounts an empty home, and explicit `BindReadOnlyPaths` expose only the four trees the runtime needs: the project venv, the uv-managed Python tree behind the venv interpreter symlink, the `mymodel` package, and the PrismML tree.
`ProtectSystem=strict`, `PrivateTmp`, `NoNewPrivileges`, and the kernel and syscall restrictions complete the profile.

## `IV` Configuration

`ternary/config.env` holds the operational session bounds.
Each variable maps to a `Worker` field with the same default:

| Variable                   | Default              | Meaning                                 |
| -------------------------- | -------------------- | --------------------------------------- |
| `TERNARY_PROJECT`          | repo path            | Project root the unit runs from         |
| `TERNARY_MODEL_ALIAS`      | `ternary-bonsai-27b` | Server alias                            |
| `TERNARY_HOST`             | `127.0.0.1`          | Must stay loopback                      |
| `TERNARY_PORT`             | `8001`               | Loopback port                           |
| `TERNARY_CONTEXT_TOKENS`   | `262144`             | Context size                            |
| `TERNARY_PARALLEL`         | `1`                  | Must stay `1`                           |
| `TERNARY_REASONING_BUDGET` | `65536`              | Reasoning budget in tokens              |
| `TERNARY_REQUEST_TIMEOUT`  | `1800`               | Per-request deadline in seconds         |
| `TERNARY_MIN_GPU_FREE_MIB` | `16000`              | GPU admission floor in MiB              |
| `LOCAL_GGUF_TOKEN`         | `local-bonsai-key`   | Loopback token, sent as `LLAMA_API_KEY` |

After you change the profile, run `task ternary:install` to refresh the unit symlink and reload systemd, then repeat the preflight.

## `V` Operating a session

The `ternary:*` Taskfile namespace drives the whole lifecycle:

1. Run `task ternary:preflight` to validate the host before you commit the GPU.
2. Run `task ternary:up` to install the user-unit symlink and start the service.
   This task never enables the unit.
3. Run `task ternary:status` to see the systemd state and the live health endpoint.
4. Run `task ternary:logs` to follow the journal.
5. Run `task ternary:down` to stop the service when the session ends.

The served endpoint is `http://127.0.0.1:8001/v1`, OpenAI-compatible, one request at a time.

Corpus supervises each local merge-gate review in a dedicated process group with a one-hour default ceiling, and a deadline never counts as a successful review.
For an operator-attached session that intentionally needs longer, raise the finite `CORPUS_CRITIC_REVIEW_TIMEOUT_SECONDS` ceiling for that invocation.
Keep the service's thirty-minute per-request bound unchanged.

## `VI` The A100 backend seam

The local RTX 4090 is not the only planned home for this contract.
An operator-attached Colab A100 backend boundary may implement the same `ternary-bonsai-27b` OpenAI-compatible contract when the local GPU is insufficient.

That backend is not built yet.
This repo ships only the local service, and nothing here creates or keeps cloud runtimes alive.
Backend selection stays explicit for each active session.

## `VII` See also

- [Quickstart](quickstart) for the gateway route that fronts this service.
- [The LiteLLM Gateway](gateway.md) for the `local-bonsai-27b` alias and client wiring.
- [Serving with vLLM](serving) for the always-on local serving path, which the session service does not replace.