# Model Catalogue

The catalogue is the single source of truth for every inference model myModel knows about.
Instead of hardcoding model paths and vLLM flags in serving code, you declare one **ModelCard** per model in a YAML file.
A **Registry** loads those cards, enriches them with HuggingFace metadata, and the serving layer (`serve.py`) reads them to construct `vllm serve` launch commands.

```{contents}
```

## What is the Model Catalogue?

Every model — local or remote — is a `ModelCard`, a pydantic record with identifiers, type information, provenance metadata, and runtime parameters.
Cards are grouped into YAML catalogue files that the `Registry` loads into memory at startup.

Two seed catalogues ship with the repo:

| File                                      | Contents                                                      |
| ----------------------------------------- | ------------------------------------------------------------- |
| `mymodel/catalogue/data/optimizers.yaml`  | Generative / transformer models (`CHAT`, `CODE`, `DRAW`, ...) |
| `mymodel/catalogue/data/classifiers.yaml` | Reductive models (`EMBED`, `SCORE`, `SEGMENT`, ...)           |

You can point the `Registry` at any YAML file, so you can maintain a private catalogue for experimental models without touching the seeds.
The flow is:

```
YAML catalogue → Registry (loads ModelCards) → serve.py (builds vllm serve) → vLLM subprocess
```

## The Ontology: Verbs, Modalities, and Sizes

Every model is placed in the ontology by two coordinates: its **Verb** (what it does) and its **modality pair** (what it consumes and produces).
A **Size** bucket records how big it is.

### Verbs

A `Verb` is a model's function — the input-to-output transformation it performs.
Each verb carries a default `(input, output)` modality pair inherited by every card with that verb (unless overridden).

| Verb                            | Input → Output              | What it does                            |
| ------------------------------- | --------------------------- | --------------------------------------- |
| `OMNI`                          | any → any                   | Omnimodal (any-to-any)                  |
| `EMBED`                         | text → vector               | Text → embedding vector                 |
| `CHAT`                          | text\|image → text\|image   | Conversational chat (multimodal)        |
| `CODE`                          | text\|code → code           | Code generation                         |
| `COMPLETE`                      | text → text                 | Text completion                         |
| `SUMMARIZE`                     | text → text                 | Summarization (collapses to `COMPLETE`) |
| `SEE`                           | image → text                | Image captioning / VQA                  |
| `DRAW`                          | text → image                | Text-to-image                           |
| `REDRAW`                        | image → image               | Image-to-image                          |
| `LISTEN`                        | audio → text                | Speech-to-text                          |
| `SPEAK`                         | text → speech               | Text-to-speech                          |
| `PLAY`                          | text → music                | Text-to-music                           |
| `REPLAY`                        | audio → audio               | Audio-to-audio                          |
| `ANIMATE`                       | image\|audio → video        | Image+audio → video                     |
| `SHOOT`                         | text → video                | Text-to-video                           |
| `SCULPT`                        | text\|image → facets        | 3D mesh generation                      |
| `TRACK`                         | text\|image → points        | 3D pointcloud tracking                  |
| `BUILD`                         | text\|image → space         | 3D scene construction                   |
| `OCR`                           | pdf\|math → text            | Document OCR                            |
| `PARSE`                         | html\|wiki → text           | Web/wikitext parsing                    |
| `TREND` / `DETREND` / `RETREND` | text↔series / series↔series | Time series                             |
| `CHART` / `DECHART` / `RECHART` | text↔graph / graph↔graph    | Visual graphs                           |
| `DESERIALIZE` / `SERIALIZE`     | text↔json                   | Text ↔ JSON                             |
| `LABEL_TEXT` / `SCORE_TEXT`     | text → class/score          | Text classification/scoring             |
| `LABEL_IMAGE` / `SCORE_IMAGE`   | image → class/score         | Image classification/scoring            |
| `SEGMENT`                       | image → segments            | Image segmentation                      |
| `SHADE`                         | image → regions             | Depth / region mapping                  |
| `LABEL` / `SCORE`               | any → class/score           | Universal classification/scoring        |
| `ENCODE`                        | text → model                | Trainable encoder                       |
| `TRAIN`                         | model → model               | Training step                           |

### Modalities

A `Modality` is a `Flag` enum — a datatype a model can consume or produce.
Models cluster by `(input, output)` modality pairs.
Composite flags combine related atomic modalities:

| Composite   | Members                                           |
| ----------- | ------------------------------------------------- |
| `BASIC`     | `TEXT \| CODE`                                    |
| `IMAGE`     | `PHOTO \| ILLUSTRATION \| DATAVIZ \| FACE`        |
| `AUDIO`     | `SPEECH \| MUSIC`                                 |
| `VIDEO`     | `AVATAR \| CLIP`                                  |
| `SPACE`     | `FACETS \| POINTS \| SCENE`                       |
| `DOCUMENT`  | `PDF \| MATH \| WIKI \| HTML`                     |
| `STRUCTURE` | `JSON \| SVG \| GRAPH \| SERIES`                  |
| `LABEL`     | `CLASS \| SCORE \| VECTOR \| SEGMENTS \| REGIONS` |
| `MODEL`     | `CLASSIFIER \| OPTIMIZER`                         |

The full set of atomic modalities is: `ANY`, `TEXT`, `CODE`, `PHOTO`, `ILLUSTRATION`, `DATAVIZ`, `FACE`, `SPEECH`, `MUSIC`, `AVATAR`, `CLIP`, `FACETS`, `POINTS`, `SCENE`, `PDF`, `MATH`, `WIKI`, `HTML`, `JSON`, `SVG`, `GRAPH`, `SERIES`, `CLASS`, `SCORE`, `VECTOR`, `SEGMENTS`, `REGIONS`, `CLASSIFIER`, `OPTIMIZER`.

#### Parsing modality pairs

Modality pairs are written as `"input -> output"` strings in YAML.
The `PAIR_RGX` regex splits on `->`, `to`, or `2` (case-insensitive): `Modality.read_pair("text -> image")` returns `(Modality.TEXT, Modality.IMAGE)`.
When a card omits `modality`, it inherits the verb's default pair.

### Sizes

A `Size` is a coarse parameter-count bucket, parsed from a model's `_xs`..`_xl` name suffix:

| Size | Suffix | Typical scale |
| ---- | ------ | ------------- |
| `XS` | `_xs`  | < 1B params   |
| `SM` | `_sm`  | 1–7B params   |
| `MD` | `_md`  | 7–13B params  |
| `LG` | `_lg`  | 13B+ params   |
| `XL` | `_xl`  | 70B+ params   |

`Size.get_size_suffix("gemma_md")` returns `Size.MD`.
Sizes are ordered (`XS < SM < MD < LG < XL`), enabling size-based filtering and display.

## The ModelCard

A `ModelCard` is a pydantic model — the persisted, HuggingFace-backed metadata record for one inference model.
Here is the full field reference, grouped by category.

### Identifiers

| Field    | Type  | Default      | Description                                      |
| -------- | ----- | ------------ | ------------------------------------------------ |
| `myname` | `str` | *(required)* | System name (e.g. `gemma_md`). Must be unique.   |
| `hfname` | `str` | *(required)* | HuggingFace repo id (e.g. `google/gemma-7b-it`). |
| `family` | `str` | `''`         | Family name for grouping related variants.       |

### Type

| Field      | Type                        | Default      | Description                                                        |
| ---------- | --------------------------- | ------------ | ------------------------------------------------------------------ |
| `verb`     | `Verb`                      | `CHAT`       | What the model does. Inherited from the YAML top-level key.        |
| `modality` | `tuple[Modality, Modality]` | verb default | `(input, output)` modality pair. Inherited from `verb` if omitted. |
| `engine`   | `str`                       | `''`         | Model framework (`transformers`, `diffusers`, `remote`, etc.).     |
| `hftask`   | `str`                       | `''`         | Auto-generated HuggingFace pipeline tag.                           |
| `usecase`  | `str`                       | `''`         | Manually-entered description of the model's purpose.               |

### Local-specific

| Field    | Type   | Default | Description                                                            |
| -------- | ------ | ------- | ---------------------------------------------------------------------- |
| `size`   | `Size` | `MD`    | Parameter-count bucket. Auto-detected from `myname` suffix if omitted. |
| `f_size` | `str`  | `''`    | Total file size (e.g. `"9GB"`).                                        |
| `params` | `str`  | `''`    | Parameter count (e.g. `"7B"`).                                         |
| `weight` | `int`  | `0`     | Datatype bits (e.g. `16` for fp16, `4` for 4-bit).                     |

### Remote-specific

| Field    | Type    | Default | Description                     |
| -------- | ------- | ------- | ------------------------------- |
| `i_cost` | `float` | `0.0`   | Input cost per million tokens.  |
| `o_cost` | `float` | `0.0`   | Output cost per million tokens. |

### Quantization-specific

| Field    | Type  | Default | Description                                                     |
| -------- | ----- | ------- | --------------------------------------------------------------- |
| `parent` | `str` | `''`    | HF repo of the parent (unquantized) model, for metadata lookup. |
| `target` | `str` | `''`    | Specific quantized file to load from the repo.                  |

### Social metadata

| Field               | Type       | Default | Description                                   |
| ------------------- | ---------- | ------- | --------------------------------------------- |
| `c_time` / `m_time` | `date`     | epoch   | Creation / last-modified date on HuggingFace. |
| `usages` / `hearts` | `int`      | `0`     | HuggingFace download count / likes.           |
| `license`           | `str`      | `''`    | License identifier.                           |
| `papers`            | `set[str]` | `set()` | arXiv paper IDs.                              |
| `tags`              | `set[str]` | `set()` | Descriptive tags (library tags filtered out). |

### Architecture config (`HFConfig`)

| Field                       | Type       | Default | Description                                                        |
| --------------------------- | ---------- | ------- | ------------------------------------------------------------------ |
| `m_type` / `m_auto`         | `str`      | `''`    | `config.json` `model_type` / `transformers_info.auto_model` class. |
| `processor`                 | `str`      | `''`    | Processor class (e.g. `AutoTokenizer`).                            |
| `libraries`                 | `set[str]` | `set()` | Known HuggingFace library tags.                                    |
| `pipeline` / `pipeline_tag` | `str`      | `''`    | Pipeline tags.                                                     |
| `architecture`              | `str`      | `''`    | Architecture class name (e.g. `LlamaForCausalLM`).                 |
| `context_window`            | `int`      | `0`     | Context window size.                                               |

### Runtime parameters

| Field        | Type        | Default | Description                                                |
| ------------ | ----------- | ------- | ---------------------------------------------------------- |
| `local`      | `bool`      | `False` | Whether the model is locally hosted.                       |
| `build_args` | `dict`      | `{}`    | Engine-init kwargs. Become `vllm serve` CLI flags.         |
| `infer_args` | `dict`      | `{}`    | Inference-time params (`temperature`, `max_tokens`, etc.). |
| `callbacks`  | `list`      | `[]`    | Async callbacks executed at lifecycle points.              |
| `files`      | `list[str]` | `[]`    | Model files on HuggingFace.                                |

### `build_args` and `infer_args`

`build_args` are engine-init kwargs that `serve.py` translates into `vllm serve` CLI flags (e.g. `dtype: half` → `--dtype half`, `trust_remote_code: true` → `--trust-remote-code`).
`infer_args` are inference-time parameters (`temperature`, `max_tokens`, etc.) consumed by clients, not the vLLM launcher.

Cards are nested under their `verb` key (top-level YAML), optionally grouped into `families`, with shared fields hoisted to the parent node.

## The Registry

The `Registry` class owns one YAML catalogue file plus its live `ModelCard` instances and a `RegistryConfig`.

### Creating, listing, and looking up

```python
from mymodel.catalogue import Registry, OPTIMIZERS, RegistryConfig, DEFAULT_MODEL_DIR

registry = Registry(OPTIMIZERS)          # from a seed catalogue path
registry = Registry(Path('~/my/catalogue.yaml').expanduser())  # or any YAML file

for myname, card in registry.models.items():   # iterate all cards
    print(f'{myname}: {card.hfname}')

card = registry.models['gemma_md']              # look up by name
```

`OPTIMIZERS` and `CLASSIFIERS` are pre-exported `Path` constants pointing at the seed YAML files in `mymodel/catalogue/data/`.
`DEFAULT_MODEL_DIR` is `~/local/models` — where local model weights are stored.

### Resolving layered args

The `resolve_args` method layers defaults over a card's own `build_args` and `infer_args` in the order: default → verb → family → myname.
This lets you set engine defaults once and override per-verb, per-family, or per-model:

```python
build_args, infer_args = registry.resolve_args(card)
```

### HuggingFace integration

```python
await registry.refresh()  # re-fetch metadata for every local model
```

See [HuggingFace Integration](#huggingface-integration) below for details.

### Adding a new model

```python
await registry.add(hfname='meta-llama/Llama-3.2-3B-Instruct',
                    myname='llama_sm', verb='chat', family='llama')
```

`add()` creates a `ModelCard`, fetches HuggingFace metadata, registers it, and rewrites the YAML index.

### Saving changes

```python
registry.write_index()  # serialize self.models back to the YAML file
```

`write_index` distills shared fields back into family and verb nodes, sorts cards by `(verb, family, myname, size)`, and writes canonical YAML with blank separator lines.

## Creating a New Model Entry

### Step 1: Choose a model from HuggingFace

Find the model on [huggingface.co](https://huggingface.co/models).
Note the repo id, e.g. `Qwen/Qwen2.5-7B-Instruct`.

### Step 2: Determine its verb and modality pair

For a conversational text model, the verb is `CHAT` (text/image → text/image).
For a code model, `CODE` (text/code → code).
For an embedding model, `EMBED` (text → vector).

### Step 3: Determine its size category

Use the parameter count to pick a `Size`: ~7B params → `MD` (`_md`).
Name your card with the suffix: `qwen_md`.

### Step 4: Write the ModelCard YAML entry

Add the card under the appropriate verb in your catalogue file:

```yaml
CHAT:
    families:
        qwen:
            engine: transformers
            license: apache-2.0
            qwen_md:
                hfname: Qwen/Qwen2.5-7B-Instruct
                size: md
                params: 7B
                weight: 16
                build_args:
                    dtype: half
                    gpu_memory_utilization: 0.9
```

### Step 5: Test that the Registry loads it

```python
from mymodel.catalogue import Registry, OPTIMIZERS
r = Registry(OPTIMIZERS)
card = r.models['qwen_md']
print(card.verb, card.modality, card.hfname)
# CHAT (TEXT|IMAGE, TEXT|IMAGE) Qwen/Qwen2.5-7B-Instruct
```

### Step 6: Test that serve.py can launch it

```sh
uv run python -m mymodel qwen_md
curl http://localhost:5000/models   # check the management API
curl http://localhost:5001/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"qwen_md","messages":[{"role":"user","content":"Hello"}]}'
```

### Step 7: Adjust build_args for your GPU

If the model OOMs or is slow, adjust `build_args` — limit `max_model_len` to save VRAM, lower `gpu_memory_utilization`, etc.

## Using the Catalogue with serve.py

The serving layer (`mymodel/serve.py`) is a catalogue-driven vLLM launcher.
It does not hardcode any model — everything comes from the `Registry`.

### Model resolution

`Semioserve.__init__` takes a list of model specs (names or `VERB:name` pairs), loads the catalogue via `Registry(catalogue)`, and resolves each name to a `ModelCard`:

```python
self.registry = Registry(catalogue)
for spec in models:
    myname = spec.split(':', 1)[-1]
    card = self.registry.models[myname]
```

The `_to_flags` function translates a card's resolved `build_args` dict into a `vllm serve` CLI flag list.
Boolean `True` values become bare flags; `False`/`None` are skipped; all other values are stringified.
So `{"trust_remote_code": True, "dtype": "half"}` becomes `['--trust-remote-code', '--dtype', 'half']`.
The full `vllm serve` argv also includes `--served-model-name` (the card's `myname`), `--host`, `--port`, and optionally `--api-key`.

### build_args → vLLM flags

The `_to_flags` function translates a card's resolved `build_args` dict into a `vllm serve` CLI flag list.
Boolean `True` values become bare flags; `False`/`None` are skipped; all other values are stringified.
So `{"trust_remote_code": True, "dtype": "half"}` becomes `['--trust-remote-code', '--dtype', 'half']`.
The full `vllm serve` argv also includes `--served-model-name` (the card's `myname`), `--host`, `--port`, and optionally `--api-key`.

### OpsStance: dev vs prd

`OpsStance` is an enum with two members:

- `DEV` — disables JWT auth on the management API, enables interactive prompts
- `PRD` — enables `AuthMiddleware` (JWT auth) on the management API

The stance is selected via the `--dev_mode` CLI flag or the `NUCLEUS_MODE` environment variable.

### Management API

`Semioserve` exposes a thin Quart management API on the main port:

- `GET /models` — active servers (with live health) and the full catalogue
- `GET /health` — returns `{"status": "ok", "models": <count>}`

In production mode, these routes are wrapped in `AuthMiddleware` (JWT validation).

See [Serving with vLLM](serving) for the full serving guide.

## HuggingFace Integration

The `Registry` fetches metadata from HuggingFace to enrich cards with social, provenance, and technical fields.

### How it works

The `huggingface_hub` library's `hf_api.repo_info(hfname, files_metadata=True)` call retrieves a `ModelInfo` object.
The registry splits this into two parse passes: **`parse_hf_metadata`** (social/provenance: creation date, last-modified date, download count, likes, license, parameter count) and **`parse_hf_data`** (technical: tags, papers, pipeline tag, engine, architecture, model type, auto-model class, processor, file listing).

### Caching

Fetched `ModelInfo` objects are pickled to `~/.local/models/hf_cache/<myname>.pkl`.
The `refresh_days` config (default 7) controls freshness.
If you are offline, the registry will use cached pickles within that window; if no cache exists, `refresh_hf_data` will fail — but the YAML catalogue itself carries enough metadata for `serve.py` to launch models.

```python
registry = Registry(OPTIMIZERS, RegistryConfig(refresh_days=30))
await registry.refresh()  # re-fetch stale cards
```

### DEFAULT_LIBRARIES

The `DEFAULT_LIBRARIES` frozenset contains well-known HuggingFace library tags (`transformers`, `safetensors`, `pytorch`, `diffusers`, etc.).
During tag parsing, the registry partitions `info.tags` into library tags (matching this set) and descriptive tags.
Library tags go into `config.libraries`; descriptive tags go into `card.tags`. arXiv paper IDs (matching `arxiv:<digits>`) are extracted into `card.papers`.