# `myModel`: Local LLM inference and serving

myModel is a collection of tools for deploying local LLMs across multiple runtimes (vLLM, llama.cpp) and platforms, with an optional tuning pipeline.
An external LiteLLM gateway fronts the local servers. See [The LiteLLM Gateway](gateway.md).
Together, these tools form one cohesive group, but they are not a single product.
They work together to serve models over HTTP from your own hardware.

## Install

The source repository is public.
Install from Git with Python 3.13 or newer:

```sh
uv tool install git+https://gitlab.com/doering-ai/apps/model.git
```

Or clone and sync the development environment:

```sh
git clone https://gitlab.com/doering-ai/apps/model.git
cd model
uv sync
```

See the [Quickstart](quickstart) for a full walkthrough.

```{toctree}
:maxdepth: 2
1. Quickstart <quickstart>
2. Serving with vLLM <serving>
3. The LiteLLM Gateway <gateway>
4. Container Deployments <pods>
5. Model Catalogue <catalogue>
6. Fine-tuning <tuning>
7. Ternary Bonsai <ternary>
8. Python API <api>
9. Auth & TLS <auth>
```