Vibra-Ingenn / Janus

# Janus — Local LLM Server & OpenAI-Compatible API
Janus is a **single Go binary** that runs `.gguf` models on your machine (GPU or CPU) and exposes an **OpenAI-compatible API**. No Python, no Docker, no Ollama required.
**Use it your way:** call it from the **command line** (`curl`, PowerShell, scripts), wire it into **Cursor / Cline / any OpenAI client** — same local models, whatever workflow fits you.
---
## What you get
- **Local inference** — llama.cpp via Vulkan (AMD / Intel / NVIDIA) or CPU fallback
- **OpenAI-compatible API** — `/v1/chat/completions`, `/v1/models`
- **Hot-swap models** — change `.gguf` without restarting
- **Thinking model support** — ` ` reasoning split into `reasoning_content`
- **Chat template auto-detection** — uses the template from GGUF metadata
- **Zero dependencies** — one `.exe` on Windows, no Python, no Docker
---
## Requirements
| Platform | What you need |
|----------|----------------|
| **Windows** (primary) | Windows 10/11, Go 1.25+, Vulkan-capable GPU recommended |
| **Linux** | Go 1.25+, `curl`, `tar`, Vulkan (Mesa/NVIDIA driver; check with `vulkaninfo`) or CPU |
| **macOS** | Go 1.25+, CPU backend (Vulkan varies by hardware) |
**Disk:** plan for the model size (often 2–8 GB per model) plus ~50 MB for Janus + llama.dll.
---
## Quick start (Windows)
### 1. Clone and build
 [code] 
`build.ps1` downloads pre-built **llama.cpp Vulkan DLLs** and compiles `dist\janus.exe`.
### 2. Download a model
Put a `.gguf` file in the `models\` folder. Use the included downloader:
 [code] 
Or download any GGUF from Hugging Face.
### 3. Configure
 [code] 
Edit `.env`:
 [code] 
| Variable | Default | Meaning |
|----------|---------|---------|
| `INFERENCE_BACKEND` | `vulkan` | `vulkan`, `cpu`, or `openrouter` |
| `JANUS_MODEL_PATH` | *(required)* | Path to your `.gguf` file |
| `JANUS_GPU_LAYERS` | `-1` | `-1` = all layers on GPU, `0` = CPU only |
| `JANUS_VRAM_CEILING_MB` | `9216` | VRAM budget hint (MiB) |
| `JANUS_MAX_TOKENS` | `4096` | Max tokens per reply |
| `JANUS_LISTEN_ADDR` | `127.0.0.1:8990` | Bind address |
### 4. Run
 [code] 
Opens **http://127.0.0.1:8990** in your browser.
### 5. Verify
 [code] 
---
## Quick start (Linux)
 [code] 
- `build.sh` pins llama.cpp **b11146** (the struct layouts in `internal/bridge/llama_dl.go` match that tag); `./build.sh --version bNNNN` tries another one, `--skip-download` reuses `lib/linux/`.
- The `.so` files are copied next to the binary (`dist/`); they find each other via `RUNPATH=$ORIGIN`, no `LD_LIBRARY_PATH` needed.
- **Go 1.25+** is required (purego v0.11 is needed to pass llama.cpp's structs by value on Linux). With `GOTOOLCHAIN=auto` (default) an older `go` downloads the right toolchain itself.
- `JANUS_GPU_LAYERS=-1` (default) offloads every layer; check the log for `offloaded N/N layers to GPU`.
Supported: linux/amd64; arm64/macOS not tested/not supported.
macOS: build with `go build -o dist/janus ./cmd/janus` and provide `libllama.dylib` next to the binary (untested).
--