ggml-org / llama.vim

# llama.vim
Local LLM-assisted text completion.
#### Fill-in-Middle (FIM) completions
#### Instruction-based editing
https://github.com/user-attachments/assets/641a6e72-f1a2-4fe5-b0fd-c2597c6f4cdc
https://github.com/user-attachments/assets/68bff15b-2d91-4800-985d-b7f110a0ccb7
---
## Features
- Auto-suggest on cursor movement in `Insert` mode
- Accept a suggestion with `Tab`
- Accept the first line of a suggestion with `Shift+Tab`
- Instruction-based editing with ` lli`
- Control max text generation time
- Configure scope of context around the cursor
- Ring context with chunks from open and edited files and yanked text
- Supports very large contexts even on low-end hardware via smart context reuse
- Display performance stats
## Installation
### Plugin setup
- vim-plug
 [code] 
- Vundle
 [code] 
 Then add `Plugin 'llama.vim'` to your *.vimrc* in the `vundle#begin()` section.
- lazy.nvim
 [code] 
### Plugin configuration
You can customize *llama.vim* by setting the `g:llama_config` variable.
Examples:
1. Disable the inline info:
 [code] 
2. Same thing but setting directly
 [code] 
3. Disable auto FIM (Fill-In-the-Middle) completion with lazy.nvim
 [code] 
4. Configure FIM keymaps:
 [code] 
5. Configure instruction-based editing keymaps
 [code] 
6. Configure named llama.cpp servers:
 [code] 
Use `:LlamaProfile` to show the available profiles and `:LlamaProfile spark`
to switch both FIM and instruction requests to another server. Profile names
support command-line completion. The selected profile is persisted and
`:LlamaProfileReset` can be used to restore the profile (or custom endpoints)
configured in `.vimrc`.
This only allows the host servers to be specified, not the models used which are
configured using `model_fim` and `model_inst`. But using consistent naming
accross servers, or using server aliases, one can make this independant of the
actual selected profile. For example, using something like:
 [code] 
And then in the server configuration, something like the following can be used:
 [code] 
And likewise for the fim_model.
Please refer to `:help llama_config` or the source
for the full list of options.
### llama.cpp setup
The plugin requires a llama.cpp server instance to be running at `g:llama_config.endpoint_fim` and/or `g:llama_config.endpoint_inst`.
#### Mac OS
 [code] 
#### Windows
 [code] 
#### Any other OS
Either build from source or use the latest binaries: https://github.com/ggml-org/llama.cpp/releases
### llama.cpp settings
Here are recommended settings, depending on the amount of VRAM that you have:
- More than 64GB VRAM:
 [code] 
- More than 16GB VRAM:
 [code] 
- Less than 16GB VRAM:
 [code] 
- Less than 8GB VRAM:
 [code] 
Use `:help llama` for more details.
### Recommended LLMs
The plugin requires FIM-compatible models: HF collection
## Examples
### Using `llama.vim` on M1 Pro (2021) with `Qwen2.5-Coder 1.5B Q8_0`:
The orange text is the generated suggestion. The green text contains performance stats for the FIM request: the curren