Llmash

An Ollama-compatible server and command line for Windows, built on llama.cpp.

It serves your GGUF files through llama-server and keeps Ollama's commands, API and model store, so anything already pointed at Ollama keeps working. What it changes is the llama.cpp settings, picked per model at launch.

Nothing needs to be installed first: the installer fetches the llama.cpp build for your GPU, puts llmash on your PATH, and starts the server. With no NVIDIA card it takes the Vulkan or CPU build instead, and models load into system RAM, which works but is slower.

Ollama's store is found and used as it is. A folder of GGUFs from llama.cpp or LM Studio is one command away:

It reads the folder where it is, subfolders included, and says how many models it found. Nothing is copied, downloaded or deleted, and rm will not touch it. llmash models shows what is being read. It is the same setting as OLLAMA_MODELS , which llmash takes the way Ollama does, pointed at either kind of folder.

One GPU, an RTX PRO 6000 Blackwell with 96 GB: same prompts, 4-bit weights in each engine's own format, every backend run as it comes with no hand tuning. Your numbers will differ.

Tokens per second while generating, median of three runs, excluding model load and prompt processing.

The server owns the model store, starts and stops llama-server processes, and decides how long each one stays resident.