What is a .gguf file?
GGML Universal File · Data · llama.cpp / Georgi Gerganov · 2023
GGUF is the file format for running large language models locally with llama.cpp and its many front-ends. One .gguf file holds the whole model: its weights, the tokenizer and all the metadata needed to load it.
What is a .gguf file used for?
GGUF replaced the older GGML format. It starts with the magic bytes GGUF, followed by a version and a key-value metadata block, then the quantized tensor data. Bundling everything in one self-describing file is what makes local models easy to share and run.
Weights are usually quantized (for example Q4_K_M) to shrink a multi-gigabyte model enough to run on consumer hardware. You do not open a .gguf directly; you load it in a runner like llama.cpp, Ollama, LM Studio or Jan.
How to open a .gguf file
The .gguf format opens in the following apps, grouped by operating system. If one app does not work, try another from the same list. For a step-by-step walkthrough, see how to open a .gguf file.
Windows
- LM Studio
- Ollama
- llama.cpp
- Jan
macOS
- LM Studio
- Ollama
- llama.cpp
- Jan
Linux
- llama.cpp
- Ollama
- text-generation-webui
How to convert a .gguf file
You can convert a .gguf file to safetensors (via conversion) using a conversion tool or the export menu of an app that opens it.
.gguf file signature (magic bytes)
Programs recognise a .gguf file by the bytes at the start of the file, not by its name. These are the signatures for GGUF. For a full breakdown, see the GGUF file signature page.
| Hex signature | Offset | Note |
|---|---|---|
| 47 47 55 46 | 0 | 'GGUF' |
See the full magic-numbers reference for every format.
Frequently asked questions
What is a .gguf file?
GGUF is the file format for running large language models locally with llama.cpp and its many front-ends. One .gguf file holds the whole model: its weights, the tokenizer and all the metadata needed to load it.
How do I open a GGUF file?
Load it in a local LLM runner such as LM Studio, Ollama or llama.cpp. It is a model file, not a document, so there is no viewer; you run it to chat with the model.
What does the quantization in a GGUF filename mean?
Tags like Q4_K_M describe how aggressively the weights are compressed. Lower numbers mean smaller files that run on less memory, with some loss of quality.
What is the difference between GGUF and safetensors?
GGUF is built for running models locally with llama.cpp and bundles tokenizer and metadata. Safetensors is a safe storage format used more in training and the Hugging Face ecosystem.
Related formats
Sources
Details on this page were checked against authoritative references: